Commit Graph

269 Commits (1f51bb68915eec6ba7e44a4fb184ff68b069d9d2)

Author SHA1 Message Date
Maisem Ali 395cb588b6 types/views: make SliceOf/MapOf panic if they see a pointer
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali d04afc697c cmd/viewer,types/views: add support for views of maps
Updates #4635

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali 9f3ad40707 tailcfg: use cmd/viewer instead of cmd/cloner.
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali c4e9739251 cmd/viewer: add codegen tool for Views
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali e409e59a54 cmd/cloner,util/codegen: refactor cloner internals to allow reuse
Also run go generate again for Copyright updates.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick cc575fe4d6 net/dns: schedule DoH upgrade explicitly, fix Resolver.Addr confusion
Two changes in one:

* make DoH upgrades an explicitly scheduled send earlier, when we come
  up with the resolvers-and-delay send plan. Previously we were
  getting e.g.  four Google DNS IPs and then spreading them out in
  time (for back when we only did UDP) but then later we added DoH
  upgrading at the UDP packet layer, which resulted in sometimes
  multiple DoH queries to the same provider running (each doing happy
  eyeballs dialing to 4x IPs themselves) for each of the 4 source IPs.
  Instead, take those 4 Google/Cloudflare IPs and schedule 5 things:
  first the DoH query (which can use all 4 IPs), and then each of the
  4 IPs as UDP later.

* clean up the dnstype.Resolver.Addr confusion; half the code was
  using it as an IP string (as documented) as half was using it as
  an IP:port (from some prior type we used), primarily for tests.
  Instead, document it was being primarily an IP string but also
  accepting an IP:port for tests, then add an accessor method on it
  to get the IPPort and use that consistently everywhere.

Change-Id: Ifdd72b9e45433a5b9c029194d50db2b9f9217b53
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Joe Tsai 01adcfa688
tailcfg: add omitempty to all fields of Hostinfo (#4360)
This reduces the noise when marshaling only a subset of this type.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
3 years ago
Maisem Ali bd073b8dd6 types/views: rename Generic to Unwrap
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Josh Bleecher Snyder 0868329936 all: use any instead of interface{}
My favorite part of generics.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Maisem Ali 6b9d938c1a types/views: add generic Slice[T] and remove StringSlice
Also make IPPrefixSliceOf use Slice[netaddr.IPPrefix] as it also
provides additional functions besides the standard ones provided by
Slice[T].

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick 61ee72940c all: use Go 1.18's strings.Cut
More remain.

Change-Id: I6ec562cc1f687600758deae1c9d7dbd0d04004cb
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Maisem Ali c85694fac4 types/views: add ContainsExitRoutes to IPPrefixSlice
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali c7a8f0992d ipn/ipnlocal: use views for Peer.PrimaryRoutes and Peer.Tags
RELNOTE=`tailscale status --json` now shows Tags and PrimaryRoutes

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Josh Bleecher Snyder 66f5aa6814 types/logger: add more reserved top level field names
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 84138450a4 types/logger, logtail: add mechanism to do structured JSON logs
e.g. the change to ipnlocal in this commit ultimately logs out:

{"logtail":{"client_time":"2022-02-17T20:40:30.511381153-08:00","server_time":"2022-02-18T04:40:31.057771504Z"},"type":"Hostinfo","val":{"GoArch":"amd64","Hostname":"tsdev","IPNVersion":"1.21.0-date.20220107","OS":"linux","OSVersion":"Debian 11.2 (bullseye); kernel=5.10.0-10-amd64"},"v":1}

Change-Id: I668646b19aeae4a2fed05170d7b279456829c844
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Maisem Ali 72d8672ef7 tailcfg: make Node.Hostinfo a HostinfoView
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali 53998e26a6 tailcfg: introduce HostinfoView
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick 57115e923e tailcfg: add start of SSH policy to be sent from control plane to nodes
Updates #3802

Change-Id: Iec58f35d445aaa267d0f7e7e2f30c049c1df4c0e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 857cd6c0d7 ipn/ipnstate: add TailnetName to status
RELNOTE=tailscale status --json | jq .TailnetName

Fixes tailscale/corp#3665

Change-Id: I85de027ba2781eb31ee1e0c5ab913b0dfa5b4c86
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 41fd4eab5c envknob: add new package for all the strconv.ParseBool(os.Getenv(..))
A new package can also later record/report which knobs are checked and
set. It also makes the code cleaner & easier to grep for env knobs.

Change-Id: Id8a123ab7539f1fadbd27e0cbeac79c2e4f09751
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder deb2f5e793 types/logger: add Context and related helpers
We often need both a log function and a context.
We can do this by adding the log function as a context value.
This commit adds helper glue to make that easy.
It is designed to allow incremental adoption.

Updates tailscale/corp#3138

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Anderson 293431aaea control/noise: use key.Machine{Public,Private} as appropriate.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 0532eb30db all: replace tailcfg.DiscoKey with key.DiscoPublic.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 17b5782b3a types/key: delete legacy NodeKey type.
Fixes #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 7e6a1ef4f1 tailcfg: use key.NodePublic in wire protocol types.
Updates #3206.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson d6e7cec6a7 types/netmap: use key.NodePublic instead of tailcfg.NodeKey.
Update #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 84c3a09a8d types/key: export constants for key size, not a method.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson ff16e58d23 tailcfg: move NodeKey type to types/key.
This leaves behind a type alias and associated constructor, to allow
for gradual switchover.

Updates #3206.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson b43362852c types/key: delete legacy undifferentiated key types.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 743293d473 types/key: remove node key AsPublic/AsPrivate compat shims.
Updates #3206.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson ef241f782e wgengine/magicsock: remove uses of tailcfg.DiscoKey.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 073a3ec416 types/key: correct ShortString representation of DiscoPublic.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 06dccea416 types/key: fix license header on disco files.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 05cc2f510b types/key: new types for disco keys.
Needed for #3206 to remove final uses of key.{Public,Private}.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 15376f975b types/wgkey: delete, no longer used.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Josh Bleecher Snyder 94fb42d4b2 all: use testingutil.MinAllocsPerRun
There are a few remaining uses of testing.AllocsPerRun:
Two in which we only log the number of allocations,
and one in which dynamically calculate the allocations
target based on a different AllocsPerRun run.

This also allows us to tighten the "no allocs"
test in wgengine/filter.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Anderson 6e5175373e types/netmap: use new node key type.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 0c546a28ba types/persist: use new node key type.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson a9c78910bd wgengine/wgcfg: convert to use new node key type.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson bc89a796ec types/key: add a dedicated type for node keys.
Complete with converters to all the other types that represent a
node key today, so the new type can gradually subsume old ones.

Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 9310713bfb all: fix some js/wasm compilation issues
Change-Id: I05a3a4835e225a1e413ec3540a7c7e4a2d477084
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Joe Tsai 9af27ba829 cmd/cloner: mangle "go:generate" in cloner.go
The "go generate" command blindly looks for "//go:generate" anywhere
in the file regardless of whether it is truly a comment.
Prevent this false positive in cloner.go by mangling the string
to look less like "//go:generate".

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
3 years ago
David Crawshaw 45f51d4fa6 types/opt: implement Bool.Scan
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Brad Fitzpatrick aae622314e tailcfg, health: add way for control plane to add problems to health check
So if the control plane knows that something's broken about the node, it can
include problem(s) in MapResponse and "tailscale status" will show it.
(and GUIs in the future, as it's in ipnstate.Status/JSON)

This also bumps the MapRequest.Version, though it's not strictly
required. Doesn't hurt.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder d5ab18b2e6 cmd/cloner: add Clone context to regen struct assignments
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 4bbf5a8636 cmd/cloner: reduce diff noise when changing command
Spelling out the command to run for every type
means that changing the command makes for a large, repetitive diff.
Stop doing that.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Crawshaw 9502b515f1 net/dns: replace resolver IPs with type for DoH
We currently plumb full URLs for DNS resolvers from the control server
down to the client. But when we pass the values into the net/dns
package, we throw away any URL that isn't a bare IP. This commit
continues the plumbing, and gets the URL all the way to the built in
forwarder. (It stops before plumbing URLs into the OS configurations
that can handle them.)

For #2596

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Dave Anderson 980acc38ba
types/key: add a special key with custom serialization for control private keys (#2792)
* Revert "Revert "types/key: add MachinePrivate and MachinePublic.""

This reverts commit 61c3b98a24.

Signed-off-by: David Anderson <danderson@tailscale.com>

* types/key: add ControlPrivate, with custom serialization.

ControlPrivate is just a MachinePrivate that serializes differently
in JSON, to be compatible with how the Tailscale control plane
historically serialized its private key.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 61c3b98a24 Revert "types/key: add MachinePrivate and MachinePublic."
Broke the tailscale control plane due to surprise different serialization.

This reverts commit 4fdb88efe1.
3 years ago
David Anderson 4fdb88efe1 types/key: add MachinePrivate and MachinePublic.
Plumb throughout the codebase as a replacement for the mixed use of
tailcfg.MachineKey and wgkey.Private/Public.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 923c98cd8f types/wgkey: add TODO for a future API change.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Josh Bleecher Snyder 4c8b5fdec4 control/controlclient: do not periodically print full netmap
The netmaps can get really large.
Printing, processing, and uploading them is expensive.
Only print the header on an ongoing basis.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Crawshaw 360223fccb types/dnstype: introduce new package for Resolver
So the type can be used in net/dns without introducing a tailcfg
dependency.

For #2596

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Josh Bleecher Snyder a5da4ed981 all: gofmt with Go 1.17
This adds "//go:build" lines and tidies up existing "// +build" lines.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 171ec9f8f4 control/{controlknobs,controlclient}: simplify knobs API, fix controlclient crash
From integration tests elsewhere:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x845c9b]

goroutine 226 [running]:
tailscale.com/control/controlclient.(*Direct).sendMapRequest(0xc00053e1e0, 0x16670f0, 0xc000353780, 0xffffffffffffffff, 0xc0003e5f10, 0x0, 0x0)
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/direct.go:803 +0x19bb
tailscale.com/control/controlclient.(*Direct).PollNetMap(...)
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/direct.go:574
tailscale.com/control/controlclient.(*Auto).mapRoutine(0xc00052a1e0)
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/auto.go:464 +0x571
created by tailscale.com/control/controlclient.(*Auto).Start
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/auto.go:151 +0x65
exit status 2

Also remove types/opt.Bool API addition which is now unnecessary.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
julianknodt 1bb6abc604 net/portmapper: add upnp port mapping
Add in UPnP portmapping, using goupnp library in order to get the UPnP client and run the
portmapping functions. This rips out anywhere where UPnP used to be in portmapping, and has a
flow separate from PMP and PCP.

RELNOTE=portmapper now supports UPnP mappings

Fixes #682
Updates #2109

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
Brad Fitzpatrick 8a4dffee07 types/logger: fix deadlock RateLimitedFn reentrancy
Fix regression from 19c3e6cc9e
which made the locking coarser.

Found while debugging #2245, which ended up looking like a tswin/Windows
issue where Crawshaw had blocked cmd.exe's output.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Crawshaw 297b3d6fa4 staticcheck.conf: turn off noisy lint errors
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Brad Fitzpatrick e66d4e4c81 tailcfg, types/wgkey: add AppendTo methods on some types
Add MarshalText-like appending variants. Like:
https://pkg.go.dev/inet.af/netaddr#IP.AppendTo

To be used by @josharian's pending deephash optimizations.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Josh Bleecher Snyder 5666663370 net/packet: use netaddr AppendTo methods
This lets us remote the types/strbuilder package,
which had only a single user.
And it's faster.

name              old time/op    new time/op    delta
String/tcp4-8        175ns ± 0%      58ns ± 1%  -66.95%  (p=0.000 n=10+9)
String/tcp6-8        226ns ± 1%     136ns ± 1%  -39.85%  (p=0.000 n=10+10)
String/udp4-8        175ns ± 1%      58ns ± 1%  -67.01%  (p=0.000 n=10+9)
String/udp6-8        230ns ± 1%     140ns ± 0%  -39.32%  (p=0.000 n=10+9)
String/icmp4-8       164ns ± 0%      50ns ± 1%  -69.89%  (p=0.000 n=10+10)
String/icmp6-8       217ns ± 1%     129ns ± 0%  -40.46%  (p=0.000 n=10+10)
String/igmp-8        196ns ± 0%      56ns ± 1%  -71.32%  (p=0.000 n=10+10)
String/unknown-8    2.06ns ± 1%    2.06ns ± 2%     ~     (p=0.985 n=10+10)

name              old alloc/op   new alloc/op   delta
String/tcp4-8        32.0B ± 0%     32.0B ± 0%     ~     (all equal)
String/tcp6-8         168B ± 0%       96B ± 0%  -42.86%  (p=0.000 n=10+10)
String/udp4-8        32.0B ± 0%     32.0B ± 0%     ~     (all equal)
String/udp6-8         168B ± 0%       96B ± 0%  -42.86%  (p=0.000 n=10+10)
String/icmp4-8       32.0B ± 0%     32.0B ± 0%     ~     (all equal)
String/icmp6-8        104B ± 0%       64B ± 0%  -38.46%  (p=0.000 n=10+10)
String/igmp-8        48.0B ± 0%     48.0B ± 0%     ~     (all equal)
String/unknown-8     0.00B          0.00B          ~     (all equal)

name              old allocs/op  new allocs/op  delta
String/tcp4-8         1.00 ± 0%      1.00 ± 0%     ~     (all equal)
String/tcp6-8         3.00 ± 0%      1.00 ± 0%  -66.67%  (p=0.000 n=10+10)
String/udp4-8         1.00 ± 0%      1.00 ± 0%     ~     (all equal)
String/udp6-8         3.00 ± 0%      1.00 ± 0%  -66.67%  (p=0.000 n=10+10)
String/icmp4-8        1.00 ± 0%      1.00 ± 0%     ~     (all equal)
String/icmp6-8        3.00 ± 0%      1.00 ± 0%  -66.67%  (p=0.000 n=10+10)
String/igmp-8         1.00 ± 0%      1.00 ± 0%     ~     (all equal)
String/unknown-8      0.00           0.00          ~     (all equal)

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder 25df067dd0 all: adapt to opaque netaddr types
This commit is a mishmash of automated edits using gofmt:

gofmt -r 'netaddr.IPPort{IP: a, Port: b} -> netaddr.IPPortFrom(a, b)' -w .
gofmt -r 'netaddr.IPPrefix{IP: a, Port: b} -> netaddr.IPPrefixFrom(a, b)' -w .

gofmt -r 'a.IP.Is4 -> a.IP().Is4' -w .
gofmt -r 'a.IP.As16 -> a.IP().As16' -w .
gofmt -r 'a.IP.Is6 -> a.IP().Is6' -w .
gofmt -r 'a.IP.As4 -> a.IP().As4' -w .
gofmt -r 'a.IP.String -> a.IP().String' -w .

And regexps:

\w*(.*)\.Port = (.*)  ->  $1 = $1.WithPort($2)
\w*(.*)\.IP = (.*)  ->  $1 = $1.WithIP($2)

And lots of manual fixups.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder e9066ee625 types/wgkey: optimize Key.ShortString
name           old time/op    new time/op    delta
ShortString-8    82.6ns ± 0%    15.6ns ± 0%  -81.07%  (p=0.008 n=5+5)

name           old alloc/op   new alloc/op   delta
ShortString-8      104B ± 0%        8B ± 0%  -92.31%  (p=0.008 n=5+5)

name           old allocs/op  new allocs/op  delta
ShortString-8      3.00 ± 0%      1.00 ± 0%  -66.67%  (p=0.008 n=5+5)

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder 7cd4766d5e types/wgkey: add BenchmarkShortString
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder 05bed64772 types/wgkey: simplify Key.UnmarshalJSON
Instead of calling ParseHex, do the hex.Decode directly.

name             old time/op    new time/op    delta
UnmarshalJSON-8    86.9ns ± 0%    42.6ns ± 0%   -50.94%  (p=0.000 n=15+14)

name             old alloc/op   new alloc/op   delta
UnmarshalJSON-8      128B ± 0%        0B       -100.00%  (p=0.000 n=15+15)

name             old allocs/op  new allocs/op  delta
UnmarshalJSON-8      2.00 ± 0%      0.00       -100.00%  (p=0.000 n=15+15)

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Brad Fitzpatrick eb06ec172f wgengine/netstack: don't pass non-subnet traffic to netstack in hybrid mode
Fixes tailscale/corp#1725

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Josh Bleecher Snyder 78d4c561b5 types/logger: add key grinder stats lines to rate-limiting exemption list
Updates #1749

Co-authored-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder f116a4c44f types/logger: fix rate limiter allowlist
Upstream wireguard-go renamed the interface method
from CreateEndpoint to ParseEndpoint.
I updated the log call site but not the allowlist.

Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>
4 years ago
Avery Pennarun 19c3e6cc9e types/logger: rate limited: more hysteresis, better messages.
- Switch to our own simpler token bucket, since x/time/rate is missing
  necessary stuff (can't provide your own time func; can't check the
  current bucket contents) and it's overkill anyway.

- Add tests that actually include advancing time.

- Don't remove the rate limit on a message until there's enough room to
  print at least two more of them. When we do, we'll also print how
  many we dropped, as a contextual reminder that some were previously
  lost. (This is more like how the Linux kernel does it.)

- Reformat the [RATE LIMITED] messages to be shorter, and to not
  corrupt original message. Instead, we print the message, then print
  its format string.

- Use %q instead of \"%s\", for more accurate parsing later, if the
  format string contained quotes.

Fixes #1772

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
4 years ago
Josh Bleecher Snyder 4037fc25c5 types/wgkey: use value receiver with MarshalJSON
Pointer receivers used with MarshalJSON are code rakes.

https://github.com/golang/go/issues/22967
https://github.com/dominikh/go-tools/issues/911

I just stepped on one, and it hurt. Turn it over.
While we're here, optimize the code a bit.

name           old time/op    new time/op    delta
MarshalJSON-8     184ns ± 0%      44ns ± 0%  -76.03%  (p=0.000 n=20+19)

name           old alloc/op   new alloc/op   delta
MarshalJSON-8      184B ± 0%       80B ± 0%  -56.52%  (p=0.000 n=20+20)

name           old allocs/op  new allocs/op  delta
MarshalJSON-8      4.00 ± 0%      1.00 ± 0%  -75.00%  (p=0.000 n=20+20)

Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>
4 years ago
Josh Bleecher Snyder 7ee891f5fd all: delete wgcfg.Key and wgcfg.PrivateKey
For historical reasons, we ended up with two near-duplicate
copies of curve25519 key types, one in the wireguard-go module
(wgcfg) and one in the tailscale module (types/wgkey).
Then we moved wgcfg to the tailscale module.
We can now remove the wgcfg key type in favor of wgkey.

Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>
4 years ago
Brad Fitzpatrick 6d64107f26 types/netmap: remove some old TODOs
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 64c80129f1 types/netmap: add some docs/warning to NetworkMap
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 2074dfa5e0 types/preftype: don't use iota for consts persisted to disk
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 01b90df2fa net/packet, wgengine/filter: support SCTP
Add proto to flowtrack.Tuple.

Add types/ipproto leaf package to break a cycle.

Server-side ACL work remains.

Updates #1516

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick c64bd587ae net/portmapper: add NAT-PMP client, move port mapping service probing
* move probing out of netcheck into new net/portmapper package
* use PCP ANNOUNCE op codes for PCP discovery, rather than causing
  short-lived (sub-second) side effects with a 1-second-expiring map +
  delete.
* track when we heard things from the router so we can be less wasteful
  in querying the router's port mapping services in the future
* use portmapper from magicsock to map a public port

Fixes #1298
Fixes #1080
Fixes #1001
Updates #864

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 1e7a35b225 types/netmap: split controlclient.NetworkMap off into its own leaf package
Updates #1278

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick ddfcc4326c types/persist: split controlclient.Persist into a small leaf package
This one alone doesn't modify the global dependency map much
(depaware.txt if anything looks slightly worse), but it leave
controlclient as only containing NetworkMap:

bradfitz@tsdev:~/src/tailscale.com/ipn$ grep -F "controlclient." *.go
backend.go:     NetMap        *controlclient.NetworkMap // new netmap received
fake_test.go:   b.notify(Notify{NetMap: &controlclient.NetworkMap{}})
fake_test.go:   b.notify(Notify{NetMap: &controlclient.NetworkMap{}})
handle.go:      netmapCache       *controlclient.NetworkMap
handle.go:func (h *Handle) NetMap() *controlclient.NetworkMap {

Once that goes into a leaf package, then ipn doesn't depend on
controlclient at all, and then the client gets smaller.

Updates #1278
4 years ago
Brad Fitzpatrick d76334d2f0 ipn: split LocalBackend off into new ipn/ipnlocal package
And move a couple other types down into leafier packages.

Now cmd/tailscale doesn't bring in netlink, magicsock, wgengine, etc.

Fixes #1181

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Josh Bleecher Snyder 1e28207a15 types/logger: fix rateFree interaction with verbosity prefixes
We log lines like this:

c.logf("[v1] magicsock: disco: %v->%v (%v, %v) sent %v", c.discoShort, dstDisco.ShortString(), dstKey.ShortString(), derpStr(dst.String()), disco.MessageSummary(m))

The leading [v1] causes it to get unintentionally rate limited.
Until we have a proper fix, work around it.

Fixes #1216

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
David Anderson 78338ac029 types/logger: trim spaces from the rate-limited example message.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
Josh Bleecher Snyder 2d837f79dc wgengine/magicsock: close test loggers once we're done with them
This is a big hammer approach to helping with #1132.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
David Anderson 22507adf54 wgengine/magicsock: stop depending on UpdateDst in legacy codepaths.
This makes connectivity between ancient and new tailscale nodes slightly
worse in some cases, but only in cases where the ancient version would
likely have failed to get connectivity anyway.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
Smitty 2bf49ddf90 Provide example when format string is rate limited
Here's an example log line in the new format:
    [RATE LIMITED] format string "open-conn-track: timeout opening %v; no associated peer node" (example: "open-conn-track: timeout opening ([ip] => [ip]); no associated peer node")
This should make debugging logging issues a bit easier, and give more
context as to why something was rate limited. This change was proposed
in a comment on #1110.

Signed-off-by: Smitty <me@smitop.com>
4 years ago
Josh Bleecher Snyder 1e4604f60e wgengine: quiet some wireguard-go logging
The log lines that wireguard-go prints as it starts
and stops its worker routines are mostly noise.
They also happen after other work is completed,
which causes failures in some of the log testing packages.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder 56a7652dc9 wgkey: new package
This is a replacement for the key-related parts
of the wireguard-go wgcfg package.

This is almost a straight copy/paste from the wgcfg package.
I have slightly changed some of the exported functions and types
to avoid stutter, added and tweaked some comments,
and removed some now-unused code.

To avoid having wireguard-go depend on this new package,
wgcfg will keep its key types.

We translate into and out of those types at the last minute.
These few remaining uses will be eliminated alongside
the rest of the wgcfg package.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Smitty f0b0a62873 Clarify that raw format strings are intentional
This caused some confusion in issue #460, since usually raw format
strings aren't printed directly. Hopefully by directly logging that
they are intended to be raw format strings, this will be more clear.
Rate limited format strings now look like:

  [RATE LIMITED] format string "control: sendStatus: %s: %v"

Closes #460.

Signed-off-by: Smitty <me@smitop.com>
4 years ago
Brad Fitzpatrick 19b0cfe89e all: prepare for GOOS=ios in Go 1.16
Work with either way for now on iOS (darwin/arm64 vs ios/arm64).

In February when Go 1.16 comes out we'll have a universal binary for
darwin/arm64 (macOS) and will drop support for Go 1.15 and its
darwin/amd64 meaning iOS. (it'll mean macOS).

Context:

* https://tip.golang.org/doc/go1.16#darwin
* https://github.com/golang/go/issues/38485
* https://github.com/golang/go/issues/42100
4 years ago
Brad Fitzpatrick 8b904b1493 types/logger: fix LogOnChange to pass through format/args to underlying logger
So they don't get interpretted as a format pattern or get rate-limited away
in the wrong way.
4 years ago
Brad Fitzpatrick 691f1d5c1d types/flagtype: fix bug showing the default port value (shown in --help) 4 years ago
Brad Fitzpatrick 86c271caba types/logger: move RusagePrefixLog to logger package, disable by default
The RusagePrefixLog is rarely useful, hasn't been useful in a long
time, is rarely the measurement we need, and is pretty spammy (and
syscall-heavy). Disable it by default. We can enable it when we're
debugging memory.
4 years ago
Brad Fitzpatrick 8b94a769be cmd/tailscaled: use the standard flag page instead of getopt
Per discussion with @crawshaw. The CLI tool already used std flag anyway.
If either of them, it would've made more sense for the CLI to use getopt.
4 years ago
Brad Fitzpatrick 309c15dfdd types/key: restore Curve25519 clamping in NewPrivate
It was lost during a copy from wgcfg.NewPresharedKey (which doesn't
clamp) instead of wgcfg.NewPrivateKey (which does).

Fortunately this was only use for discovery messages (not WireGuard)
and only for ephemeral process-lifetime keys.
4 years ago
Elias Naur fa45d606fa types/logger: fix go test vet error
Silences

types/logger/logger_test.go:63:30: conversion from int to string yields a string of one rune

Signed-off-by: Elias Naur <mail@eliasnaur.com>
4 years ago
Brad Fitzpatrick 6c74065053 wgengine/magicsock, tstest/natlab: start hooking up natlab to magicsock
Also adds ephemeral port support to natlab.

Work in progress.

Pairing with @danderson.
4 years ago
Brad Fitzpatrick 0ea51872c9 types/logger: add rateFreePrefix rate-limiting-exempt log format prefixes
Per conversation with @danderson.
4 years ago
Brad Fitzpatrick 0071888a17 types/opt: add Bool.EqualBool method
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 53fb25fc2f all: generate discovery key, plumb it around
Not actually used yet.

Updates #483
4 years ago
Dmytro Shynkevych c12d87c54b
Fix concurrency issues in controlclient, ipn, types/logger (#456)
Signed-Off-By: Dmytro Shynkevych <dmytro@tailscale.com>
5 years ago
Brad Fitzpatrick dd6b96ba68 types/logger: add TS_DEBUG_LOG_RATE knob to easily turn off rate limiting 5 years ago
Brad Fitzpatrick 3f4a567032 types/strbuilder: add a variant of strings.Builder that uses sync.Pool
... and thus does not need to worry about when it escapes into
unprovable fmt interface{} land.

Also, add some convenience methods for efficiently writing integers.
5 years ago
Avery Pennarun af9328c1b7 log rate limiting: reformat limiter messages, and use nonempty burst size.
- Reformat the warning about a message being rate limited to print the
  format string, rather than the formatted message. This helps give a
  clue what "type" of message is being limited.

- Change the rate limit warning to be [RATE LIMITED] in all caps. This
  uses less space on each line, plus is more noticeable.

- In tailscaled, change the frequency to be less often (once every 5
  seconds per format string) but to allow bursts of up to 5 messages.
  This greatly reduces the number of messages that are rate limited
  during startup, but allows us to tighten the limit even further during
  normal runtime.

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
Wendi Yu bb55694c95
wgengine: log node IDs when peers are added/removed (#381)
Also stop logging data sent/received from nodes we're not connected to (ie all those `x`s being logged in the `peers: ` line)
Signed-off-by: Wendi <wendi.yu@yahoo.ca>
5 years ago
Brad Fitzpatrick fe97bedf67 types/logger: add ArgWriter 5 years ago
Brad Fitzpatrick 8eda667aa1 types/logger: simplify mutex locking in rate-limited logger
Updates #365

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Anderson 48b1e85e8a types/logger: fix deadlock in the burst case.
Fixes #365.

Signed-off-by: David Anderson <danderson@tailscale.com>
5 years ago
Wendi Yu 0c69b4e00d
Implement rate limiting on log messages (#356)
Implement rate limiting on log messages

Addresses issue #317, where logs can get spammed with the same message
nonstop. Created a rate limiting closure on logging functions, which
limits the number of messages being logged per second based on format
string. To keep memory usage as constant as possible, the previous cache
purging at periodic time intervals has been replaced by an LRU that
discards the oldest string when the capacity of the cache is reached.


Signed-off-by: Wendi Yu <wendi.yu@yahoo.ca>
5 years ago
Brad Fitzpatrick fefd7e10dc types/structs: add structs.Incomparable annotation, use it where applicable
Shotizam before and output queries:

sqlite> select sum(size) from bin where func like 'type..%';
129067
=>
120216
5 years ago
Brad Fitzpatrick 45f2b53aca all: remove unnecessary trailing newlines in format patterns for consistency
And document on logger.Logf that it's unnecessary.
5 years ago
Brad Fitzpatrick 922d9546bf wgengine: don't allocate so much in userspaceEngine.getStatus
It was one of the top garbage producers on my phone.

It's slated to be deleted and replaced anyway, but this helps in the
meantime.

The go.sum changes look scary, but the new dep only adds 240 bytes to
the binary. The go.sum noise is just cmd/go being aggressive in
including a lot of stuff (which is being fixed in Go 1.15, for what I
understand). And I ran a go mod tidy, which added some too. (I had to
write a custom wrapper around go mod tidy because this mod tidy
normally breaks on tailscale.io/control being missing but referenced
in tests)

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Anderson 96b2f20c5b types/logger: add Discard helper. 5 years ago
Brad Fitzpatrick 810c1e9704 types/key: make Public implement TextMarshaler, TextUnmarshaler
So it can be a map key with encoding/json
5 years ago
Brad Fitzpatrick 322499473e cmd/tailscaled, wgengine, ipn: add /debug/ipn handler with world state
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 946df89fa6 types/logger: add adapters for Logf to std Logger/Writer 5 years ago
Brad Fitzpatrick a6c695ba6b types/key: add IsZero methods
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick a07af762e4 types/opt: add opt package for a new opt.Bool JSON type
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 3988ddc85d types/logger: add WithPrefix, use it in two places
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 769e25e37b derp: use new types/key package
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 2896be60db Move "logger" package to under types, now that we have it.
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 747c7d7ce2 types/empty: add Message, stop using mysterious *struct{}
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago