Commit Graph

2906 Commits (fd4838dc57846ef15b51d23a867874efb93a8eb6)
 

Author SHA1 Message Date
julianknodt 3377089583 tsweb: add float64 to logged metrics
A previously added metric which was float64 was being ignored in tsweb, because it previously
only accepted int64 and ints. It can be handled in the same way as ints.

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
Brad Fitzpatrick 53a2f63658 net/dns/resolver: race well-known resolvers less aggressively
Instead of blasting away at all upstream resolvers at the same time,
make a timing plan upon reconfiguration and have each upstream have an
associated start delay, depending on the overall forwarding config.

So now if you have two or four upstream Google or Cloudflare DNS
servers (e.g. two IPv4 and two IPv6), we now usually only send a
query, not four.

This is especially nice on iOS where we start fewer DoH queries and
thus fewer HTTP/1 requests (because we still disable HTTP/2 on iOS),
fewer sockets, fewer goroutines, and fewer associated HTTP buffers,
etc, saving overall memory burstiness.

Fixes #2436
Updates tailscale/corp#2250
Updates tailscale/corp#2238

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick e94ec448a7 net/dns/resolver: add forwardQuery type as race work prep
Add a place to hang state in a future change for #2436.
For now this just simplifies the send signature without
any functional change.

Updates #2436

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 064b916b1a net/dns/resolver: fix func used as netaddr.IP in printf
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Joe Tsai d145c594ad
util/deephash: improve cycle detection (#2470)
The previous algorithm used a map of all visited pointers.
The strength of this approach is that it quickly prunes any nodes
that we have ever visited before. The detriment of the approach
is that pruning is heavily dependent on the order that pointers
were visited. This is especially relevant for hashing a map
where map entries are visited in a non-deterministic manner,
which would cause the map hash to be non-deterministic
(which defeats the point of a hash).

This new algorithm uses a stack of all visited pointers,
similar to how github.com/google/go-cmp performs cycle detection.
When we visit a pointer, we push it onto the stack, and when
we leave a pointer, we pop it from the stack.
Before visiting a pointer, we first check whether the pointer exists
anywhere in the stack. If yes, then we prune the node.
The detriment of this approach is that we may hash a node more often
than before since we do not prune as aggressively.

The set of visited pointers up until any node is only the
path of nodes up to that node and not any other pointers
that may have been visited elsewhere. This provides us
deterministic hashing regardless of visit order.
We can now delete hashMapFallback and associated complexity,
which only exists because the previous approach was non-deterministic
in the presence of cycles.

This fixes a failure of the old algorithm where obviously different
values are treated as equal because the pruning was too aggresive.
See https://github.com/tailscale/tailscale/issues/2443#issuecomment-883653534

The new algorithm is slightly slower since it prunes less aggresively:
	name              old time/op    new time/op    delta
	Hash-8              66.1µs ± 1%    68.8µs ± 1%   +4.09%        (p=0.000 n=19+19)
	HashMapAcyclic-8    63.0µs ± 1%    62.5µs ± 1%   -0.76%        (p=0.000 n=18+19)
	TailcfgNode-8       9.79µs ± 2%    9.88µs ± 1%   +0.95%        (p=0.000 n=19+17)
	HashArray-8          643ns ± 1%     653ns ± 1%   +1.64%        (p=0.000 n=19+19)
However, a slower but more correct algorithm seems
more favorable than a faster but incorrect algorithm.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
3 years ago
Brad Fitzpatrick 7b295f3d21 net/portmapper: disable UPnP on iOS for now
Updates #2495

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 4a2c3e2a0a control/controlclient: grow goroutine debug buffer as needed
To not allocate 1MB up front on iOS.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 1986d071c3 control/controlclient: don't use regexp in goroutine stack scrubbing
To reduce binary size on iOS.

Updates tailscale/corp#2238

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill 60f34c70a2
tstest/integration/vms: disable rDNS for sshd on centos (#2492)
This prevents centos tests from timing out because sshd does reverse dns
lookups on every session being established instead of doing it once on
the acutal ssh connection being established. This is odd. Appending this
to the sshd config and restarting it seems to fix it though.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 8db26a2261
tstest/integration/vms: disable nixos unstable (#2491)
cloud-init broke with the upgrade to python 3.9:
https://github.com/NixOS/nixpkgs/issues/131098

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick cecfc14875 net/dns: don't build init*.go on non-windows
To remove the regexp dep on iOS, notably.

Updates tailscale/corp#2238

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 2968893add net/dns/resolver: bound DoH usage on iOS
Updates tailscale/corp#2238

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 95a9adbb97 wgengine/netstack: implement UDP relaying to advertised subnets
TCP was done in 662fbd4a09.

This does the same for UDP.

Tested by hand. Integration tests will have to come later. I'd wanted
to do it in this commit, but the SOCKS5 server needed for interop
testing between two userspace nodes doesn't yet support UDP and I
didn't want to invent some whole new userspace packet injection
interface at this point, as SOCKS seems like a better route, but
that's its own bug.

Fixes #2302

RELNOTE=netstack mode can now UDP relay to subnets

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 3daf27eaad net/dns/resolver: fall back to IPv6 for well-known DoH servers if v4 fails
Should help with IPv6-only environments when the tailnet admin
only specified IPv4 DNS IPs.

See https://github.com/tailscale/tailscale/issues/2447#issuecomment-884188562

Co-Author: Adrian Dewhurst <adrian@tailscale.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 74eee4de1c net/dns/resolver: use correct Cloudflare DoH hostnames
We were using the wrong ones for the malware & adult content
variants. Docs:

https://developers.cloudflare.com/1.1.1.1/1.1.1.1-for-families/setup-instructions/dns-over-https

Earlier commit which added them:
236eb4d04d

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Joe Tsai d666bd8533
util/deephash: disambiguate hashing of AppendTo (#2483)
Prepend size to AppendTo output.

Fixes #2443

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
3 years ago
Joe Tsai 23ad028414
util/deephash: include type as part of hash for interfaces (#2476)
A Go interface may hold any number of different concrete types.
Just because two underlying values hash to the same thing
does not mean the two values are identical if they have different
concrete types. As such, include the type in the hash.
3 years ago
julianknodt 3a4201e773 net/portmapper: return correct upnp port
Previously, this was incorrectly returning the internal port, and using that with the external
exposed IP when it did not use WANIPConnection2. In the case when we must provide a port, we
return it instead.

Noticed this while implementing the integration test for upnp.

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
Joe Tsai a5fb8e0731
util/deephash: introduce deliberate instability (#2477)
Seed the hash upon first use with the current time.
This ensures that the stability of the hash is bounded within
the lifetime of one program execution.
Hopefully, this prevents future bugs where someone assumes that
this hash is stable.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
3 years ago
Brad Fitzpatrick ecac74bb65 wgengine/netstack: fix doc comment
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick e4fecfe31d wgengine/{monitor,router}: restore Linux ip rules when systemd deletes them
Thanks.

Fixes #1591

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 0aa77ba80f tstest/integration: fix filch test flake
Filch doesn't like having multiple processes competing
for the same log files (#937).

Parallel integration tests were all using the same log files.

Add a TS_LOGS_DIR env var that the integration test can use
to use separate log files per test.

Fixes #2269

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick ed8587f90d wgengine/router: take a link monitor
Prep for #1591 which will need to make Linux's router react to changes
that the link monitor observes.

The router package already depended on the monitor package
transitively. Now it's explicit.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 24db1a3c9b safesocket: print full lsof command on failure
This makes it easier to manually run the command
to discover why it is failing.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 130c5e727b safesocket: reduce log spam while running integration tests
Instead of logging lsof execution failures to stdout,
incorporate them into the returned error.

While we're here, make it clear that the file
success case always returns a nil error.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder f80193fa4c tstest/integration: shorten test names
The maximum unix domain socket path length on darwin is 104 bytes,
including the trailing NUL.

On my machine, the path created by some newly added tests (6eecf3c9)
was too long, resulting in cryptic test failures.

Shorten the names of the tests, and add a check to make
the diagnosis easier next time.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Joe Tsai 81cdd2f26c
Merge pull request #2464 from tailscale/dsnet/opaque-hash
util/deephash: make hash type opaque
3 years ago
Joe Tsai 9a0c8bdd20 util/deephash: make hash type opaque
The fact that Hash returns a [sha256.Size]byte leaks details about
the underlying hash implementation. This could very well be any other
hashing algorithm with a possible different block size.

Abstract this implementation detail away by declaring an opaque type
that is comparable. While we are changing the signature of UpdateHash,
rename it to just Update to reduce stutter (e.g., deephash.Update).

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
3 years ago
Brad Fitzpatrick a909d37a59 derp: rate limit how often same-key clients can kick each other off server
Updates #392
Updates #506

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick e74d37d30f net/dns{,/resolver}: quiet DNS output logging
It was a huge chunk of the overall log output and made debugging
difficult. Omit and summarize the spammy *.arpa parts instead.

Fixes tailscale/corp#2066 (to which nobody had opinions, so)
3 years ago
Brad Fitzpatrick b6d70203d3 ipn/ipnlocal: fix 'tailscale up' on Windows without GUI
With this, I can now:

* install Tailscale
* stop the GUI
* net stop Tailscale
* net start Tailscale
* tailscale up --unattended

(where the middle three steps simulate what would happen on a Windows
Server Core machine without a GUI)

Fixes #2137

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 7f7a81e5ae cmd/tailscaled: add func to create ipnserver.Opts
To unify the Windows service and non-service/non-Windows paths a bit.

And provides a way to make Linux act like Windows for testing.
(notably, for testing the fix to #2137)

One perhaps visible change of this is that tailscaled.exe when run in
cmd.exe/powershell (not as a Windows Service) no longer uses the
"_daemon" autostart key. But in addition to being naturally what falls
out of this change, that's also what Windows users would likely want,
as otherwise the unattended mode user is ignored when the "_daemon"
autostart key is specified. Notably, this would let people debug what
their normally-run-as-a-service tailscaled is doing, even when they're
running in Unattended Mode.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 87244eda3f cmd/tailscale/cli: allow effective GOOS to be changed for integration tests
Adds TS_DEBUG_UP_FLAG_GOOS for integration tests to make "tailscale
up" act like other OSes.

For an upcoming change to test #2137.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 787939a60c .github/workflows: add 'go generate' CI job
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 84a6dcd9a9 net/dnsfallback: regenerate
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 4dbbd0aa4a cmd/addlicense: add command to add licenseheaders to generated code
And use it to make our stringer invocations match the existing code.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 0ec9040c5e scripts: remove special case for _strings.go files in check license headers
And add a license header for derp/dropreason_string.go to make it happy.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 4b1f2ae382 tstest/integration: generate deps for all platforms in deps generator
We have different deps depending on the platform.
If we pick a privileged platform, we'll miss some deps.
If we use the union of all platforms, the integration test
won't compile on some platforms, because it'll import
packages that don't compile on that platform.

Give in to the madness and give each platform its own deps file.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 41d06bdf86 tempfork/wireguard-windows: remove unnecessary build tag
The _windows.go suffix suffices.
This allows go:generate to run without creating a diff.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder c179580599 wgengine/magicsock: add debug envvar to force all traffic over DERP
This would have been useful during debugging DERP issues recently.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick e41193ec4d wgengine/monitor: don't spam about Linux RTM_NEWRULE events
The earlier 2ba36c294b started listening
for ip rule changes and only cared about DELRULE events, buts its subscription
included all rule events, including new ones, which meant we were then
catching our own ip rule creations and logging about how they were unknown.

Stop that log spam.

Updates #1591

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick ec4d721572 cmd/tailscaled: use state key constant from ipn package
Rather than redefining it again.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick e2eaae8224 cmd/derpprobe: add in a delay to wait for mesh info to sync 3 years ago
Brad Fitzpatrick 2ba36c294b wgengine/monitor: subscribe to Linux ip rule events, log on rule deletes
For debugging & working on #1591 where certain versions of systemd-networkd
delete Tailscale's ip rule entries.

Updates #1591

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Avery Pennarun 14135dd935 ipn/ipnlocal: make state_test catch the bug fixed by #2445
Updates #2434

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
3 years ago
Brad Fitzpatrick 6eecf3c9d1 ipn/ipnlocal: stay out of map poll when down
Fixes #2434

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill 798b0da470
tstest/integration/vms: codegen for top level tests (#2441)
This moves the distribution definitions into a maintainable hujson file
instead of just existing as constants in `distros.go`. Comments are
maintained from the inline definitions.

This uses jennifer[1] for hygenic source tree creation. This allows us
to generate a unique top-level test for each VM run. This should
hopefully help make the output of `go test` easier to read.

This also separates each test out into its own top-level test so that we
can better track the time that each distro takes. I really wish there
was a way to have the `test_codegen.go` file _always_ run as a part of
the compile process instead of having to rely on people remembering to
run `go generate`, but I am limited by my tools.

This will let us remove the `-distro-regex` flag and use `go test -run`
to pick which distros are run.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 391207bbcf
tstest/integration/vms: use one testcontrol instance per VM (#2437)
This paves the way for future MagicDNS tests.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 171ec9f8f4 control/{controlknobs,controlclient}: simplify knobs API, fix controlclient crash
From integration tests elsewhere:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x70 pc=0x845c9b]

goroutine 226 [running]:
tailscale.com/control/controlclient.(*Direct).sendMapRequest(0xc00053e1e0, 0x16670f0, 0xc000353780, 0xffffffffffffffff, 0xc0003e5f10, 0x0, 0x0)
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/direct.go:803 +0x19bb
tailscale.com/control/controlclient.(*Direct).PollNetMap(...)
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/direct.go:574
tailscale.com/control/controlclient.(*Auto).mapRoutine(0xc00052a1e0)
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/auto.go:464 +0x571
created by tailscale.com/control/controlclient.(*Auto).Start
   /home/runner/go/pkg/mod/tailscale.com@v1.1.1-0.20210715222212-1bb6abc604c1/control/controlclient/auto.go:151 +0x65
exit status 2

Also remove types/opt.Bool API addition which is now unnecessary.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
julianknodt 1bb6abc604 net/portmapper: add upnp port mapping
Add in UPnP portmapping, using goupnp library in order to get the UPnP client and run the
portmapping functions. This rips out anywhere where UPnP used to be in portmapping, and has a
flow separate from PMP and PCP.

RELNOTE=portmapper now supports UPnP mappings

Fixes #682
Updates #2109

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago