Commit Graph

409 Commits (41aac261064602c5eb14ccbacd0a684ffe3ae533)

Author SHA1 Message Date
Tom DNetto 3b3d1b9350 tstest/integration/vms: consistently use two dashes for command-line switches
Signed-off-by: Tom DNetto <tom@tailscale.com>
3 years ago
Tom DNetto 858ab80172 tstest/integration/vms: fix docs, qemu-img invocation
Signed-off-by: Tom DNetto <tom@tailscale.com>
3 years ago
Josh Bleecher Snyder 32fd42430b all: use cibuild.On
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 0868329936 all: use any instead of interface{}
My favorite part of generics.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Maisem Ali 0f37317664 control/controlclient: make RegisterRequest go over Noise
Updates #3488

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Maisem Ali 249758df90 control/controlclient: start fetching the server noise key
Updates #3488

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick d5f8f38ac6 tailcfg: rename map request version to "capability version"
And add a CapabilityVersion type, primarily for documentation.

This makes MapRequest.Version, RegisterRequest.Version, and
SetDNSRequest.Version all use the same version, which will avoid
confusing in the future if Register or SetDNS ever changed their
semantics on Version change. (Currently they're both always 1)

This will requre a control server change to allow a
SetDNSRequest.Version value other than 1 to be deployed first.

Change-Id: I073042a216e0d745f52ee2dbc45cf336b9f84b7c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Maisem Ali 497324ddf6 ipn/store: add common package for instantiating ipn.StateStores
Also move KubeStore and MemStore into their own package.

RELNOTE: tsnet now supports providing a custom ipn.StateStore.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick 4cbdc84d27 cmd/tailscaled/childproc: add be-child registration mechanism
For ssh and maybe windows service babysitter later.

Updates #3802

Change-Id: I7492b98df98971b3fb72d148ba92c2276cca491f
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Maisem Ali f9a50779e2 cmd/tailscaled: add `-state=mem:` to support creation of an ephemeral node.
RELNOTE=`tailscaled --state=mem:` registers as an ephemeral node and
does not store state to disk.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Josh Bleecher Snyder 823d970d60 control/controlclient: use structured logging for MapResponse.ControlTime
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 8c3c5e80b7 tailcfg: make MapResponse.ControlTime a pointer
Otherwise omitempty doesn't work.

This is wire-compatible with a non-pointer type, so switching
is safe, now and in the future.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 8cf6d0a17b tailcfg: add MapResponse.ControlTime field
And log it when provided in map responses.

The test uses the date on which I joined Tailscale. :)

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Maisem Ali 72d8672ef7 tailcfg: make Node.Hostinfo a HostinfoView
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick 02bdc654d5 cmd/tailscale: fix up --reset, again
Also fix a somewhat related printing bug in the process where
some paths would print "Success." inconsistently even
when there otherwise was no output (in the EditPrefs path)

Fixes #3830
Updates #3702 (which broke it once while trying to fix it)

Change-Id: Ic51e14526ad75be61ba00084670aa6a98221daa5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 1af26222b6 go.mod: bump netstack, switch to upstream netstack
Now that Go 1.17 has module graph pruning
(https://go.dev/doc/go1.17#go-command), we should be able to use
upstream netstack without breaking our private repo's build
that then depends on the tailscale.com Go module.

This is that experiment.

Updates #1518 (the original bug to break out netstack to own module)
Updates #2642 (this updates netstack, but doesn't remove workaround)

Change-Id: I27a252c74a517053462e5250db09f379de8ac8ff
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 41fd4eab5c envknob: add new package for all the strconv.ParseBool(os.Getenv(..))
A new package can also later record/report which knobs are checked and
set. It also makes the code cleaner & easier to grep for env knobs.

Change-Id: Id8a123ab7539f1fadbd27e0cbeac79c2e4f09751
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 92dfaf53bb cmd/tailscaled: include Go runtime metrics in /debug/metrics
Fixes #3772

Change-Id: I237ea23268664d99e83d27890146018b04474556
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 681d4897cc tstest/integration: remove t param in testNode methods
testNodes have a reference to a testing.TB via their env.
Use it instead of making the caller pass theirs.
We did this in some methods but not others; finish the job.
This simplifies the call sites.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 93ae11105d ipn/ipnlocal: clear magicsock's netmap on logout
magicsock was hanging onto its netmap on logout,
which caused tailscale status to display partial
information about a bunch of zombie peers.
After logout, there should be no peers.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 84a1106fa7 tstest/integration: make -verbose-tailscaled pass -verbose=2 to tailscaled
If you're using -verbose-tailscaled, you're doing in-the-weeds debugging,
so you probably want the verbose output.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 486059589b all: gofmt -w -s (simplify) tests
And it updates the build tag style on a couple files.

Change-Id: I84478d822c8de3f84b56fa1176c99d2ea5083237
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 4691e012a9 tstest/integration: build binaries only once
The existing code relied on the Go build cache to avoid
needless work when obtaining the tailscale binaries.

For non-obvious reasons, the binaries were getting re-linked
every time, which added 600ms or so on my machine to every test.

Instead, build the binaries exactly once, on demand.
This reduces the time to run 'go test -count=5' from 34s to 10s
on my machine.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 63cd581c3f safesocket: add ConnectionStrategy, provide control over fallbacks
fee2d9fad added support for cmd/tailscale to connect to IPNExtension.
It came in two parts: If no socket was provided, dial IPNExtension first,
and also, if dialing the socket failed, fall back to IPNExtension.

The second half of that support caused the integration tests to fail
when run on a machine that was also running IPNExtension.
The integration tests want to wait until the tailscaled instances
that they spun up are listening. They do that by dialing the new
instance. But when that dial failed, it was falling back to IPNExtension,
so it appeared (incorrectly) that tailscaled was running.
Hilarity predictably ensued.

If a user (or a test) explicitly provides a socket to dial,
it is a reasonable assumption that they have a specific tailscaled
in mind and don't want to fall back to IPNExtension.
It is certainly true of the integration tests.

Instead of adding a bool to Connect, split out the notion of a
connection strategy. For now, the implementation remains the same,
but with the details hidden a bit. Later, we can improve that.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder a5235e165c tstest/integration: fix running with -verbose-tailscale
Without this fix, any run with -verbose-tailscale fails.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder c8829b742b all: minor code cleanup
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 768baafcb5 tstest/integration: use t.Cleanup
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick c37af58ea4 net/tsdial: move more weirdo dialing into new tsdial package, plumb
Not done yet, but this move more of the outbound dial special casing
from random packages into tsdial, which aspires to be the one unified
place for all outbound dialing shenanigans.

Then this plumbs it all around, so everybody is ultimately
holding on to the same dialer.

As of this commit, macOS/iOS using an exit node should be able to
reach to the exit node's DoH DNS proxy over peerapi, doing the sockopt
to stay within the Network Extension.

A number of steps remain, including but limited to:

* move a bunch more random dialing stuff

* make netstack-mode tailscaled be able to use exit node's DNS proxy,
  teaching tsdial's resolver to use it when an exit node is in use.

Updates #1713

Change-Id: I1e8ee378f125421c2b816f47bc2c6d913ddcd2f5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Artyom Pervukhin 49a9e62d58 Replace AWS SDK v1 dependency with v2
This change drops AWS SDKv1 dependency, leaving only SDK v2 in use.

Closes #3461

Signed-off-by: Artyom Pervukhin <github@artyom.dev>
3 years ago
Brad Fitzpatrick d5405c66b7 net/tsdial: start of new package to unify all outbound dialing complexity
For now this just deletes the net/socks5/tssocks implementation (and
the DNSMap stuff from wgengine/netstack) and moves it into net/tsdial.

Then initialize a Dialer early in tailscaled, currently only use for the
outbound and SOCKS5 proxies. It will be plumbed more later. Notably, it
needs to get down into the DNS forwarder for exit node DNS forwading
in netstack mode. But it will also absorb all the peerapi setsockopt
and netns Dial and tlsdial complexity too.

Updates #1713

Change-Id: Ibc6d56ae21a22655b2fa1002d8fc3f2b2ae8b6df
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder b800663779 tstest/integration: stop leaking zstd.Decoders
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Anderson 6e584ffa33 cmd/tailscaled: allow running the SOCKS5 and HTTP proxies on the same port.
Fixes #3248

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 3b541c833e util/clientmetric, logtail: log metric changes
Updates #3307

Change-Id: I1399ebd786f6ff7defe6e11c0eb651144c071574
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 57b039c51d util/clientmetrics: add new package to add metrics to the client
And annotate magicsock as a start.

And add localapi and debug handlers with the Prometheus-format
exporter.

Updates #3307

Change-Id: I47c5d535fe54424741df143d052760387248f8d3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick f7da8c77bd tstest/integration/testcontrol: fix data race
Fix race from 1ec99e99f4

Fixes #3289

Change-Id: I58158d3f82339ac171fb14827c5f158d602327f4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Maisem Ali d6dde5a1ac ipn/ipnlocal: handle key extensions after key has already expired
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick 5f36ab8a90 tstest/integration: go generate
Change-Id: I49d19007a16261e447240e149deac24c15c93fce
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 2b082959db safesocket: add WindowsLocalPort const
Remove all the 41112 references.

Change-Id: I2d7ed330d457e3bb91b7e6416cfb2667611e50c4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Denton Gentry 1ec99e99f4 tstest: extend node key expiration integration test.
Can produce the problem in #2515, preparing to test a fix.
Marked as t.Skip() until we have a fix.

Updates https://github.com/tailscale/tailscale/issues/2515

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
3 years ago
David Anderson 0532eb30db all: replace tailcfg.DiscoKey with key.DiscoPublic.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 6d82a18916 tstest/integration: don't include stdlib deps in go generate output
Causes too much churn for zero benefit.

Change-Id: I838f8cdb5723f122f11dd4bbce5e9c07755c3cd9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 3fd5f4380f util/multierr: new package
github.com/go-multierror/multierror served us well.
But we need a few feature from it (implement Is),
and it's not worth maintaining a fork of such a small module.

Instead, I did a clean room implementation inspired by its API.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Anderson 7e6a1ef4f1 tailcfg: use key.NodePublic in wire protocol types.
Updates #3206.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 0303ec44c3 go.mod: bump netstack for mipsle fix
Fixes #3233

Change-Id: I18d1af886402774ce0ecc77dae3bc71eb8ba5c9d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick c18b9d58aa tstest/archtest: add GOARCH-specific tests, run via qemu-user
Updates #3233

Change-Id: Ia224c90490d41e50a1d547eeea709b0d9171c1f9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Anderson 8d14bc32d1 tstest/integration: use key.NodePublic instead of tailcfg.NodeKey.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 418adae379 various: use NodePublic.AsNodeKey() instead of tailcfg.NodeKeyFromNodePublic()
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson eeb97fd89f various: remove remaining uses of key.NewPrivate.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 37c150aee1 derp: use new node key type.
Update #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Josh Bleecher Snyder 4bb2c6980d util/testingutil: new package with MinAllocsPerRun
testing.AllocsPerRun measures the total allocations performed
by the entire program while repeatedly executing a function f.
If some unrelated part of the rest of the program happens to
allocate a lot during that period, you end up with a test failure.

Ideally, the rest of the program would be silent while
testing.AllocsPerRun executes.

Realistically, that is often unachievable.

AllocsPerRun attempts to mitigate this by setting GOMAXPROCS to 1,
but that doesn't prevent other code from running;
it only makes it less likely.

You can also mitigate this by passing a large iteration count to
AllocsPerRun, but that is unreliable and needlessly expensive.

Unlike most of package testing, AllocsPerRun doesn't use any
toolchain magic, so we can just write a replacement.

One wild idea is to change how we count mallocs.
Instead of using runtime.MemStats, turn on memory profiling with a
memprofilerate of 1. Discard all samples from the profile whose stack
does not contain testing.AllocsPerRun. Count the remaining samples to
determine the number of mallocs.

That's fun, but overkill.

Instead, this change adds a simple API that attempts to get f to
run at least once with a target number of allocations.
This is useful when you know that f should allocate consistently.
We can then assume that any iterations with too many allocations
are probably due to one-time costs or background noise.

This suits most uses of AllocsPerRun.

Ratcheting tests tend to be significantly less flaky,
because they are biased towards success.
They can also be faster, because they can exit early,
once success has been reached.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Maisem Ali 81cabf48ec control/controlclient,tailcfg: propagate registration errors to the frontend
Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
nicksherron f01ff18b6f all: fix spelling mistakes
Signed-off-by: nicksherron <nsherron90@gmail.com>
3 years ago
Brad Fitzpatrick 7634af5c6f all: gofmt
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Aaron Klotz e016eaf410 cmd/tailscaled: conditionally flush Windows DNS cache on SessionChange
For the service, all we need to do is handle the `svc.SessionChange` command.
Upon receipt of a `windows.WTS_SESSION_UNLOCK` event, we fire off a goroutine to flush the DNS cache.
(Windows expects responses to service requests to be quick, so we don't want to do that synchronously.)

This is gated on an integral registry value named `FlushDNSOnSessionUnlock`,
whose value we obtain during service initialization.

(See [this link](https://docs.microsoft.com/en-us/windows/win32/api/winsvc/nc-winsvc-lphandler_function_ex) for information re: handling `SERVICE_CONTROL_SESSIONCHANGE`.)

Fixes #2956

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
3 years ago
Brad Fitzpatrick a7cb241db1 cmd/tailscaled: add support for running an HTTP proxy
This adds support for tailscaled to be an HTTP proxy server.
It shares the same backend dialing code as the SOCK5 server, but the
client protocol is HTTP (including CONNECT), rather than SOCKS.

Fixes #2289

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 382b349c54 cmd/tailscaled: disable netns in userspace-networking mode
Updates #2827
Updates #2822

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick a353fbd3b4 tstest: make MemLogger.String acquire its mutex
Updates #2781 (might even fix it, but its real issue is that
SetPrivateKey starts a ReSTUN goroutines which then logs, and
that bug and data race existed prior to MemLogger existing)
3 years ago
Brad Fitzpatrick 640134421e all: update tests to use tstest.MemLogger
And give MemLogger a mutex, as one caller had, which does match the logf
contract better.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 4c68b7df7c tstest: add MemLogger bytes.Buffer wrapper with Logf method
We use it tons of places. Updated three at least in this PR.

Another use in next commit.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Crawshaw b2a3d1da13 tstest/integration/vms: use fork of goexpect to avoid proto/grpc dep
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Dave Anderson 980acc38ba
types/key: add a special key with custom serialization for control private keys (#2792)
* Revert "Revert "types/key: add MachinePrivate and MachinePublic.""

This reverts commit 61c3b98a24.

Signed-off-by: David Anderson <danderson@tailscale.com>

* types/key: add ControlPrivate, with custom serialization.

ControlPrivate is just a MachinePrivate that serializes differently
in JSON, to be compatible with how the Tailscale control plane
historically serialized its private key.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 61c3b98a24 Revert "types/key: add MachinePrivate and MachinePublic."
Broke the tailscale control plane due to surprise different serialization.

This reverts commit 4fdb88efe1.
3 years ago
David Anderson 4fdb88efe1 types/key: add MachinePrivate and MachinePublic.
Plumb throughout the codebase as a replacement for the mixed use of
tailcfg.MachineKey and wgkey.Private/Public.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Crawshaw 4e18cca62e testcontrol: replace panic with error
I have seen this once in the VM test (caused by an EOF, I believe on
shutdown) that didn't need to cause the test to fail.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw f53792026e tstest/integration/vms: move build tags from linux to !windows
The tests build fine on other Unix's, they just can't run there.
But there is already a t.Skip by default, so `go test` ends up
working fine elsewhere and checks the code compiles.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Christine Dodrill 0b9e938152 tstest/integration/vms: test DNS configuration
This uses a neat little tool to dump the output of DNS queries to
standard out. This is the first end-to-end test of DNS that runs against
actual linux systems. The /etc/resolv.conf test may look superflous,
however this will help for correlating system state if one of the DNS
tests fails.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
David Crawshaw 9b7fc2ed1f .github: add Ubuntu VM test
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw debaaebf3b tstest/integration/vms: turn on logcatcher logging by default
Absolutely vital to debugging failures.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw a1f1020042 tstest/integration/vms: avoid log after test completion
Avoids a panic in the Go testing package if a late log comes in.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw 583af7c1a6 tstest/integration/vms: give guest multiple cores and use generic machine
Speeds up tests.
Allows the use of more version of qemu.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw 8668103f06 tstest/integration/vms: print qemu console output, fix printing issues
Fix a few test printing issues when tests fail.

Qemu console output is super useful when something is wrong in the
harness and we cannot even bring up the tests.
Also useful for figuring out where all the time goes in tests.

A little noisy, but not too noisy as long as you're only running one VM
as part of the tests, which is my plan.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw 1a9fba5b04 tstest/integration/vms: fix ubuntu URLs
Also remove extra distros for now.
We can bring them back later if useful.
Though our most important distros are these two Ubuntu, debian stable,
and Raspbian (not currently supported).
And before doing more Linux, we should do Windows.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw bb47feca44 tstest/integration: prefix logs with logid
The VM test has two tailscaled instances running and interleaves the
logs. Without a prefix it is impossible to figure out what is going on.

It might be even better to include the [ABCD] node prefix here as well.
Unfortunately lots of interesting logs happen before tailscaled has a
node key, so it wouldn't be a replacement for a short ID.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Maisem Ali fd4838dc57 wgengine/userspace: add support to automatically enable/disable the tailscale
protocol in BIRD, when the node is a primary subnet router as determined
by control.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
David Crawshaw 0b962567fa vms: make ssh-keygen quiet by default
Always succeeds. Its output clutters the log.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw 4fcce70df5 tstest/integration: listen on the specified IP
By default httptest listens only on the loopback adapter.
Instead, listen on the IP the user asked for.
The VM test needs this, as it wants to start DERP and STUN
servers on the host that can be reached by guest VMs.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
David Crawshaw 0b2761ca92 testcontrol: plumb through DERP
Without this tailscaled does not know the DERP of a peer.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
slowy07 ac0353e982 fix: typo spelling grammar
Signed-off-by: slowy07 <slowy.arfy@gmail.com>
3 years ago
Josh Bleecher Snyder 4ff0757d44 cmd/testcontrol: add test control server
This is useful for manual performance testing
of networks with many nodes.
I imagine it'll grow more knobs over time.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder fcca374fa7 tstest/integration/testcontrol: sort peers in map response
This is part of the control protocol.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder a5da4ed981 all: gofmt with Go 1.17
This adds "//go:build" lines and tidies up existing "// +build" lines.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 0858673f1f tstest/integration: regenerate test deps
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 24ee0ed3c3 tstest/integration: update test deps
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Crawshaw 3e039daf95 logpolicy: actually collect panics
(Written with Josh)

For #2544

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Christine Dodrill 60f34c70a2
tstest/integration/vms: disable rDNS for sshd on centos (#2492)
This prevents centos tests from timing out because sshd does reverse dns
lookups on every session being established instead of doing it once on
the acutal ssh connection being established. This is odd. Appending this
to the sshd config and restarting it seems to fix it though.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 8db26a2261
tstest/integration/vms: disable nixos unstable (#2491)
cloud-init broke with the upgrade to python 3.9:
https://github.com/NixOS/nixpkgs/issues/131098

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Josh Bleecher Snyder 0aa77ba80f tstest/integration: fix filch test flake
Filch doesn't like having multiple processes competing
for the same log files (#937).

Parallel integration tests were all using the same log files.

Add a TS_LOGS_DIR env var that the integration test can use
to use separate log files per test.

Fixes #2269

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder f80193fa4c tstest/integration: shorten test names
The maximum unix domain socket path length on darwin is 104 bytes,
including the trailing NUL.

On my machine, the path created by some newly added tests (6eecf3c9)
was too long, resulting in cryptic test failures.

Shorten the names of the tests, and add a check to make
the diagnosis easier next time.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick b6d70203d3 ipn/ipnlocal: fix 'tailscale up' on Windows without GUI
With this, I can now:

* install Tailscale
* stop the GUI
* net stop Tailscale
* net start Tailscale
* tailscale up --unattended

(where the middle three steps simulate what would happen on a Windows
Server Core machine without a GUI)

Fixes #2137

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 4b1f2ae382 tstest/integration: generate deps for all platforms in deps generator
We have different deps depending on the platform.
If we pick a privileged platform, we'll miss some deps.
If we use the union of all platforms, the integration test
won't compile on some platforms, because it'll import
packages that don't compile on that platform.

Give in to the madness and give each platform its own deps file.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 6eecf3c9d1 ipn/ipnlocal: stay out of map poll when down
Fixes #2434

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill 798b0da470
tstest/integration/vms: codegen for top level tests (#2441)
This moves the distribution definitions into a maintainable hujson file
instead of just existing as constants in `distros.go`. Comments are
maintained from the inline definitions.

This uses jennifer[1] for hygenic source tree creation. This allows us
to generate a unique top-level test for each VM run. This should
hopefully help make the output of `go test` easier to read.

This also separates each test out into its own top-level test so that we
can better track the time that each distro takes. I really wish there
was a way to have the `test_codegen.go` file _always_ run as a part of
the compile process instead of having to rely on people remembering to
run `go generate`, but I am limited by my tools.

This will let us remove the `-distro-regex` flag and use `go test -run`
to pick which distros are run.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 391207bbcf
tstest/integration/vms: use one testcontrol instance per VM (#2437)
This paves the way for future MagicDNS tests.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
julianknodt 1bb6abc604 net/portmapper: add upnp port mapping
Add in UPnP portmapping, using goupnp library in order to get the UPnP client and run the
portmapping functions. This rips out anywhere where UPnP used to be in portmapping, and has a
flow separate from PMP and PCP.

RELNOTE=portmapper now supports UPnP mappings

Fixes #682
Updates #2109

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
Christine Dodrill 3ebe16558c
tstest/integration/vms: attempt to send more UDP packets (#2433)
Apparently this test was flaking because I critically misunderstood how
the kernel buffers UDP packets for senders. I'm trying to send more UDP
packets and will see if that helps.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 09e81b8ba1
tstest/integration/vms: deflake the tailscale_status test (#2427)
This test used to try to run this only once, but this variant of the
test attempts to run `tailscale status` up to 6 times in a loop with
exponential backoff.

This fixes the flakiness found in previous instances of this test.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 01e159b610 ipn/ipnlocal: save prefs to disk on UpdatePrefs
Regression from 6d10655dc3, which added
UpdatePrefs but didn't write it out to disk.

I'd planned on adding tests to state_test.go which is why I'd earlier
added 46896a9311 to prepare for making
such persistence tests easier to write, but turns out state_test.go
didn't even test UpdatePrefs, so I'm staying out of there.
Instead, this is tested using integration tests.

Fixes #2321

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill 5b845631ce
tstest/integration/vms: delete a log file i was using to debug the tests (#2416)
Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill ede8ec1e20
tstest/integration/vms: split up test framework into files (#2396)
My spatial memory functions poorly with large files and the vms_test.go
file recently surpassed the point where it functions adequately. This
patch splits up vms_test.go into more files to make my spatial memory
function like I need it to.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 1f2a877c61
tstest/integration/testcontrol: add ipv6 support (#2394)
Split from https://github.com/tailscale/tailscale/pull/2376.

This adds IPv6 support to testcontrol so each member of the tailscale
network gets an IPv6 address too.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 1b1c85d069
Revert "Revert "tstest/integration/vms: end-to-end UDP test (#2361)" (#2364)" (#2365)
This reverts commit 254fc7885b.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Simeng He 3d049a9d71 tstest/integration: change return type of AwaitIP from string to netaddr.IP
Signed-off-by: Simeng He <simeng@tailscale.com>
3 years ago
Brad Fitzpatrick 7e7c4c1bbe tailcfg: break DERPNode.DERPTestPort into DERPPort & InsecureForTests
The DERPTestPort int meant two things before: which port to use, and
whether to disable TLS verification. Users would like to set the port
without disabling TLS, so break it into two options.

Updates #1264

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill afbd35482d
tstest/integration/vms: add NixOS unstable back (#2330)
The upstream NixOS issue was fixed.
https://github.com/NixOS/nixpkgs/issues/128783

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 33cacb5284
tstest/integration/vms: unbreak nixos builds (#2372)
To avoid the generated nixos disk images from becoming immune from the
GC, I delete the symlink to the nix store at the end of tests.
`t.Cleanup` runs at the end of a test. I changed this part of the code
to have a separate timer for how long it takes to run NixOS builds, but
I did that by using a subtest. This means that it was creating the NixOS
image, deleting its symlink and then trying to use that symlink to find
the resulting disk image, making the whole thing ineffectual.

This was a mistake. I am reverting this change made in
https://github.com/tailscale/tailscale/pull/2360 to remove this layer of
subtesting.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 254fc7885b
Revert "tstest/integration/vms: end-to-end UDP test (#2361)" (#2364)
This reverts commit dc78be12c5.
3 years ago
Christine Dodrill dc78be12c5
tstest/integration/vms: end-to-end UDP test (#2361)
This tests incoming and outgoing UDP traffic. It would test incoming UDP
traffic however our socks server doesn't seem to allow for connecting to
destinations over UDP. When the socks server gets that support the
incoming test should pass without issue.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill a19eea965f
tstest/integration/vms: use an in-process logcatcher (#2360)
This adapts the existing in-process logcatcher from tstest/integration
into a public type and uses it on the side of testcontrol. This also
fixes a bug in the Alpine Linux OpenRC unit that makes every value in
`/etc/default/tailscaled` exported into tailscaled's environment, a-la
systemd [Service].EnviromentFile.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 1e83b97498
tstest/integration/vms: outgoing SSH test (#2349)
This does a few things:

1. Rewrites the tests so that we get a log of what individual tests
   failed at the end of a test run.
2. Adds a test that runs an HTTP server via the tester tailscale node and
   then has the VMs connect to that over Tailscale.
3. Dials the VM over Tailscale and ensures it answers SSH requests.
4. Other minor framework refactoring.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 97279a0fe0
tstest/integration/vms: add Oracle Linux image (#2328)
Oracle Linux[1] is a CentOS fork. It is not very special. I am adding it
to the integration jungle because I am adding it to pkgs and the website
directions.

[1]: https://www.oracle.com/linux/

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 7d417586a8 tstest/integration: help bust cmd/go's test caching
It was caching too aggressively, as it didn't see our deps due to our
running "go install tailscaled" as a child process.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill a8360050e7
tstest/integration/vms: make first end to end test (#2332)
This makes sure `tailscale status` and `tailscale ping` works. It also
switches goexpect to use a batch instead of manually banging out each
line, which makes the tests so much easier to read.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 5813da885c
tstest/integration/vms: verbosify nixos logs to fs, disable unstable (#2294)
This puts nix build logs on the filesystem so that we can debug them
later. This also disables nixos unstable until
https://github.com/NixOS/nixpkgs/issues/128783 is fixed.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
julianknodt 506c2fe8e2 cmd/tailscale: make netcheck use active DERP map, delete static copy
After allowing for custom DERP maps, it's convenient to be able to see their latency in
netcheck. This adds a query to the local tailscaled for the current DERPMap.

Updates #1264

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
Brad Fitzpatrick 5e19ac7adc tstest/integration: always run SOCK5 server, parse out its listening address
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill b131a74f99
tstest/integration/vms: build and run NixOS (#2190)
Okay, so, at a high level testing NixOS is a lot different than
other distros due to NixOS' determinism. Normally NixOS wants packages to
be defined in either an overlay, a custom packageOverrides or even
yolo-inline as a part of the system configuration. This is going to have
us take a different approach compared to other distributions. The overall
plan here is as following:

1. make the binaries as normal
2. template in their paths as raw strings to the nixos system module
3. run `nixos-generators -f qcow -o $CACHE_DIR/tailscale/nixos/version -c generated-config.nix`
4. pass that to the steps that make the virtual machine

It doesn't really make sense for us to use a premade virtual machine image
for this as that will make it harder to deterministically create the image.

Nix commands generate a lot of output, so their output is hidden behind the
`-verbose-nix-output` flag.

This unfortunately makes this test suite have a hard dependency on
Nix/NixOS, however the test suite has only ever been run on NixOS (and I
am not sure if it runs on other distros at all), so this probably isn't too
big of an issue.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 194d5b8412
tstest/integration/vms: add in-process DERP server (#2108)
Previously this test would reach out to the public DERP servers in order
to help machines connect with eachother. This is not ideal given our
plans to run these tests completely disconnected from the internet. This
patch introduces an in-process DERP server running on its own randomly
assigned HTTP port.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 6b234323a0
tstest/integration/vms: fix flake when testing (#2145)
Occasionally the test framework would fail with a timeout due to a
virtual machine not phoning home in time. This seems to be happen
whenever qemu can't bind the VNC or SSH ports for a virtual machine.
This was fixed by taking the following actions:

1. Don't listen on VNC unless the `-use-vnc` flag is passed, this
   removes the need to listen on VNC at all in most cases. The option to
   use VNC is still left in for debugging virtual machines, but removing
   this makes it easier to deal with (VNC uses this odd system of
   "displays" that are mapped to ports above 5900, and qemu doesn't
   offer a decent way to use a normal port number, so we just disable
   VNC by default as a compromise).
2. Use a (hopefully) inactive port for SSH. In an ideal world I'd just
   have the VM's SSH port be exposed via a Unix socket, however the QEMU
   documentation doesn't really say if you can do this or not. While I
   do more research, this stopgap will have to make do.
3. Strictly tie more VM resource lifetimes to the tests themselves.
   Previously the disk image layers for virtual machines were only
   cleaned up at the end of the test and existed in the parent
   test-scoped temporary folder. This can make your tmpfs run out of
   space, which is not ideal. This should minimize the use of temporary
   storage as much as I know how to.
4. Strictly tie the qemu process lifetime to the lifetime of the test
   using testing.T#Cleanup. Previously it used a defer statement to
   clean up the qemu process, however if the tests timed out this defer
   was not run. This left around an orphaned qemu process that had to be
   killed manually. This change ensures that all qemu processes exit
   when their relevant tests finish.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 45e64f2e1a net/dns{,/resolver}: refactor DNS forwarder, send out of right link on macOS/iOS
Fixes #2224
Fixes tailscale/corp#2045

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick a0c632f6b5 tstest/integration: fix a race
Noticed on a CI failure.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Crawshaw 297b3d6fa4 staticcheck.conf: turn off noisy lint errors
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
3 years ago
Brad Fitzpatrick 2f4817fe20 tstest/integration: fix race flake
Fixes #2172

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill e0f0d10672
tstest/integration/vms: log to t.Logf directly (#2147)
Previously we used t.Logf indirectly via package log. This worked, but
it was not ideal for our needs. It could cause the streams of output to
get crossed. This change uses a logger.FuncWriter every place log.Output
was previously used, which will more correctly write log information to
the right test output stream.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 082cc1b0a7 tstest/integration: reenable TestAddPingRequest
Failure understood now; see:
https://github.com/tailscale/tailscale/pull/2088#issuecomment-859896598

As of 333e9e75d4, PingRequest is
now safe for the server to send multiple times, without fear
of the client handling it multiple times.

Fixes #2079

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Denton Gentry c61d777705 tstest/integration: disable TestAddPingRequest
Failing often now, we don't want people to get used to
routinely ignoring test failures.

Can be re-enabled when
https://github.com/tailscale/tailscale/issues/2079
is resolved.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
3 years ago
Christine Dodrill 8b2b899989
tstest/integration: test Alpine Linux (#2098)
Alpine Linux[1] is a minimal Linux distribution built around musl libc.
It boots very quickly, requires very little ram and is as close as you
can get to an ideal citizen for testing Tailscale on musl. Alpine has a
Tailscale package already[2], but this patch also makes it easier for us
to provide an Alpine Linux package off of pkgs in the future.

Alpine only offers Tailscale on the rolling-release edge branch.

[1]: https://alpinelinux.org/
[2]: https://pkgs.alpinelinux.org/packages?name=tailscale&branch=edge

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 0affcd4e12 tstest/integration: add some debugging for TestAddPingRequest flakes
This fails pretty reliably with a lot of output now showing what's
happening:

TS_DEBUG_MAP=1 go test --failfast -v -run=Ping -race -count=20 ./tstest/integration --verbose-tailscaled

I haven't dug into the details yet, though.

Updates #2079
3 years ago
Brad Fitzpatrick ee3df2f720 tstest/integration: rename ambiguous --verbose test flag
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill b402e76185
.github/workflows: add integration test with a custom runner (#2044)
This runner is in my homelab while we muse about a better, more
permanent home for these tests.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 622dc7b093
tstest/integration/vms: download images from s3 (#2035)
This makes integration tests pull pristine VM images from Amazon S3 if
they don't exist on disk. If the S3 fetch fails, it will fall back to
grabbing the image from the public internet. The VM images on the public
internet are known to be updated without warning and thusly change their
SHA256 checksum. This is not ideal for a test that we want to be able to
fire and forget, then run reliably for a very long time.

This requires an AWS profile to be configured at the default path. The
S3 bucket is rigged so that the requester pays. The VM images are
currently about 6.9 gigabytes. Please keep this in mind when running
these tests on your machine.

Documentation was added to the integration test folder to aid others in
running these tests on their machine.

Some wording in the logs of the tests was altered.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 3f1405fa2a
tstest/integration/vms: bump images, fix caching bug (#2052)
Before this redownloaded the image every time. Now it only redownloads
it when it needs to.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Simeng He 0141390365 tstest/integration/testcontrol: add Server.AddPingRequest
Signed-off-by: Simeng He <simeng@tailscale.com>
3 years ago
Christine Dodrill adaecd83c8
tstest/integration/vms: add DownloadImages test to download images (#2039)
The image downloads can take a significant amount of time for the tests.
This creates a new test that will download every distro image into the
local cache in parallel, optionally matching the distribution regex.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 607b7ab692
tstest/integration/vms: aggressively re-verify shasums (#2050)
I've run into a couple issues where the tests time out while a VM image
is being downloaded, making the cache poisoned for the next run. This
moves the hash checking into its own function and calls it much sooner
in the testing chain. If the hash check fails, the OS is redownloaded.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 6ce77b8eca
tstest/integration/vms: log qemu output (#2047)
Most of the time qemu will output nothing when it is running. This is
expected behavior. However when qemu is unable to start due to some
problem, it prints that to either stdout or stderr. Previously this
output wasn't being captured. This patch captures that output to aid in
debugging qemu issues.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick 58cc2cc921 tstest/integration/testcontrol: add Server.nodeLocked 3 years ago
Christine Dodrill 0a655309c6
tstest/integration/vms: only build binaries once (#2042)
Previously this built the binaries for every distro. This is a bit
overkill given we are using static binaries. This patch makes us only
build once.

There was also a weird issue with how processes were being managed.
Previously we just killed qemu with Process.Kill(), however that was
leaving behind zombies. This has been mended to not only kill qemu but
also waitpid() the process so it doesn't become a zombie.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill a282819026
tstest/integration/vms: fix OpenSUSE Leap 15.1 (#2038)
The OpenSUSE 15.1 image we are using (and conseqentially the only one
that is really available easily given it is EOL) has cloud-init
hardcoded to use the OpenStack metadata thingy. Other OpenSUSE Leap
images function fine with the NoCloud backend, but this one seems to
just not work with it. No bother, we can just pretend to be OpenStack.

Thanks to Okami for giving me an example OpenStack configuration seed
image.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 4da5e79c39
tstest/integration/vms: test on Arch Linux (#2040)
Arch is a bit of a weirder distro, however as a side effect it is much
more of a systemd purist experience. Adding it to our test suite will
make sure that we are working in the systemd happy path.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill ca96357d4b
tstest/integration/vms: add OpenSUSE Leap 15.3 (#2026)
This distro is about to be released. OpenSUSE has historically had the
least coverage for functional testing, so this may prove useful in the
future.

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 2802a01b81
tstest/integration/vms: test vms as they are ready (#2022)
Instead of testing all the VMs at once when they are all ready, this
patch changes the testing logic so that the vms are tested as soon as
they register with testcontrol. Also limit the amount of VM ram used at
once with the `-ram-limit` flag. That uses a semaphore to guard resource
use.

Also document CentOS' sins.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 36cb69002a
tstest/integration/vms: regex-match distros using a flag (#2021)
If you set `-distro-regex` to match a subset of distros, only those
distros will be tested. Ex:

    $ go test -run-vm-tests -distro-regex='opensuse'

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill e1b994f7ed
tstest/integration/vms: maintain distro info (#2020)
This lets us see the names of distros in our tests.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick fa548c5b96
tstest/integration/vms: fix bindhost lookup (#2012)
Don't try to do heuristics on the name. Use the net/interfaces package
which we already have to do this sort of stuff.

Fixes #2011

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill 14c1113d2b
tstest/integration/vms: copy locally built binaries (#2006)
Instead of pulling packages from pkgs.tailscale.com, we should use the
tailscale binaries that are local to this git commit. This exposes a bit
of the integration testing stack in order to copy the binaries
correctly.

This commit also bumps our version of github.com/pkg/sftp to the latest
commit.

If you run into trouble with yaml, be sure to check out the
commented-out alpine linux image complete with instructions on how to
use it.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Brad Fitzpatrick f21982f854 tstest/integration/vms: skip a test for now
Updates #2011

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Christine Dodrill 4cfaf489ac
tstest/integration/vms: t.Log for VM output (#2007)
Previously we spewed a lot of output to stdout and stderr, even when
`-v` wasn't set. This is sub-optimal for various reasons. This patch
shunts that output to test logs so it only shows up when `-v` is set.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 1f72b6f812
tstest/integration/vms: use dynamically discovered bindhost (#1992)
Instead of relying on a libvirtd bridge address that you probably won't
have on your system.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill 35749ec297
tstest/integration/vms: small cleanups (#1989)
Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
Christine Dodrill ba59c0391b
tstest/integration: add experimental integration test (#1966)
This will spin up a few vms and then try and make them connect to a
testcontrol server.

Updates #1988

Signed-off-by: Christine Dodrill <xe@tailscale.com>
3 years ago
simenghe dd0b690e7b
Added new Addresses / AllowedIPs fields to testcontrol when creating tailcfg.Node (#1948)
* Added new Addresses / AllowedIPs fields to testcontrol when creating new &tailcfg.Node

Signed-off-by: Simeng He <simeng@tailscale.com>

* Added single node test to check Addresses and AllowedIPs

Signed-off-by: Simeng He <simeng@tailscale.com>

Co-authored-by: Simeng He <simeng@tailscale.com>
4 years ago
Josh Bleecher Snyder 25df067dd0 all: adapt to opaque netaddr types
This commit is a mishmash of automated edits using gofmt:

gofmt -r 'netaddr.IPPort{IP: a, Port: b} -> netaddr.IPPortFrom(a, b)' -w .
gofmt -r 'netaddr.IPPrefix{IP: a, Port: b} -> netaddr.IPPrefixFrom(a, b)' -w .

gofmt -r 'a.IP.Is4 -> a.IP().Is4' -w .
gofmt -r 'a.IP.As16 -> a.IP().As16' -w .
gofmt -r 'a.IP.Is6 -> a.IP().Is6' -w .
gofmt -r 'a.IP.As4 -> a.IP().As4' -w .
gofmt -r 'a.IP.String -> a.IP().String' -w .

And regexps:

\w*(.*)\.Port = (.*)  ->  $1 = $1.WithPort($2)
\w*(.*)\.IP = (.*)  ->  $1 = $1.WithIP($2)

And lots of manual fixups.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Brad Fitzpatrick 5a7c6f1678 tstest/integration{,/testcontrol}: add node update support, two node test
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick d32667011d tstest/integration: build test binaries with -race if test itself is
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick ed9d825552 tstest/integration: fix integration test on linux/386
Apparently can't use GOBIN with GOARCH.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick c0158bcd0b tstest/integration{,/testcontrol}: add testcontrol.RequireAuth mode, new test
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Josh Bleecher Snyder 78d4c561b5 types/logger: add key grinder stats lines to rate-limiting exemption list
Updates #1749

Co-authored-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Brad Fitzpatrick 68fb51b833 tstest/integration: misc cleanups
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 3237e140c4 tstest/integration: add testNode.AwaitListening, DERP+STUN, improve proxy trap
Updates #1840
4 years ago
Brad Fitzpatrick 98d7c28faa tstest/integration: start factoring test types out to clean things up
To enable easy multi-node testing (including inter-node traffic) later.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 5e9e11a77d tstest/integration/testcontrol: add start of test control server
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick a07a504b16 tstest/integration: use go binary from runtime.GOROOT
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick f342d10dc5 tstest/integration: set an HTTP_PROXY to catch bogus requests
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 80429b97e5 testing: add start of an integration test
Only minimal tailscale + tailscaled for now.

And a super minimal in-memory logcatcher.

No control ... yet.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 08782b92f7 tstest: add WaitFor helper
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Josh Bleecher Snyder d31eff8473 tstest/natlab: use net.ErrClosed
We are now on 1.16.
And wgconn.NetErrClosed has been removed upstream.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Filippo Valsorda 39f7a61e9c tstest/staticcheck: import the main package to fix "go mod tidy"
Importing the non-main package was missing some dependencies that
"go mod tidy" would then cleanup. Also added a non-ignore build tag to
avoid other tools getting upset about importing a main package.

Signed-off-by: Filippo Valsorda <hi@filippo.io>
4 years ago
Josh Bleecher Snyder 138055dd70 tstest/natlab: use net.ErrClosed instead of a new error
Upstream wireguard-go decided to use errors.Is(err, net.ErrClosed)
instead of checking the error string.

It also provided an unsafe linknamed version of net.ErrClosed
for clients running Go 1.15. Switch to that.

This reduces the time required for the wgengine/magicsock tests
on my machine from ~35s back to the ~13s it was before
456cf8a376.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder e8cd7bb66f tstest: simplify goroutine leak tests
Use tb.Cleanup to simplify both the API and the implementation.

One behavior change: When the number of goroutines shrinks, don't log.
I've never found these logs to be useful, and they frequently add noise.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder 18471a8792 ipn: close logger at the end of TestLocalLogLines
If any goroutine continues to use the logger in TestLocalLogLines
after the test finishes, the test panics.

The culprit for this was wireguard-go; the previous commit fixed that.
This commit adds suspenders: When the test is done, make logging calls
into no-ops.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Brad Fitzpatrick fd2a30cd32 wgengine/magicsock: make test pass on Windows and without firewall dialog box
Updates #50
4 years ago
Josh Bleecher Snyder e1596d655a tstest: skip resource check when test has failed
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder 2e40c4b564 tstest: don't log on success
This is particularly important for benchmarks,
where stray output disrupts tooling.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Brad Fitzpatrick 7fddc33481 tstest: make LogLineTracker pass variadic format args through
Omitting the "..." passed a literal []interface{} to the underlying
logger always.
4 years ago
Brad Fitzpatrick 68c42530e9 tstest: rename LogListener to LogLineTracker
But mostly to rename tstest.ListenFor which has no mention of log lines in it.
It sounded like a net.Listener or something.
4 years ago
David Crawshaw 3baa084548 tstest: take testing.TB in helper for benchmarks
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
4 years ago
Wendi Yu a3fb422a39
ipn: tag and test for grinder log lines (#711)
Signed-off-by: Wendi <wendi.yu@yahoo.ca>
4 years ago
David Anderson f794493b4f wgengine/magicsock: explicitly check path discovery, add a firewall test.
The test proves that active discovery can traverse two facing firewalls.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 5a370d545a tstest/natlab: drop packets that can't be routed in a LAN.
LANs are authoritative for their prefixes, so we should not bounce
packets back and forth to the default gateway in that case.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 45578b47f3 tstest/natlab: refactor PacketHandler into a larger interface.
The new interface lets implementors more precisely distinguish
local traffic from forwarded traffic, and applies different
forwarding logic within Machines for each type. This allows
Machines to be packet forwarders, which didn't quite work
with the implementation of Inject.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 23123907c0 tstest/natlab: add a configurable SNAT44 translator.
This lets us implement the most common kinds of NAT in the wild.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 39ecb37fd6 tstest/natlab: support different firewall selectivities.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 88e8456e9b wgengine/magicsock: add a connectivity test for facing firewalls.
The test demonstrates that magicsock can traverse two stateful
firewalls facing each other, that each require localhost to
initiate connections.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson b3d65ba943 tstest/natlab: refactor, expose a Packet type.
HandlePacket and Inject now receive/take Packets. This is a handy
container for the packet, and the attached Trace method can be used
to print traces from custom packet handlers that integrate nicely
with natlab's internal traces.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 5eedbcedd1 tstest/natlab: add a stateful firewall.
The firewall provides a ProcessPacket handler, and implements an
address-and-port endpoint dependent firewall that allows all
traffic to egress from the trusted interface, and only allows
inbound traffic if corresponding outbound traffic was previously
seen.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 0ed9f62ed0 tstest/natlab: provide inbound interface to HandlePacket.
Requires a bunch of refactoring so that Networks only ever
refer to Interfaces that have been attached to them, and
Interfaces know about both their Network and Machine.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
Brad Fitzpatrick 6c74065053 wgengine/magicsock, tstest/natlab: start hooking up natlab to magicsock
Also adds ephemeral port support to natlab.

Work in progress.

Pairing with @danderson.
4 years ago
Brad Fitzpatrick 7883e5c5e7 go.mod: restore staticcheck module, make it stick around, go mod tidy
It kept coming & going as different people ran go mod tidy and others
ran staticcheck.

Make it stop going away with go mod tidy by adding a dep to it.
4 years ago
David Anderson 0aea087766 tstest/natlab: add PacketHandler and Inject.
Together, they can be used to plug custom packet processors into
Machines.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 73db7e99ab tstest/natlab: make Machine constructible directly.
This is a prelude to adding more fields, which would otherwise
become more unnamed function params.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson d94593e884 tstest/natlab: unregister conn4 if registration of conn6 fails.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson d7bc4ec029 tstest/natlab: use common helper for conn registration.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 80a14c49c6 tstest/natlab: add comments to conns4/conns6.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson c53b154171 tstest/natlab: use &Network in test.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 622c0d0cb3 tstest/natlab: print trace data when NATLAB_TRACE is set.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 1d4f9852a7 tstest/natlab: correctly handle dual-stacked PacketConns.
Adds a test with multiple networks, one of which is v4-only.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson 771eb05bcb tstest/natlab: first network attached becomes the default route.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson f2e5da916a tstest/natlab: allow sensible default construction of networks.
Add a test for LAN->LAN traffic.

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
Brad Fitzpatrick 97910ce712 tstest/natlab: remove unused PacketConner type 4 years ago
Brad Fitzpatrick 14b4213c17 tstest/natlab: add missing tests from earlier commits
Now you can actually see that packet delivery works.

Pairing with @danderson
4 years ago
Brad Fitzpatrick 3f4f1cfe66 tstest/natlab: basic NAT-free packet delivery works
Pairing with @danderson
4 years ago
Brad Fitzpatrick a477e70632 tstest/natlab: network address allocation
Pairing with @danderson
4 years ago
Brad Fitzpatrick bb1a9e4700 tstest/natlab: bit more of in-memory network testing package
Pairing with @danderson
4 years ago
Brad Fitzpatrick 23c93da942 tstest/natlab: start of in-memory network testing package
Pairing with @danderson
4 years ago
Brad Fitzpatrick 5ec7ac1d02 tstest: document PanicOnLog 5 years ago
Avery Pennarun 4f128745d8 magicsock/test: oops, fix a data race in nested-test logf hack.
Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
Avery Pennarun 08acb502e5 Add tstest.PanicOnLog(), and fix various problems detected by this.
If a test calls log.Printf, 'go test' horrifyingly rearranges the
output to no longer be in chronological order, which makes debugging
virtually impossible. Let's stop that from happening by making
log.Printf panic if called from any module, no matter how deep, during
tests.

This required us to change the default error handler in at least one
http.Server, as well as plumbing a bunch of logf functions around,
especially in magicsock and wgengine, but also in logtail and backoff.

To add insult to injury, 'go test' also rearranges the output when a
parent test has multiple sub-tests (all the sub-test's t.Logf is always
printed after all the parent tests t.Logf), so we need to screw around
with a special Logf that can point at the "current" t (current_t.Logf)
in some places. Probably our entire way of using subtests is wrong,
since 'go test' would probably like to run them all in parallel if you
called t.Parallel(), but it definitely can't because the're all
manipulating the shared state created by the parent test. They should
probably all be separate toplevel tests instead, with common
setup/teardown logic. But that's a job for another time.

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
Avery Pennarun e0b666c5d2 tstest.ResourceCheck: clarify success message.
Inclusion of the word "assert" made it seem like a failure, even though
it was supposed to be identifying the name of the function (Assert()).

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
David Anderson 0038223632 tstest: rename from testy.
Signed-off-by: David Anderson <danderson@tailscale.com>
5 years ago