412 Commits (532b26145a088c3946c37040dc4731dc4edcb7cf)

Author	SHA1	Message	Date
Anton Tolchanov	8546ff98fb	tsweb: move varz handler(s) into separate modules ... This splits Prometheus metric handlers exposed by tsweb into two modules: - `varz.Handler` exposes Prometheus metrics generated by our expvar converter; - `promvarz.Handler` combines our expvar-converted metrics and native Prometheus metrics. By default, tsweb will use the promvarz handler, however users can keep using only the expvar converter. Specifically, `tailscaled` now uses `varz.Handler` explicitly, which avoids a dependency on the (heavyweight) Prometheus client. Updates https://github.com/tailscale/corp/issues/10205 Signed-off-by: Anton Tolchanov <anton@tailscale.com>	2 years ago
Maisem Ali	f6ea6863de	tstest/integration: add ping test w/ masquerades ... Updates tailscale/corp#8020 Co-authored-by: Melanie Warrick <warrick@tailscale.com> Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Will Norris	57a008a1e1	all: pass log IDs as the proper type rather than strings ... This change focuses on the backend log ID, which is the mostly commonly used in the client. Tests which don't seem to make use of the log ID just use the zero value. Signed-off-by: Will Norris <will@tailscale.com>	2 years ago
Maisem Ali	f34590d9ed	tsnet: add test for Funnel connections ... For the logic added in b797f77. Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Maisem Ali	1a30b2d73f	all: use tstest.Replace more ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Maisem Ali	b9ebf7cf14	tstest: add method to Replace values for tests ... We have many function pointers that we replace for the duration of test and restore it on test completion, add method to do that. Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Andrew Dunham	3f8e8b04fd	cmd/tailscale, cmd/tailscaled: move portmapper debugging into tailscale CLI ... The debug flag on tailscaled isn't available in the macOS App Store build, since we don't have a tailscaled binary; move it to the 'tailscale debug' CLI that is available on all platforms instead, accessed over LocalAPI. Updates #7377 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I47bffe4461e036fab577c2e51e173f4003592ff7	2 years ago
Joe Tsai	0d19f5d421	all: replace logtail.{Public,Private}ID with logid.{Public,Private}ID (#7404) ... The log ID types were moved to a separate package so that code that only depend on log ID types do not need to link in the logic for the logtail client itself. Not all code need the logtail client. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
James Tucker	ee471ca1c8	tstest/integration: enable go lookups from $PATH ... Fixes tailscale/corp#9261 Signed-off-by: James Tucker <james@tailscale.com>	2 years ago
Brad Fitzpatrick	cf8dd7aa09	all: use Go 1.20's bytes.Clone ... Updates #7123 Updates #6257 (more to do in other repos) Change-Id: I073e2a6d81a5d7fbecc29caddb7e057ff65239d0 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Will Norris	947c14793a	all: update tools that manage copyright headers ... Update all code generation tools, and those that check for license headers to use the new standard header. Also update copyright statement in LICENSE file. Fixes #6865 Signed-off-by: Will Norris <will@tailscale.com>	2 years ago
Will Norris	71029cea2d	all: update copyright and license headers ... This updates all source files to use a new standard header for copyright and license declaration. Notably, copyright no longer includes a date, and we now use the standard SPDX-License-Identifier header. This commit was done almost entirely mechanically with perl, and then some minimal manual fixes. Updates #6865 Signed-off-by: Will Norris <will@tailscale.com>	2 years ago
Andrew Dunham	b74db24149	tstest/integration: mark all integration tests as flaky ... Updates #7036 Change-Id: I3aec5ad680078199ba984bf8afc20b2f2eb37257 Signed-off-by: Andrew Dunham <andrew@du.nham.ca>	2 years ago
Andrew Dunham	4fb663fbd2	various: mark more tests as flaky ... Updates #2855 Updates #3598 Updates #7008 Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I2b849e04646456b9f0c8a01563f2add752f4b2a4	2 years ago
Brad Fitzpatrick	1116602d4c	ssh/tailssh: add OpenBSD support for Tailscale SSH ... And bump go.mod for https://github.com/u-root/u-root/pull/2593 Change-Id: I36ec94c5b2b76d671cb739f1e9a1a43ca1d9d1b1 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
James Tucker	b2c55e62c8	net/tlsdial,tstest,version: use go command from $PATH ... Go now includes the GOROOT bin directory in $PATH while running tests and generate, so it is no longer necessary to construct a path using runtime.GOROOT(). Fixes #6689 Signed-off-by: James Tucker <james@tailscale.com>	2 years ago
Jordan Whited	ea5ee6f87c	all: update golang.zx2c4.com/wireguard to github.com/tailscale/wireguard-go (#6692) ... This is temporary while we work to upstream performance work in https://github.com/WireGuard/wireguard-go/pull/64. A replace directive is less ideal as it breaks dependent code without duplication of the directive. Signed-off-by: Jordan Whited <jordan@tailscale.com>	2 years ago
Jordan Whited	76389d8baf	net/tstun, wgengine/magicsock: enable vectorized I/O on Linux (#6663) ... This commit updates the wireguard-go dependency and implements the necessary changes to the tun.Device and conn.Bind implementations to support passing vectors of packets in tailscaled. This significantly improves throughput performance on Linux. Updates #414 Signed-off-by: Jordan Whited <jordan@tailscale.com> Signed-off-by: James Tucker <james@tailscale.com> Co-authored-by: James Tucker <james@tailscale.com>	2 years ago
Aaron Klotz	98f21354c6	cmd/tailscaled: add a special command to tailscaled's Windows service for removing WinTun ... WinTun is installed lazily by tailscaled while it is running as LocalSystem. Based upon what we're seeing in bug reports and support requests, removing WinTun as a lesser user may fail under certain Windows versions, even when that user is an Administrator. By adding a user-defined command code to tailscaled, we can ask the service to do the removal on our behalf while it is still running as LocalSystem. * The uninstall code is basically the same as it is in corp; * The command code will be sent as a service control request and is protected by the SERVICE_USER_DEFINED_CONTROL access right, which requires Administrator. I'll be adding follow-up patches in corp to engage this functionality. Updates https://github.com/tailscale/tailscale/issues/6433 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	2 years ago
Brad Fitzpatrick	0cb2ccce7f	safesocket: remove the IPN protocol support ... Updates #6417 Change-Id: I78908633de842d83b2cc8b10a864a0f88ab1b113 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	3b0de97e07	cmd/tailscaled: unify the two Windows paths + separate IPN server path ... tailscaled on Windows had two entirely separate start-up paths for running as a service vs in the foreground. It's been causing problems for ages. This unifies the two paths, making them be the same as the path used for every other platform. Also, it uses the new async LocalBackend support in ipnserver.Server so the Server can start serving HTTP immediately, even if tun takes awhile to come up. Updates #6535 Change-Id: Icc8c4f96d4887b54a024d7ac15ad11096b5a58cf Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	e8cc78b1af	ipn/ipnserver: change Server to let LocalBackend be supplied async ... This is step 1 of de-special-casing of Windows and letting the LocalAPI HTTP server start serving immediately, even while the rest of the world (notably the Engine and its TUN device) are being created, which can take a few to dozens of seconds on Windows. With this change, the ipnserver.New function changes to not take an Engine and to return immediately, not returning an error, and let its Run run immediately. If its ServeHTTP is called when it doesn't yet have a LocalBackend, it returns an error. A TODO in there shows where a future handler will serve status before an engine is available. Future changes will: * delete a bunch of tailscaled_windows.go code and use this new API * add the ipnserver.Server ServerHTTP handler to await the engine being available * use that handler in the Windows GUI client Updates #6522 Change-Id: Iae94e68c235e850b112a72ea24ad0e0959b568ee Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Aaron Klotz	033bd94d4c	cmd/tailscaled, wgengine/router: use wingoes/com for COM initialization instead of go-ole ... This patch removes the crappy, half-backed COM initialization used by `go-ole` and replaces that with the `StartRuntime` function from `wingoes`, a library I have started which, among other things, initializes COM properly. In particular, we should always be initializing COM to use the multithreaded apartment. Every single OS thread in the process becomes implicitly initialized as part of the MTA, so we do not need to concern ourselves as to whether or not any particular OS thread has initialized COM. Furthermore, we no longer need to lock the OS thread when calling methods on COM interfaces. Single-threaded apartments are designed solely for working with Win32 threads that have a message pump; any other use of the STA is invalid. Fixes https://github.com/tailscale/tailscale/issues/3137 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	2 years ago
Brad Fitzpatrick	4d3713f631	envknob: add GOOS func ... Centralize the fake GOOS stuff, start to use it more. To be used more in the future. Change-Id: Iabacfbeaf5fca0b53bf4d5dbcdc0367f05a205f9 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Maisem Ali	41dd49391f	tstest/integration: add --accept-risk=all to tailscale down ... The test would fail if I was running it over SSH. Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Maisem Ali	26d1fc867e	ipn/ipnlocal: delete profile on logout ... Updates #713 Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Maisem Ali	4d330bac14	ipn/ipnlocal: add support for multiple user profiles ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Pat Maddox	9bf3ef4167	ssh/tailssh: add Tailscale SSH (server) support on FreeBSD ... Change-Id: I607194b6ef99205e777f3df93a74ffe1a2e0344c Signed-off-by: Pat Maddox <pat@ratiopbc.com>	2 years ago
Brad Fitzpatrick	da8def8e13	all: remove old +build tags ... The //go:build syntax was introduced in Go 1.17: https://go.dev/doc/go1.17#build-lines gofmt has kept the +build and go:build lines in sync since then, but enough time has passed. Time to remove them. Done with: perl -i -npe 's,^// \+build.*\n,,' $(git grep -l -F '+build') Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Andrew Dunham	e975cb6b05	ipn/ipnlocal: fix test flake when we log after a test completes ... This switches from using an atomic.Bool to a mutex for reasons that are described in the commit, and should address the flakes that we're still seeing. Fixes #3020 Change-Id: I4e39471c0eb95886db03020ea1ccf688c7564a11 Signed-off-by: Andrew Dunham <andrew@tailscale.com>	2 years ago
Brad Fitzpatrick	79755d3ce5	tstest/natlab: add Firewall.Reset method to drop firewall state ... For future use in magicsock tests. Updates #540 Change-Id: I2f07d1a2924f20b36e357c4533ff0a1a974d5061 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Josh Soref	d4811f11a0	all: fix spelling mistakes ... Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>	2 years ago
Andrew Dunham	c6162c2a94	net/netcheck: add check for captive portal (#5593) ... This doesn't change any behaviour for now, other than maybe running a full netcheck more often. The intent is to start gathering data on captive portals, and additionally, seeing this in the 'tailscale netcheck' command should provide a bit of additional information to users. Updates #1634 Change-Id: I6ba08f9c584dc0200619fa97f9fde1a319f25c76 Signed-off-by: Andrew Dunham <andrew@du.nham.ca>	2 years ago
Eng Zer Jun	f0347e841f	refactor: move from io/ioutil to io and os packages ... The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Reference: https://golang.org/doc/go1.16#ioutil Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	2 years ago
Brad Fitzpatrick	c66f99fcdc	tailcfg, control/controlclient, ipn/ipnlocal: add c2n (control-to-node) system ... This lets the control plane can make HTTP requests to nodes. Then we can use this for future things rather than slapping more stuff into MapResponse, etc. Change-Id: Ic802078c50d33653ae1f79d1e5257e7ade4408fd Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Maisem Ali	a9f6cd41fd	all: use syncs.AtomicValue ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Brad Fitzpatrick	8725b14056	all: migrate more code code to net/netip directly ... Instead of going through the tailscale.com/net/netaddr transitional wrappers. Updates #5162 Change-Id: I3dafd1c2effa1a6caa9b7151ecf6edd1a3fda3dd Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Maisem Ali	8fd5d3eaf3	tstest: do not error if the there are fewer goroutines than at start ... This fixes test failures like: ``` resource.go:34: goroutine count: expected 3, got 2 ``` Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Brad Fitzpatrick	a12aad6b47	all: convert more code to use net/netip directly ... perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	6a396731eb	all: use various net/netip parse funcs directly ... Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	7eaf5e509f	net/netaddr: start migrating to net/netip via new netaddr adapter package ... Updates #5162 Change-Id: Id7bdec303b25471f69d542f8ce43805328d56c12 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Maisem Ali	9514ed33d2	go.mod: bump gvisor.dev/gvisor ... Pick up https://github.com/google/gvisor/pull/7787 Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Mihai Parparita	de2dcda2e0	.github/workflows: update module that we try to build in cross-wasm ... We now have the actual module that we need to build, so switch to building it directly instead of its (expected) dependencies. Also fix a copy/paste error in a jsdeps comment. Signed-off-by: Mihai Parparita <mihai@tailscale.com>	2 years ago
Denton Gentry	7fffddce8e	net/portmapper: enable for iOS ... In the 1.27 unstable releases we set the min-version to iOS15, which means we have 50 MBytes of RAM in the Network Extension. https://tailscale.com/blog/go-linker/ Include the UPnP/NAT-PMP/PCP portmapper support now that there is memory for it. Fixes https://github.com/tailscale/tailscale/issues/2495 Signed-off-by: Denton Gentry <dgentry@tailscale.com>	2 years ago
Mihai Parparita	6246fa32f0	wasm: add test for disallowed JS/WASM dependencies ... Ensures that binary size gains like the ones from #4802 and #4813 don't regress. Updates #3517 Signed-off-by: Mihai Parparita <mihai@tailscale.com>	2 years ago
Maisem Ali	bf2fa7b184	go.mod: pin github.com/tailscale/mkctr (try #2) ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Joe Tsai	741ae9956e	tstest/integration/vms: use hujson.Standardize instead of hujson.Unmarshal (#4520) ... The hujson package transition to just being a pure AST parser and formatter for HuJSON and not an unmarshaler. Thus, parse HuJSON as such, convert it to JSON, and then use the standard JSON unmarshaler. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Maisem Ali	fd99c54e10	tailcfg,all: change structs to []*dnstype.Resolver ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Aaron Klotz	d915e0054c	cmd/tailscaled: change Windows service shutdown and add optional event logging ... Once a stop request is received and the service updates its status to `svc.StopPending`, it should continue running *until the shutdown sequence is complete*, and then return out of `(*ipnService).Execute`, which automatically sends a `svc.Stopped` notification to Windows. To make this happen, I changed the loop so that it runs until `doneCh` is closed, and then returns. I also removed a spurious `svc.StopPending` notification that the Windows Service Control Manager might be interpreting as a request for more time to shut down. Finally, I added some optional logging that sends a record of service notifications to the Windows event log, allowing us to more easily correlate with any Service Control Manager errors that are sent to the same log. Change-Id: I5b596122e5e89c4c655fe747a612a52cb4e8f1e0 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Brad Fitzpatrick	cf9b9a7fec	tstest/iosdeps: add test for forbidden iOS dependencies ... Fixes #4463 Change-Id: I8305710e8a075263ae9a88a29624b19032d5beeb Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	8ee044ea4a	ssh/tailssh: make the SSH server a singleton, register with LocalBackend ... Remove the weird netstack -> tailssh dependency and instead have tailssh register itself with ipnlocal when linked. This makes tailssh.server a singleton, so we can have a global map of all sessions. Updates #3802 Change-Id: Iad5caec3a26a33011796878ab66b8e7b49339f29 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Tom DNetto	dec68166e4	tstest/integration/vms: smoke test derphttp through mitm proxies ... Updates #4377 Very smoky/high-level test to ensure that derphttp internals play well with an agressive (stare + bump) meddler-in-the-middle proxy. Signed-off-by: Tom DNetto <tom@tailscale.com>	3 years ago
Tom DNetto	8f6d8cf979	tstest/integration/vms: test on stable nixos (21.11) ... I would like to do some more customized integration tests in the future, (specifically, bringing up a mitm proxy and testing tailscaled through that) so hoping to bring back the nixos wiring to support that. Signed-off-by: Tom DNetto <tom@tailscale.com>	3 years ago
Tom DNetto	3b3d1b9350	tstest/integration/vms: consistently use two dashes for command-line switches ... Signed-off-by: Tom DNetto <tom@tailscale.com>	3 years ago
Tom DNetto	858ab80172	tstest/integration/vms: fix docs, qemu-img invocation ... Signed-off-by: Tom DNetto <tom@tailscale.com>	3 years ago
Josh Bleecher Snyder	32fd42430b	all: use cibuild.On ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	0868329936	all: use any instead of interface{} ... My favorite part of generics. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Maisem Ali	0f37317664	control/controlclient: make RegisterRequest go over Noise ... Updates #3488 Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Maisem Ali	249758df90	control/controlclient: start fetching the server noise key ... Updates #3488 Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Brad Fitzpatrick	d5f8f38ac6	tailcfg: rename map request version to "capability version" ... And add a CapabilityVersion type, primarily for documentation. This makes MapRequest.Version, RegisterRequest.Version, and SetDNSRequest.Version all use the same version, which will avoid confusing in the future if Register or SetDNS ever changed their semantics on Version change. (Currently they're both always 1) This will requre a control server change to allow a SetDNSRequest.Version value other than 1 to be deployed first. Change-Id: I073042a216e0d745f52ee2dbc45cf336b9f84b7c Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Maisem Ali	497324ddf6	ipn/store: add common package for instantiating ipn.StateStores ... Also move KubeStore and MemStore into their own package. RELNOTE: tsnet now supports providing a custom ipn.StateStore. Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Brad Fitzpatrick	4cbdc84d27	cmd/tailscaled/childproc: add be-child registration mechanism ... For ssh and maybe windows service babysitter later. Updates #3802 Change-Id: I7492b98df98971b3fb72d148ba92c2276cca491f Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Maisem Ali	f9a50779e2	cmd/tailscaled: add `-state=mem:` to support creation of an ephemeral node. ... RELNOTE=`tailscaled --state=mem:` registers as an ephemeral node and does not store state to disk. Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Josh Bleecher Snyder	823d970d60	control/controlclient: use structured logging for MapResponse.ControlTime ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	8c3c5e80b7	tailcfg: make MapResponse.ControlTime a pointer ... Otherwise omitempty doesn't work. This is wire-compatible with a non-pointer type, so switching is safe, now and in the future. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	8cf6d0a17b	tailcfg: add MapResponse.ControlTime field ... And log it when provided in map responses. The test uses the date on which I joined Tailscale. :) Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Maisem Ali	72d8672ef7	tailcfg: make Node.Hostinfo a HostinfoView ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Brad Fitzpatrick	02bdc654d5	cmd/tailscale: fix up --reset, again ... Also fix a somewhat related printing bug in the process where some paths would print "Success." inconsistently even when there otherwise was no output (in the EditPrefs path) Fixes #3830 Updates #3702 (which broke it once while trying to fix it) Change-Id: Ic51e14526ad75be61ba00084670aa6a98221daa5 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	1af26222b6	go.mod: bump netstack, switch to upstream netstack ... Now that Go 1.17 has module graph pruning (https://go.dev/doc/go1.17#go-command), we should be able to use upstream netstack without breaking our private repo's build that then depends on the tailscale.com Go module. This is that experiment. Updates #1518 (the original bug to break out netstack to own module) Updates #2642 (this updates netstack, but doesn't remove workaround) Change-Id: I27a252c74a517053462e5250db09f379de8ac8ff Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	41fd4eab5c	envknob: add new package for all the strconv.ParseBool(os.Getenv(..)) ... A new package can also later record/report which knobs are checked and set. It also makes the code cleaner & easier to grep for env knobs. Change-Id: Id8a123ab7539f1fadbd27e0cbeac79c2e4f09751 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	92dfaf53bb	cmd/tailscaled: include Go runtime metrics in /debug/metrics ... Fixes #3772 Change-Id: I237ea23268664d99e83d27890146018b04474556 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	681d4897cc	tstest/integration: remove t param in testNode methods ... testNodes have a reference to a testing.TB via their env. Use it instead of making the caller pass theirs. We did this in some methods but not others; finish the job. This simplifies the call sites. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	93ae11105d	ipn/ipnlocal: clear magicsock's netmap on logout ... magicsock was hanging onto its netmap on logout, which caused tailscale status to display partial information about a bunch of zombie peers. After logout, there should be no peers. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	84a1106fa7	tstest/integration: make -verbose-tailscaled pass -verbose=2 to tailscaled ... If you're using -verbose-tailscaled, you're doing in-the-weeds debugging, so you probably want the verbose output. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	486059589b	all: gofmt -w -s (simplify) tests ... And it updates the build tag style on a couple files. Change-Id: I84478d822c8de3f84b56fa1176c99d2ea5083237 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	4691e012a9	tstest/integration: build binaries only once ... The existing code relied on the Go build cache to avoid needless work when obtaining the tailscale binaries. For non-obvious reasons, the binaries were getting re-linked every time, which added 600ms or so on my machine to every test. Instead, build the binaries exactly once, on demand. This reduces the time to run 'go test -count=5' from 34s to 10s on my machine. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	63cd581c3f	safesocket: add ConnectionStrategy, provide control over fallbacks ... fee2d9fad added support for cmd/tailscale to connect to IPNExtension. It came in two parts: If no socket was provided, dial IPNExtension first, and also, if dialing the socket failed, fall back to IPNExtension. The second half of that support caused the integration tests to fail when run on a machine that was also running IPNExtension. The integration tests want to wait until the tailscaled instances that they spun up are listening. They do that by dialing the new instance. But when that dial failed, it was falling back to IPNExtension, so it appeared (incorrectly) that tailscaled was running. Hilarity predictably ensued. If a user (or a test) explicitly provides a socket to dial, it is a reasonable assumption that they have a specific tailscaled in mind and don't want to fall back to IPNExtension. It is certainly true of the integration tests. Instead of adding a bool to Connect, split out the notion of a connection strategy. For now, the implementation remains the same, but with the details hidden a bit. Later, we can improve that. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	a5235e165c	tstest/integration: fix running with -verbose-tailscale ... Without this fix, any run with -verbose-tailscale fails. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	c8829b742b	all: minor code cleanup ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	768baafcb5	tstest/integration: use t.Cleanup ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	c37af58ea4	net/tsdial: move more weirdo dialing into new tsdial package, plumb ... Not done yet, but this move more of the outbound dial special casing from random packages into tsdial, which aspires to be the one unified place for all outbound dialing shenanigans. Then this plumbs it all around, so everybody is ultimately holding on to the same dialer. As of this commit, macOS/iOS using an exit node should be able to reach to the exit node's DoH DNS proxy over peerapi, doing the sockopt to stay within the Network Extension. A number of steps remain, including but limited to: * move a bunch more random dialing stuff * make netstack-mode tailscaled be able to use exit node's DNS proxy, teaching tsdial's resolver to use it when an exit node is in use. Updates #1713 Change-Id: I1e8ee378f125421c2b816f47bc2c6d913ddcd2f5 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Artyom Pervukhin	49a9e62d58	Replace AWS SDK v1 dependency with v2 ... This change drops AWS SDKv1 dependency, leaving only SDK v2 in use. Closes #3461 Signed-off-by: Artyom Pervukhin <github@artyom.dev>	3 years ago
Brad Fitzpatrick	d5405c66b7	net/tsdial: start of new package to unify all outbound dialing complexity ... For now this just deletes the net/socks5/tssocks implementation (and the DNSMap stuff from wgengine/netstack) and moves it into net/tsdial. Then initialize a Dialer early in tailscaled, currently only use for the outbound and SOCKS5 proxies. It will be plumbed more later. Notably, it needs to get down into the DNS forwarder for exit node DNS forwading in netstack mode. But it will also absorb all the peerapi setsockopt and netns Dial and tlsdial complexity too. Updates #1713 Change-Id: Ibc6d56ae21a22655b2fa1002d8fc3f2b2ae8b6df Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	b800663779	tstest/integration: stop leaking zstd.Decoders ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
David Anderson	6e584ffa33	cmd/tailscaled: allow running the SOCKS5 and HTTP proxies on the same port. ... Fixes #3248 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Brad Fitzpatrick	3b541c833e	util/clientmetric, logtail: log metric changes ... Updates #3307 Change-Id: I1399ebd786f6ff7defe6e11c0eb651144c071574 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	57b039c51d	util/clientmetrics: add new package to add metrics to the client ... And annotate magicsock as a start. And add localapi and debug handlers with the Prometheus-format exporter. Updates #3307 Change-Id: I47c5d535fe54424741df143d052760387248f8d3 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	f7da8c77bd	tstest/integration/testcontrol: fix data race ... Fix race from 1ec99e99f4 Fixes #3289 Change-Id: I58158d3f82339ac171fb14827c5f158d602327f4 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Maisem Ali	d6dde5a1ac	ipn/ipnlocal: handle key extensions after key has already expired ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Brad Fitzpatrick	5f36ab8a90	tstest/integration: go generate ... Change-Id: I49d19007a16261e447240e149deac24c15c93fce Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	2b082959db	safesocket: add WindowsLocalPort const ... Remove all the 41112 references. Change-Id: I2d7ed330d457e3bb91b7e6416cfb2667611e50c4 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Denton Gentry	1ec99e99f4	tstest: extend node key expiration integration test. ... Can produce the problem in #2515, preparing to test a fix. Marked as t.Skip() until we have a fix. Updates https://github.com/tailscale/tailscale/issues/2515 Signed-off-by: Denton Gentry <dgentry@tailscale.com>	3 years ago
David Anderson	0532eb30db	all: replace tailcfg.DiscoKey with key.DiscoPublic. ... Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Brad Fitzpatrick	6d82a18916	tstest/integration: don't include stdlib deps in go generate output ... Causes too much churn for zero benefit. Change-Id: I838f8cdb5723f122f11dd4bbce5e9c07755c3cd9 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	3fd5f4380f	util/multierr: new package ... github.com/go-multierror/multierror served us well. But we need a few feature from it (implement Is), and it's not worth maintaining a fork of such a small module. Instead, I did a clean room implementation inspired by its API. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
David Anderson	7e6a1ef4f1	tailcfg: use key.NodePublic in wire protocol types. ... Updates #3206. Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Brad Fitzpatrick	0303ec44c3	go.mod: bump netstack for mipsle fix ... Fixes #3233 Change-Id: I18d1af886402774ce0ecc77dae3bc71eb8ba5c9d Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	c18b9d58aa	tstest/archtest: add GOARCH-specific tests, run via qemu-user ... Updates #3233 Change-Id: Ia224c90490d41e50a1d547eeea709b0d9171c1f9 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Anderson	8d14bc32d1	tstest/integration: use key.NodePublic instead of tailcfg.NodeKey. ... Updates #3206 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	418adae379	various: use NodePublic.AsNodeKey() instead of tailcfg.NodeKeyFromNodePublic() ... Updates #3206 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	eeb97fd89f	various: remove remaining uses of key.NewPrivate. ... Updates #3206 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	37c150aee1	derp: use new node key type. ... Update #3206 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Josh Bleecher Snyder	4bb2c6980d	util/testingutil: new package with MinAllocsPerRun ... testing.AllocsPerRun measures the total allocations performed by the entire program while repeatedly executing a function f. If some unrelated part of the rest of the program happens to allocate a lot during that period, you end up with a test failure. Ideally, the rest of the program would be silent while testing.AllocsPerRun executes. Realistically, that is often unachievable. AllocsPerRun attempts to mitigate this by setting GOMAXPROCS to 1, but that doesn't prevent other code from running; it only makes it less likely. You can also mitigate this by passing a large iteration count to AllocsPerRun, but that is unreliable and needlessly expensive. Unlike most of package testing, AllocsPerRun doesn't use any toolchain magic, so we can just write a replacement. One wild idea is to change how we count mallocs. Instead of using runtime.MemStats, turn on memory profiling with a memprofilerate of 1. Discard all samples from the profile whose stack does not contain testing.AllocsPerRun. Count the remaining samples to determine the number of mallocs. That's fun, but overkill. Instead, this change adds a simple API that attempts to get f to run at least once with a target number of allocations. This is useful when you know that f should allocate consistently. We can then assume that any iterations with too many allocations are probably due to one-time costs or background noise. This suits most uses of AllocsPerRun. Ratcheting tests tend to be significantly less flaky, because they are biased towards success. They can also be faster, because they can exit early, once success has been reached. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Maisem Ali	81cabf48ec	control/controlclient,tailcfg: propagate registration errors to the frontend ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
nicksherron	f01ff18b6f	all: fix spelling mistakes ... Signed-off-by: nicksherron <nsherron90@gmail.com>	3 years ago
Brad Fitzpatrick	7634af5c6f	all: gofmt ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Aaron Klotz	e016eaf410	cmd/tailscaled: conditionally flush Windows DNS cache on SessionChange ... For the service, all we need to do is handle the `svc.SessionChange` command. Upon receipt of a `windows.WTS_SESSION_UNLOCK` event, we fire off a goroutine to flush the DNS cache. (Windows expects responses to service requests to be quick, so we don't want to do that synchronously.) This is gated on an integral registry value named `FlushDNSOnSessionUnlock`, whose value we obtain during service initialization. (See [this link](https://docs.microsoft.com/en-us/windows/win32/api/winsvc/nc-winsvc-lphandler_function_ex) for information re: handling `SERVICE_CONTROL_SESSIONCHANGE`.) Fixes #2956 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Brad Fitzpatrick	a7cb241db1	cmd/tailscaled: add support for running an HTTP proxy ... This adds support for tailscaled to be an HTTP proxy server. It shares the same backend dialing code as the SOCK5 server, but the client protocol is HTTP (including CONNECT), rather than SOCKS. Fixes #2289 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	382b349c54	cmd/tailscaled: disable netns in userspace-networking mode ... Updates #2827 Updates #2822 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	a353fbd3b4	tstest: make MemLogger.String acquire its mutex ... Updates #2781 (might even fix it, but its real issue is that SetPrivateKey starts a ReSTUN goroutines which then logs, and that bug and data race existed prior to MemLogger existing)	3 years ago
Brad Fitzpatrick	640134421e	all: update tests to use tstest.MemLogger ... And give MemLogger a mutex, as one caller had, which does match the logf contract better. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	4c68b7df7c	tstest: add MemLogger bytes.Buffer wrapper with Logf method ... We use it tons of places. Updated three at least in this PR. Another use in next commit. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Crawshaw	b2a3d1da13	tstest/integration/vms: use fork of goexpect to avoid proto/grpc dep ... Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Dave Anderson	980acc38ba	types/key: add a special key with custom serialization for control private keys (#2792) ... * Revert "Revert "types/key: add MachinePrivate and MachinePublic."" This reverts commit 61c3b98a24. Signed-off-by: David Anderson <danderson@tailscale.com> * types/key: add ControlPrivate, with custom serialization. ControlPrivate is just a MachinePrivate that serializes differently in JSON, to be compatible with how the Tailscale control plane historically serialized its private key. Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	61c3b98a24	Revert "types/key: add MachinePrivate and MachinePublic." ... Broke the tailscale control plane due to surprise different serialization. This reverts commit 4fdb88efe1.	3 years ago
David Anderson	4fdb88efe1	types/key: add MachinePrivate and MachinePublic. ... Plumb throughout the codebase as a replacement for the mixed use of tailcfg.MachineKey and wgkey.Private/Public. Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Crawshaw	4e18cca62e	testcontrol: replace panic with error ... I have seen this once in the VM test (caused by an EOF, I believe on shutdown) that didn't need to cause the test to fail. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	f53792026e	tstest/integration/vms: move build tags from linux to !windows ... The tests build fine on other Unix's, they just can't run there. But there is already a t.Skip by default, so `go test` ends up working fine elsewhere and checks the code compiles. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Christine Dodrill	0b9e938152	tstest/integration/vms: test DNS configuration ... This uses a neat little tool to dump the output of DNS queries to standard out. This is the first end-to-end test of DNS that runs against actual linux systems. The /etc/resolv.conf test may look superflous, however this will help for correlating system state if one of the DNS tests fails. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
David Crawshaw	9b7fc2ed1f	.github: add Ubuntu VM test ... Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	debaaebf3b	tstest/integration/vms: turn on logcatcher logging by default ... Absolutely vital to debugging failures. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	a1f1020042	tstest/integration/vms: avoid log after test completion ... Avoids a panic in the Go testing package if a late log comes in. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	583af7c1a6	tstest/integration/vms: give guest multiple cores and use generic machine ... Speeds up tests. Allows the use of more version of qemu. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	8668103f06	tstest/integration/vms: print qemu console output, fix printing issues ... Fix a few test printing issues when tests fail. Qemu console output is super useful when something is wrong in the harness and we cannot even bring up the tests. Also useful for figuring out where all the time goes in tests. A little noisy, but not too noisy as long as you're only running one VM as part of the tests, which is my plan. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	1a9fba5b04	tstest/integration/vms: fix ubuntu URLs ... Also remove extra distros for now. We can bring them back later if useful. Though our most important distros are these two Ubuntu, debian stable, and Raspbian (not currently supported). And before doing more Linux, we should do Windows. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	bb47feca44	tstest/integration: prefix logs with logid ... The VM test has two tailscaled instances running and interleaves the logs. Without a prefix it is impossible to figure out what is going on. It might be even better to include the [ABCD] node prefix here as well. Unfortunately lots of interesting logs happen before tailscaled has a node key, so it wouldn't be a replacement for a short ID. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Maisem Ali	fd4838dc57	wgengine/userspace: add support to automatically enable/disable the tailscale ... protocol in BIRD, when the node is a primary subnet router as determined by control. Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
David Crawshaw	0b962567fa	vms: make ssh-keygen quiet by default ... Always succeeds. Its output clutters the log. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	4fcce70df5	tstest/integration: listen on the specified IP ... By default httptest listens only on the loopback adapter. Instead, listen on the IP the user asked for. The VM test needs this, as it wants to start DERP and STUN servers on the host that can be reached by guest VMs. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
David Crawshaw	0b2761ca92	testcontrol: plumb through DERP ... Without this tailscaled does not know the DERP of a peer. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
slowy07	ac0353e982	fix: typo spelling grammar ... Signed-off-by: slowy07 <slowy.arfy@gmail.com>	3 years ago
Josh Bleecher Snyder	4ff0757d44	cmd/testcontrol: add test control server ... This is useful for manual performance testing of networks with many nodes. I imagine it'll grow more knobs over time. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	fcca374fa7	tstest/integration/testcontrol: sort peers in map response ... This is part of the control protocol. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	a5da4ed981	all: gofmt with Go 1.17 ... This adds "//go:build" lines and tidies up existing "// +build" lines. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	0858673f1f	tstest/integration: regenerate test deps ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	24ee0ed3c3	tstest/integration: update test deps ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Crawshaw	3e039daf95	logpolicy: actually collect panics ... (Written with Josh) For #2544 Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Christine Dodrill	60f34c70a2	tstest/integration/vms: disable rDNS for sshd on centos (#2492) ... This prevents centos tests from timing out because sshd does reverse dns lookups on every session being established instead of doing it once on the acutal ssh connection being established. This is odd. Appending this to the sshd config and restarting it seems to fix it though. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	8db26a2261	tstest/integration/vms: disable nixos unstable (#2491) ... cloud-init broke with the upgrade to python 3.9: https://github.com/NixOS/nixpkgs/issues/131098 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Josh Bleecher Snyder	0aa77ba80f	tstest/integration: fix filch test flake ... Filch doesn't like having multiple processes competing for the same log files (#937). Parallel integration tests were all using the same log files. Add a TS_LOGS_DIR env var that the integration test can use to use separate log files per test. Fixes #2269 Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	f80193fa4c	tstest/integration: shorten test names ... The maximum unix domain socket path length on darwin is 104 bytes, including the trailing NUL. On my machine, the path created by some newly added tests (6eecf3c9) was too long, resulting in cryptic test failures. Shorten the names of the tests, and add a check to make the diagnosis easier next time. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	b6d70203d3	ipn/ipnlocal: fix 'tailscale up' on Windows without GUI ... With this, I can now: * install Tailscale * stop the GUI * net stop Tailscale * net start Tailscale * tailscale up --unattended (where the middle three steps simulate what would happen on a Windows Server Core machine without a GUI) Fixes #2137 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	4b1f2ae382	tstest/integration: generate deps for all platforms in deps generator ... We have different deps depending on the platform. If we pick a privileged platform, we'll miss some deps. If we use the union of all platforms, the integration test won't compile on some platforms, because it'll import packages that don't compile on that platform. Give in to the madness and give each platform its own deps file. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	6eecf3c9d1	ipn/ipnlocal: stay out of map poll when down ... Fixes #2434 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	798b0da470	tstest/integration/vms: codegen for top level tests (#2441) ... This moves the distribution definitions into a maintainable hujson file instead of just existing as constants in `distros.go`. Comments are maintained from the inline definitions. This uses jennifer[1] for hygenic source tree creation. This allows us to generate a unique top-level test for each VM run. This should hopefully help make the output of `go test` easier to read. This also separates each test out into its own top-level test so that we can better track the time that each distro takes. I really wish there was a way to have the `test_codegen.go` file _always_ run as a part of the compile process instead of having to rely on people remembering to run `go generate`, but I am limited by my tools. This will let us remove the `-distro-regex` flag and use `go test -run` to pick which distros are run. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	391207bbcf	tstest/integration/vms: use one testcontrol instance per VM (#2437) ... This paves the way for future MagicDNS tests. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
julianknodt	1bb6abc604	net/portmapper: add upnp port mapping ... Add in UPnP portmapping, using goupnp library in order to get the UPnP client and run the portmapping functions. This rips out anywhere where UPnP used to be in portmapping, and has a flow separate from PMP and PCP. RELNOTE=portmapper now supports UPnP mappings Fixes #682 Updates #2109 Signed-off-by: julianknodt <julianknodt@gmail.com>	3 years ago
Christine Dodrill	3ebe16558c	tstest/integration/vms: attempt to send more UDP packets (#2433) ... Apparently this test was flaking because I critically misunderstood how the kernel buffers UDP packets for senders. I'm trying to send more UDP packets and will see if that helps. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	09e81b8ba1	tstest/integration/vms: deflake the tailscale_status test (#2427) ... This test used to try to run this only once, but this variant of the test attempts to run `tailscale status` up to 6 times in a loop with exponential backoff. This fixes the flakiness found in previous instances of this test. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	01e159b610	ipn/ipnlocal: save prefs to disk on UpdatePrefs ... Regression from 6d10655dc3, which added UpdatePrefs but didn't write it out to disk. I'd planned on adding tests to state_test.go which is why I'd earlier added 46896a9311 to prepare for making such persistence tests easier to write, but turns out state_test.go didn't even test UpdatePrefs, so I'm staying out of there. Instead, this is tested using integration tests. Fixes #2321 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	5b845631ce	tstest/integration/vms: delete a log file i was using to debug the tests (#2416) ... Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	ede8ec1e20	tstest/integration/vms: split up test framework into files (#2396) ... My spatial memory functions poorly with large files and the vms_test.go file recently surpassed the point where it functions adequately. This patch splits up vms_test.go into more files to make my spatial memory function like I need it to. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	1f2a877c61	tstest/integration/testcontrol: add ipv6 support (#2394) ... Split from https://github.com/tailscale/tailscale/pull/2376. This adds IPv6 support to testcontrol so each member of the tailscale network gets an IPv6 address too. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	1b1c85d069	Revert "Revert "tstest/integration/vms: end-to-end UDP test (#2361)" (#2364)" (#2365) ... This reverts commit 254fc7885b. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Simeng He	3d049a9d71	tstest/integration: change return type of AwaitIP from string to netaddr.IP ... Signed-off-by: Simeng He <simeng@tailscale.com>	3 years ago
Brad Fitzpatrick	7e7c4c1bbe	tailcfg: break DERPNode.DERPTestPort into DERPPort & InsecureForTests ... The DERPTestPort int meant two things before: which port to use, and whether to disable TLS verification. Users would like to set the port without disabling TLS, so break it into two options. Updates #1264 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	afbd35482d	tstest/integration/vms: add NixOS unstable back (#2330) ... The upstream NixOS issue was fixed. https://github.com/NixOS/nixpkgs/issues/128783 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	33cacb5284	tstest/integration/vms: unbreak nixos builds (#2372) ... To avoid the generated nixos disk images from becoming immune from the GC, I delete the symlink to the nix store at the end of tests. `t.Cleanup` runs at the end of a test. I changed this part of the code to have a separate timer for how long it takes to run NixOS builds, but I did that by using a subtest. This means that it was creating the NixOS image, deleting its symlink and then trying to use that symlink to find the resulting disk image, making the whole thing ineffectual. This was a mistake. I am reverting this change made in https://github.com/tailscale/tailscale/pull/2360 to remove this layer of subtesting. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	254fc7885b	Revert "tstest/integration/vms: end-to-end UDP test (#2361)" (#2364) ... This reverts commit dc78be12c5.	3 years ago
Christine Dodrill	dc78be12c5	tstest/integration/vms: end-to-end UDP test (#2361) ... This tests incoming and outgoing UDP traffic. It would test incoming UDP traffic however our socks server doesn't seem to allow for connecting to destinations over UDP. When the socks server gets that support the incoming test should pass without issue. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	a19eea965f	tstest/integration/vms: use an in-process logcatcher (#2360) ... This adapts the existing in-process logcatcher from tstest/integration into a public type and uses it on the side of testcontrol. This also fixes a bug in the Alpine Linux OpenRC unit that makes every value in `/etc/default/tailscaled` exported into tailscaled's environment, a-la systemd [Service].EnviromentFile. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	1e83b97498	tstest/integration/vms: outgoing SSH test (#2349) ... This does a few things: 1. Rewrites the tests so that we get a log of what individual tests failed at the end of a test run. 2. Adds a test that runs an HTTP server via the tester tailscale node and then has the VMs connect to that over Tailscale. 3. Dials the VM over Tailscale and ensures it answers SSH requests. 4. Other minor framework refactoring. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	97279a0fe0	tstest/integration/vms: add Oracle Linux image (#2328) ... Oracle Linux[1] is a CentOS fork. It is not very special. I am adding it to the integration jungle because I am adding it to pkgs and the website directions. [1]: https://www.oracle.com/linux/ Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	7d417586a8	tstest/integration: help bust cmd/go's test caching ... It was caching too aggressively, as it didn't see our deps due to our running "go install tailscaled" as a child process. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	a8360050e7	tstest/integration/vms: make first end to end test (#2332) ... This makes sure `tailscale status` and `tailscale ping` works. It also switches goexpect to use a batch instead of manually banging out each line, which makes the tests so much easier to read. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	5813da885c	tstest/integration/vms: verbosify nixos logs to fs, disable unstable (#2294) ... This puts nix build logs on the filesystem so that we can debug them later. This also disables nixos unstable until https://github.com/NixOS/nixpkgs/issues/128783 is fixed. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
julianknodt	506c2fe8e2	cmd/tailscale: make netcheck use active DERP map, delete static copy ... After allowing for custom DERP maps, it's convenient to be able to see their latency in netcheck. This adds a query to the local tailscaled for the current DERPMap. Updates #1264 Signed-off-by: julianknodt <julianknodt@gmail.com>	3 years ago
Brad Fitzpatrick	5e19ac7adc	tstest/integration: always run SOCK5 server, parse out its listening address ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	b131a74f99	tstest/integration/vms: build and run NixOS (#2190) ... Okay, so, at a high level testing NixOS is a lot different than other distros due to NixOS' determinism. Normally NixOS wants packages to be defined in either an overlay, a custom packageOverrides or even yolo-inline as a part of the system configuration. This is going to have us take a different approach compared to other distributions. The overall plan here is as following: 1. make the binaries as normal 2. template in their paths as raw strings to the nixos system module 3. run `nixos-generators -f qcow -o $CACHE_DIR/tailscale/nixos/version -c generated-config.nix` 4. pass that to the steps that make the virtual machine It doesn't really make sense for us to use a premade virtual machine image for this as that will make it harder to deterministically create the image. Nix commands generate a lot of output, so their output is hidden behind the `-verbose-nix-output` flag. This unfortunately makes this test suite have a hard dependency on Nix/NixOS, however the test suite has only ever been run on NixOS (and I am not sure if it runs on other distros at all), so this probably isn't too big of an issue. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	194d5b8412	tstest/integration/vms: add in-process DERP server (#2108) ... Previously this test would reach out to the public DERP servers in order to help machines connect with eachother. This is not ideal given our plans to run these tests completely disconnected from the internet. This patch introduces an in-process DERP server running on its own randomly assigned HTTP port. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	6b234323a0	tstest/integration/vms: fix flake when testing (#2145) ... Occasionally the test framework would fail with a timeout due to a virtual machine not phoning home in time. This seems to be happen whenever qemu can't bind the VNC or SSH ports for a virtual machine. This was fixed by taking the following actions: 1. Don't listen on VNC unless the `-use-vnc` flag is passed, this removes the need to listen on VNC at all in most cases. The option to use VNC is still left in for debugging virtual machines, but removing this makes it easier to deal with (VNC uses this odd system of "displays" that are mapped to ports above 5900, and qemu doesn't offer a decent way to use a normal port number, so we just disable VNC by default as a compromise). 2. Use a (hopefully) inactive port for SSH. In an ideal world I'd just have the VM's SSH port be exposed via a Unix socket, however the QEMU documentation doesn't really say if you can do this or not. While I do more research, this stopgap will have to make do. 3. Strictly tie more VM resource lifetimes to the tests themselves. Previously the disk image layers for virtual machines were only cleaned up at the end of the test and existed in the parent test-scoped temporary folder. This can make your tmpfs run out of space, which is not ideal. This should minimize the use of temporary storage as much as I know how to. 4. Strictly tie the qemu process lifetime to the lifetime of the test using testing.T#Cleanup. Previously it used a defer statement to clean up the qemu process, however if the tests timed out this defer was not run. This left around an orphaned qemu process that had to be killed manually. This change ensures that all qemu processes exit when their relevant tests finish. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	45e64f2e1a	net/dns{,/resolver}: refactor DNS forwarder, send out of right link on macOS/iOS ... Fixes #2224 Fixes tailscale/corp#2045 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	a0c632f6b5	tstest/integration: fix a race ... Noticed on a CI failure. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Crawshaw	297b3d6fa4	staticcheck.conf: turn off noisy lint errors ... Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Brad Fitzpatrick	2f4817fe20	tstest/integration: fix race flake ... Fixes #2172 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	e0f0d10672	tstest/integration/vms: log to t.Logf directly (#2147) ... Previously we used t.Logf indirectly via package log. This worked, but it was not ideal for our needs. It could cause the streams of output to get crossed. This change uses a logger.FuncWriter every place log.Output was previously used, which will more correctly write log information to the right test output stream. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	082cc1b0a7	tstest/integration: reenable TestAddPingRequest ... Failure understood now; see: https://github.com/tailscale/tailscale/pull/2088#issuecomment-859896598 As of 333e9e75d4, PingRequest is now safe for the server to send multiple times, without fear of the client handling it multiple times. Fixes #2079 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Denton Gentry	c61d777705	tstest/integration: disable TestAddPingRequest ... Failing often now, we don't want people to get used to routinely ignoring test failures. Can be re-enabled when https://github.com/tailscale/tailscale/issues/2079 is resolved. Signed-off-by: Denton Gentry <dgentry@tailscale.com>	3 years ago
Christine Dodrill	8b2b899989	tstest/integration: test Alpine Linux (#2098) ... Alpine Linux[1] is a minimal Linux distribution built around musl libc. It boots very quickly, requires very little ram and is as close as you can get to an ideal citizen for testing Tailscale on musl. Alpine has a Tailscale package already[2], but this patch also makes it easier for us to provide an Alpine Linux package off of pkgs in the future. Alpine only offers Tailscale on the rolling-release edge branch. [1]: https://alpinelinux.org/ [2]: https://pkgs.alpinelinux.org/packages?name=tailscale&branch=edge Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	0affcd4e12	tstest/integration: add some debugging for TestAddPingRequest flakes ... This fails pretty reliably with a lot of output now showing what's happening: TS_DEBUG_MAP=1 go test --failfast -v -run=Ping -race -count=20 ./tstest/integration --verbose-tailscaled I haven't dug into the details yet, though. Updates #2079	3 years ago
Brad Fitzpatrick	ee3df2f720	tstest/integration: rename ambiguous --verbose test flag ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	b402e76185	.github/workflows: add integration test with a custom runner (#2044) ... This runner is in my homelab while we muse about a better, more permanent home for these tests. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	622dc7b093	tstest/integration/vms: download images from s3 (#2035) ... This makes integration tests pull pristine VM images from Amazon S3 if they don't exist on disk. If the S3 fetch fails, it will fall back to grabbing the image from the public internet. The VM images on the public internet are known to be updated without warning and thusly change their SHA256 checksum. This is not ideal for a test that we want to be able to fire and forget, then run reliably for a very long time. This requires an AWS profile to be configured at the default path. The S3 bucket is rigged so that the requester pays. The VM images are currently about 6.9 gigabytes. Please keep this in mind when running these tests on your machine. Documentation was added to the integration test folder to aid others in running these tests on their machine. Some wording in the logs of the tests was altered. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	3f1405fa2a	tstest/integration/vms: bump images, fix caching bug (#2052) ... Before this redownloaded the image every time. Now it only redownloads it when it needs to. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Simeng He	0141390365	tstest/integration/testcontrol: add Server.AddPingRequest ... Signed-off-by: Simeng He <simeng@tailscale.com>	3 years ago
Christine Dodrill	adaecd83c8	tstest/integration/vms: add DownloadImages test to download images (#2039) ... The image downloads can take a significant amount of time for the tests. This creates a new test that will download every distro image into the local cache in parallel, optionally matching the distribution regex. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	607b7ab692	tstest/integration/vms: aggressively re-verify shasums (#2050) ... I've run into a couple issues where the tests time out while a VM image is being downloaded, making the cache poisoned for the next run. This moves the hash checking into its own function and calls it much sooner in the testing chain. If the hash check fails, the OS is redownloaded. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	6ce77b8eca	tstest/integration/vms: log qemu output (#2047) ... Most of the time qemu will output nothing when it is running. This is expected behavior. However when qemu is unable to start due to some problem, it prints that to either stdout or stderr. Previously this output wasn't being captured. This patch captures that output to aid in debugging qemu issues. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	58cc2cc921	tstest/integration/testcontrol: add Server.nodeLocked	3 years ago
Christine Dodrill	0a655309c6	tstest/integration/vms: only build binaries once (#2042) ... Previously this built the binaries for every distro. This is a bit overkill given we are using static binaries. This patch makes us only build once. There was also a weird issue with how processes were being managed. Previously we just killed qemu with Process.Kill(), however that was leaving behind zombies. This has been mended to not only kill qemu but also waitpid() the process so it doesn't become a zombie. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	a282819026	tstest/integration/vms: fix OpenSUSE Leap 15.1 (#2038) ... The OpenSUSE 15.1 image we are using (and conseqentially the only one that is really available easily given it is EOL) has cloud-init hardcoded to use the OpenStack metadata thingy. Other OpenSUSE Leap images function fine with the NoCloud backend, but this one seems to just not work with it. No bother, we can just pretend to be OpenStack. Thanks to Okami for giving me an example OpenStack configuration seed image. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	4da5e79c39	tstest/integration/vms: test on Arch Linux (#2040) ... Arch is a bit of a weirder distro, however as a side effect it is much more of a systemd purist experience. Adding it to our test suite will make sure that we are working in the systemd happy path. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	ca96357d4b	tstest/integration/vms: add OpenSUSE Leap 15.3 (#2026) ... This distro is about to be released. OpenSUSE has historically had the least coverage for functional testing, so this may prove useful in the future. Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	2802a01b81	tstest/integration/vms: test vms as they are ready (#2022) ... Instead of testing all the VMs at once when they are all ready, this patch changes the testing logic so that the vms are tested as soon as they register with testcontrol. Also limit the amount of VM ram used at once with the `-ram-limit` flag. That uses a semaphore to guard resource use. Also document CentOS' sins. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	36cb69002a	tstest/integration/vms: regex-match distros using a flag (#2021) ... If you set `-distro-regex` to match a subset of distros, only those distros will be tested. Ex: $ go test -run-vm-tests -distro-regex='opensuse' Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Christine Dodrill	e1b994f7ed	tstest/integration/vms: maintain distro info (#2020) ... This lets us see the names of distros in our tests. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	fa548c5b96	tstest/integration/vms: fix bindhost lookup (#2012) ... Don't try to do heuristics on the name. Use the net/interfaces package which we already have to do this sort of stuff. Fixes #2011 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	14c1113d2b	tstest/integration/vms: copy locally built binaries (#2006) ... Instead of pulling packages from pkgs.tailscale.com, we should use the tailscale binaries that are local to this git commit. This exposes a bit of the integration testing stack in order to copy the binaries correctly. This commit also bumps our version of github.com/pkg/sftp to the latest commit. If you run into trouble with yaml, be sure to check out the commented-out alpine linux image complete with instructions on how to use it. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago
Brad Fitzpatrick	f21982f854	tstest/integration/vms: skip a test for now ... Updates #2011 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Christine Dodrill	4cfaf489ac	tstest/integration/vms: t.Log for VM output (#2007) ... Previously we spewed a lot of output to stdout and stderr, even when `-v` wasn't set. This is sub-optimal for various reasons. This patch shunts that output to test logs so it only shows up when `-v` is set. Updates #1988 Signed-off-by: Christine Dodrill <xe@tailscale.com>	3 years ago