Commit Graph

189 Commits (b1bd96f114ddf33f841a621af025cccd6a776e5b)

Author SHA1 Message Date
Brad Fitzpatrick c1cabe75dc derp: fix server struct fielfd alignment on 32-bit
Mostly so the GitHub CI will pass on 32-bit.
4 years ago
Brad Fitzpatrick 4732722b87 derp: add frameClosePeer to move around clients within a region
For various reasons (mostly during rollouts or config changes on our
side), nodes may end up connecting to a fallback DERP node in a
region, rather than the primary one we tell them about in the DERP
map.

Connecting to the "wrong" node is fine, but it's in our best interest
for all nodes in a domain to connect to the same node, to reduce
intra-region packet forwarding.

This adds a privileged frame type used by the control system that can
kick off a client connection when they're connected to the wrong node
in a region. Then they hopefully reconnect immediately to the correct
location. (If not, we can leave them alone and stop closing them.)

Updates tailscale/corp#372
4 years ago
Brad Fitzpatrick dd43d9bc5f derp: fix varz typo
Updates tailscale/corp#391
4 years ago
Brad Fitzpatrick b87396b5d9 cmd/derper, derp: add some more varz and consistency check handler
I'm trying to hunt down a slow drift in numbers not agreeing.
4 years ago
Brad Fitzpatrick 6fbd1abcd3 derp: update peerGone code to work with regional DERP mesh clusters too
Updates #150
Updates #388
4 years ago
Brad Fitzpatrick c8cf3169ba cmd/derper, derp/derphttp: move bulk of derp mesh code into derphttp
To be reused in various other tools.
5 years ago
Brad Fitzpatrick 6757c990a8 Fix staticcheck warning, add Makefile with staticcheck targets, lock in staticcheck version in go.mod 5 years ago
Brad Fitzpatrick abd79ea368 derp: reduce DERP memory use; don't require callers to pass in memory to use
The magicsock derpReader was holding onto 65KB for each DERP
connection forever, just in case.

Make the derp{,http}.Client be in charge of memory instead. It can
reuse its bufio.Reader buffer space.
5 years ago
Brad Fitzpatrick 07ca0c1c29 derp: fix tracking problem if conn starts local, then also joins mesh peer 5 years ago
Brad Fitzpatrick 1cb7dab881 cmd/derper: support forwarding packets amongst set of peer DERP servers
Updates #388

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 5e0ff494a5 derp: change NewClient constructor to an option pattern
(The NewMeshClient constructor I added recently was gross in
retrospect at call sites, especially when it wasn't obvious that a
meshKey empty string meant a regular client)
5 years ago
Brad Fitzpatrick 4d599d194f derp, derp/derphttp: add key accessors, add Client.RecvDetail
Client.RecvDetail returns a connection generation so interested clients
can detect when a reconnect happened. (Will be needed for #388)
5 years ago
Brad Fitzpatrick b33c86b542 derp: add an unexported key.Public zero value variable to be less verbose 5 years ago
Brad Fitzpatrick b663ab4685 cmd/derper: treat self-connection connection watch as no-op
Updates #388
5 years ago
Brad Fitzpatrick 484b7fc9a3 derp, cmd/derper: add frameWatchConns, framePeerPresent for inter-DERP routing
This lets a trusted DERP client that knows a pre-shared key subscribe
to the connection list. Upon subscribing, they get the current set
of connected public keys, and then all changes over time.

This lets a set of DERP server peers within a region all stay connected to
each other and know which clients are connected to which nodes.

Updates #388

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 24009241bf net/netns: move SOCKS dialing to netns for now
This lets control & logs also use SOCKS dials.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick cf0d19f0ab net/tlsdial, derp/derphttp: finish DERPNode.CertName validation 5 years ago
Brad Fitzpatrick becce82246 net/netns, misc tests: remove TestOnlySkipPrivilegedOps, argv checks
The netns UID check is sufficient for now. We can do something else
later if/when needed.
5 years ago
Brad Fitzpatrick 7f68e097dd net/netcheck: fix HTTPS fallback bug from earlier today
My earlier 3fa58303d0 tried to implement
the net/http.Tranhsport.DialTLSContext hook, but I didn't return a
*tls.Conn, so we ended up sending a plaintext HTTP request to an HTTPS
port. The response ended up being Go telling as such, not the
/derp/latency-check handler's response (which is currently still a
404). But we didn't even get the 404.

This happened to work well enough because Go's built-in error response
was still a valid HTTP response that we can measure for timing
purposes, but it's not a great answer. Notably, it means we wouldn't
be able to get a future handler to run server-side and count those
latency requests.
5 years ago
David Anderson 5114df415e net/netns: set the bypass socket mark on linux.
This allows tailscaled's own traffic to bypass Tailscale-managed routes,
so that things like tailscale-provided default routes don't break
tailscaled itself.

Progress on #144.

Signed-off-by: David Anderson <danderson@tailscale.com>
5 years ago
Brad Fitzpatrick 3fa58303d0 netcheck: address some HTTP fallback measurement TODOs 5 years ago
David Anderson e9f7d01b91 derp/derphttp: make DERP client use netns for dial-outs. 5 years ago
Brad Fitzpatrick e6b84f2159 all: make client use server-provided DERP map, add DERP region support
Instead of hard-coding the DERP map (except for cmd/tailscale netcheck
for now), get it from the control server at runtime.

And make the DERP map support multiple nodes per region with clients
picking the first one that's available. (The server will balance the
order presented to clients for load balancing)

This deletes the stunner package, merging it into the netcheck package
instead, to minimize all the config hooks that would've been
required.

Also fix some test flakes & races.

Fixes #387 (Don't hard-code the DERP map)
Updates #388 (Add DERP region support)
Fixes #399 (wgengine: flaky tests)

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 040a0d5121 derp/derphttp: don't use x/net/proxy for SOCKS on iOS
We don't want those extra dependencies on iOS, at least yet.

Especially since there's no way to set the relevant environment
variables so it's just bloat with no benefits. Perhaps we'll need to
do SOCKS on iOS later, but probably differently if/when so.

Updates #227

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick e42ec4efba derp/derphttp: use SOCKS/etc proxies for derphttp dials
Updates #227

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 495796fff1 derp/derpmap: add World.ForeachServer, check STUN server validity earlier 5 years ago
Brad Fitzpatrick fefd7e10dc types/structs: add structs.Incomparable annotation, use it where applicable
Shotizam before and output queries:

sqlite> select sum(size) from bin where func like 'type..%';
129067
=>
120216
5 years ago
Brad Fitzpatrick b6fa5a69be net/tlsdial: add package for TLS dials, and make DERP & controlclient use it
This will do the iOS-optimized cert checking in a following change.
5 years ago
Brad Fitzpatrick 614eec174f derp/derphttp: avoid endless reconnect race on failure
Originally from @stablebits (Dmitry Adamushko) in:
https://github.com/tailscale/tailscale/pull/264
5 years ago
Dmitry Adamushko 806645ea0e derp: prevent readFrame() from reading more than len(b) bytes.
Signed-off-by: Dmitry Adamushko <da@stablebits.net>
5 years ago
Dmitry Adamushko f2c2d0de68 derp/derp_server: unregisterClient() for replaced client connections.
When unregistering a replaced client connection, move the
still-connected peers to the current client connecition. Inform
the peers that we've gone only when unregistering the active
client connection.

Signed-off-by: Dmitry Adamushko <da@stablebits.net>
5 years ago
Dmitry Adamushko 44434fdc82 derp/derp_server: fixed unbalanced {register,unregister}Client() calls.
Signed-off-by: Dmitry Adamushko <da@stablebits.net>
5 years ago
David Anderson 4e0ee141e8 derp/derpmap: add derp5 (Sydney) to prod map.
Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
Brad Fitzpatrick 1df3c8d02a derp, netcheck: make tests listen on localhost only
avoid macOS firewall pop-ups
5 years ago
Brad Fitzpatrick e749377a56 derp/derphttp: remove a redundant log message 5 years ago
Brad Fitzpatrick 1d3f45f9be derp/derphttp: ignore PeerGoneMessage in test 5 years ago
Brad Fitzpatrick 1ab5b31c4b derp, magicsock: send new "peer gone" frames when previous sender disconnects
Updates #150 (not yet enabled by default in magicsock)

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 8c4cef60f8 derp: wait for send goroutine to finish before returning from run
I saw a test flake due to the sender goroutine logging (ultimately to
t.Logf) after the server was closed.

This makes sure the all goroutines are cleaned up before Server.Close
returns.
5 years ago
Brad Fitzpatrick 521ad7b0fc derp: only flush writes to clients when we're out of things to write
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 0a25abcdaa derp: remove TODO comment
It was done in bf704a5218
5 years ago
Brad Fitzpatrick 8519a1b29f derp: revert rename of pkt to sendMsg
New plan, sorry.

Most reverts 77921a31b1, except some doc changes.
5 years ago
Brad Fitzpatrick c34b350efa derp: remove the mutex around and closing of send channel
Makes it less complicated.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 77921a31b1 derp: rename pkt to sendMsg, in prep for other types of writes
Updates #150
5 years ago
Brad Fitzpatrick 1453aecb44 derp: add sclient.done channel, simplify some context passing
This is mostly prep for a few future CLs, making sure we always have a
close-on-dead done channel available to select on when doing other
channel operations.
5 years ago
David Anderson ea90780066 derp: specify type of the by-reason drop varz.
Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
Brad Fitzpatrick fd824df1fa derp: export metric for server's initial MemStats.Sys reading
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Anderson 64b5248929 derp: sprinkle comments and docstrings.
Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson 58b8ebd290 derp: add per-reason packet drop counter.
In parallel with the summed counter for now, for dashboard migration.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson 1e031001db derp: move write timeout to package constant.
Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson dbfc916273 derp: add short queues between reader and writer, drop on overload.
This avoids the server blocking on misbehaving or heavily contended
clients. We attempt to drop from the head of the queue to keep
overall queueing time lower.

Also:
 - fixes server->client keepalives, which weren't happening.
 - removes read rate-limiter, deferring instead to kernel-level
   global limiter/fair queuer.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Crawshaw 131541c06d derp: deflake test
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
David Crawshaw 72dbf26f63 derp: test that client a->b and a->c relaying do not interfere
Without the recent write deadline introduction, this test fails.

They still do interfere, but the interference is now bound by
the write deadline. Many improvements are possible.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
David Crawshaw e838b3fb59 derp: use a write timeout when sending to clients
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
David Crawshaw 3df1b97ea8 derp: do not treat failure to relay as the fault of the sender
If Alice attempts to send a packet to Bob and the DERP server
encounters an error on the socket to Bob, we should not disconnect
Alice for that.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
David Crawshaw 43aa8595dd derp: introduce Conn interface
This lets us test with something other than a net.Conn.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
Brad Fitzpatrick 4fd29349b9 derp: add clients_replaced counter
Updates #109
5 years ago
Brad Fitzpatrick 39c0ae1dba derp/derpmap: new DERP config package, merge netcheck into magicsock more
Fixes #153
Updates #162
Updates #163

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 5d8001d0ad derp: add varz for home moves
Updates #162
Updates #163 (maybe)

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick e733fa6f7e derp: add some varz tests for active/home conns 5 years ago
Brad Fitzpatrick dd456f04c5 derp: actually record client's preferred bit 5 years ago
Brad Fitzpatrick 6978b93bdd derp, magicsock: track home (preferred) vs visiting connections for stats 5 years ago
Brad Fitzpatrick cf4aacde57 derp/derphttp: appease staticcheck 5 years ago
Brad Fitzpatrick 2cff9016e4 net/dnscache: add overly simplistic DNS cache package for selective use
I started to write a full DNS caching resolver and I realized it was
overkill and wouldn't work on Windows even in Go 1.14 yet, so I'm
doing this tiny one instead for now, just for all our netcheck STUN
derp lookups, and connections to DERP servers. (This will be caching a
exactly 8 DNS entries, all ours.)

Fixes #145 (can be better later, of course)
5 years ago
Brad Fitzpatrick bf704a5218 derp: protocol negotiation, add v2: send src pub keys to clients in packets
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Crawshaw 9be9738f62 derphttp: add TLSConfig field
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
Brad Fitzpatrick e371520cc5 tsweb, derp: add expvar http.Handler for Prometheus's format
And add some opinions.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 517a91de75 derp/derphttp: fix reconnect behavior
I broke it in today's 752146a70f.

Added some comments to prevent recurrences.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick d59d0576af derp: don't log on all dropped packets; super spammy
Signed-off-by: Brad Fitzpatrick <brad@danga.com>
5 years ago
Brad Fitzpatrick 38acccad09 derp: remove some debug noise I left in
:(
5 years ago
Brad Fitzpatrick 752146a70f derp: clean up derphttp client code, use contexts
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 3317531021 derp: fix JSON field typo, sort fields to make it easier to see inconsistencies 5 years ago
Brad Fitzpatrick 6cd81d5d1f derp: add more derp stats for dropped and received packets
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 2612e54ad1 derp, cmd/derper: add debug handlers, stats
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 379a3125fd derp, wgengine/magicsock: support more than just packets from Client.Recv
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 88f1cc0c98 derp, cmd/derper: add rate limiting support, add default 5Mbps limit
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 1166c34f6c derp: fix staticcheck warning
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 322cb58b14 derp: deflake test I flaked up in earlier change
I broke an invariant in 11048b8932 (it was even nicely
documented then).

Also clean up the test a bit from while I was debugging it.

Fixes #84

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick f029c4c82d derp: change the protocol framing to always include a length
Addresses one of crawshaw's TODOs.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 11048b8932 derp: add docs on current protocol overview, break accept apart a bit
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 79af6054bf derp: don't start sending keepalives to client until after serverInfo sent
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 1155716318 derp: rename some things in the client, add some docs
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 4b461c2e77 derp: rename some things in the server, add some docs 5 years ago
Brad Fitzpatrick 619697063e derp: more misc cleanups
And add an explicit derphttp.Client.Connect in the test now that it's
done lazily.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick e06ca40650 wgengine, magicsock, derp: misc cleanups, docs
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 259406e797 derp: move away from [32]byte key types
And some minor cleanup in the process.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 769e25e37b derp: use new types/key package
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 2896be60db Move "logger" package to under types, now that we have it.
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick a59dc5f155 Use logger.Logf consistently. It was used in most places.
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Earl Lee a8d8b8719a Move Linux client & common packages into a public repo. 5 years ago