138 Commits (f205efcf1856c2338989c5e76e2d23fde24fd532)

Author	SHA1	Message	Date
Brad Fitzpatrick	fc160f80ee	metrics: move currentFDs code to the metrics package ... Updates #2784 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	73280595a8	derp: accept dup clients without closing prior's connection ... A public key should only have max one connection to a given DERP node (or really: one connection to a node in a region). But if people clone their machine keys (e.g. clone their VM, Raspbery Pi SD card, etc), then we can get into a situation where a public key is connected multiple times. Originally, the DERP server handled this by just kicking out a prior connections whenever a new one came. But this led to reconnect fights where 2+ nodes were in hard loops trying to reconnect and kicking out their peer. Then a909d37a59 tried to add rate limiting to how often that dup-kicking can happen, but empirically it just doesn't work and ~leaks a bunch of goroutines and TCP connections, tying them up for hour+ while more and more accumulate and waste memory. Mostly because we were doing a time.Sleep forever while not reading from their TCP connections. Instead, just accept multiple connections per public key but track which is the most recent. And if two both are writing back & forth, then optionally disable them both. That last part is only enabled in tests for now. The current default policy is just last-sender-wins while we gather the next round of stats. Updates #2751 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	ffd22050c0	derp: export current file descriptor metric	3 years ago
slowy07	ac0353e982	fix: typo spelling grammar ... Signed-off-by: slowy07 <slowy.arfy@gmail.com>	3 years ago
Brad Fitzpatrick	f35b8c3ead	derp: fix meshing accounting edge case bug ... If a peer is connected to multiple nodes in a region (so multiForwarder is in use) and then a node restarts and re-sends all its additions, this bug about whether an element is in the multiForwarder could cause a one-time flip in the which peer node we forward to. Note a huge deal, but not written as intended. Thanks to @lewgun for the bug report in #2141. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	fd7b738e5b	derp: use pad32 package for padding, reduce duplication ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	7298e777d4	derp: reduce server memory by 30% by removing persistent bufio.Writer ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	b622c60ed0	derp,wgengine/magicsock: don't assume stringer is in $PATH for go:generate ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	a909d37a59	derp: rate limit how often same-key clients can kick each other off server ... Updates #392 Updates #506 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	4dbbd0aa4a	cmd/addlicense: add command to add licenseheaders to generated code ... And use it to make our stringer invocations match the existing code. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
maddie	d976a84d7e	derp: allow self node when verifying clients ... Fixes #2408 Signed-off-by: Maddie Zhan <maddie.zhan@cynovan.com>	3 years ago
Brad Fitzpatrick	4c0494185b	derp: remove "fine for now" intentional slow memory leak from derp server ... It was once believed that it might be useful. It wasn't. We never used it. Remove it so we don't slowly leak memory. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Anderson	d98829583a	derp: use a dedicated queue for disco traffic. ... Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	67158549ab	derp: actually export the new drop counter. ... Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	36492ace9d	derp: add counters to track the type of dropped packets. ... Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
julianknodt	148602a89a	derp,cmd/derper: allow server to verify clients ... This adds a flag to the DERP server which specifies to verify clients through a local tailscaled. It is opt-in, so should not affect existing clients, and is mainly intended for users who want to run their own DERP servers. It assumes there is a local tailscaled running and will attempt to hit it for peer status information. Updates #1264 Signed-off-by: julianknodt <julianknodt@gmail.com>	3 years ago
julianknodt	3687e5352b	derp: fix traffic handler peer addresses ... Before it was using the local address and port, so fix that. The fields in the response from `ss` are: State, Recv-Q, Send-Q, Local Address:Port, Peer Address:Port, Process Signed-off-by: julianknodt <julianknodt@gmail.com>	3 years ago
julianknodt	3728634af9	derp: add debug traffic handler ... This adds a handler on the DERP server for logging bytes send and received by clients of the server, by holding open a connection and recording if there is a difference between the number of bytes sent and received. It sends a JSON marshalled object if there is an increase in the number of bytes. Signed-off-by: julianknodt <julianknodt@gmail.com>	3 years ago
julianknodt	fe54721e31	derp: add pkt queue latency timer ... It would be useful to know the time that packets spend inside of a queue before they are sent off, as that can be indicative of the load the server is handling (and there was also an existing TODO). This adds a simple exponential moving average metric to track the average packet queue duration. Changes during review: Add CAS loop for recording queue timing w/ expvar.Func, rm snake_case, annotate in milliseconds, convert Signed-off-by: julianknodt <julianknodt@gmail.com>	3 years ago
David Anderson	54e6c3a290	version: use OSS repo's version when building. ... When building with redo, also include the git commit hash from the proprietary repo, so that we have a precise commit that identifies all build info (including Go toolchain version). Add a top-level build script demonstrating to downstream distros how to burn the right information into builds. Adjust `tailscale version` to print commit hashes when available. Fixes #841. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
Brad Fitzpatrick	169ff22a84	derp: set NotBefore and NotAfter in DERP server's metacert ... Fixes regression from e415991256 that only affected Windows users because Go only on Windows delegates x509 cert validation to the OS and Windows as unhappy with our "metacert" lacking NotBefore and NotAfter. Fixes #705	4 years ago
Brad Fitzpatrick	e415991256	derp, derp/derphttp: remove one RTT from DERP setup ... * advertise server's DERP public key following its ServerHello * have client look for that DEPR public key in the response PeerCertificates * let client advertise it's going into a "fast start" mode if it finds it * modify server to support that fast start mode, just not sending the HTTP response header Cuts down another round trip, bringing the latency of being able to write our first DERP frame from SF to Bangalore from ~725ms (3 RTT) to ~481ms (2 RTT: TCP and TLS). Fixes #693 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	805850add9	derp: remove JSON struct tags in comments ... They don't work in comments. Added a test too to show that there's no change in behavior. (It does case insensitive matching on parse anyway)	4 years ago
Brad Fitzpatrick	93ffc565e5	derp: remove protocol version 1 support ... It hasn't existed for a long time and there are no current users. Fixes #199	4 years ago
Josh Bleecher Snyder	062bd67d3b	derp: use rand instead of crypto/rand to generate jitter ... We don't need crypto/rand. Let the OS keep its entropy bits. Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>	4 years ago
David Anderson	37c19970b3	derp: add a debug option to verbosely log drops to a destination. ... Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
Josh Bleecher Snyder	909c165382	derp: remove two key.Public allocations ... Reading and writing a [32]byte key to a bufio.Reader/bufio.Writer can easily by done without allocating. Do so. It is slower; on my machine, it adds about 100ns per read/write. However, the overall request takes a minimum of several µs, and it cuts allocations meaningfully, so it is probably worth it. name old time/op new time/op delta SendRecv/msgsize=10-8 9.21µs ± 9% 9.08µs ± 8% ~ (p=0.250 n=15+15) SendRecv/msgsize=100-8 6.51µs ± 9% 6.60µs ± 7% ~ (p=0.259 n=15+13) SendRecv/msgsize=1000-8 7.24µs ±13% 7.61µs ±36% ~ (p=1.000 n=11+15) SendRecv/msgsize=10000-8 19.5µs ±15% 19.9µs ±25% ~ (p=0.890 n=14+15) name old speed new speed delta SendRecv/msgsize=10-8 1.09MB/s ± 8% 1.10MB/s ± 8% ~ (p=0.286 n=15+15) SendRecv/msgsize=100-8 15.4MB/s ± 8% 15.1MB/s ± 6% ~ (p=0.129 n=15+12) SendRecv/msgsize=1000-8 139MB/s ±15% 135MB/s ±28% ~ (p=1.000 n=11+15) SendRecv/msgsize=10000-8 516MB/s ±17% 506MB/s ±21% ~ (p=0.880 n=14+15) name old alloc/op new alloc/op delta SendRecv/msgsize=10-8 170B ± 1% 108B ± 1% -36.63% (p=0.000 n=15+15) SendRecv/msgsize=100-8 265B ± 1% 203B ± 1% -23.34% (p=0.000 n=15+15) SendRecv/msgsize=1000-8 1.18kB ± 1% 1.12kB ± 0% -5.31% (p=0.000 n=14+14) SendRecv/msgsize=10000-8 18.8kB ± 2% 18.8kB ± 2% ~ (p=0.443 n=12+12) name old allocs/op new allocs/op delta SendRecv/msgsize=10-8 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=15+15) SendRecv/msgsize=100-8 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=15+15) SendRecv/msgsize=1000-8 4.00 ± 0% 2.00 ± 0% -50.00% (p=0.000 n=15+15) SendRecv/msgsize=10000-8 5.00 ± 0% 3.00 ± 0% -40.00% (p=0.000 n=13+14) Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>	4 years ago
Brad Fitzpatrick	75225368a4	derp: fix 32-bit struct field alignment	4 years ago
David Anderson	15949ad77d	derp: export the new expvar.	4 years ago
David Anderson	13661e195a	derp: rename "wireguard" packet type to "other". ... Strictly speaking, we don't know that it's a wireguard packet, just that it doesn't look like a disco packet. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	1b5b59231b	derp: break down received packets by kind (disco vs. wireguard). ... Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
Josh Bleecher Snyder	b23f2263c1	derp: add server version to /debug, expvars ... This will make it easier for a human to tell what version is deployed, for (say) correlating line numbers in profiles or panics to corresponding source code. It'll also let us observe version changes in prometheus. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	4 years ago
David Anderson	c3994fd77c	derp: remove OnlyDisco option. ... Active discovery lets us introspect the state of the network stack precisely enough that it's unnecessary, and dropping the initial DERP packets greatly slows down tests. Additionally, it's unrealistic since our production network will never deliver _only_ discovery packets, it'll be all or nothing. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
Brad Fitzpatrick	52969bdfb0	derp: fix atomic padding on 32-bit again ... Broken by earlier OnlyDisco addition.	4 years ago
Brad Fitzpatrick	a6559a8924	wgengine/magicsock: run test DERP in mode where only disco packets allowed ... So we don't accidentally pass a NAT traversal test by having DERP pick up our slack when we really just wanted DERP as an OOB messaging channel.	4 years ago
Brad Fitzpatrick	c1cabe75dc	derp: fix server struct fielfd alignment on 32-bit ... Mostly so the GitHub CI will pass on 32-bit.	4 years ago
Brad Fitzpatrick	4732722b87	derp: add frameClosePeer to move around clients within a region ... For various reasons (mostly during rollouts or config changes on our side), nodes may end up connecting to a fallback DERP node in a region, rather than the primary one we tell them about in the DERP map. Connecting to the "wrong" node is fine, but it's in our best interest for all nodes in a domain to connect to the same node, to reduce intra-region packet forwarding. This adds a privileged frame type used by the control system that can kick off a client connection when they're connected to the wrong node in a region. Then they hopefully reconnect immediately to the correct location. (If not, we can leave them alone and stop closing them.) Updates tailscale/corp#372	4 years ago
Brad Fitzpatrick	dd43d9bc5f	derp: fix varz typo ... Updates tailscale/corp#391	4 years ago
Brad Fitzpatrick	b87396b5d9	cmd/derper, derp: add some more varz and consistency check handler ... I'm trying to hunt down a slow drift in numbers not agreeing.	4 years ago
Brad Fitzpatrick	6fbd1abcd3	derp: update peerGone code to work with regional DERP mesh clusters too ... Updates #150 Updates #388	4 years ago
Brad Fitzpatrick	07ca0c1c29	derp: fix tracking problem if conn starts local, then also joins mesh peer	5 years ago
Brad Fitzpatrick	1cb7dab881	cmd/derper: support forwarding packets amongst set of peer DERP servers ... Updates #388 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	4d599d194f	derp, derp/derphttp: add key accessors, add Client.RecvDetail ... Client.RecvDetail returns a connection generation so interested clients can detect when a reconnect happened. (Will be needed for #388)	5 years ago
Brad Fitzpatrick	b33c86b542	derp: add an unexported key.Public zero value variable to be less verbose	5 years ago
Brad Fitzpatrick	b663ab4685	cmd/derper: treat self-connection connection watch as no-op ... Updates #388	5 years ago
Brad Fitzpatrick	484b7fc9a3	derp, cmd/derper: add frameWatchConns, framePeerPresent for inter-DERP routing ... This lets a trusted DERP client that knows a pre-shared key subscribe to the connection list. Upon subscribing, they get the current set of connected public keys, and then all changes over time. This lets a set of DERP server peers within a region all stay connected to each other and know which clients are connected to which nodes. Updates #388 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Dmitry Adamushko	f2c2d0de68	derp/derp_server: unregisterClient() for replaced client connections. ... When unregistering a replaced client connection, move the still-connected peers to the current client connecition. Inform the peers that we've gone only when unregistering the active client connection. Signed-off-by: Dmitry Adamushko <da@stablebits.net>	5 years ago
Dmitry Adamushko	44434fdc82	derp/derp_server: fixed unbalanced {register,unregister}Client() calls. ... Signed-off-by: Dmitry Adamushko <da@stablebits.net>	5 years ago
Brad Fitzpatrick	1ab5b31c4b	derp, magicsock: send new "peer gone" frames when previous sender disconnects ... Updates #150 (not yet enabled by default in magicsock) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	8c4cef60f8	derp: wait for send goroutine to finish before returning from run ... I saw a test flake due to the sender goroutine logging (ultimately to t.Logf) after the server was closed. This makes sure the all goroutines are cleaned up before Server.Close returns.	5 years ago
Brad Fitzpatrick	521ad7b0fc	derp: only flush writes to clients when we're out of things to write ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	0a25abcdaa	derp: remove TODO comment ... It was done in bf704a5218	5 years ago
Brad Fitzpatrick	8519a1b29f	derp: revert rename of pkt to sendMsg ... New plan, sorry. Most reverts 77921a31b1, except some doc changes.	5 years ago
Brad Fitzpatrick	c34b350efa	derp: remove the mutex around and closing of send channel ... Makes it less complicated. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	77921a31b1	derp: rename pkt to sendMsg, in prep for other types of writes ... Updates #150	5 years ago
Brad Fitzpatrick	1453aecb44	derp: add sclient.done channel, simplify some context passing ... This is mostly prep for a few future CLs, making sure we always have a close-on-dead done channel available to select on when doing other channel operations.	5 years ago
David Anderson	ea90780066	derp: specify type of the by-reason drop varz. ... Signed-off-by: David Anderson <dave@natulte.net>	5 years ago
Brad Fitzpatrick	fd824df1fa	derp: export metric for server's initial MemStats.Sys reading ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
David Anderson	64b5248929	derp: sprinkle comments and docstrings. ... Signed-off-by: David Anderson <dave@natulte.net>	5 years ago
David Anderson	58b8ebd290	derp: add per-reason packet drop counter. ... In parallel with the summed counter for now, for dashboard migration. Signed-off-by: David Anderson <dave@natulte.net>	5 years ago
David Anderson	1e031001db	derp: move write timeout to package constant. ... Signed-off-by: David Anderson <dave@natulte.net>	5 years ago
David Anderson	dbfc916273	derp: add short queues between reader and writer, drop on overload. ... This avoids the server blocking on misbehaving or heavily contended clients. We attempt to drop from the head of the queue to keep overall queueing time lower. Also: - fixes server->client keepalives, which weren't happening. - removes read rate-limiter, deferring instead to kernel-level global limiter/fair queuer. Signed-off-by: David Anderson <dave@natulte.net>	5 years ago
David Crawshaw	e838b3fb59	derp: use a write timeout when sending to clients ... Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	5 years ago
David Crawshaw	3df1b97ea8	derp: do not treat failure to relay as the fault of the sender ... If Alice attempts to send a packet to Bob and the DERP server encounters an error on the socket to Bob, we should not disconnect Alice for that. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	5 years ago
David Crawshaw	43aa8595dd	derp: introduce Conn interface ... This lets us test with something other than a net.Conn. Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	5 years ago
Brad Fitzpatrick	4fd29349b9	derp: add clients_replaced counter ... Updates #109	5 years ago
Brad Fitzpatrick	5d8001d0ad	derp: add varz for home moves ... Updates #162 Updates #163 (maybe) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	dd456f04c5	derp: actually record client's preferred bit	5 years ago
Brad Fitzpatrick	6978b93bdd	derp, magicsock: track home (preferred) vs visiting connections for stats	5 years ago
Brad Fitzpatrick	bf704a5218	derp: protocol negotiation, add v2: send src pub keys to clients in packets ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	e371520cc5	tsweb, derp: add expvar http.Handler for Prometheus's format ... And add some opinions. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	d59d0576af	derp: don't log on all dropped packets; super spammy ... Signed-off-by: Brad Fitzpatrick <brad@danga.com>	5 years ago
Brad Fitzpatrick	752146a70f	derp: clean up derphttp client code, use contexts ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	3317531021	derp: fix JSON field typo, sort fields to make it easier to see inconsistencies	5 years ago
Brad Fitzpatrick	6cd81d5d1f	derp: add more derp stats for dropped and received packets ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	2612e54ad1	derp, cmd/derper: add debug handlers, stats ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	379a3125fd	derp, wgengine/magicsock: support more than just packets from Client.Recv ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	88f1cc0c98	derp, cmd/derper: add rate limiting support, add default 5Mbps limit ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	322cb58b14	derp: deflake test I flaked up in earlier change ... I broke an invariant in 11048b8932 (it was even nicely documented then). Also clean up the test a bit from while I was debugging it. Fixes #84 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	f029c4c82d	derp: change the protocol framing to always include a length ... Addresses one of crawshaw's TODOs. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	11048b8932	derp: add docs on current protocol overview, break accept apart a bit ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	79af6054bf	derp: don't start sending keepalives to client until after serverInfo sent ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	4b461c2e77	derp: rename some things in the server, add some docs	5 years ago
Brad Fitzpatrick	619697063e	derp: more misc cleanups ... And add an explicit derphttp.Client.Connect in the test now that it's done lazily. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	259406e797	derp: move away from [32]byte key types ... And some minor cleanup in the process. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	2896be60db	Move "logger" package to under types, now that we have it. ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	a59dc5f155	Use logger.Logf consistently. It was used in most places. ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Earl Lee	a8d8b8719a	Move Linux client & common packages into a public repo.	5 years ago