Commit Graph

48 Commits (9c07f4f512f71799a796cb3cdb5532657df8444f)

Author SHA1 Message Date
Brad Fitzpatrick 6c791f7d60 derp: include src IPs in mesh watch messages
Updates tailscale/corp#13945

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Claire Wang 90a7d3066c
derp: use tstime (#8634)
Updates #8587

Signed-off-by: Claire Wang <claire@tailscale.com>
1 year ago
valscale 74eb99aed1
derp, derphttp, magicsock: send new unknown peer frame when destination is unknown (#7552)
* wgengine/magicsock: add envknob to send CallMeMaybe to non-existent peer

For testing older client version responses to the PeerGone packet format change.

Updates #4326

Signed-off-by: Val <valerie@tailscale.com>

* derp: remove dead sclient struct member replaceLimiter

Leftover from an previous solution to the duplicate client problem.

Updates #2751

Signed-off-by: Val <valerie@tailscale.com>

* derp, derp/derphttp, wgengine/magicsock: add new PeerGone message type Not Here

Extend the PeerGone message type by adding a reason byte. Send a
PeerGone "Not Here" message when an endpoint sends a disco message to
a peer that this server has no record of.

Fixes #4326

Signed-off-by: Val <valerie@tailscale.com>

---------

Signed-off-by: Val <valerie@tailscale.com>
2 years ago
Anton Tolchanov 654b5a0616 derp: add optional debug logging for prober clients
This allows tracking packet flow via logs for prober clients. Note that
the new sclient.debug() function is called on every received packet, but
will do nothing for most clients.

I have adjusted sclient logging to print public keys in short format
rather than full. This takes effect even for existing non-debug logging
(mostly client disconnect messages).

Example logs for a packet being sent from client [SbsJn] (connected to
derper [dM2E3]) to client [10WOo] (connected to derper [AVxvv]):

```
derper [dM2E3]:
derp client 10.0.0.1:35470[SbsJn]: register single client mesh("10.0.1.1"): 4 peers
derp client 10.0.0.1:35470[SbsJn]: read frame type 4 len 40 err <nil>
derp client 10.0.0.1:35470[SbsJn]: SendPacket for [10WOo], forwarding via <derphttp_client.Client [AVxvv] url=https://10.0.1.1/derp>: <nil>
derp client 10.0.0.1:35470[SbsJn]: read frame type 0 len 0 err EOF
derp client 10.0.0.1:35470[SbsJn]: read EOF
derp client 10.0.0.1:35470[SbsJn]: sender failed: context canceled
derp client 10.0.0.1:35470[SbsJn]: removing connection

derper [AVxvv]:
derp client 10.0.1.1:50650[10WOo]: register single client
derp client 10.0.1.1:50650[10WOo]: received forwarded packet from [SbsJn] via [dM2E3]
derp client 10.0.1.1:50650[10WOo]: sendPkt attempt 0 enqueued
derp client 10.0.1.1:50650[10WOo]: sendPacket from [SbsJn]: <nil>
derp client 10.0.1.1:50650[10WOo]: read frame type 0 len 0 err EOF
derp client 10.0.1.1:50650[10WOo]: read EOF
derp client 10.0.1.1:50650[10WOo]: sender failed: context canceled
derp client 10.0.1.1:50650[10WOo]: removing connection
```

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Brad Fitzpatrick cf8dd7aa09 all: use Go 1.20's bytes.Clone
Updates #7123
Updates #6257 (more to do in other repos)

Change-Id: I073e2a6d81a5d7fbecc29caddb7e057ff65239d0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Maisem Ali 5bba65e978 net/memnet: rename from net/nettest
This is just #cleanup to resolve a TODO

Also add a package doc.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Will Norris 71029cea2d all: update copyright and license headers
This updates all source files to use a new standard header for copyright
and license declaration.  Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.

This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.

Updates #6865

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Anton Tolchanov 6cc6c70d70 derp: prevent concurrent access to multiForwarder map
Instead of iterating over the map to determine the preferred forwarder
on every packet (which could happen concurrently with map mutations),
store it separately in an atomic variable.

Fixes #6445

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Josh Soref d4811f11a0 all: fix spelling mistakes
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2 years ago
Eng Zer Jun f0347e841f refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Reference: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2 years ago
Brad Fitzpatrick 4950fe60bd syncs, all: move to using Go's new atomic types instead of ours
Fixes #5185

Change-Id: I850dd532559af78c3895e2924f8237ccc328449d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Charlotte Brandhorst-Satzkorn 4c0feba38e
derp: plumb '/derp' request context through (#5083)
This change is required to implement tracing for derp.

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Brad Fitzpatrick 18818763d1 derp: set Basic Constraints on metacert
See https://github.com/golang/go/issues/51759#issuecomment-1071147836

Once we deploy this, tailscaled should work again for macOS users with
Go 1.18.

Updates golang/go#51759

Change-Id: I869b6ddc556a2de885e96ccf9f335dfc8f6f6a7e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 0868329936 all: use any instead of interface{}
My favorite part of generics.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 434af15a04 derp: support client->server ping (and server->client pong)
In prep for a future change to have client ping derp connections
when their state is questionable, rather than aggressively tearing
them down and doing a heavy reconnect when their state is unknown.

We already support ping/pong in the other direction (servers probing
clients) so we already had the two frame types, but I'd never finished
this direction.

Updates #3619

Change-Id: I024b815d9db1bc57c20f82f80f95fb55fc9e2fcc
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Anderson 1f06f77dcb derp: remove package shadowing of types/key.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 37c150aee1 derp: use new node key type.
Update #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 73f177e4d5 derp: throttle client sends if server advertises rate limits
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 3759fb8987 derp: deflake TestSendFreeze
On about 1 out of 500 runs, TestSendFreeze failed:

    derp_test.go:416: bob: unexpected message type derp.PeerGoneMessage

Closing alice before bob created a race.
If bob closed promptly, the test passed.
If bob closed slowly, and alice's disappearance caused
bob to receive a PeerGoneMessage before closing, the test failed.

Deflake the test by closing bob first.
With this fix, the test passed 12,000 times locally.

Fixes #2668

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick a59b389a6a derp: add new health update and server restarting frame types
Updates #2756
Updates #2746

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 73280595a8 derp: accept dup clients without closing prior's connection
A public key should only have max one connection to a given
DERP node (or really: one connection to a node in a region).

But if people clone their machine keys (e.g. clone their VM, Raspbery
Pi SD card, etc), then we can get into a situation where a public key
is connected multiple times.

Originally, the DERP server handled this by just kicking out a prior
connections whenever a new one came. But this led to reconnect fights
where 2+ nodes were in hard loops trying to reconnect and kicking out
their peer.

Then a909d37a59 tried to add rate
limiting to how often that dup-kicking can happen, but empirically it
just doesn't work and ~leaks a bunch of goroutines and TCP
connections, tying them up for hour+ while more and more accumulate
and waste memory. Mostly because we were doing a time.Sleep forever
while not reading from their TCP connections.

Instead, just accept multiple connections per public key but track
which is the most recent. And if two both are writing back & forth,
then optionally disable them both. That last part is only enabled in
tests for now. The current default policy is just last-sender-wins
while we gather the next round of stats.

Updates #2751

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick f35b8c3ead derp: fix meshing accounting edge case bug
If a peer is connected to multiple nodes in a region (so
multiForwarder is in use) and then a node restarts and re-sends all
its additions, this bug about whether an element is in the
multiForwarder could cause a one-time flip in the which peer node we
forward to.  Note a huge deal, but not written as intended.

Thanks to @lewgun for the bug report in #2141.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick a909d37a59 derp: rate limit how often same-key clients can kick each other off server
Updates #392
Updates #506

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
julianknodt 3728634af9 derp: add debug traffic handler
This adds a handler on the DERP server for logging bytes send and received by clients of the
server, by holding open a connection and recording if there is a difference between the number
of bytes sent and received. It sends a JSON marshalled object if there is an increase in the
number of bytes.

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
Maisem Ali 4d142ebe06 derp: handle net.ErrClosed in TestSendFreeze
Signed-off-by: Maisem Ali <maisem@tailscale.com>
4 years ago
Brad Fitzpatrick 79d8288f0a wgengine/magicsock, derp, derp/derphttp: respond to DERP server->client pings
No server support yet, but we want Tailscale 1.6 clients to be able to respond
to them when the server can do it.

Updates #1310

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick e415991256 derp, derp/derphttp: remove one RTT from DERP setup
* advertise server's DERP public key following its ServerHello
* have client look for that DEPR public key in the response
  PeerCertificates
* let client advertise it's going into a "fast start" mode
  if it finds it
* modify server to support that fast start mode, just not
  sending the HTTP response header

Cuts down another round trip, bringing the latency of being able to
write our first DERP frame from SF to Bangalore from ~725ms
(3 RTT) to ~481ms (2 RTT: TCP and TLS).

Fixes #693

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 805850add9 derp: remove JSON struct tags in comments
They don't work in comments.

Added a test too to show that there's no change in behavior.
(It does case insensitive matching on parse anyway)
4 years ago
Brad Fitzpatrick 6b80bcf112 derp: remove a client round-trip waiting on serverInfo
It just has a version number in it and it's not really needed.
Instead just return it as a normal Recv message type for those
that care (currently only tests).

Updates #150 (in that it shares the same goal: initial DERP latency)
Updates #199 (in that it removes some DERP versioning)
4 years ago
Josh Bleecher Snyder 355c6296f0 derp: add readUint32 and writeUint32 benchmarks
These aren't particularly performance critical,
but since I have an optimization pending for them,
it's worth having a corresponding benchmark.

Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>
4 years ago
Josh Bleecher Snyder d65e2632ab derp: add basic benchmark
This benchmark is far from perfect: It mixes together
client and server. Still, it provides a starting point
for easy profiling.

Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>
4 years ago
Brad Fitzpatrick abd79ea368 derp: reduce DERP memory use; don't require callers to pass in memory to use
The magicsock derpReader was holding onto 65KB for each DERP
connection forever, just in case.

Make the derp{,http}.Client be in charge of memory instead. It can
reuse its bufio.Reader buffer space.
5 years ago
Brad Fitzpatrick 07ca0c1c29 derp: fix tracking problem if conn starts local, then also joins mesh peer 5 years ago
Brad Fitzpatrick 1cb7dab881 cmd/derper: support forwarding packets amongst set of peer DERP servers
Updates #388

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 5e0ff494a5 derp: change NewClient constructor to an option pattern
(The NewMeshClient constructor I added recently was gross in
retrospect at call sites, especially when it wasn't obvious that a
meshKey empty string meant a regular client)
5 years ago
Brad Fitzpatrick 484b7fc9a3 derp, cmd/derper: add frameWatchConns, framePeerPresent for inter-DERP routing
This lets a trusted DERP client that knows a pre-shared key subscribe
to the connection list. Upon subscribing, they get the current set
of connected public keys, and then all changes over time.

This lets a set of DERP server peers within a region all stay connected to
each other and know which clients are connected to which nodes.

Updates #388

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 1df3c8d02a derp, netcheck: make tests listen on localhost only
avoid macOS firewall pop-ups
5 years ago
Brad Fitzpatrick 1ab5b31c4b derp, magicsock: send new "peer gone" frames when previous sender disconnects
Updates #150 (not yet enabled by default in magicsock)

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Crawshaw 131541c06d derp: deflake test
Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
David Crawshaw 72dbf26f63 derp: test that client a->b and a->c relaying do not interfere
Without the recent write deadline introduction, this test fails.

They still do interfere, but the interference is now bound by
the write deadline. Many improvements are possible.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
David Crawshaw 43aa8595dd derp: introduce Conn interface
This lets us test with something other than a net.Conn.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
5 years ago
Brad Fitzpatrick e733fa6f7e derp: add some varz tests for active/home conns 5 years ago
Brad Fitzpatrick bf704a5218 derp: protocol negotiation, add v2: send src pub keys to clients in packets
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 379a3125fd derp, wgengine/magicsock: support more than just packets from Client.Recv
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 1166c34f6c derp: fix staticcheck warning
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 322cb58b14 derp: deflake test I flaked up in earlier change
I broke an invariant in 11048b8932 (it was even nicely
documented then).

Also clean up the test a bit from while I was debugging it.

Fixes #84

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 259406e797 derp: move away from [32]byte key types
And some minor cleanup in the process.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Earl Lee a8d8b8719a Move Linux client & common packages into a public repo. 5 years ago