Commit Graph

174 Commits (13f75b9667c0522323b7b4c8a1762debbf0e7866)

Author SHA1 Message Date
Tom DNetto c8f4dfc8c0 derp/derphttp,net/netcheck: improve netcheck behavior under MITM proxies
In cases where tailscale is operating behind a MITM proxy, we need to consider
that a lot more of the internals of our HTTP requests are visible and may be
used as part of authorization checks. As such, we need to 'behave' as closely
as possible to ideal.

 - Some proxies do authorization or consistency checks based the on Host header
   or HTTP URI, instead of just the IP/hostname/SNI. As such, we need to
   construct a `*http.Request` with a valid URI everytime HTTP is going to be
   used on the wire, even if its over TLS.
   Aside from the singular instance in net/netcheck, I couldn't find anywhere
   else a http.Request was constructed incorrectly.

 - Some proxies may deny requests, typically by returning a 403 status code. We
   should not consider these requests as a valid latency check, so netcheck
   semantics have been updated to consider >299 status codes as a failed probe.

Signed-off-by: Tom DNetto <tom@tailscale.com>
3 years ago
Maisem Ali 9f604f2bd3
derp: add (*Server).IsClientConnectedForTest func. (#4331)
This allows tests to verfiy that a DERP connection was actually
established.

Related to #4326
Updates tailscale/corp#2579

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 years ago
Brad Fitzpatrick 18818763d1 derp: set Basic Constraints on metacert
See https://github.com/golang/go/issues/51759#issuecomment-1071147836

Once we deploy this, tailscaled should work again for macOS users with
Go 1.18.

Updates golang/go#51759

Change-Id: I869b6ddc556a2de885e96ccf9f335dfc8f6f6a7e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 0868329936 all: use any instead of interface{}
My favorite part of generics.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 730aa1c89c derp/derphttp, wgengine/magicsock: prefer IPv6 to DERPs when IPv6 works
Fixes #3838

Change-Id: Ie47a2a30c7e8e431512824798d2355006d72fb6a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 70d71ba1e7 cmd/derpprobe: check derper TLS certs too
Change-Id: If8c48e012b294570ebbb1a46bacdc58fafbfbcc5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 41fd4eab5c envknob: add new package for all the strconv.ParseBool(os.Getenv(..))
A new package can also later record/report which knobs are checked and
set. It also makes the code cleaner & easier to grep for env knobs.

Change-Id: Id8a123ab7539f1fadbd27e0cbeac79c2e4f09751
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 2aeb93003f derp: add metrics to server got pings, sent pongs
Updates #3652

Change-Id: I1d350bcaee39ea36b0c71912028624d18fb541b4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 2c94e3c4ad wgengine/magicsock: don't unconditionally close DERP connections on rebind
Only if the source address isn't on the currently active interface or
a ping of the DERP server fails.

Updates #3619

Change-Id: I6bf06503cff4d781f518b437c8744ac29577acc8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 63d9c7b9b3 derp: add Client.LocalAddr method
So magicsock can later ask a DERP connection whether its source IP
would've changed if it reconnected.

Updates #3619

Change-Id: Ibc8810340c511d6786b60c78c1a61c09f5800e40
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 535b925d1b derp/derphttp: add Client.Ping, SendPing methods
Continuing work in 434af15a04, to make it possible for magicsock to
probe whether a DERP server is still there.

Updates #3619

Change-Id: I366a77c27e93b876734e64f445b85ef01eb590f2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 434af15a04 derp: support client->server ping (and server->client pong)
In prep for a future change to have client ping derp connections
when their state is questionable, rather than aggressively tearing
them down and doing a heavy reconnect when their state is unknown.

We already support ping/pong in the other direction (servers probing
clients) so we already had the two frame types, but I'd never finished
this direction.

Updates #3619

Change-Id: I024b815d9db1bc57c20f82f80f95fb55fc9e2fcc
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 758c37b83d net/netns: thread logf into control functions
So that darwin can log there without panicking during tests.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Anderson 84c3a09a8d types/key: export constants for key size, not a method.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson c1d009b9e9 ipn/ipnstate: use key.NodePublic instead of the generic key.Public.
Updates #3206.

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 1f06f77dcb derp: remove package shadowing of types/key.
Updates #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 37c150aee1 derp: use new node key type.
Update #3206

Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick eebe7afad7 derp/derphttp: only log about a weird upgrade if any was specified
Otherwise random browser requests to /derp cause log spam.

Change-Id: I7bdf991d2106f0323868e651156c788a877a90d5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 505f844a43 cmd/derper, derp/derphttp: add websocket support
Updates #3157

Change-Id: I337a919a3b350bc7bd9af567b49c4d5d6616abdd
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 7cf8ec8108 net/tlsdial: bake in LetsEncrypt's ISRG Root X1 root
We still try the host's x509 roots first, but if that fails (like if
the host is old), we fall back to using LetsEncrypt's root and
retrying with that.

tlsdial was used in the three main places: logs, control, DERP. But it
was missing in dnsfallback. So added it there too, so we can run fine
now on a machine with no DNS config and no root CAs configured.

Also, move SSLKEYLOGFILE support out of DERP. tlsdial is the logical place
for that support.

Fixes #1609

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 73f177e4d5 derp: throttle client sends if server advertises rate limits
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 3759fb8987 derp: deflake TestSendFreeze
On about 1 out of 500 runs, TestSendFreeze failed:

    derp_test.go:416: bob: unexpected message type derp.PeerGoneMessage

Closing alice before bob created a race.
If bob closed promptly, the test passed.
If bob closed slowly, and alice's disappearance caused
bob to receive a PeerGoneMessage before closing, the test failed.

Deflake the test by closing bob first.
With this fix, the test passed 12,000 times locally.

Fixes #2668

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick fc160f80ee metrics: move currentFDs code to the metrics package
Updates #2784

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick a59b389a6a derp: add new health update and server restarting frame types
Updates #2756
Updates #2746

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 73280595a8 derp: accept dup clients without closing prior's connection
A public key should only have max one connection to a given
DERP node (or really: one connection to a node in a region).

But if people clone their machine keys (e.g. clone their VM, Raspbery
Pi SD card, etc), then we can get into a situation where a public key
is connected multiple times.

Originally, the DERP server handled this by just kicking out a prior
connections whenever a new one came. But this led to reconnect fights
where 2+ nodes were in hard loops trying to reconnect and kicking out
their peer.

Then a909d37a59 tried to add rate
limiting to how often that dup-kicking can happen, but empirically it
just doesn't work and ~leaks a bunch of goroutines and TCP
connections, tying them up for hour+ while more and more accumulate
and waste memory. Mostly because we were doing a time.Sleep forever
while not reading from their TCP connections.

Instead, just accept multiple connections per public key but track
which is the most recent. And if two both are writing back & forth,
then optionally disable them both. That last part is only enabled in
tests for now. The current default policy is just last-sender-wins
while we gather the next round of stats.

Updates #2751

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick ffd22050c0 derp: export current file descriptor metric 3 years ago
slowy07 ac0353e982 fix: typo spelling grammar
Signed-off-by: slowy07 <slowy.arfy@gmail.com>
3 years ago
Brad Fitzpatrick f35b8c3ead derp: fix meshing accounting edge case bug
If a peer is connected to multiple nodes in a region (so
multiForwarder is in use) and then a node restarts and re-sends all
its additions, this bug about whether an element is in the
multiForwarder could cause a one-time flip in the which peer node we
forward to.  Note a huge deal, but not written as intended.

Thanks to @lewgun for the bug report in #2141.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick fd7b738e5b derp: use pad32 package for padding, reduce duplication
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 7298e777d4 derp: reduce server memory by 30% by removing persistent bufio.Writer
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick b622c60ed0 derp,wgengine/magicsock: don't assume stringer is in $PATH for go:generate
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick e422e9f4c9 cmd/derper: mesh over VPC network
Updates #2414

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick e299300b48 net/dnscache: cache all IPs per hostname
Not yet used in the dialer, but plumbed around.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick a909d37a59 derp: rate limit how often same-key clients can kick each other off server
Updates #392
Updates #506

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 4dbbd0aa4a cmd/addlicense: add command to add licenseheaders to generated code
And use it to make our stringer invocations match the existing code.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 0ec9040c5e scripts: remove special case for _strings.go files in check license headers
And add a license header for derp/dropreason_string.go to make it happy.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
maddie d976a84d7e derp: allow self node when verifying clients
Fixes #2408

Signed-off-by: Maddie Zhan <maddie.zhan@cynovan.com>
3 years ago
Brad Fitzpatrick 05da2691a5 cmd/derper/derpprobe: add derp prober
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 4c0494185b derp: remove "fine for now" intentional slow memory leak from derp server
It was once believed that it might be useful. It wasn't. We never used it.

Remove it so we don't slowly leak memory.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
David Anderson d98829583a derp: use a dedicated queue for disco traffic.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 67158549ab derp: actually export the new drop counter.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 36492ace9d derp: add counters to track the type of dropped packets.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 7e7c4c1bbe tailcfg: break DERPNode.DERPTestPort into DERPPort & InsecureForTests
The DERPTestPort int meant two things before: which port to use, and
whether to disable TLS verification. Users would like to set the port
without disabling TLS, so break it into two options.

Updates #1264

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
julianknodt 506c2fe8e2 cmd/tailscale: make netcheck use active DERP map, delete static copy
After allowing for custom DERP maps, it's convenient to be able to see their latency in
netcheck. This adds a query to the local tailscaled for the current DERPMap.

Updates #1264

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
julianknodt 148602a89a derp,cmd/derper: allow server to verify clients
This adds a flag to the DERP server which specifies to verify clients through a local
tailscaled. It is opt-in, so should not affect existing clients, and is mainly intended for
users who want to run their own DERP servers. It assumes there is a local tailscaled running and
will attempt to hit it for peer status information.

Updates #1264

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
julianknodt 3687e5352b derp: fix traffic handler peer addresses
Before it was using the local address and port, so fix that.
The fields in the response from `ss` are:

State, Recv-Q, Send-Q, Local Address:Port, Peer Address:Port, Process

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
julianknodt 3728634af9 derp: add debug traffic handler
This adds a handler on the DERP server for logging bytes send and received by clients of the
server, by holding open a connection and recording if there is a difference between the number
of bytes sent and received. It sends a JSON marshalled object if there is an increase in the
number of bytes.

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
julianknodt fe54721e31 derp: add pkt queue latency timer
It would be useful to know the time that packets spend inside of a queue before they are sent
off, as that can be indicative of the load the server is handling (and there was also an
existing TODO). This adds a simple exponential moving average metric to track the average packet
queue duration.
Changes during review:
Add CAS loop for recording queue timing w/ expvar.Func, rm snake_case, annotate in milliseconds,
convert

Signed-off-by: julianknodt <julianknodt@gmail.com>
4 years ago
Brad Fitzpatrick 3d4d97601a derp/derpmap: add São Paulo (derp11)
Updates #1499
4 years ago
Maisem Ali 4d142ebe06 derp: handle net.ErrClosed in TestSendFreeze
Signed-off-by: Maisem Ali <maisem@tailscale.com>
4 years ago