Commit Graph

103 Commits (50bf32a0ba13935273e200d52b9327821f25efc5)

Author SHA1 Message Date
James Tucker e1e22785b4 net/netcheck: ensure prior preferred DERP is always in netchecks
In an environment with unstable latency, such as upstream bufferbloat,
there are cases where a full netcheck could drop the prior preferred
DERP (likely home DERP) from future netcheck probe plans. This will then
likely result in a home DERP having a missing sample on the next
incremental netcheck, ultimately resulting in a home DERP move.

This change does not fix our overall response to highly unstable
latency, but it is an incremental improvement to prevent single spurious
samples during a full netcheck from alone triggering a flapping
condition, as now the prior changes to include historical latency will
still provide the desired resistance, and the home DERP should not move
unless latency is consistently worse over a 5 minute period.

Note that there is a nomenclature and semantics issue remaining in the
difference between a report preferred DERP and a home DERP. A report
preferred DERP is aspirational, it is what will be picked as a home DERP
if a home DERP connection needs to be established. A nodes home DERP may
be different than a recent preferred DERP, in which case a lot of
netcheck logic is fallible. In future enhancements much of the DERP move
logic should move to consider the home DERP, rather than recent report
preferred DERP.

Updates #8603
Updates #13969

Signed-off-by: James Tucker <james@tailscale.com>
4 weeks ago
Andrew Dunham b2665d9b89 net/netcheck: add a Now field to the netcheck Report
This allows us to print the time that a netcheck was run, which is
useful in debugging.

Updates #10972

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Id48d30d4eb6d5208efb2b1526a71d83fe7f9320b
1 month ago
Jordan Whited 33029d4486
net/netcheck: fix netcheck cli-triggered nil pointer deref (#13782)
Updates #13780

Signed-off-by: Jordan Whited <jordan@tailscale.com>
1 month ago
Andrew Dunham 8ee7f82bf4 net/netcheck: don't panic if a region has no Nodes
Updates #13728

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I1e8319d6b2da013ae48f15113b30c9333e69cc0b
2 months ago
Jordan Whited 951884b077
net/netcheck,wgengine/magicsock: plumb OnlyTCP443 controlknob through netcheck (#13491)
Updates tailscale/corp#17879

Signed-off-by: Jordan Whited <jordan@tailscale.com>
2 months ago
Jordan Whited afec2d41b4
wgengine/magicsock: remove redundant deadline from netcheck report call (#13395)
netcheck.Client.GetReport() applies its own deadlines. This 2s deadline
was causing GetReport() to never fall back to HTTPS/ICMP measurements
as it was shorter than netcheck.stunProbeTimeout, leaving no time
for fallbacks.

Updates #13394
Updates #6187

Signed-off-by: Jordan Whited <jordan@tailscale.com>
2 months ago
Andrea Gottardo 90be06bd5b
health: introduce captive-portal-detected Warnable (#12707)
Updates tailscale/tailscale#1634

This PR introduces a new `captive-portal-detected` Warnable which is set to an unhealthy state whenever a captive portal is detected on the local network, preventing Tailscale from connecting.



ipn/ipnlocal: fix captive portal loop shutdown


Change-Id: I7cafdbce68463a16260091bcec1741501a070c95

net/captivedetection: fix mutex misuse

ipn/ipnlocal: ensure that we don't fail to start the timer


Change-Id: I3e43fb19264d793e8707c5031c0898e48e3e7465

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
4 months ago
Brad Fitzpatrick 1ca323ac65 net/netcheck: fix probeProto.String result for IPv6 probes
This bug was introduced in e6b84f215 (May 2020) but was only used in
tests when stringifying probeProto values on failure so it wasn't
noticed for a long time.

But then it was moved into non-test code in 8450a18aa (Jun 2024) and I
didn't notice during the code movement that it was wrong. It's still
only used in failure paths in logs, but having wrong/ambiguous
debugging information isn't the best.

Whoops.

Updates tailscale/corp#20654

Change-Id: I296c727ed1c292a04db7b46ecc05c07fc1abc774
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 months ago
Brad Fitzpatrick 8450a18aa9 net/netcheck: flesh out some logging in error paths
Updates tailscale/corp#20654

Change-Id: Ie190f956b864985668f79b5b986438bbe07ce905
6 months ago
Maisem Ali 4a8cb1d9f3 all: use math/rand/v2 more
Updates #11058

Signed-off-by: Maisem Ali <maisem@tailscale.com>
6 months ago
James Tucker 87ee559b6f net/netcheck: apply some polish suggested from #12161
Apply some post-submit code review suggestions.

Updates #12161
Updates tailscale/corp#19106

Signed-off-by: James Tucker <james@tailscale.com>
6 months ago
James Tucker 9351eec3e1 net/netcheck: remove hairpin probes
Palo Alto reported interpreting hairpin probes as LAND attacks, and the
firewalls may be responding to this by shutting down otherwise in use NAT sessions
prematurely. We don't currently make use of the outcome of the hairpin
probes, and they contribute to other user confusion with e.g. the
AirPort Extreme hairpin session workaround. We decided in response to
remove the whole probe feature as a result.

Updates #188
Updates tailscale/corp#19106
Updates tailscale/corp#19116

Signed-off-by: James Tucker <james@tailscale.com>
6 months ago
James Tucker 8d1249550a net/netcheck,wgengine/magicsock: add potential workaround for Palo Alto DIPP misbehavior
Palo Alto firewalls have a typically hard NAT, but also have a mode
called Persistent DIPP that is supposed to provide consistent port
mapping suitable for STUN resolution of public ports. Persistent DIPP
works initially on most Palo Alto firewalls, but some models/software
versions have a bug which this works around.

The bug symptom presents as follows:

- STUN sessions resolve a consistent public IP:port to start with
- Much later netchecks report the same IP:Port for a subset of
  sessions, most often the users active DERP, and/or the port related
  to sustained traffic.
- The broader set of DERPs in a full netcheck will now consistently
  observe a new IP:Port.
- After this point of observation, new inbound connections will only
  succeed to the new IP:Port observed, and existing/old sessions will
  only work to the old binding.

In this patch we now advertise the lowest latency global endpoint
discovered as we always have, but in addition any global endpoints that
are observed more than once in a single netcheck report. This should
provide viable endpoints for potential connection establishment across
a NAT with this behavior.

Updates tailscale/corp#19106

Signed-off-by: James Tucker <james@tailscale.com>
6 months ago
Maisem Ali 85b9a6c601 net/netcheck: do not add derps if IPv4/IPv6 is set to "none"
It was documented as such but seems to have been dropped in a
refactor, restore the behavior. This brings down the time it
takes to run a single integration test by 2s which adds up
quite a bit.

Updates tailscale/corp#19786

Signed-off-by: Maisem Ali <maisem@tailscale.com>
7 months ago
Brad Fitzpatrick c3c18027c6 all: make more tests pass/skip in airplane mode
Updates tailscale/corp#19786

Change-Id: Iedc6730fe91c627b556bff5325bdbaf7bf79d8e6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick e42c4396cf net/netcheck: don't spam on ICMP socket permission denied errors
While debugging a failing test in airplane mode on macOS, I noticed
netcheck logspam about ICMP socket creation permission denied errors.

Apparently macOS just can't do those, or at least not in airplane
mode. Not worth spamming about.

Updates #cleanup

Change-Id: I302620cfd3c8eabb25202d7eef040c01bd8a843c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick ee3bd4dbda derp/derphttp, net/netcheck: plumb netmon.Monitor to derp netcheck client
Fixes #11981

Change-Id: I0e15a09f93aefb3cfddbc12d463c1c08b83e09fd
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick b9adbe2002 net/{interfaces,netmon}, all: merge net/interfaces package into net/netmon
In prep for most of the package funcs in net/interfaces to become
methods in a long-lived netmon.Monitor that can cache things.  (Many
of the funcs are very heavy to call regularly, whereas the long-lived
netmon.Monitor can subscribe to things from the OS and remember
answers to questions it's asked regularly later)

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: Ie4e8dedb70136af2d611b990b865a822cd1797e5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick 6b95219e3a net/netmon, add: add netmon.State type alias of interfaces.State
... in prep for merging the net/interfaces package into net/netmon.

This is a no-op change that updates a bunch of the API signatures ahead of
a future change to actually move things (and remove the type alias)

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: I477613388f09389214db0d77ccf24a65bff2199c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick 3672f29a4e net/netns, net/dns/resolver, etc: make netmon required in most places
The goal is to move more network state accessors to netmon.Monitor
where they can be cheaper/cached. But first (this change and others)
we need to make sure the one netmon.Monitor is plumbed everywhere.

Some notable bits:

* tsdial.NewDialer is added, taking a now-required netmon

* because a tsdial.Dialer always has a netmon, anything taking both
  a Dialer and a NetMon is now redundant; take only the Dialer and
  get the NetMon from that if/when needed.

* netmon.NewStatic is added, primarily for tests

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: I877f9cb87618c4eb037cee098241d18da9c01691
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick 7a62dddeac net/netcheck, wgengine/magicsock: make netmon.Monitor required
This has been a TODO for ages. Time to do it.

The goal is to move more network state accessors to netmon.Monitor
where they can be cheaper/cached.

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: I60fc6508cd2d8d079260bda371fc08b6318bcaf1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
James Tucker 6e334e64a1 net/netcheck,wgengine/magicsock: align DERP frame receive time heuristics
The netcheck package and the magicksock package coordinate via the
health package, but both sides have time based heuristics through
indirect dependencies. These were misaligned, so the implemented
heuristic aimed at reducing DERP moves while there is active traffic
were non-operational about 3/5ths of the time.

It is problematic to setup a good test for this integration presently,
so instead I added comment breadcrumbs along with the initial fix.

Updates #8603

Signed-off-by: James Tucker <james@tailscale.com>
8 months ago
Brad Fitzpatrick 2bd3c1474b util/cmpx: delete now that we're using Go 1.22
Updates #11058

Change-Id: I09dea8e86f03ec148b715efca339eab8b1f0f644
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
10 months ago
Andrew Dunham 727acf96a6 net/netcheck: use DERP frames as a signal for home region liveness
This uses the fact that we've received a frame from a given DERP region
within a certain time as a signal that the region is stil present (and
thus can still be a node's PreferredDERP / home region) even if we don't
get a STUN response from that region during a netcheck.

This should help avoid DERP flaps that occur due to losing STUN probes
while still having a valid and active TCP connection to the DERP server.

RELNOTE=Reduce home DERP flapping when there's still an active connection

Updates #8603

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: If7da6312581e1d434d5c0811697319c621e187a0
12 months ago
Andrea Barisani affe11c503 net/netcheck: only run HTTP netcheck for tamago clients
Signed-off-by: Andrea Barisani <andrea@inversepath.com>
12 months ago
James Tucker de8e55fda6 net/netcheck,wgengine/magicsock: reduce coupling between netcheck and magicsock
Netcheck no longer performs I/O itself, instead it makes requests via
SendPacket and expects users to route reply traffic to
ReceiveSTUNPacket.

Netcheck gains a Standalone function that stands up sockets and
goroutines to implement I/O when used in a standalone fashion.

Magicsock now unconditionally routes STUN traffic to the netcheck.Client
that it hosts, and plumbs the send packet sink.

The CLI is updated to make use of the Standalone mode.

Fixes #8723

Signed-off-by: James Tucker <james@tailscale.com>
1 year ago
Brad Fitzpatrick 66e46bf501 ipnlocal, net/*: deprecate interfaces.GetState, use netmon more for it
Updates #cleanup

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Andrew Dunham 9b5e29761c net/netcheck: ignore PreferredDERP changes that are small
If the absolute value of the difference between the current
PreferredDERP's latency and the best latency is <= 10ms, don't change
it and instead prefer the previous value.

This is in addition to the existing hysteresis that tries to remain
on the previous DERP region if the relative improvement is small, but
handles nodes that have low latency to >1 DERP region better.

Updates #8603

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I1e34c94178f8c9a68a69921c5bc0227337514c70
1 year ago
Andrew Dunham 7aba0b0d78 net/netcheck, tailcfg: add DERPHomeParams and use it
This allows providing additional information to the client about how to
select a home DERP region, such as preferring a given DERP region over
all others.

Updates #8603

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I7c4a270f31d8585112fab5408799ffba5b75266f
1 year ago
Brad Fitzpatrick eefee6f149 all: use cmpx.Or where it made sense
I left a few out where writing it explicitly was better
for various reasons.

Updates #8296

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Charlotte Brandhorst-Satzkorn 161d1d281a
net/ping,netcheck: add v6 pinging capabilities to pinger (#7971)
This change adds a v6conn to the pinger to enable sending pings to v6
addrs.

Updates #7826

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Brad Fitzpatrick 90ba26cea1 net/netcheck: fix crash when IPv6 kinda but not really works
Looks like on some systems there's an IPv6 address, but then opening
a IPv6 UDP socket fails later. Probably some firewall. Tolerate it
better and don't crash.

To repro: check the "udp6" to something like "udp7" (something that'll
fail) and run "go run ./cmd/tailscale netcheck" on a machine with
active IPv6. It used to crash and now it doesn't.

Fixes #7949

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Mihai Parparita 7330aa593e all: avoid repeated default interface lookups
On some platforms (notably macOS and iOS) we look up the default
interface to bind outgoing connections to. This is both duplicated
work and results in logspam when the default interface is not available
(i.e. when a phone has no connectivity, we log an error and thus cause
more things that we will try to upload and fail).

Fixed by passing around a netmon.Monitor to more places, so that we can
use its cached interface state.

Fixes #7850
Updates #7621

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Brad Fitzpatrick 10f1c90f4d wgengine/magicsock, types/nettype, etc: finish ReadFromUDPAddrPort netip migration
So we're staying within the netip.Addr/AddrPort consistently and
avoiding allocs/conversions to the legacy net addr types.

Updates #5162

Change-Id: I59feba60d3de39f773e68292d759766bac98c917
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 29f7df9d8f wgengine/magicsock, etc: remove mostly unused WriteTo methods
Updates #2331
Updates #5162

Change-Id: I8291884425481eeaedde38a54adfd8ed7292a497
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 83c41f3697 net/netcheck: remove unused method from interface
Updates #2331
Updates #5162

Change-Id: I77ff956c2d59bde611d47523659a07afb4a6da2d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Andrew Dunham 228d0c6aea net/netcheck: use dnscache.Resolver when resolving DERP IPs
This also adds a bunch of tests for this function to ensure that we're
returning the proper IP(s) in all cases.

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I0d9d57170dbab5f2bf07abdf78ecd17e0e635399
2 years ago
Mihai Parparita edb02b63f8 net/sockstats: pass in logger to sockstats.WithSockStats
Using log.Printf may end up being printed out to the console, which
is not desirable. I noticed this when I was investigating some client
logs with `sockstats: trace "NetcheckClient" was overwritten by another`.
That turns to be harmless/expected (the netcheck client will fall back
to the DERP client in some cases, which does its own sockstats trace).

However, the log output could be visible to users if running the
`tailscale netcheck` CLI command, which would be needlessly confusing.

Updates tailscale/corp#9230

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Andrew Dunham 4cb1bfee44 net/netcheck: improve determinism in hairpinning test
If multiple Go channels have a value (or are closed), receiving from
them all in a select will nondeterministically return one of the two
arms. In this case, it's possible that the hairpin check timer will have
expired between when we start checking and before we check at all, but
the hairpin packet has already been received. In such cases, we'd
nondeterministically set report.HairPinning.

Instead, check if we have a value in our results channel first, then
select on the value and timeout channel after. Also, add a test that
catches this particular failure.

Fixes #1795

Change-Id: I842ab0bd38d66fabc6cabf2c2c1bb9bd32febf35
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
2 years ago
Mihai Parparita 6ac6ddbb47 sockstats: switch label to enum
Makes it cheaper/simpler to persist values, and encourages reuse of
labels as opposed to generating an arbitrary number.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 9cb332f0e2 sockstats: instrument networking code paths
Uses the hooks added by tailscale/go#45 to instrument the reads and
writes on the major code paths that do network I/O in the client. The
convention is to use "<package>.<type>:<label>" as the annotation for
the responsible code path.

Enabled on iOS, macOS and Android only, since mobile platforms are the
ones we're most interested in, and we are less sensitive to any
throughput degradation due to the per-I/O callback overhead (macOS is
also enabled for ease of testing during development).

For now just exposed as counters on a /v0/sockstats PeerAPI endpoint.

We also keep track of the current interface so that we can break out
the stats by interface.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Will Norris 71029cea2d all: update copyright and license headers
This updates all source files to use a new standard header for copyright
and license declaration.  Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.

This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.

Updates #6865

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Andrew Dunham e85613aa2d net/netcheck: don't use a space in the captive portal challenge
The derpers don't allow whitespace in the challenge.

Change-Id: I93a8b073b846b87854fba127b5c1d80db205f658
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
2 years ago
Brad Fitzpatrick 036334e913 net/netcheck: deflake (maybe) magicsock's TestNewConn
Updates #6207

Change-Id: I51d200d0b42b9a1ef799d0abfc8d4bd871c50cf2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Andrew Dunham 74693793be net/netcheck, tailcfg: track whether OS supports IPv6
We had previously added this to the netcheck report in #5087 but never
copied it into the NetInfo struct. Additionally, add it to log lines so
it's visible to support.

Change-Id: Ib6266f7c6aeb2eb2a28922aeafd950fe1bf5627e
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
2 years ago
Andrew Dunham 223126fe5b cmd/derper, net/netcheck: add challenge/response to generate_204 endpoint
The Lufthansa in-flight wifi generates a synthetic 204 response to the
DERP server's /generate_204 endpoint. This PR adds a basic
challenge/response to the endpoint; something sufficiently complicated
that it's unlikely to be implemented by a captive portal. We can then
check for the expected response to verify whether we're being MITM'd.

Follow-up to #5601

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I94a68c9a16a7be7290200eea6a549b64f02ff48f
2 years ago
Brad Fitzpatrick 9a264dac01 net/netcheck: fix crash in checkCaptivePortal
If netcheck happens before there's a derpmap.

This seems to only affect Headscale because it doesn't send a derpmap
as early?

Change-Id: I51e0dfca8e40623e04702bc9cc471770ca20d2c2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Josh Soref d4811f11a0 all: fix spelling mistakes
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2 years ago
Emmanuel T Odeke f981b1d9da all: fix resource leaks with missing .Close() calls
Fixes #5706

Signed-off-by: Emmanuel T Odeke <emmanuel@orijtech.com>
2 years ago
Andrew Dunham c6162c2a94
net/netcheck: add check for captive portal (#5593)
This doesn't change any behaviour for now, other than maybe running a
full netcheck more often. The intent is to start gathering data on
captive portals, and additionally, seeing this in the 'tailscale
netcheck' command should provide a bit of additional information to
users.

Updates #1634

Change-Id: I6ba08f9c584dc0200619fa97f9fde1a319f25c76
Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
2 years ago