Commit Graph

27 Commits (ebeb5da202c00c41a3c87ebf687f89a2fc70bb90)

Author SHA1 Message Date
Andrea Gottardo 6985369479
net/sockstats: prevent crash in setNetMon (#13985) 4 weeks ago
Brad Fitzpatrick 841eaacb07 net/sockstats: quiet some log spam in release builds
Updates #13731

Change-Id: Ibee85426827ebb9e43a1c42a9c07c847daa50117
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick b9adbe2002 net/{interfaces,netmon}, all: merge net/interfaces package into net/netmon
In prep for most of the package funcs in net/interfaces to become
methods in a long-lived netmon.Monitor that can cache things.  (Many
of the funcs are very heavy to call regularly, whereas the long-lived
netmon.Monitor can subscribe to things from the OS and remember
answers to questions it's asked regularly later)

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: Ie4e8dedb70136af2d611b990b865a822cd1797e5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Brad Fitzpatrick 7c1d6e35a5 all: use Go 1.22 range-over-int
Updates #11058

Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Andrew Dunham 530aaa52f1 net/dns: retry forwarder requests over TCP
We weren't correctly retrying truncated requests to an upstream DNS
server with TCP. Instead, we'd return a truncated request to the user,
even if the user was querying us over TCP and thus able to handle a
large response.

Also, add an envknob and controlknob to allow users/us to disable this
behaviour if it turns out to be buggy ( DNS ).

Updates #9264

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ifb04b563839a9614c0ba03e9c564e8924c1a2bfd
1 year ago
Brad Fitzpatrick 9089efea06 net/netmon: make ChangeFunc's signature take new ChangeDelta, not bool
Updates #9040

Change-Id: Ia43752064a1a6ecefc8802b58d6eaa0b71cf1f84
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Will Norris ea84fc9ad2 net/sockstats: wait before reporting battery usage
Wait 2 minutes before we start reporting battery usage. There is always
radio activity on initial startup, which gets reported as 100% high
power usage.  Let that settle before we report usage data.

Updates tailscale/corp#9230

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Mihai Parparita 7330aa593e all: avoid repeated default interface lookups
On some platforms (notably macOS and iOS) we look up the default
interface to bind outgoing connections to. This is both duplicated
work and results in logspam when the default interface is not available
(i.e. when a phone has no connectivity, we log an error and thus cause
more things that we will try to upload and fail).

Fixed by passing around a netmon.Monitor to more places, so that we can
use its cached interface state.

Fixes #7850
Updates #7621

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 4722f7e322 all: move network monitoring from wgengine/monitor to net/netmon
We're using it in more and more places, and it's not really specific to
our use of Wireguard (and does more just link/interface monitoring).

Also removes the separate interface we had for it in sockstats -- it's
a small enough package (we already pull in all of its dependencies
via other paths) that it's not worth the extra complexity.

Updates #7621
Updates #7850

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita d0906cda97 net/sockstats: expose debug info
Exposes some internal state of the sockstats package via the C2N and
PeerAPI endpoints, so that it can be used for debugging. For now this
includes the estimated radio on percentage and a second-by-second view
of the times the radio was active.

Also fixes another off-by-one error in the radio on percentage that
was leading to >100% values (if n seconds have passed since we started
to monitor, there may be n + 1 possible seconds where the radio could
have been on).

Updates tailscale/corp#9230

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Will Norris 7c386ca6d2 net/sockstats: fix calculation of radio power usage
When splitting the radio monitor usage array, we were splitting at now %
3600 to get values into chronological order.  This caused the value for
the final second to be included at the beginning of the ordered slice
rather than the end.  If there was activity during that final second, an
extra five seconds of high power usage would get recorded in some cases.
This could result in a final calculation of greater than 100% usage.

This corrects that by splitting values at (now+1 % 3600).

This also simplifies the percentage calculation by always rounding
values down, which is sufficient for our usage.

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Mihai Parparita edb02b63f8 net/sockstats: pass in logger to sockstats.WithSockStats
Using log.Printf may end up being printed out to the console, which
is not desirable. I noticed this when I was investigating some client
logs with `sockstats: trace "NetcheckClient" was overwritten by another`.
That turns to be harmless/expected (the netcheck client will fall back
to the DERP client in some cases, which does its own sockstats trace).

However, the log output could be visible to users if running the
`tailscale netcheck` CLI command, which would be needlessly confusing.

Updates tailscale/corp#9230

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 782ccb5655 .github/workflows: run one set of tests with the tailscale_go build tag
We use it to gate code that depends on custom Go toolchain, but it's
currently only passed in the corp runners. Add a set on OSS so that we
can catch regressions earlier.

To specifically test sockstats this required adding a build tag to
explicitly enable them -- they're normally on for iOS, macOS and Android
only, and we don't run tests on those platforms normally.

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita e978299bf0 net/sockstats: disable deltas for the cell radio power state metric
Updates tailscale/corp#9230

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Will Norris 22680a11ae net/sockstats: return early if no radio period length
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Will Norris 75784e10e2 sockstats: add client metrics for radio power state
power state is very roughly approximated based on observed network
activity and AT&T's state transition timings for a typical 3G radio.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Will Norris e99c7c3ee5 sockstats: add labels for netlog and sockstatlog packages
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Mihai Parparita d2dec13392 net/sockstats: export cellular-only clientmetrics
Followup to #7518 to also export client metrics when the active interface
is cellular.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 97b6d3e917 sockstats: remove per-interface stats from Get
They're not needed for the sockstats logger, and they're somewhat
expensive to return (since they involve the creation of a map per
label). We now have a separate GetInterfaces() method that returns
them instead (which we can still use in the PeerAPI debug endpoint).

If changing sockstatlog to sample at 10,000 Hz (instead of the default
of 10Hz), the CPU usage would go up to 59% on a iPhone XS. Removing the
per-interface stats drops it to 20% (a no-op implementation of Get that
returns a fixed value is 16%).

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Will Norris a1d9f65354 ipn,log: add logger for sockstat deltas
Signed-off-by: Will Norris <will@tailscale.com>
Co-authored-by: Melanie Warrick <warrick@tailscale.com>
2 years ago
Mihai Parparita b64d78d58f sockstats: refactor validation to be opt-in
Followup to #7499 to make validation a separate function (
GetWithValidation vs. Get). This way callers that don't need it don't
pay the cost of a syscall per active TCP socket.

Also clears the conn on close, so that we don't double-count the stats.

Also more consistently uses Go doc comments for the exported API of the
sockstats package.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita ea81bffdeb sockstats: export as client metrics
Though not fine-grained enough to be useful for detailed analysis, we
might as well export that we gather as client metrics too, since we have
an upload/analysis pipeline for them.

clientmetric.Metric.Add is an atomic add, so it's pretty cheap to also
do per-packet.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 4c2f67a1d0 net/sockstat: fix per-interface statistics not always being available
withSockStats may be called before setLinkMonitor, in which case we
don't have a populated knownInterfaces map. Since we pre-populate the
per-interface counters at creation time, we would end up with an
empty map. To mitigate this, we do an on-demand request for the list of
interfaces.

This would most often happen with the logtail instrumentation, since we
initialize it very early on.

Updates tailscale/corp#9230

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita f4f8ed98d9 sockstats: add validation for TCP socket stats
We can use the TCP_CONNECTION_INFO getsockopt() on Darwin to get
OS-collected tx/rx bytes for TCP sockets. Since this API is not available
for UDP sockets (or on Linux/Android), we can't rely on it for actual
stats gathering.

However, we can use it to validate the stats that we collect ourselves
using read/write hooks, so that we can be more confident in them. We
do need additional hooks from the Go standard library (added in
tailscale/go#59) to be able to collect them.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 6ac6ddbb47 sockstats: switch label to enum
Makes it cheaper/simpler to persist values, and encourages reuse of
labels as opposed to generating an arbitrary number.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 3e71e0ef68
net/sockstats: remove explicit dependency on wgengine/monitor
Followup to #7177 to avoid adding extra dependencies to the CLI. We
instead declare an interface for the link monitor.

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Mihai Parparita 9cb332f0e2 sockstats: instrument networking code paths
Uses the hooks added by tailscale/go#45 to instrument the reads and
writes on the major code paths that do network I/O in the client. The
convention is to use "<package>.<type>:<label>" as the annotation for
the responsible code path.

Enabled on iOS, macOS and Android only, since mobile platforms are the
ones we're most interested in, and we are less sensitive to any
throughput degradation due to the per-I/O callback overhead (macOS is
also enabled for ease of testing during development).

For now just exposed as counters on a /v0/sockstats PeerAPI endpoint.

We also keep track of the current interface so that we can break out
the stats by interface.

Updates tailscale/corp#9230
Updates #3363

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago