Commit Graph

4698 Commits (b2035a1dca8d3c89f1237ff127dd7be8a6e8f685)
 

Author SHA1 Message Date
Joe Tsai b2035a1dca
cmd/netlogfmt: handle any stream of network logs (#6108)
Make netlogfmt useful regardless of the exact schema of the input.
If a JSON object looks like a network log message,
then unmarshal it as one and then print it.
This allows netlogfmt to support both a stream of JSON objects
directly serialized from netlogtype.Message, or the schema
returned by the /api/v2/tailnet/{{tailnet}}/network-logs API endpoint.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Joe Tsai 48ddb3af2a
wgengine/netlog: enforce hard limit on network log message sizes (#6109)
This is a temporary hack to prevent logtail getting stuck
uploading the same excessive message over and over.
A better solution will be discussed and implemented.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Joe Tsai a3602c28bd
wgengine/netlog: embed the StableNodeID of the authoring node (#6105)
This allows network messages to be annotated with which node it came from.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Joe Tsai 81fd259133
wgengine/magicsock: gather physical-layer statistics (#5925)
There is utility in logging traffic statistics that occurs at the physical layer.
That is, in order to send packets virtually to a particular tailscale IP address,
what physical endpoints did we need to communicate with?

This functionality logs IP addresses identical to
what had always been logged in magicsock prior to #5823,
so there is no increase in PII being logged.

ExtractStatistics returns a mapping of connections to counts.
The source is always a Tailscale IP address (without port),
while the destination is some endpoint reachable on WAN or LAN.
As a special case, traffic routed through DERP will use 127.3.3.40
as the destination address with the port being the DERP region.

This entire feature is only enabled if data-plane audit logging
is enabled on the tailnet (by default it is disabled).

Example of type of information logged:

	------------------------------------  Tx[P/s]    Tx[B/s]  Rx[P/s]   Rx[B/s]
	PhysicalTraffic:                       25.80      3.39Ki   38.80     5.57Ki
	    100.1.2.3 -> 143.11.22.33:41641    15.40      2.00Ki   23.20     3.37Ki
	    100.4.5.6 -> 192.168.0.100:41641   10.20      1.38Ki   15.60     2.20Ki
	    100.7.8.9 -> 127.3.3.40:2           0.20      6.40      0.00     0.00

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Brad Fitzpatrick 5e9e57ecf5 control/controlhttp: add AcceptHTTP hook to add coalesced Server->Client write
New plan for #5972. Instead of sending the public key in the clear
(from earlier unreleased 246274b8e9) where the client might have to
worry about it being dropped or tampered with and retrying, we'll
instead send it post-Noise handshake but before the HTTP/2 connection
begins.

This replaces the earlier extraHeaders hook with a different sort of
hook that allows us to combine two writes on the wire in one packet.

Updates #5972

Change-Id: I42cdf7c1859b53ca4dfa5610bd1b840c6986e09c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Joe Tsai c21a3c4733
types/netlogtype: new package for network logging types (#6092)
The netlog.Message type is useful to depend on from other packages,
but doing so would transitively cause gvisor and other large packages
to be linked in.

Avoid this problem by moving all network logging types to a single package.

We also update staticcheck to take in:

	003d277bcf

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Aaron Klotz a44687e71f wgengine/winnet: invoke some COM methods directly instead of through IDispatch.
Intermittently in the wild we are seeing failures when calling
`INetworkConnection::GetNetwork`. It is unclear what the root cause is, but what
is clear is that the error is happening inside the object's `IDispatch` invoker
(as opposed to the method implementation itself).

This patch replaces our wrapper for `INetworkConnection::GetNetwork` with an
alternate implementation that directly invokes the method, instead of using
`IDispatch`. I also replaced the implementations of `INetwork::SetCategory` and
`INetwork::GetCategory` while I was there.

This patch is speculative and tightly-scoped so that we could possibly add it
to a dot-release if necessary.

Updates https://github.com/tailscale/tailscale/issues/4134
Updates https://github.com/tailscale/tailscale/issues/6037

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
2 years ago
Brad Fitzpatrick 4021ae6b9d types/key: add missing ChallengePublic.UnmarshalText
Forgot it when adding the Challenge types earlier.

Change-Id: Ie0872c4e6dc25e5d832aa58c7b3f66d450bf6b71
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Adrian Dewhurst 8c09ae9032 tka, types/key: add NLPublic.KeyID
This allows direct use of NLPublic with tka.Authority.KeyTrusted() and
similar without using tricks like converting the return value of Verifier.

Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
2 years ago
Sonia Appasamy 944f43f1c8 docs/webhooks: add sample endpoint code
Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
2 years ago
Andrew Dunham 95f3dd1346 net/interfaces: don't dereference null pointer if no destination/netmask
Fixes #6065

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I7159b8cbb8d5f47c0668cf83e59167f182f1defd
2 years ago
Maisem Ali 19b5586573 cmd/tailscale/cli: add beginnings of `tailscale set`
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Jordan Whited a471681e28
wgengine/netstack: enable TCP SACK (#6066)
TCP selective acknowledgement can improve throughput by an order
of magnitude in the presence of loss.

Signed-off-by: Jordan Whited <jordan@tailscale.com>
2 years ago
Mihai Parparita d60f7fe33f cmd/tsconnect: run wasm-opt on the generated wasm file
Saves about 1.4MB from the generated wasm file. The Brotli size is
basically unchanged (it's actually slightly larger, by 40K), suggesting
that most of the size delta is due to not inlining and other changes
that were easily compressible.

However, it still seems worthwhile to have a smaller final binary, to
reduce parse time and increase likelihood that we fit in the browser's
disk cache. Actual performance appears to be unchanged.

Updates #5142

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Maisem Ali 2a9ba28def ipn/ipnlocal: set prefs before calling tkaSyncIfNeeded
Caught this in a test in a different repo.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Will Norris bff202a290 cmd/nginx-auth: add experimental status badge to README 2 years ago
Brad Fitzpatrick 35bee36549 portlist: use win32 calls instead of running netstat process [windows]
Turns out using win32 instead of shelling out to child processes is a
bit faster:

    name                  old time/op    new time/op    delta
    GetListIncremental-4     278ms ± 2%       0ms ± 7%  -99.93%  (p=0.000 n=8+10)

    name                  old alloc/op   new alloc/op   delta
    GetListIncremental-4     238kB ± 0%       9kB ± 0%  -96.12%  (p=0.000 n=10+8)

    name                  old allocs/op  new allocs/op  delta
    GetListIncremental-4     1.19k ± 0%     0.02k ± 0%  -98.49%  (p=0.000 n=10+10)

Fixes #3876 (sadly)

Change-Id: I1195ac5de21a8a8b3cdace5871d263e81aa27e91
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Andrew Dunham 527741d41f shell.nix: add graphviz
Change-Id: Ic25e11a056a7624ebba923d2b87d947e24c41a20
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
2 years ago
License Updater a1a2c165e9 licenses: update win/apple licenses
Signed-off-by: License Updater <noreply@tailscale.com>
2 years ago
Denton Gentry dafc822654 cmd/nginx-auth: increment version.
We need a new release to handle TCD changes
after MagicDNS GA

Updates https://github.com/tailscale/tailscale/issues/6048

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
2 years ago
Maisem Ali 9f39c3b10f ipn/ipnlocal: make EditPrefs strip private keys before returning
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Maisem Ali a2d15924fb types/persist: add PublicNodeKey helper
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Maisem Ali 0957bc5af2 ipn/ipnlocal: use ipn.PrefsView
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Maisem Ali 20324eeebc ipn/prefs: add views
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Andrew Dunham ba459aeef5 net/interfaces: don't call GetList in List.ForeachInterface
It looks like this was left by mistake in 4a3e2842.

Change-Id: Ie4e3d5842548cd2e8533b3552298fb1ce9ba761a
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
2 years ago
Mihai Parparita 660abd7309 cmd/tsconnect: add README to generated NPM package
Makes the landing page at https://www.npmjs.com/package/@tailscale/connect
look slightly nicer.

Fixes #5976

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Denton Gentry 9beb07b4ff scripts/install.sh: add Ubuntu Kinetic Kudu
Signed-off-by: Denton Gentry <dgentry@tailscale.com>
2 years ago
Brad Fitzpatrick 575c599410 portlist: add a test that verifies changes are picked up over time
To avoid annoying firewall dialogs on macOS and Windows, only run it
on Linux by default without the flag.

Change-Id: If8486c31d4243ade54b0131f673237c6c9184c08
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 036f70b7b4 portlist: refactor, introduce OS-specific types
Add an osImpl interface that can be stateful and thus more efficient
between calls. It will later be implemented by all OSes but for now
this change only adds a Linux implementation.

Remove Port.inode. It was only used by Linux and moves into its osImpl.

Don't reopen /proc/net/* files on each run. Turns out you can just
keep then open and seek to the beginning and reread and the contents
are fresh.

    name                    old time/op    new time/op    delta
    GetListIncremental-8    7.29ms ± 2%    6.53ms ± 1%  -10.50%  (p=0.000 n=9+9)

    name                   old alloc/op   new alloc/op   delta
    GetListIncremental-8    1.30kB ±13%    0.70kB ± 5%  -46.38%  (p=0.000 n=9+10)

    name                  old allocs/op  new allocs/op  delta
    GetListIncremental-8      33.2 ±11%      18.0 ± 0%  -45.82%  (p=0.000 n=9+10)

Updates #5958

Change-Id: I4be83463cbd23c2e2fa5d0bdf38560004f53401b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Peter Cai 4597ec1037 net/dnscache: Handle 4-in-6 addresses in DNS responses
On Android, the system resolver can return IPv4 addresses as IPv6-mapped
addresses (i.e. `::ffff:a.b.c.d`). After the switch to `net/netip`
(19008a3), this case is no longer handled and a response like this will
be seen as failure to resolve any IPv4 addresses.

Handle this case by simply calling `Unmap()` on the returned IPs. Fixes #5698.

Signed-off-by: Peter Cai <peter@typeblog.net>
2 years ago
Brad Fitzpatrick 70dde89c34 portlist: add package doc, file comments, move a method to the right file
And respect envknob earlier. NewPoller has one caller and ignores
errors; they just signal ipnlocal to log a warning and not use the
portlist poller.

Change-Id: I4a33af936fe780cca8c7197d4d74ac31a1dc01e3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 774fa72d32 portlist: add BenchmarkGetListIncremental
In contrast to BenchmarkGetList, this new BenchmarkGetListIncremental
acts like what happens in practice, remembering the previous run and
avoiding work that's already been done previously.

Currently:

    BenchmarkGetList
    BenchmarkGetList-8                           100          11011100 ns/op           68411 B/op       2211 allocs/op
    BenchmarkGetList-8                           100          11443410 ns/op           69073 B/op       2223 allocs/op
    BenchmarkGetList-8                           100          11217311 ns/op           68421 B/op       2197 allocs/op
    BenchmarkGetList-8                           100          11035559 ns/op           68801 B/op       2220 allocs/op
    BenchmarkGetList-8                           100          10921596 ns/op           69226 B/op       2225 allocs/op
    BenchmarkGetListIncremental
    BenchmarkGetListIncremental-8                168           7187217 ns/op            1192 B/op         28 allocs/op
    BenchmarkGetListIncremental-8                172           7004525 ns/op            1194 B/op         28 allocs/op
    BenchmarkGetListIncremental-8                162           7235889 ns/op            1221 B/op         29 allocs/op
    BenchmarkGetListIncremental-8                164           7035671 ns/op            1219 B/op         29 allocs/op
    BenchmarkGetListIncremental-8                174           7095448 ns/op            1114 B/op         27 allocs/op

Updates #5958

Change-Id: I1bd5a4b206df4173e2cb8e8a780429d9daa6ef1d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Pontus Leitzler f36ddd9275
words: even the odds with something that reminds you of vacation (#6025)
The cute little salak belongs there. It also evens the odds if tails
start a mutiny against scales. Even though they outnumber scales, they
should still know their place. Behind.

Signed-off-by: Pontus Leitzler <leitzler@gmail.com>
2 years ago
Brad Fitzpatrick 3697609aaa portlist: remove unix.Readlink allocs on Linux
name       old time/op    new time/op    delta
    GetList-8    11.2ms ± 5%    11.1ms ± 3%     ~     (p=0.661 n=10+9)

    name       old alloc/op   new alloc/op   delta
    GetList-8    83.3kB ± 1%    67.4kB ± 1%  -19.05%  (p=0.000 n=10+10)

    name       old allocs/op  new allocs/op  delta
    GetList-8     2.89k ± 2%     2.19k ± 1%  -24.24%  (p=0.000 n=10+10)

(real issue is we're calling this code as much as we are, but easy
enough to make it efficient because it'll still need to be called
sometimes in any case)

Updates #5958

Change-Id: I90c20278d73e80315a840aed1397d24faa308d93
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 7149155b80 portlist: further reduce allocations on Linux
Make Linux parsePorts also an append-style API and attach it to
caller's provided append base memory.

And add a little string intern pool in front of the []byte to string
for inode names.

    name       old time/op    new time/op    delta
    GetList-8    11.1ms ± 4%     9.8ms ± 6%  -11.68%  (p=0.000 n=9+10)

    name       old alloc/op   new alloc/op   delta
    GetList-8    92.8kB ± 2%    79.7kB ± 0%  -14.11%  (p=0.000 n=10+9)

    name       old allocs/op  new allocs/op  delta
    GetList-8     2.94k ± 1%     2.76k ± 0%   -6.16%  (p=0.000 n=10+10)

More coming. (the bulk of the allocations are in addProcesses and
filesystem operations, most of which we should usually be able to
skip)

Updates #5958

Change-Id: I3f0c03646d314a16fef7f8346aefa7d5c96701e7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick def089f9c9 portlist: unexport all Poller fields, removing unused one, rework channels
Poller.C and Poller.c were duplicated for one caller. Add an accessor
returning the receive-only version instead. It'll inline.

Poller.Err was unused. Remove.

Then Poller is opaque.

The channel usage and shutdown was a bit sketchy. Clean it up.

And document some things.

Change-Id: I5669e54f51a6a13492cf5485c83133bda7ea3ce9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 46ce80758d portlist: update some internals to use append-style APIs
In prep for reducing garbage, being able to reuse memory.  So far this
doesn't actually reuse much. This is just changing signatures around.

But some improvement in any case:

    bradfitz@tsdev:~/src/tailscale.com$ ~/go/bin/benchstat before after
    name       old time/op    new time/op    delta
    GetList-8    11.8ms ± 9%     9.9ms ± 3%  -15.98%  (p=0.000 n=10+10)

    name       old alloc/op   new alloc/op   delta
    GetList-8    99.5kB ± 2%    91.9kB ± 0%   -7.62%  (p=0.000 n=9+9)

    name       old allocs/op  new allocs/op  delta
    GetList-8     3.05k ± 1%     2.93k ± 0%   -3.83%  (p=0.000 n=8+9)

More later, once parsers can reuse strings from previous parses.

Updates #5958

Change-Id: I76cd5048246dd24d11c4e263d8bb8041747fb2b0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 67597bfc9e portlist: unexport GetList
It's an internal implementation detail, and I plan to refactor it
for performance (garbage) reasons anyway, so start by hiding it.

Updates #5958

Change-Id: I2c0d1f743d3495c5f798d1d8afc364692cd9d290
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Will Norris 7b745a1a50 .github: run CI on release branches
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Andrew Dunham 74693793be net/netcheck, tailcfg: track whether OS supports IPv6
We had previously added this to the netcheck report in #5087 but never
copied it into the NetInfo struct. Additionally, add it to log lines so
it's visible to support.

Change-Id: Ib6266f7c6aeb2eb2a28922aeafd950fe1bf5627e
Signed-off-by: Andrew Dunham <andrew@tailscale.com>
2 years ago
Maisem Ali 42d9e7171c Makefile: add publishdevimage target
This builds and publishes the tailscale container image for dev testing.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Anton Tolchanov bd47e28638 prober: optionally spread probes over time
By default all probes with the same probe interval that have been added
together will run on a synchronized schedule, which results in spiky
resource usage and potential throttling by third-party systems (for
example, OCSP servers used by the TLS probes).

To address this, prober can now run in "spread" mode that will
introduce a random delay before the first run of each probe.

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Charlotte Brandhorst-Satzkorn adec726fee
words: for charlotte, by charlotte (#6002)
What's better than getting a community request?

A community request from another Charlotte!

Bun and hops!

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Maisem Ali 74637f2c15 wgengine/router: [linux] add before deleting interface addrs
Deleting may temporarily result in no addrs on the interface, which results in
all other rules (like routes) to get dropped by the OS.

I verified this fixes the problem.

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Charlotte Brandhorst-Satzkorn 95f630ced0
words: beaver, the cutest of them all (#6001)
Beavers have scales, on their tails.

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
phirework d13c9cdfb4
wgengine/magicsock: set up pathfinder (#5994)
Sets up new file for separate silent disco goroutine, tentatively named
pathfinder for now.

Updates #540

Co-authored-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Signed-off-by: Jenny Zhang <jz@tailscale.com>
2 years ago
Brad Fitzpatrick deac82231c wgengine/magicsock: add start of alternate send path
During development of silent disco (#540), an alternate send policy
for magicsock that doesn't wake up the radio frequently with
heartbeats, we want the old & new policies to coexist, like we did
previously pre- and post-disco.

We started to do that earlier in 5c42990c2f but only set up the
env+control knob plumbing to set a bool about which path should be
used.

This starts to add a way for the silent disco code to update the send
path from a separate goroutine. (Part of the effort is going to
de-state-machinify the event based soup that is the current disco
code and make it more Go synchronous style.)

So far this does nothing. (It does add an atomic load on each send
but that should be noise in the grand scheme of things, and a even more
rare atomic store of nil on node config changes.)

Baby steps.

Updates #540

Co-authored-by: Jenny Zhang <jz@tailscale.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Anton Tolchanov 69f61dcad8 prober: add a DERP probe manager based on derpprobe
This ensures that each DERP server is probed individually (TLS and STUN)
and also manages per-region mesh probing. Actual probing code has been
copied from cmd/derpprobe.

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Charlotte Brandhorst-Satzkorn f39847aa52
words: double double tails and trouble, scales aflame and puns abubble (#5992)
Months upon months I ponder about this,
Adding new words onto our little lists.
Given our integrity I should not have missed,
Including the creatures from folklore and myth.
Carefully curated, many of them hiss,
Don't forget about the ones hiding in the abyss.
Now they are added, I cannot resist,
Searching for more words for me to enlist.

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Brad Fitzpatrick afce773aae ipn: remove handle.go
It was unused in this repo. The Windows client used it, but it can move there.

Change-Id: I572816fd80cbbf1b8db734879b6280857d5bd2a7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago