Compare commits

...

382 Commits

Author SHA1 Message Date
Aaron Klotz d7a4f9d31c net/dns: ensure multiple hosts with the same IP address are combined into a single HostEntry
This ensures that each line has a unique IP address.

Fixes #11939

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
1 day ago
Jordan Whited 0d6e71df70
cmd/stunstamp: add explicit metric to track timeout events (#12564)
Timeouts could already be identified as NaN values on
stunstamp_derp_stun_rtt_ns, but we can't use NaN effectively with
promql to visualize them. So, this commit adds a timeouts metric that
we can use with rate/delta/etc promql functions.

Updates tailscale/corp#20689

Signed-off-by: Jordan Whited <jordan@tailscale.com>
1 day ago
Kristoffer Dalby dcb0f189cc cmd/proxy-to-grafana: add flag for alternative control server
Fixes #12571

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2 days ago
Brad Fitzpatrick 5ec01bf3ce wgengine/filter: support FilterRules matching on srcIP node caps [capver 100]
See #12542 for background.

Updates #12542

Change-Id: Ida312f700affc00d17681dc7551ee9672eeb1789
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 days ago
Irbe Krumina 07063bc5c7
ssh/tailssh: fix integration test (#12562)
Updates#cleanup

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 days ago
Brad Fitzpatrick fd3efd9bad
control/controlclient: add more Screen Time blocking detection
Updates #9658
Updates #12545

Change-Id: Iec1dad354a75f145567b4055d77b1c1db27c89e2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Co-authored-by: Andrea Gottardo <andrea@gottardo.me>
2 days ago
Keli bd50a3457d
wgengine/filter: add "Accept" TCP log lines to verbose logging (#12525)
Changes "Accept" TCP logs to display in verbose logs only,
and removes lines from default logging behavior.

Updates #12158

Signed-off-by: Keli Velazquez <keli@tailscale.com>
2 days ago
Percy Wegmann 730f0368d0 ssh/tailssh: replace incubator process with su instead of running su as child
This allows the SSH_AUTH_SOCK environment variable to work inside of
su and agent forwarding to succeed.

Fixes #12467

Signed-off-by: Percy Wegmann <percy@tailscale.com>
3 days ago
Andrew Dunham 24976b5bfd cmd/tailscale/cli: actually perform Noise request in 'debug ts2021'
This actually performs a Noise request in the 'debug ts2021' command,
instead of just exiting once we've dialed a connection. This can help
debug certain forms of captive portals and deep packet inspection that
will allow a connection, but will RST the connection when trying to send
data on the post-upgraded TCP connection.

Updates #1634

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I1e46ca9c9a0751c55f16373a6a76cdc24fec1f18
3 days ago
Andrew Dunham 732605f961 control/controlclient: move noiseConn to internal package
So that it can be later used in the 'tailscale debug ts2021' function in
the CLI, to aid in debugging captive portals/WAFs/etc.

Updates #1634

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Iec9423f5e7570f2c2c8218d27fc0902137e73909
3 days ago
Brad Fitzpatrick 0004827681
control/controlhttp: add health warning for macOS filtering blocking Tailscale (#12546)
Updates #9658
Updates #12545

Change-Id: I6612b9b65eb193a1a651e219b5198c7c20ed94e1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Co-authored-by: Andrea Gottardo <andrea@tailscale.com>
3 days ago
Brad Fitzpatrick 1023b2a82c util/deephash: fix test regression on 32-bit
Fix regression from bd93c3067e where I didn't notice the
32-bit test failure was real and not its usual slowness-related
regression. Yay failure blindness.

Updates #12526

Change-Id: I00e33bba697e2cdb61a0d76a71b62406f6c2eeb9
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 days ago
Andrea Gottardo d7619d273b health: fix nil DERPMap dereference panic
Looks like a DERPmap might not be available when we try to get the
name associated with a region ID, and that was causing an intermittent
panic in CI.

Fixes #12534

Change-Id: I4ace53681bf004df46c728cff830b27339254243
Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
3 days ago
Brad Fitzpatrick 25eeafde23 derp: don't verify mesh peers when --verify-clients is set
Updates tailscale/corp#20654

Change-Id: I33c7ca3c7a3c4e492797b73c66eefb699376402c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 days ago
Brad Fitzpatrick 4b39b6f7ce derp: fix fmt verb for nodekeys
It was hex-ifying the String() form of key.NodePublic, which was already hex.
I noticed in some logs:

    "client 6e6f64656b65793a353537353..."

And thought that 6x6x6x6x looked strange. It's "nodekey:" in hex.

Updates tailscale/corp#20844

Change-Id: Ib9f2d63b37e324420b86efaa680668a9b807e465
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 days ago
Brad Fitzpatrick 21460a5b14 tailcfg, wgengine/filter: remove most FilterRule.SrcBits code
The control plane hasn't sent it to clients in ages.

Updates tailscale/corp#20965

Change-Id: I1d71a4b6dd3f75010a05c544ee39827837c30772
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 days ago
Brad Fitzpatrick 162d593514 net/flowtrack: fix, test String method
I meant to do this in the earlier change and had a git fail.

To atone, add a test too while I'm here.

Updates #12486
Updates #12507

Change-Id: I4943b454a2530cb5047636f37136aa2898d2ffc7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 days ago
Brad Fitzpatrick 9e0a5cc551 net/flowtrack: optimize Tuple type for use as map key
This gets UDP filter overhead closer to TCP. Still ~2x, but no longer ~3x.

    goos: darwin
    goarch: arm64
    pkg: tailscale.com/wgengine/filter
                                       │   before    │                after                │
                                       │   sec/op    │   sec/op     vs base                │
    FilterMatch/tcp-not-syn-v4-8         15.43n ± 3%   15.38n ± 5%        ~ (p=0.339 n=10)
    FilterMatch/udp-existing-flow-v4-8   42.45n ± 0%   34.77n ± 1%  -18.08% (p=0.000 n=10)
    geomean                              25.59n        23.12n        -9.65%

Updates #12486

Change-Id: I595cfadcc6b7234604bed9c4dd4261e087c0d4c4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 days ago
Andrea Gottardo d6a8fb20e7
health: include DERP region name in bad derp notifications (#12530)
Fixes tailscale/corp#20971

We added some Warnables for DERP failure situations, but their Text currently spits out the DERP region ID ("10") in the UI, which is super ugly. It would be better to provide the RegionName of the DERP region that is failing. We can do so by storing a reference to the last-known DERP map in the health package whenever we fetch one, and using it when generating the notification text.

This way, the following message...

> Tailscale could not connect to the relay server '10'. The server might be temporarily unavailable, or your Internet connection might be down.

becomes:

> Tailscale could not connect to the 'Seattle' relay server. The server might be temporarily unavailable, or your Internet connection might be down.

which is a lot more user-friendly.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
4 days ago
Andrea Gottardo 8eb15d3d2d
cli/netcheck: fail with output if we time out fetching a derpmap (#12528)
Updates tailscale/corp#20969

Right now, when netcheck starts, it asks tailscaled for a copy of the DERPMap. If it doesn't have one, it makes a HTTPS request to controlplane.tailscale.com to fetch one.

This will always fail if you're on a network with a captive portal actively blocking HTTPS traffic. The code appears to hang entirely because the http.Client doesn't have a Timeout set. It just sits there waiting until the request succeeds or fails.

This adds a timeout of 10 seconds, and logs more details about the status of the HTTPS request.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
4 days ago
Jordan Whited a93173b56a
cmd/xdpderper,derp/xdp: implement mode that drops STUN packets (#12527)
This is useful during maintenance as a method for shedding home client
load.

Updates tailscale/corp#20689

Signed-off-by: Jordan Whited <jordan@tailscale.com>
4 days ago
Andrea Gottardo d55b105dae
health: expose DependsOn to local API via UnhealthyState (#12513)
Updates #4136

Small PR to expose the health Warnables dependencies to the GUI via LocalAPI, so that we can only show warnings for root cause issues, and filter out unnecessary messages before user presentation.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
4 days ago
Brad Fitzpatrick bd93c3067e wgengine/filter/filtertype: make Match.IPProto a view
I noticed we were allocating these every time when they could just
share the same memory. Rather than document ownership, just lock it
down with a view.

I was considering doing all of the fields but decided to just do this
one first as test to see how infectious it became.  Conclusion: not
very.

Updates #cleanup (while working towards tailscale/corp#20514)

Change-Id: I8ce08519de0c9a53f20292adfbecd970fe362de0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 days ago
Flakes Updater bfb775ce62 go.mod.sri: update SRI hash for go.mod changes
Signed-off-by: Flakes Updater <noreply+flakes-updater@tailscale.com>
4 days ago
Tom Proctor 3099323976
cmd/k8s-operator,k8s-operator,go.{mod,sum}: publish proxy status condition for annotated services (#12463)
Adds a new TailscaleProxyReady condition type for use in corev1.Service
conditions.

Also switch our CRDs to use metav1.Condition instead of
ConnectorCondition. The Go structs are seralized identically, but it
updates some descriptions and validation rules. Update k8s
controller-tools and controller-runtime deps to fix the documentation
generation for metav1.Condition so that it excludes comments and
TODOs.

Stop expecting the fake client to populate TypeMeta in tests. See
kubernetes-sigs/controller-runtime#2633 for details of the change.

Finally, make some minor improvements to validation for service hostnames.

Fixes #12216

Co-authored-by: Irbe Krumina <irbe@tailscale.com>
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
4 days ago
Andrew Dunham 45d2f4301f proxymap, various: distinguish between different protocols
Previously, we were registering TCP and UDP connections in the same map,
which could result in erroneously removing a mapping if one of the two
connections completes while the other one is still active.

Add a "proto string" argument to these functions to avoid this.
Additionally, take the "proto" argument in LocalAPI, and plumb that
through from the CLI and add a new LocalClient method.

Updates tailscale/corp#20600

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I35d5efaefdfbf4721e315b8ca123f0c8af9125fb
4 days ago
Aaron Klotz 2cb408f9b1 hostinfo: update Windows hostinfo to include MSIDist registry value
We need to expand our enviornment information to include info about
the Windows store. Thinking about future plans, it would be nice
to include both the packaging mechanism and the distribution mechanism.

In this PR we change packageTypeWindows to check a new registry value
named MSIDist, and concatenate that value to "msi/" when present.

We also remove vestigial NSIS detection.

Updates https://github.com/tailscale/corp/issues/2790

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
4 days ago
James Tucker 87c5ad4c2c derp: add a verifyClients check to the consistency check
Only implemented for the local tailscaled variant for now.

Updates tailscale/corp#20844

Signed-off-by: James Tucker <james@tailscale.com>
5 days ago
Joe Tsai 2db2d04a37
types/logid: add Add method (#12478)
The Add method derives a new ID by adding a signed integer
to the ID, treating it as an unsigned 256-bit big-endian integer.

We also add Less and Compare methods to PrivateID to provide
feature parity with existing methods on PublicID.

Updates tailscale/corp#11038

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
5 days ago
Jordan Whited 315f3d5df1
derp/xdp: fix handling of zero value UDP checksums (#12510)
validate_udp_checksum was previously indeterminate (not zero) at
declaration, and IPv4 zero value UDP checksum packets were being passed
to the kernel.

Updates tailscale/corp#20689

Signed-off-by: Jordan Whited <jordan@tailscale.com>
5 days ago
Irbe Krumina 8cc2738609
cmd/{containerboot,k8s-operator}: store proxy device ID early to help with cleanup for broken proxies (#12425)
* cmd/containerboot: store device ID before setting up proxy routes.

For containerboot instances whose state needs to be stored
in a Kubernetes Secret, we additonally store the device's
ID, FQDN and IPs.
This is used, between other, by the Kubernetes operator,
who uses the ID to delete the device when resources need
cleaning up and writes the FQDN and IPs on various kube
resource statuses for visibility.

This change shifts storing device ID earlier in the proxy setup flow,
to ensure that if proxy routing setup fails,
the device can still be deleted.

Updates tailscale/tailscale#12146

Signed-off-by: Irbe Krumina <irbe@tailscale.com>

* code review feedback

Signed-off-by: Irbe Krumina <irbe@tailscale.com>

---------

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
5 days ago
Andrew Lytvynov 674c998e93
cmd/tailscale/cli: do not allow update --version on macOS (#12508)
We do not support specific version updates or track switching on macOS.
Do not populate the flag to avoid confusion.

Updates #cleanup

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
5 days ago
Andrew Lytvynov be54dde0eb
clientupdate: allow switching from unstable to stable tracks (#12477)
Previously, we would only compare the current version to resolved latest
version for track. When running `tailscale update --track=stable` from
an unstable build, it would almost always fail because the stable
version is "older". But we should support explicitly switching tracks
like that.

Fixes #12347

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
5 days ago
Kristoffer Dalby a1ab7f7c94 client/tailscale: add NodeID to device
Updates tailscale/corp#20514

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
6 days ago
Brad Fitzpatrick 1f6645b19f net/ipset: skip the loop over Prefixes when there's only one
For pprof cosmetic/confusion reasons more than performance, but it
might have tiny speed benefit.

Updates #12486

Change-Id: I40e03714f3afa3a7e7f5e1fa99b81c7e889b91b6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick 20a5f939ba wgengine/filter: add UDP flow benchmark
To show the effects of the flow LRU accounting on e.g. QUIC traffic.

For an open TCP connection:

    BenchmarkFilterMatch/tcp-not-syn-v4-8           66602070                16.74 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           67718179                16.60 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           68403351                16.84 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           66076416                16.87 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           67159012                16.67 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           65009526                16.58 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           66588055                16.62 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           63037071                16.58 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           69124975                21.15 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           54482922                20.41 ns/op

And an open UDP connection:

    BenchmarkFilterMatch/udp-existing-flow-v4-8             25570020                44.09 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             26725958                46.99 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             25936412                47.11 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             25418325                45.99 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             25759848                44.73 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             25212488                46.26 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             25344370                44.55 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             26399372                45.26 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             26274159                47.51 ns/op
    BenchmarkFilterMatch/udp-existing-flow-v4-8             26070472                46.79 ns/op

Updates #12486

Change-Id: Ica4263fb77972cf43db5a2e9433b4429506edfde
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick bf2d13cfa0 net/ipset: return all closures from named wrappers
So profiles show more useful names than just func1, func2, func3, etc.
There will still be func1 on them all, but the symbol before will say
what the lookup type is.

Updates #12486

Change-Id: I910b024a7861394eb83d07f5a899eae338cb1f22
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick 86e0f9b912 net/ipset, wgengine/filter/filtertype: add split-out packages
This moves NewContainsIPFunc from tsaddr to new ipset package.

And wgengine/filter types gets split into wgengine/filter/filtertype,
so netmap (and thus the CLI, etc) doesn't need to bring in ipset,
bart, etc.

Then add a test making sure the CLI deps don't regress.

Updates #1278

Change-Id: Ia246d6d9502bbefbdeacc4aef1bed9c8b24f54d5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick 36b1b4af2f wgengine/filter: split local+logging lookups by IPv4-vs-IPv6
If we already know it's an incoming IPv4 packet, no need to match
against the set of IPv6s and vice versa.

    goos: darwin
    goarch: arm64
    pkg: tailscale.com/wgengine/filter
                                         │   before    │                after                │
                                         │   sec/op    │   sec/op     vs base                │
    FilterMatch/not-local-v4-8             21.40n ± 3%   16.04n ± 1%  -25.09% (p=0.000 n=10)
    FilterMatch/not-local-v6-8             20.75n ± 9%   15.71n ± 0%  -24.31% (p=0.000 n=10)
    FilterMatch/no-match-v4-8              81.37n ± 1%   78.57n ± 3%   -3.43% (p=0.005 n=10)
    FilterMatch/no-match-v6-8              77.73n ± 2%   73.71n ± 3%   -5.18% (p=0.002 n=10)
    FilterMatch/tcp-not-syn-v4-8           21.41n ± 3%   16.86n ± 0%  -21.25% (p=0.000 n=10)
    FilterMatch/tcp-not-syn-v4-no-logs-8   10.04n ± 0%   10.05n ± 0%        ~ (p=0.446 n=10)
    geomean                                29.07n        25.05n       -13.84%

Updates #12486

Change-Id: I70e5024af03893327d26629a994ab2aa9811f4f3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick d4220a76da wgengine/filter: add TCP non-SYN benchmarks
To show performance during heavy flows on established connections.

    BenchmarkFilterMatch/tcp-not-syn-v4-8           52125848                21.46 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           52388781                21.43 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           52916954                21.32 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           52590730                21.43 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-8           53015923                21.32 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-no-logs-8   122795029                9.783 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-no-logs-8   100000000               10.09 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-no-logs-8   120090948                9.747 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-no-logs-8   122350448               10.55 ns/op
    BenchmarkFilterMatch/tcp-not-syn-v4-no-logs-8   122943025                9.813 ns/op

Updates #12486

Change-Id: I8e7c9380bf969ad646851d53f8a4c287717694ea
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick 10e8a2a05c wgengine/filter: fix copy/pasteo in new benchmark's v6 CIDR
I noticed the not-local-v6 numbers were nowhere near the v4 numbers
(they should be identical) and then saw this. It meant the
Addr().Next() wasn't picking an IP that was no longer local, as
assumed.

Updates #12486

Change-Id: I18dfb641f00c74c6252666bc41bd2248df15fadd
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Brad Fitzpatrick 64ac64fb66 net/tsaddr: use bart in NewContainsIPFunc, add tests, benchmarks
NewContainsIPFunc was previously documented as performing poorly if
there were many netip.Prefixes to search over. As such, we never it used it
in such cases.

This updates it to use bart at a certain threshold (over 6 prefixes,
currently), at which point the bart lookup overhead pays off.

This is currently kinda useless because we're not using it. But now we
can and get wins elsewhere. And we can remove the caveat in the docs.

    goos: darwin
    goarch: arm64
    pkg: tailscale.com/net/tsaddr
                                     │    before    │                after                 │
                                     │    sec/op    │    sec/op     vs base                │
    NewContainsIPFunc/empty-8          2.215n ± 11%   2.239n ±  1%   +1.08% (p=0.022 n=10)
    NewContainsIPFunc/cidr-list-1-8    17.44n ±  0%   17.59n ±  6%   +0.89% (p=0.000 n=10)
    NewContainsIPFunc/cidr-list-2-8    27.85n ±  0%   28.13n ±  1%   +1.01% (p=0.000 n=10)
    NewContainsIPFunc/cidr-list-3-8    36.05n ±  0%   36.56n ± 13%   +1.41% (p=0.000 n=10)
    NewContainsIPFunc/cidr-list-4-8    43.73n ±  0%   44.38n ±  1%   +1.50% (p=0.000 n=10)
    NewContainsIPFunc/cidr-list-5-8    51.61n ±  2%   51.75n ±  0%        ~ (p=0.101 n=10)
    NewContainsIPFunc/cidr-list-10-8   95.65n ±  0%   68.92n ±  0%  -27.94% (p=0.000 n=10)
    NewContainsIPFunc/one-ip-8         4.466n ±  0%   4.469n ±  1%        ~ (p=0.491 n=10)
    NewContainsIPFunc/two-ip-8         8.002n ±  1%   7.997n ±  4%        ~ (p=0.697 n=10)
    NewContainsIPFunc/three-ip-8       27.98n ±  1%   27.75n ±  0%   -0.82% (p=0.012 n=10)
    geomean                            19.60n         19.07n         -2.71%

Updates #12486

Change-Id: I2e2320cc4384f875f41721374da536bab995c1ce
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 days ago
Maisem Ali 491483d599 cmd/viewer,type/views: add MapSlice for maps of slices
This abstraction provides a nicer way to work with
maps of slices without having to write out three long type
params.

This also allows it to provide an AsMap implementation which
copies the map and the slices at least.

Updates tailscale/corp#20910

Signed-off-by: Maisem Ali <maisem@tailscale.com>
7 days ago
Brad Fitzpatrick 7574f586aa wgengine/filter: add more benchmarks, make names more explicit
Updates #12486

Change-Id: If2e6d9c70212644eb4a0bc8ec6768512894a646a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 days ago
Brad Fitzpatrick 21ed31e33a wgengine/filter: use NewContainsIPFunc for Srcs matches
NewContainsIPFunc returns a contains matcher optimized for its
input. Use that instead of what this did before, always doing a test
over each of a list of netip.Prefixes.

    goos: darwin
    goarch: arm64
    pkg: tailscale.com/wgengine/filter
                        │   before    │                after                │
                        │   sec/op    │   sec/op     vs base                │
    FilterMatch/file1-8   32.60n ± 1%   18.87n ± 1%  -42.12% (p=0.000 n=10)

Updates #12486

Change-Id: I8f902bc064effb431e5b46751115942104ff6531
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 days ago
Brad Fitzpatrick e2c0d69c9c wgengine/filter: add filter benchmark
Baseline, on 2020 M1 Macbook Pro, on power:

    goos: darwin
    goarch: arm64
    pkg: tailscale.com/wgengine/filter
    BenchmarkFilterMatch/file1-8    34089133                32.79 ns/op
    BenchmarkFilterMatch/file1-8    35423917                32.59 ns/op
    BenchmarkFilterMatch/file1-8    35208598                32.80 ns/op
    BenchmarkFilterMatch/file1-8    35180470                33.39 ns/op
    BenchmarkFilterMatch/file1-8    36671608                32.82 ns/op
    BenchmarkFilterMatch/file1-8    35435991                33.13 ns/op
    BenchmarkFilterMatch/file1-8    34689181                33.29 ns/op
    BenchmarkFilterMatch/file1-8    34786053                32.94 ns/op
    BenchmarkFilterMatch/file1-8    35366235                32.56 ns/op
    BenchmarkFilterMatch/file1-8    35342799                32.47 ns/op

Updates #12486

Change-Id: I8f902bc064effb431e5b46751115942104ff6531
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 days ago
Brad Fitzpatrick 7bc9d453c2 health: fix data race in new warnable code
Fixes #12479

Change-Id: Ice84d5eb12d835eeddf6fc8cc337ea6b4dddcf6c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 week ago
Nick Khyl c32efd9118 various: create a catch-all NRPT rule when "Override local DNS" is enabled on Windows
Without this rule, Windows 8.1 and newer devices issue parallel DNS requests to DNS servers
associated with all network adapters, even when "Override local DNS" is enabled and/or
a Mullvad exit node is being used, resulting in DNS leaks.

This also adds "disable-local-dns-override-via-nrpt" nodeAttr that can be used to disable
the new behavior if needed.

Fixes tailscale/corp#20718

Signed-off-by: Nick Khyl <nickk@tailscale.com>
1 week ago
Aaron Klotz 7354547bd8 util/winutil: update UserProfile to ensure any environment variables in the roaming profile path are expanded
Updates #12383

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
1 week ago
Andrea Gottardo a8ee83e2c5
health: begin work to use structured health warnings instead of strings, pipe changes into ipn.Notify (#12406)
Updates tailscale/tailscale#4136

This PR is the first round of work to move from encoding health warnings as strings and use structured data instead. The current health package revolves around the idea of Subsystems. Each subsystem can have (or not have) a Go error associated with it. The overall health of the backend is given by the concatenation of all these errors.

This PR polishes the concept of Warnable introduced by @bradfitz a few weeks ago. Each Warnable is a component of the backend (for instance, things like 'dns' or 'magicsock' are Warnables). Each Warnable has a unique identifying code. A Warnable is an entity we can warn the user about, by setting (or unsetting) a WarningState for it. Warnables have:

- an identifying Code, so that the GUI can track them as their WarningStates come and go
- a Title, which the GUIs can use to tell the user what component of the backend is broken
- a Text, which is a function that is called with a set of Args to generate a more detailed error message to explain the unhappy state

Additionally, this PR also begins to send Warnables and their WarningStates through LocalAPI to the clients, using ipn.Notify messages. An ipn.Notify is only issued when a warning is added or removed from the Tracker.

In a next PR, we'll get rid of subsystems entirely, and we'll start using structured warnings for all errors affecting the backend functionality.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
1 week ago
Andrea Gottardo e8ca30a5c7
xcode/iOS: support serial number collection via MDM on iOS (#11429)
Fixes tailscale/corp#18366.

This PR provides serial number collection on iOS, by allowing system administrators to pass a `DeviceSerialNumber` MDM key which can be read by the `posture` package in Go.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
1 week ago
Aaron Klotz bd2a6d5386 util/winutil: add UserProfile type for (un)loading user profiles
S4U logons do not automatically load the associated user profile. In this
PR we add UserProfile to handle that part. Windows docs indicate that
we should try to resolve a remote profile path when present, so we attempt
to do so when the local computer is joined to a domain.

Updates #12383

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
1 week ago
Jordan Whited 9189fe007b
cmd/stunc: support user-specified port (#12469)
Updates tailscale/corp#20689

Signed-off-by: Jordan Whited <jordan@tailscale.com>
1 week ago
James Tucker 85ad0c276c tailcfg: update PeerAPIDNS Port value documentation
We do not intend to use this value for feature support communication in
the future, and have applied changes elsewhere that now fix the expected
value.

Updates tailscale/corp#19391
Updates tailscale/corp#20398

Signed-off-by: James Tucker <james@tailscale.com>
1 week ago
Jordan Whited 65888d95c9
derp/xdp,cmd/xdpderper: initial skeleton (#12390)
This commit introduces a userspace program for managing an experimental
eBPF XDP STUN server program. derp/xdp contains the eBPF pseudo-C along
with a Go pkg for loading it and exporting its metrics.
cmd/xdpderper is a package main user of derp/xdp.

Updates tailscale/corp#20689

Signed-off-by: Jordan Whited <jordan@tailscale.com>
1 week ago
Brad Fitzpatrick 6908fb0de3 ipn/localapi,client/tailscale,cmd/derper: add WhoIs lookup by nodekey, use in derper
Fixes #12465

Change-Id: I9b7c87315a3d2b2ecae2b8db9e94b4f5a1eef74a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 week ago
Andrew Dunham 72c8f7700b wgengine/netstack: add test for #12448
This refactors the logic for determining whether a packet should be sent
to the host or not into a function, and then adds tests for it.

Updates #11304
Updates #12448

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ief9afa98eaffae00e21ceb7db073c61b170355e5
1 week ago
Irbe Krumina 88f2d234a4
wgengine/netstack: fix 4via6 subnet routes (#12454)
Fix a bug where, for a subnet router that advertizes
4via6 route, all packets with a source IP matching
the 4via6 address were being sent to the host itself.
Instead, only send to host packets whose destination
address is host's local address.

Fixes tailscale/tailscale#12448

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Co-authored-by: Andrew Dunham <andrew@du.nham.ca>
1 week ago
Brad Fitzpatrick ccdd2e6650 cmd/derper: add a README
Updates tailscale/corp#20844

Change-Id: Ie3ca5dd7f582f4f298339dd3cd2039243c204ef8
Co-authored-by: James Tucker <james@tailscale.com>
Co-authored-by: Maisem Ali <maisem@tailscale.com>
Co-authored-by: Andrew Dunham <andrew@tailscale.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 week ago
Percy Wegmann d7fdc01f7f ssh/tailssh: check IsSELinuxEnforcing in tailscaled process
Checking in the incubator as this used to do fails because
the getenforce command is not on the PATH.

Updates #12442

Signed-off-by: Percy Wegmann <percy@tailscale.com>
1 week ago
Jonathan Nobels 02e3c046aa
net/dns: re-query system resolvers on no-upstream resolver failure on apple platforms (#12398)
Fixes tailscale/corp#20677

On macOS sleep/wake, we're encountering a condition where reconfigure the network
a little bit too quickly - before apple has set the nameservers for our interface.
This results in a persistent condition where we have no upstream resolver and
fail all forwarded DNS queries.

No upstream nameservers is a legitimate configuration, and we have no  (good) way
of determining when Apple is ready - but if we need to forward a query, and we
have no nameservers, then something has gone badly wrong and the network is
very broken.

A simple fix here is to simply inject a netMon event, which will go through the
configuration dance again when we hit the SERVFAIL condition.

Tested by artificially/randomly returning [] for the list of nameservers in the bespoke
ipn-bridge code responsible for getting the nameservers.

Signed-off-by: Jonathan Nobels <jonathan@tailscale.com>
1 week ago
Andrew Dunham d0f1a838a6 net/dnscache: use parent context to perform lookup
As an alterative to #11935 using #12003.

Updates #11935

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I05f643fe812ceeaec5f266e78e3e529cab3a1ac3
1 week ago
Mario Minardi 5f121396e9
VERSION.txt: this is v1.69.0 (#12441)
Signed-off-by: Mario Minardi <mario@tailscale.com>
1 week ago
JunYanBJSS 4c01ce9f43 tsnet: fix error formatting bug
Fixes #12411

Signed-off-by: JunYanBJSS <johnnycocoyan@hotmail.com>
1 week ago
Irbe Krumina f5936d132a
kube: fix typo (#12437)
Updates#cleanup

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Irbe Krumina a95ea31a4e
kube,tailcfg: store parsed recorder tags in a separate field (#12429)
Add an additional RecorderAddrs field to tailscale.com/cap/kubernetes
capability. RecorderAddrs will only be populated by control
with the addresses of any tsrecorder tags set via Recorder.

Updates tailscale/corp#19821

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Aaron Klotz 3511d1f8a2 cmd/tailscaled, net/dns, wgengine/router: start Windows child processes with DETACHED_PROCESS when I/O is being piped
When we're starting child processes on Windows that are CLI programs that
don't need to output to a console, we should pass in DETACHED_PROCESS as a
CreationFlag on SysProcAttr. This prevents the OS from even creating a console
for the child (and paying the associated time/space penalty for new conhost
processes). This is more efficient than letting the OS create the console
window and then subsequently trying to hide it, which we were doing at a few
callsites.

Fixes #12270

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
2 weeks ago
Nick Khyl 4cdc4ed7db net/dns/resolver: return an empty successful response instead of NXDomain when resolving A records for 4via6 domains
As quad-100 is an authoritative server for 4via6 domains, it should always return responses
with a response code of 0 (indicating no error) when resolving records for these domains.
If there's no resource record of the specified type (e.g. A), it should return a response
with an empty answer section rather than NXDomain. Such a response indicates that there
is at least one RR of a different type (e.g., AAAA), suggesting the Windows stub resolver
to look for it.

Fixes tailscale/corp#20767

Signed-off-by: Nick Khyl <nickk@tailscale.com>
2 weeks ago
Maisem Ali 4b6a0c42c8 safesocket: add ConnectContext
This adds a variant for Connect that takes in a context.Context
which allows passing through cancellation etc by the caller.

Updates tailscale/corp#18266

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 weeks ago
Nick Khyl 3672f66c74 tailcfg: bump capver for NodeAttrDisableSplitDNSWhenNoCustomResolvers
Missed in b65221999c.

Updates tailscale/corp#15802

Signed-off-by: Nick Khyl <nickk@tailscale.com>
2 weeks ago
Andrew Dunham 93cd2ab224 util/singleflight: add DoChanContext
This is a variant of DoChan that supports context propagation, such that
the context provided to the inner function will only be canceled when
there are no more waiters for a given key. This can be used to
deduplicate expensive and cancelable calls among multiple callers
safely.

Updates #11935

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ibe1fb67442a854babbc6924fd8437b02cc9e7bcf
2 weeks ago
Irbe Krumina bc53ebd4a0
ipn/{ipnlocal,localapi},net/netkernelconf,client/tailscale,cmd/containerboot: optionally enable UDP GRO forwarding for containers (#12410)
Add a new TS_EXPERIMENTAL_ENABLE_FORWARDING_OPTIMIZATIONS env var
that can be set for tailscale/tailscale container running as
a subnet router or exit node to enable UDP GRO forwarding
for improved performance.
See https://tailscale.com/kb/1320/performance-best-practices#linux-optimizations-for-subnet-routers-and-exit-nodes
This is currently considered an experimental approach;
the configuration support is partially to allow further experimentation
with containerized environments to evaluate the performance
improvements.

Updates tailscale/tailscale#12295

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Irbe Krumina 6f2bae019f
cmd/k8s-nameserver: fix AAAA record query response (#12412)
Return empty response and NOERROR for AAAA record queries
for DNS names for which we have an A record.
This is to allow for callers that might be first sending an AAAA query and then,
if that does not return a response, follow with an A record query.
Previously we were returning NOTIMPL that caused some callers
to potentially not follow with an A record query or misbehave in different ways.

Also return NXDOMAIN for AAAA record queries for names
that we DO NOT have an A record for to ensure that the callers
do not follow up with an A record query.

Returning an empty response and NOERROR is the behaviour
that RFC 4074 recommends:
https://datatracker.ietf.org/doc/html/rfc4074

Updates tailscale/tailscale#12321

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Aaron Klotz df86576989 util/winutil: add AllocateContiguousBuffer and SetNTString helper funcs
AllocateContiguousBuffer is for allocating structs with trailing buffers
containing additional data. It is to be used for various Windows structures
containing pointers to data located immediately after the struct.

SetNTString performs in-place setting of windows.NTString and
windows.NTUnicodeString.

Updates #12383

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
2 weeks ago
Irbe Krumina c3e2b7347b
tailcfg,cmd/k8s-operator,kube: move Kubernetes cap to a location that can be shared with control (#12236)
This PR is in prep of adding logic to control to be able to parse
tailscale.com/cap/kubernetes grants in control:
- moves the type definition of PeerCapabilityKubernetes cap to a location
shared with control.
- update the Kubernetes cap rule definition with fields for granting
kubectl exec session recording capabilities.
- adds a convenience function to produce tailcfg.RawMessage from an
arbitrary cap rule and a test for it.

An example grant defined via ACLs:
"grants": [{
      "src": ["tag:eng"],
      "dst": ["tag:k8s-operator"],
      "app": {
        "tailscale.com/cap/kubernetes": [{
            "recorder": ["tag:my-recorder"]
	    “enforceRecorder”: true
        }],
      },
    }
]
This grant enforces `kubectl exec` sessions from tailnet clients,
matching `tag:eng` via API server proxy matching `tag:k8s-operator`
to be recorded and recording to be sent to a tsrecorder instance,
matching `tag:my-recorder`.

The type needs to be shared with control because we want
control to parse this cap and resolve tags to peer IPs.

Updates tailscale/corp#19821

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Fran Bull ba46495e11 appc: log how many routes are being written
So that we can debug customer problems more easily.

Updates #11008

Signed-off-by: Fran Bull <fran@tailscale.com>
2 weeks ago
Irbe Krumina 807934f00c
cmd/k8s-operator,k8s-operator: allow proxies accept advertized routes. (#12388)
Add a new .spec.tailscale.acceptRoutes field to ProxyClass,
that can be optionally set to true for the proxies to
accept routes advertized by other nodes on tailnet (equivalent of
setting --accept-routes to true).

Updates tailscale/tailscale#12322,tailscale/tailscale#10684

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Irbe Krumina 53d9cac196
k8s-operator/apis/v1alpha1,cmd/k8s-operator/deploy/examples: update DNSConfig description (#11971)
Also removes hardcoded image repo/tag from example DNSConfig resource
as the operator now knows how to default those.

Updates tailscale/tailscale#11019

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Tom Proctor 23e26e589f
cmd/k8s-operator,k8s-opeerator: include Connector's MagicDNS name and tailnet IPs in status (#12359)
Add new fields TailnetIPs and Hostname to Connector Status. These
contain the addresses of the Tailscale node that the operator created
for the Connector to aid debugging.

Fixes #12214

Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
2 weeks ago
Irbe Krumina 3a6d3f1a5b
cmd/k8s-operator,k8s-operator,go.{mod,sum}: make individual proxy images/image pull policies configurable (#11928)
cmd/k8s-operator,k8s-operator,go.{mod,sum}: make individual proxy images/image pull policies configurable

Allow to configure images and image pull policies for individual proxies
via ProxyClass.Spec.StatefulSet.Pod.{TailscaleContainer,TailscaleInitContainer}.Image,
and ProxyClass.Spec.StatefulSet.Pod.{TailscaleContainer,TailscaleInitContainer}.ImagePullPolicy
fields.
Document that we have images in ghcr.io on the relevant Helm chart fields.

Updates tailscale/tailscale#11675

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 weeks ago
Brad Fitzpatrick 916c4db75b net/dns: fix crash in tests
Looks like #12346 as submitted with failing tests.

Updates #12346

Change-Id: I582cd0dfb117686330d935d763d972373c5ae598
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 weeks ago
Adrian Dewhurst 0219317372 ipn/ipnlocal: improve sticky last suggestion
The last suggested exit node needs to be incorporated in the decision
making process when a new suggestion is requested, but currently it is
not quite right: it'll be used if the suggestion code has an error or a
netmap is unavailable, but it won't be used otherwise.

Instead, this makes the last suggestion into a tiebreaker when making a
random selection between equally-good options. If the last suggestion
does not make it to the final selection pool, then a different
suggestion will be made.

Since LocalBackend.SuggestExitNode is back to being a thin shim that
sets up the parameters to suggestExitNode, it no longer needs a test.
Its test was unable to be comprehensive anyway as the code being tested
contains an uncontrolled random number generator.

Updates tailscale/corp#19681

Change-Id: I94ecc9a0d1b622de3df4ef90523f1d3e67b4bfba
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
2 weeks ago
Andrew Lytvynov 7a7e314096
ipn/ipnlocal,clientupdate: allow auto-updates in contaienrs (#12391)
We assume most containers are immutable and don't expect tailscale
running in them to auto-update. But there's no reason to prohibit it
outright.

Ignore the tailnet-wide default auto-update setting in containers, but
allow local users to turn on auto-updates via the CLI.

RELNOTE=Auto-updates are allowed in containers, but ignore the tailnet-wide default.

Fixes #12292

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 weeks ago
Andrea Gottardo b65221999c
tailcfg,net/dns: add controlknob to disable battery split DNS on iOS (#12346)
Updates corp#15802.

Adds the ability for control to disable the recently added change that uses split DNS in more cases on iOS. This will allow us to disable the feature if it leads to regression in production. We plan to remove this knob once we've verified that the feature works properly.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
2 weeks ago
Andrew Dunham e88a5dbc92 various: fix lint warnings
Some lint warnings caught by running 'make lint' locally.

Updates #cleanup

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I1534ed6f2f5e1eb029658906f9d62607dad98ca3
2 weeks ago
Aaron Klotz 34e8820301 util/winutil: add conpty package and helper for building windows.StartupInfoEx
StartupInfoBuilder is a helper for constructing StartupInfoEx structures
featuring proc/thread attribute lists. Calling its setters triggers the
appropriate setting of fields, adjusting flags as necessary, and populating
the proc/thread attribute list as necessary. Currently it supports four
features: setting std handles, setting pseudo-consoles, specifying handles
for inheritance, and specifying jobs.

The conpty package simplifies creation of pseudo-consoles, their associated
pipes, and assignment of the pty to StartupInfoEx proc/thread attributes.

Updates #12383

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
2 weeks ago
Brad Fitzpatrick 8a11a43c28 cmd/derpprobe: support 'local' derpmap to get derp map via LocalAPI
To make it easier for people to monitor their custom DERP fleet.

Updates tailscale/corp#20654

Change-Id: Id8af22936a6d893cc7b6186d298ab794a2672524
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 weeks ago
Jordan Whited 6e106712f6
cmd/stunstamp: support probing multiple ports (#12356)
Updates tailscale/corp#20344

Signed-off-by: Jordan Whited <jordan@tailscale.com>
2 weeks ago
Brad Fitzpatrick 1ca323ac65 net/netcheck: fix probeProto.String result for IPv6 probes
This bug was introduced in e6b84f215 (May 2020) but was only used in
tests when stringifying probeProto values on failure so it wasn't
noticed for a long time.

But then it was moved into non-test code in 8450a18aa (Jun 2024) and I
didn't notice during the code movement that it was wrong. It's still
only used in failure paths in logs, but having wrong/ambiguous
debugging information isn't the best.

Whoops.

Updates tailscale/corp#20654

Change-Id: I296c727ed1c292a04db7b46ecc05c07fc1abc774
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 weeks ago
Brad Fitzpatrick 8450a18aa9 net/netcheck: flesh out some logging in error paths
Updates tailscale/corp#20654

Change-Id: Ie190f956b864985668f79b5b986438bbe07ce905
2 weeks ago
Kristoffer Dalby 95f266f1ce tsweb: add optional on completion callback func
Updates corp#17075

Co-Authored-By: Anton Tolchanov <anton@tailscale.com>
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2 weeks ago
Andrew Lytvynov b8cf852881
go.toolchain.rev: update to go 1.22.4 (#12365)
Updates https://github.com/tailscale/corp/issues/20635

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 weeks ago
Maisem Ali 36e8e8cd64 wgengine/magicsock: use math/rands/v2
Updates #11058

Co-authored-by: James Tucker <james@tailscale.com>
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 weeks ago
Fran Bull 573c8bd8c7 cmd/natc: add --wg-port flag
Updates tailscale/corp#20503

Signed-off-by: Fran Bull <fran@tailscale.com>
2 weeks ago
Maisem Ali 4a8cb1d9f3 all: use math/rand/v2 more
Updates #11058

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 weeks ago
Fran Bull d2d459d442 cmd/natc: add --ignore-destinations flag
Updates tailscale/corp#20503

Signed-off-by: Fran Bull <fran@tailscale.com>
2 weeks ago
Jun 9cdb33e2a4
tsnet: add a new error when HTTPS enabled but MagicDNC Disabled (#12364)
Fixes tailscale#12303

Signed-off-by: Jun <johnnycocoyan@hotmail.com>
2 weeks ago
Jordan Whited cf1e6c6e55
cmd/stunstamp: fix remote write retry (#12348)
Evaluation of remote write errors was using errors.Is() where it should
have been using errors.As().

Updates tailscale/corp#20344

Signed-off-by: Jordan Whited <jordan@tailscale.com>
3 weeks ago
Brad Fitzpatrick 6d3c10579e gokrazy: update breakglass with now-upstreamed ec2 change
This updates breakglass to use the now-upsteamed
https://github.com/gokrazy/breakglass/pull/18 change
so we're not using our fork now.

It also adds a gok wrapper tool, because doing it by hand
was tedious.

Updates #1866

Change-Id: Ifacbf5fbf0e377b3bd95c5f76c18751c2e1af7d7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 weeks ago
Andrew Lytvynov 347e3f3d9a
go.mod,ipn/ipnlocal: update the ACME fork (#12343)
Update our fork of golang.org/x/crypto to pick up a fix for ACME ARI:
3fde5e568a

Fixes #12278

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
3 weeks ago
Irbe Krumina 82576190a7
tailcfg,cmd/k8s-operator: moves tailscale.com/cap/kubernetes peer cap to tailcfg (#12235)
This is done in preparation for adding kubectl
session recording rules to this capability grant that will need to
be unmarshalled by control, so will also need to be
in a shared location.

Updates tailscale/corp#19821

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
3 weeks ago
Andrea Gottardo d636407f14
net/dns: don't set MatchDomains on Apple platforms when no upstream nameservers available (#12334)
This PR addresses a DNS issue on macOS as discussed this morning.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
3 weeks ago
Adrian Dewhurst cf9f507d47 ipn/ipnlocal: only build allowed suggested node list once
Rather than building a new suggested exit node set every time, compute
it once on first use. Currently, syspolicy ensures that values do not
change without a restart anyway.

Since the set is being constructed in a separate func now, the test code
that manipulates syspolicy can live there, and the TestSuggestExitNode
can now run in parallel with other tests because it does not have global
dependencies.

Updates tailscale/corp#19681

Change-Id: Ic4bb40ccc91b671f9e542bd5ba9c96f942081515
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
3 weeks ago
signed-long 1dc3136a24
cmd/k8s-operator: Support image 'repo' or 'repository' keys in helm values file (#12285)
cmd/k8s-operator/deploy/chart: Support image 'repo' or 'repository' keys in helm values

Fixes #12100

Signed-off-by: Michael Long <michaelongdev@gmail.com>
3 weeks ago
Andrew Lytvynov 379e2bf189
ipn/ipnlocal: stop offline auto-updates on shutdown (#12342)
Clean up the updater goroutine on shutdown, in addition to doing that on
backend state change. This fixes a goroutine leak on shutdown in tests.

Updates #cleanup
3 weeks ago
Jordan Whited ba0dd493c8
cmd/stunstamp: validate STUN tx ID in responses (#12339)
Extremely late arriving responses may leak across probing intervals.

Updates tailscale/corp#20344

Signed-off-by: Jordan Whited <jordan@tailscale.com>
3 weeks ago
Andrew Lytvynov bc4c8b65c7
ipn/ipnlocal: periodically run auto-updates when "offline" (#12118)
When the client is disconnected from control for any reason (typically
just turned off), we should still attempt to update if auto-updates are
enabled. This may help users who turn tailscale on infrequently for
accessing resources.

RELNOTE: Apply auto-updates even if the node is down or disconnected
from the coordination server.

Updates #12117

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
3 weeks ago
Maisem Ali 2f2f588c80 cmd/natc: use ListenPacket
Now that tsnet supports it, use it.

Updates tailscale/corp#20503

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 weeks ago
Brad Fitzpatrick e84751217a gokrazy: add prototype Tailscale appliance, build tooling, docs
Updates #1866

Change-Id: I546316cb833bf2919e0d6f55cdc9951f375f165b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 weeks ago
Maisem Ali 0b1a8586eb cmd/natc: initial implementation of a NAT based connector
This adds a new prototype `cmd/natc` which can be used
to expose a services/domains to the tailnet.

It requires the user to specify a set of IPv4 prefixes
from the CGNAT range. It advertises these as normal subnet
routes. It listens for DNS on the first IP of the first range
provided to it.

When it gets a DNS query it allocates an IP for that domain
from the v4 range. Subsequent connections to the assigned IP
are then tcp proxied to the domain.

It is marked as a WIP prototype and requires the use of the
`TAILSCALE_USE_WIP_CODE` env var.

Updates tailscale/corp#20503

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 weeks ago
Maisem Ali 7b193de6b9 tsnet: return net.Listener from s.listen
A `*listener` implements net.Listener which breaks
a test in another repo.

Regressed in 42cfbf427c.

Updates #12182

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 weeks ago
Adrian Dewhurst 3bf2bddbb5 ipn/ipnlocal: improve testability of random node selection
In order to test the sticky last suggestion code, a test was written for
LocalBackend.SuggestExitNode but it contains a random number generator
which makes writing comprehensive tests very difficult. This doesn't
change how the last suggestion works, but it adds some infrastructure to
make that easier in a later PR.

This adds func parameters for the two randomized parts: breaking ties
between DERP regions and breaking ties between nodes. This way tests can
validate the entire list of tied options, rather than expecting a
particular outcome given a particular random seed.

As a result of this, the global random number generator can be used
rather than seeding a local one each time.

In order to see the tied nodes for the location based (i.e. Mullvad)
case, pickWeighted needed to return a slice instead of a single
arbitrary option, so there is a small change in how that works.

Updates tailscale/corp#19681

Change-Id: I83c48a752abdec0f59c58ccfd8bfb3f3f17d0ea8
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
3 weeks ago
Jordan Whited d21c00205d
cmd/stunstamp: implement service to measure DERP STUN RTT (#12241)
stunstamp timestamping includes userspace and SO_TIMESTAMPING kernel
timestamping where available. Measurements are written locally to a
sqlite DB, exposed over an HTTP API, and written to prometheus
via remote-write protocol.

Updates tailscale/corp#20344

Signed-off-by: Jordan Whited <jordan@tailscale.com>
3 weeks ago
License Updater 1fad06429e licenses: update license notices
Signed-off-by: License Updater <noreply+license-updater@tailscale.com>
3 weeks ago
Fran Bull e06862b8d8 appc: log how often routeInfo is stored
So that we have some debugging info if users have trouble with storing
the routeInfo.

Updates #11008

Signed-off-by: Fran Bull <fran@tailscale.com>
3 weeks ago
Adrian Dewhurst db6447ce63 ipn/ipnlocal: simplify suggest exit node tests
This mostly removes a lot of repetition by predefining some nodes and
other data structures, plus adds some helpers for creating Peer entries
in the netmap. Several existing test cases were reworked to ensure
better coverage of edge cases, and several new test cases were added to
handle some additional responsibility that is in (or will be shortly
moving in) suggestExitNode().

Updates tailscale/corp#19681

Change-Id: Ie14c2988d7fd482f7d6a877f78525f7788669b85
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
3 weeks ago
Andrew Dunham ced9a0d413
net/dns: fix typo in OSConfig logging (#12330)
Updates tailscale/corp#20530

Change-Id: I48834a0a5944ed35509c63bdd2830aa34e1bddeb

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
3 weeks ago
Anton Tolchanov 01847e0123 ipn/ipnlocal: discard node keys that have been rotated out
A non-signing node can be allowed to re-sign its new node keys following
key renewal/rotation (e.g. via `tailscale up --force-reauth`). To be
able to do this, node's TLK is written into WrappingPubkey field of the
initial SigDirect signature, signed by a signing node.

The intended use of this field implies that, for each WrappingPubkey, we
typically expect to have at most one active node with a signature
tracing back to that key. Multiple valid signatures referring to the
same WrappingPubkey can occur if a client's state has been cloned, but
it's something we explicitly discourage and don't support:
https://tailscale.com/s/clone

This change propagates rotation details (wrapping public key, a list
of previous node keys that have been rotated out) to netmap processing,
and adds tracking of obsolete node keys that, when found, will get
filtered out.

Updates tailscale/corp#19764

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
3 weeks ago
Maisem Ali 42cfbf427c tsnet,wgengine/netstack: add ListenPacket and tests
This adds a new ListenPacket function on tsnet.Server
which acts mostly like `net.ListenPacket`.

Unlike `Server.Listen`, this requires listening on a
specific IP and does not automatically listen on both
V4 and V6 addresses of the Server when the IP is unspecified.

To test this, it also adds UDP support to tsdial.Dialer.UserDial
and plumbs it through the localapi. Then an associated test
to make sure the UDP functionality works from both sides.

Updates #12182

Signed-off-by: Maisem Ali <maisem@tailscale.com>
3 weeks ago
Andrew Lytvynov bcb55fdeb6
clientupdate: mention when Alpine system upgrade is needed (#12306)
Alpine APK repos are versioned, and contain different package sets.
Older APK releases and repos don't have the latest tailscale package.
When we report "no update available", check whether pkgs.tailscale.com
has a newer tarball release. If it does, it's possible that the system
is on an older Alpine release. Print additional messages to suggest the
user to upgrade their OS.

Fixes #11309

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
3 weeks ago
Irbe Krumina c2a4719e9e
cmd/tailscale/cli: allow 'tailscale up' to succeed if --stateful-filtering is not explicitly set on linux (#12312)
This fixes an issue where, on containerized environments an upgrade
1.66.3 -> 1.66.4 failed with default containerboot configuration.
This was because containerboot by default runs 'tailscale up'
that requires all previously set flags to be explicitly provided
on subsequent runs and we explicitly set --stateful-filtering
to true on 1.66.3, removed that settingon 1.66.4.

Updates tailscale/tailscale#12307

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Co-authored-by: Andrew Lytvynov <awly@tailscale.com>
3 weeks ago
Andrew Dunham 36d0ac6f8e tailcfg: use strings.CutPrefix for CheckTag; add test
Updates #cleanup

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I42eddc7547a6dd50c4d5b2a9fc88a19aac9767aa
3 weeks ago
ChandonPierre 0a5bd63d32
ipn/store/kubestore, cmd/containerboot: allow overriding client api server URL via ENV (#12115)
Updates tailscale/tailscale#11397

Signed-off-by: Chandon Pierre <cpierre@coreweave.com>
3 weeks ago
Irbe Krumina 1ec0273473
docs/k8s: fix subnet router manifests (#12305)
In https://github.com/tailscale/tailscale/pull/11363
I changed the subnet router manifest to run in tun
mode (for performance reasons), but did not
change the security context to give it net_admin,
which is required to for the tailscale socket.

Updates tailscale/tailscale#12083

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
3 weeks ago
Brad Fitzpatrick f227083539 derp: add some guardrails for derpReason metrics getting out of sync
The derp metrics got out of sync in 74eb99aed1 (2023-03).

They were fixed in 0380cbc90d (2024-05).

This adds some further guardrails (atop the previous fix) to make sure
they don't get out of sync again.

Updates #12288

Change-Id: I809061a81f8ff92f45054d0253bc13871fc71634
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 weeks ago
Marwan Sulaiman 7e357e1636 tsweb: rename AccessLogRecord's When to Time
This change makes our access log record more consistent with the
new log/tslog package formatting of "time". Note that we can
change slog itself to call "time" "when" but we're chosing
to make this breaking change to be consistent with the std lib's
defaults.

Updates tailscale/corp#17071

Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
3 weeks ago
Spike Curtis 0380cbc90d
derp: fix dropReason metrics labels (#12288)
Updates #2745
Updates #7552

Signed-off-by: Spike Curtis <spike@coder.com>
3 weeks ago
Anton Tolchanov 32120932a5 cmd/tailscale/cli: print node signature in `tailscale lock status`
- Add current node signature to `ipnstate.NetworkLockStatus`;
- Print current node signature in a human-friendly format as part
  of `tailscale lock status`.

Examples:

```
$ tailscale lock status
Tailnet lock is ENABLED.

This node is accessible under tailnet lock. Node signature:
SigKind: direct
Pubkey: [OTB3a]
KeyID: tlpub:44a0e23cd53a4b8acc02f6732813d8f5ba8b35d02d48bf94c9f1724ebe31c943
WrappingPubkey: tlpub:44a0e23cd53a4b8acc02f6732813d8f5ba8b35d02d48bf94c9f1724ebe31c943

This node's tailnet-lock key: tlpub:44a0e23cd53a4b8acc02f6732813d8f5ba8b35d02d48bf94c9f1724ebe31c943

Trusted signing keys:
	tlpub:44a0e23cd53a4b8acc02f6732813d8f5ba8b35d02d48bf94c9f1724ebe31c943	1	(self)
	tlpub:6fa21d242a202b290de85926ba3893a6861888679a73bc3a43f49539d67c9764	1	(pre-auth key kq3NzejWoS11KTM59)
```

For a node created via a signed auth key:

```
This node is accessible under tailnet lock. Node signature:
SigKind: rotation
Pubkey: [e3nAO]
Nested:
  SigKind: credential
  KeyID: tlpub:6fa21d242a202b290de85926ba3893a6861888679a73bc3a43f49539d67c9764
  WrappingPubkey: tlpub:3623b0412cab0029cb1918806435709b5947ae03554050f20caf66629f21220a
```

For a node that rotated its key a few times:

```
This node is accessible under tailnet lock. Node signature:
SigKind: rotation
Pubkey: [DOzL4]
Nested:
  SigKind: rotation
  Pubkey: [S/9yU]
  Nested:
    SigKind: rotation
    Pubkey: [9E9v4]
    Nested:
      SigKind: direct
      Pubkey: [3QHTJ]
      KeyID: tlpub:44a0e23cd53a4b8acc02f6732813d8f5ba8b35d02d48bf94c9f1724ebe31c943
      WrappingPubkey: tlpub:2faa280025d3aba0884615f710d8c50590b052c01a004c2b4c2c9434702ae9d0
```

Updates tailscale/corp#19764

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
3 weeks ago
Andrew Lytvynov 776a05223b
ipn/ipnlocal: support c2n updates with old systemd versions (#12296)
The `--wait` flag for `systemd-run` was added in systemd 232. While it
is quite old, it doesn't hurt to special-case them and skip the `--wait`
flag. The consequence is that we lose the update command output in logs,
but at least auto-updates will work.

Fixes #12136

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
3 weeks ago
Brad Fitzpatrick 1ea100e2e5 cmd/tailscaled, ipn/conffile: support ec2 user-data config file
Updates #1412
Updates #1866

Change-Id: I4d08fb233b80c2078b3b28ffc18559baabb4a081
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 weeks ago
Brad Fitzpatrick 2d2b62c400 wgengine/router: probe generally-unused "ip" command style lazily
This busybox fwmaskWorks check was added before we moved away from
using the "ip" command to using netlink directly.

So it's now just wasted work (and log spam on Gokrazy) to check the
"ip" command capabilities if we're never going to use it.

Do it lazily instead.

Updates #12277

Change-Id: I8ab9acf64f9c0d8240ce068cb9ec8c0f6b1ecee7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 weeks ago
Brad Fitzpatrick 909a292a8d util/linuxfw: don't try cleaning iptables on gokrazy
It just generates log spam.

Updates #12277

Change-Id: I5f65c0859e86de0a5349f9d26c9805e7c26b9371
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 weeks ago
Walter Poupore 0acb61fbf8
serve.go, tsnet.go: Fix "in in" typo (#12279)
Fixes #cleanup

Signed-off-by: Walter Poupore <walterp@tailscale.com>
3 weeks ago
Andrea Gottardo dd77111462
xcode/iOS: set MatchDomains when no route requires a custom DNS resolver (#10576)
Updates https://github.com/tailscale/corp/issues/15802.

On iOS exclusively, this PR adds logic to use a split DNS configuration in more cases, with the goal of improving battery life. Acting as the global DNS resolver on iOS should be avoided, as it leads to frequent wakes of IPNExtension.

We try to determine if we can have Tailscale only handle DNS queries for resources inside the tailnet, that is, all routes in the DNS configuration do not require a custom resolver (this is the case for app connectors, for instance).

If so, we set all Routes as MatchDomains. This enables a split DNS configuration which will help preserve battery life. Effectively, for the average Tailscale user who only relies on MagicDNS to resolve *.ts.net domains, this means that Tailscale DNS will only be used for those domains.

This PR doesn't affect users with Override Local DNS enabled. For these users, there should be no difference and Tailscale will continue acting as a global DNS resolver.

Signed-off-by: Andrea Gottardo <andrea@tailscale.com>
3 weeks ago
Percy Wegmann 08a9551a73 ssh/tailssh: fall back to using su when no TTY available on Linux
This allows pam authentication to run for ssh sessions, triggering
automation like pam_mkhomedir.

Updates #11854

Signed-off-by: Percy Wegmann <percy@tailscale.com>
3 weeks ago
Claire Wang f1d10c12ac
ipn/ipnlocal: allowed suggested exit nodes policy (#12240)
Updates tailscale/corp#19681

Signed-off-by: Claire Wang <claire@tailscale.com>
4 weeks ago
signed-long 5ad0dad15e
go generate directives reorder for 'make kube-generate-all' (#12210)
Fixes #11980

Signed-off-by: Michael Long <michaelongdev@gmail.com>
4 weeks ago
Irbe Krumina d0d33f257f
cmd/k8s-operator: add a note pointing at ProxyClass (#12246)
Updates tailscale/tailscale#12242

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
4 weeks ago
Andrew Dunham 8e4a29433f util/pool: add package for storing and using a pool of items
This can be used to implement a persistent pool (i.e. one that isn't
cleared like sync.Pool is) of items–e.g. database connections.

Some benchmarks vs. a naive implementation that uses a single map
iteration show a pretty meaningful improvement:

    $ benchstat -col /impl ./bench.txt
    goos: darwin
    goarch: arm64
    pkg: tailscale.com/util/pool
                       │    Pool     │                   map                    │
                       │   sec/op    │     sec/op      vs base                  │
    Pool_AddDelete-10    10.56n ± 2%     15.11n ±  1%    +42.97% (p=0.000 n=10)
    Pool_TakeRandom-10   56.75n ± 4%   1899.50n ± 20%  +3246.84% (p=0.000 n=10)
    geomean              24.49n          169.4n         +591.74%

Updates tailscale/corp#19900

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ie509cb65573c4726cfc3da9a97093e61c216ca18
4 weeks ago
James Tucker 87ee559b6f net/netcheck: apply some polish suggested from #12161
Apply some post-submit code review suggestions.

Updates #12161
Updates tailscale/corp#19106

Signed-off-by: James Tucker <james@tailscale.com>
4 weeks ago
Maisem Ali 9a64c06a20 all: do not depend on the testing package
Discovered while looking for something else.

Updates tailscale/corp#18935

Signed-off-by: Maisem Ali <maisem@tailscale.com>
4 weeks ago
Jordan Whited 4214e5f71b
logtail/backoff: update Backoff.BackOff docs (#12229)
Update #cleanup

Signed-off-by: Jordan Whited <jordan@tailscale.com>
1 month ago
James Tucker 538c2e8f7c tool/gocross: add debug data to CGO builds
We don't build a lot of tools with CGO, but we do build some, and it's
extremely valuable for production services in particular to have symbols
included - for perf and so on.

I tested various other builds that could be affected negatively, in
particular macOS/iOS, but those use split-dwarf already as part of their
build path, and Android which does not currently use gocross.

One binary which is normally 120mb only grew to 123mb, so the trade-off
is definitely worthwhile in context.

Updates tailscale/corp#20296

Signed-off-by: James Tucker <james@tailscale.com>
1 month ago
Brad Fitzpatrick 3c9be07214 cmd/derper: support TXT-mediated unpublished bootstrap DNS rollouts
Updates tailscale/coral#127

Change-Id: I2712c50630d0d1272c30305fa5a1899a19ffacef
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 month ago
Irbe Krumina 72f0f53ed0
cmd/k8s-operator: fix typo (#12217)
Fixes#cleanup

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
1 month ago
James Tucker 9351eec3e1 net/netcheck: remove hairpin probes
Palo Alto reported interpreting hairpin probes as LAND attacks, and the
firewalls may be responding to this by shutting down otherwise in use NAT sessions
prematurely. We don't currently make use of the outcome of the hairpin
probes, and they contribute to other user confusion with e.g. the
AirPort Extreme hairpin session workaround. We decided in response to
remove the whole probe feature as a result.

Updates #188
Updates tailscale/corp#19106
Updates tailscale/corp#19116

Signed-off-by: James Tucker <james@tailscale.com>
1 month ago
Andrew Lytvynov c9179bc261
various: disable stateful filtering by default (#12197)
After some analysis, stateful filtering is only necessary in tailnets
that use `autogroup:danger-all` in `src` in ACLs. And in those cases
users explicitly specify that hosts outside of the tailnet should be
able to reach their nodes. To fix local DNS breakage in containers, we
disable stateful filtering by default.

Updates #12108

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
1 month ago
License Updater 6db1219185 licenses: update license notices
Signed-off-by: License Updater <noreply+license-updater@tailscale.com>
1 month ago
Charlotte Brandhorst-Satzkorn 4f4f317174 api.md: direct TOC links to new publicapi docs location
This change updates the existing api.md TOC links to point at the new
publicapi folder/files. It also removes the body of the docs from the
file, to avoid the docs becoming out of sync.

This change also renames overview.md to readme.md.

Updates tailscale/corp#19526

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
1 month ago
Brad Fitzpatrick 964282d34f ipn,wgengine: remove vestigial Prefs.AllowSingleHosts
It was requested by the first customer 4-5 years ago and only used
for a brief moment of time. We later added netmap visibility trimming
which removes the need for this.

It's been hidden by the CLI for quite some time and never documented
anywhere else.

This keeps the CLI flag, though, out of caution. It just returns an
error if it's set to anything but true (its default).

Fixes #12058

Change-Id: I7514ba572e7b82519b04ed603ff9f3bdbaecfda7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 month ago
Brad Fitzpatrick 1384c24e41 control/controlclient: delete unused Client.Login Oauth2Token field
Updates #12172 (then need to update other repos)

Change-Id: I439f65e0119b09e00da2ef5c7a4f002f93558578
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 month ago
Andrew Dunham 47b3476eb7 util/lru: add Clear method
Updates tailscale/corp#20109

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I751a669251a70f0134dd1540c19b274a97608a93
1 month ago
Charlotte Brandhorst-Satzkorn c56e0c4934
publicapi: include device and user invites API documentation (#12168)
This change includes the device and user invites API docs in the
new publicapi documentation structure.

Updates tailscale/corp#19526

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
1 month ago
Jordan Whited adb7a86559
cmd/stunc: support ipv6 address targets (#12166)
Updates #cleanup

Signed-off-by: Jordan Whited <jordan@tailscale.com>
1 month ago
James Tucker 8d1249550a net/netcheck,wgengine/magicsock: add potential workaround for Palo Alto DIPP misbehavior
Palo Alto firewalls have a typically hard NAT, but also have a mode
called Persistent DIPP that is supposed to provide consistent port
mapping suitable for STUN resolution of public ports. Persistent DIPP
works initially on most Palo Alto firewalls, but some models/software
versions have a bug which this works around.

The bug symptom presents as follows:

- STUN sessions resolve a consistent public IP:port to start with
- Much later netchecks report the same IP:Port for a subset of
  sessions, most often the users active DERP, and/or the port related
  to sustained traffic.
- The broader set of DERPs in a full netcheck will now consistently
  observe a new IP:Port.
- After this point of observation, new inbound connections will only
  succeed to the new IP:Port observed, and existing/old sessions will
  only work to the old binding.

In this patch we now advertise the lowest latency global endpoint
discovered as we always have, but in addition any global endpoints that
are observed more than once in a single netcheck report. This should
provide viable endpoints for potential connection establishment across
a NAT with this behavior.

Updates tailscale/corp#19106

Signed-off-by: James Tucker <james@tailscale.com>
1 month ago
Charlotte Brandhorst-Satzkorn 6831a29f8b
publicapi: create new home for API docs and split into catagory files (#12116)
This change creates a new folder called publicapi that will become the
future home to the Tailscale public API docs.

This change also splits the existing API docs (still located in api.md)
into separate files, for easier reading and contribution.

Updates tailscale/corp#19526

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
1 month ago
Andrea Gottardo e5f67f90a2
xcode: allow ICMP ping relay on macOS + iOS platforms (#12048)
Fixes tailscale/tailscale#10393
Fixes tailscale/corp#15412
Fixes tailscale/corp#19808

On Apple platforms, exit nodes and subnet routers have been unable to relay pings from Tailscale devices to non-Tailscale devices due to sandbox restrictions imposed on our network extensions by Apple. The sandbox prevented the code in netstack.go from spawning the `ping` process which we were using.

Replace that exec call with logic to send an ICMP echo request directly, which appears to work in userspace, and not trigger a sandbox violation in the syslog.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
1 month ago
Percy Wegmann 59848fe14b drive: rewrite LOCK paths
Fixes #12097

Signed-off-by: Percy Wegmann <percy@tailscale.com>
1 month ago
James Tucker 87f00d76c4 tool/gocross: treat empty GOOS/GOARCH as native GOOS/GOARCH
Tracking down the side effect can otherwise be a pain, for example on
Darwin an empty GOOS resulted in CGO being implicitly disabled. The user
intended for `export GOOS=` to act like unset, and while this is a
misunderstanding, the main toolchain would treat it this way.

Fixes tailscale/corp#20059

Signed-off-by: James Tucker <james@tailscale.com>
1 month ago
Irbe Krumina 76c30e014d
cmd/containerboot: warn when an ingress proxy with an IPv4 tailnet address is being created for an IPv6 backend(s) (#12159)
Updates tailscale/tailscale#12156

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
1 month ago
Maisem Ali 8feb4ff5d2 version: add GitCommitTime to Meta
Updates tailscale/corp#1297

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Maisem Ali 359ef61263 Revert "version: add Info func to expose EmbeddedInfo"
This reverts commit e3dec086e6.

Going to reuse Meta instead as that is already exported.

Updates tailscale/corp#1297

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Sonia Appasamy 89947606b2 api.md: document device invite apis
Updates tailscale/corp#18153

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
1 month ago
Sonia Appasamy b094e8c925 api.md: document user invite apis
Updates tailscale/corp#18153

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
1 month ago
Maisem Ali e3dec086e6 version: add Info func to expose EmbeddedInfo
To be used to in a different repo.

Updates tailscale/corp#1297

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Kevin Liang 7f83f9fc83 Net/DNS/Publicdns: update the IPv6 range that we use to recreate route endpoint for control D
In this commit I updated the Ipv6 range we use to generate Control D DOH ip, we were using the NextDNSRanges to generate Control D DOH ip, updated to use the correct range.

Updates: #7946
Signed-off-by: Kevin Liang <kevinliang@tailscale.com>
1 month ago
Brad Fitzpatrick 6877d44965 prober: plumb a now-required netmon to derphttp
Updates #11896

Change-Id: Ie2f9cd024d85b51087d297aa36c14a9b8a2b8129
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 month ago
Maisem Ali 1f51bb6891 net/tstun: do SNAT after filterPacketOutboundToWireGuard
In a configuration where the local node (ip1) has a different IP (ip2)
that it uses to communicate with a peer (ip3) we would do UDP flow
tracking on the `ip2->ip3` tuple. When we receive the response from
the peer `ip3->ip2` we would dnat it back to `ip3->ip1` which would
then not match the flow track state and the packet would get dropped.

To fix this, we should do flow tracking on the `ip1->ip3` tuple instead
of `ip2->ip3` which requires doing SNAT after the running filterPacketOutboundToWireGuard.

Updates tailscale/corp#19971, tailscale/corp#8020

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Andrea Gottardo 60266be298
version: fix macOS uploads by increasing build number prefix (#12134)
Fixes tailscale/corp#19979

A build with version number 275 was uploaded to the App Store without bumping OSS first. The presence of that build is causing any 274.* build to be rejected. To address this, added -1 to the year component, which means new builds will use the 275.* prefix.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
1 month ago
Andrew Dunham c6d42b1093 derp: remove stats goroutine, use a timer
Without changing behaviour, don't create a goroutine per connection that
sits and sleeps, but rather use a timer that wakes up and gathers
statistics on a regular basis.

Fixes #12127

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ibc486447e403070bdc3c2cd8ae340e7d02854f21
1 month ago
Irbe Krumina 7ef2f72135
util/linuxfw: fix IPv6 availability check for nftables (#12009)
* util/linuxfw: fix IPv6 NAT availability check for nftables

When running firewall in nftables mode,
there is no need for a separate NAT availability check
(unlike with iptables, there are no hosts that support nftables, but not IPv6 NAT - see tailscale/tailscale#11353).
This change fixes a firewall NAT availability check that was using the no-longer set ipv6NATAvailable field
by removing the field and using a method that, for nftables, just checks that IPv6 is available.

Updates tailscale/tailscale#12008

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
1 month ago
Brad Fitzpatrick 8aa5c3534d ipn/ipnlocal: simplify authURL vs authURLSticky, remove interact field
The previous LocalBackend & CLI 'up' changes improved some stuff, but
might've been too aggressive in some edge cases.

This simplifies the authURL vs authURLSticky distinction and removes
the interact field, which seemed to just just be about duplicate URL
suppression in IPN bus, back from when the IPN bus was a single client
at a time. This moves that suppression to a different spot.

Fixes #12119
Updates #12028
Updates #12042

Change-Id: I1f8800b1e82ccc1c8a0d7abba559e7404ddf41e4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 month ago
Parker Higgins 7b3e30f391
words: add some fruit with scales (#8460)
Signed-off-by: Parker Higgins <parker@tailscale.com>
1 month ago
Maisem Ali 79b2d425cf types/views: move AsMap to Map from *Map
This was a typo in 2e19790f61.
It should have been on `Map` and not on `*Map` as otherwise
it doesn't allow for chaining like `someView.SomeMap().AsMap()`
and requires first assigning it to a variable.

Updates #typo

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Charlotte Brandhorst-Satzkorn fc1ae97e10
words: I had a feline we were missing some words (#12098)
pspspsps

Updates #tailscale/corp#14698

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
1 month ago
Maisem Ali 486a423716 tsnet: split user facing and backend logging
This adds a new `UserLogf` field to the `Server` struct.
When set this any logs generated by Server are logged using
`UserLogf` and all spammy backend logs are logged to `Logf`.

If it `UserLogf` is unset, we default to `log.Printf` and
if `Logf` is unset we discard all the spammy logs.

Fixes #12094

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Percy Wegmann 7209c4f91e drive: parse depth 1 PROPFIND results to include children in cache
Clients often perform a PROPFIND for the parent directory before
performing PROPFIND for specific children within that directory.
The PROPFIND for the parent directory is usually done at depth 1,
meaning that we already have information for all of the children.
By immediately adding that to the cache, we save a roundtrip to
the remote peer on the PROPFIND for the specific child.

Updates tailscale/corp#19779

Signed-off-by: Percy Wegmann <percy@tailscale.com>
1 month ago
Irbe Krumina d86d1e7601
cmd/k8s-operator,cmd/containerboot,ipn,k8s-operator: turn off stateful filter for egress proxies. (#12075)
Turn off stateful filtering for egress proxies to allow cluster
traffic to be forwarded to tailnet.

Allow configuring stateful filter via tailscaled config file.

Deprecate EXPERIMENTAL_TS_CONFIGFILE_PATH env var and introduce a new
TS_EXPERIMENTAL_VERSIONED_CONFIG env var that can be used to provide
containerboot a directory that should contain one or more
tailscaled config files named cap-<tailscaled-cap-version>.hujson.
Containerboot will pick the one with the newest capability version
that is not newer than its current capability version.

Proxies with this change will not work with older Tailscale
Kubernetes operator versions - users must ensure that
the deployed operator is at the same version or newer (up to
4 version skew) than the proxies.

Updates tailscale/tailscale#12061

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Co-authored-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Claire Wang e070af7414
ipnlocal, magicsock: add more description to storing last suggested exit (#11998)
node related functions
Updates tailscale/corp#19681

Signed-off-by: Claire Wang <claire@tailscale.com>
1 month ago
Andrew Dunham 5708fc0639 wgengine/router: print Docker warning when stateful filtering is enabled
When Docker is detected on the host and stateful filtering is enabled,
Docker containers may be unable to reach Tailscale nodes (depending on
the network settings of a container). Detect Docker when stateful
filtering is enabled and print a health warning to aid users in noticing
this issue.

We avoid printing the warning if the current node isn't advertising any
subnet routes and isn't an exit node, since without one of those being
true, the node wouldn't have the correct AllowedIPs in WireGuard to
allow a Docker container to connect to another Tailscale node anyway.

Updates #12070

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Idef538695f4d101b0ef6f3fb398c0eaafc3ae281
1 month ago
Andrew Dunham 25e32cc3ae util/linuxfw: fix table name in DelStatefulRule
Updates #12061
Follow-up to #12072

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I2ba8c4bff14d93816760ff5eaa1a16f17bad13c1
1 month ago
Maisem Ali 21abb7f402 cmd/tailscale: add missing set flags for linux
We were missing `snat-subnet-routes`, `stateful-filtering`
and `netfilter-mode`. Add those to set too.

Fixes #12061

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 month ago
Anton Tolchanov ac638f32c0 util/linuxfw: fix stateful packet filtering in nftables mode
To match iptables:
b5dbf155b1/util/linuxfw/iptables_runner.go (L536)

Updates #12066

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
1 month ago
Irbe Krumina b5dbf155b1
cmd/k8s-operator: default nameserver image to tailscale/k8s-nameserver:unstable (#11991)
We are now publishing nameserver images to tailscale/k8s-nameserver,
so we can start defaulting the images if users haven't set
them explicitly, same as we already do with proxy images.

The nameserver images are currently only published for unstable
track, so we have to use the static 'unstable' tag.
Once we start publishing to stable, we can make the operator
default to its own tag (because then we'll know that for each
operator tag X there is also a nameserver tag X as we always
cut all images for a given tag.

Updates tailscale/tailscale#10499

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
1 month ago
Andrew Dunham 8f7f9ac17e wgengine/netstack: handle 4via6 routes that are advertised by the same node
Previously, a node that was advertising a 4via6 route wouldn't be able
to make use of that same route; the packet would be delivered to
Tailscale, but since we weren't accepting it in handleLocalPackets, the
packet wouldn't be delivered to netstack and would never hit the 4via6
logic. Let's add that support so that usage of 4via6 is consistent
regardless of where the connection is initiated from.

Updates #11304

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ic28dc2e58080d76100d73b93360f4698605af7cb
2 months ago
Nick O'Neill 7901925ad3
VERSION.txt: this is v1.67.0 (#12063)
Signed-off-by: Nick O'Neill <nick@tailscale.com>
2 months ago
Sonia Appasamy 8130656780 api.md: remove extraneous commas in json examples
Updates #cleanup

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
2 months ago
Anton Tolchanov 6f4a1dc6bf ipn/ipnlocal: fix another read of keyExpired outside mutex
Updates #12039

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 months ago
Brad Fitzpatrick e968b0ecd7 cmd/tailscale,controlclient,ipnlocal: fix 'up', deflake tests more
The CLI's "up" is kinda chaotic and LocalBackend.Start is kinda
chaotic and they both need to be redone/deleted (respectively), but
this fixes some buggy behavior meanwhile. We were previously calling
StartLoginInteractive (to start the controlclient's RegisterRequest)
redundantly in some cases, causing test flakes depending on timing and
up's weird state machine.

We only need to call StartLoginInteractive in the client if Start itself
doesn't. But Start doesn't tell us that. So cheat a bit and a put the
information about whether there's a current NodeKey in the ipn.Status.
It used to be accessible over LocalAPI via GetPrefs as a private key but
we removed that for security. But a bool is fine.

So then only call StartLoginInteractive if that bool is false and don't
do it in the WatchIPNBus loop.

Fixes #12028
Updates #12042

Change-Id: I0923c3f704a9d6afd825a858eb9a63ca7c1df294
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick e5ef35857f ipn/ipnlocal: fix read of keyExpired outside mutex
Fixes #12039

Change-Id: I28c8a282ce12619f17103e9535841f15394ce685
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 21509db121 ipn/ipnlocal, all: plumb health trackers in tests
I saw some panics in CI, like:

    2024-05-08T04:30:25.9553518Z ## WARNING: (non-fatal) nil health.Tracker (being strict in CI):
    2024-05-08T04:30:25.9554043Z goroutine 801 [running]:
    2024-05-08T04:30:25.9554489Z tailscale.com/health.(*Tracker).nil(0x0)
    2024-05-08T04:30:25.9555086Z 	tailscale.com/health/health.go:185 +0x70
    2024-05-08T04:30:25.9555688Z tailscale.com/health.(*Tracker).SetUDP4Unbound(0x0, 0x0)
    2024-05-08T04:30:25.9556373Z 	tailscale.com/health/health.go:532 +0x2f
    2024-05-08T04:30:25.9557296Z tailscale.com/wgengine/magicsock.(*Conn).bindSocket(0xc0003b4808, 0xc0003b4878, {0x1fbca53, 0x4}, 0x0)
    2024-05-08T04:30:25.9558301Z 	tailscale.com/wgengine/magicsock/magicsock.go:2481 +0x12c5
    2024-05-08T04:30:25.9559026Z tailscale.com/wgengine/magicsock.(*Conn).rebind(0xc0003b4808, 0x0)
    2024-05-08T04:30:25.9559874Z 	tailscale.com/wgengine/magicsock/magicsock.go:2510 +0x16f
    2024-05-08T04:30:25.9561038Z tailscale.com/wgengine/magicsock.NewConn({0xc000063c80, 0x0, 0xc000197930, 0xc000197950, 0xc000197960, {0x0, 0x0}, 0xc000197970, 0xc000198ee0, 0x0, ...})
    2024-05-08T04:30:25.9562402Z 	tailscale.com/wgengine/magicsock/magicsock.go:476 +0xd5f
    2024-05-08T04:30:25.9563779Z tailscale.com/wgengine.NewUserspaceEngine(0xc000063c80, {{0x22c8750, 0xc0001976b0}, 0x0, {0x22c3210, 0xc000063c80}, {0x22c31d8, 0x2d3c900}, 0x0, 0x0, ...})
    2024-05-08T04:30:25.9564982Z 	tailscale.com/wgengine/userspace.go:389 +0x159d
    2024-05-08T04:30:25.9565529Z tailscale.com/ipn/ipnlocal.newTestBackend(0xc000358b60)
    2024-05-08T04:30:25.9566086Z 	tailscale.com/ipn/ipnlocal/serve_test.go:675 +0x2a5
    2024-05-08T04:30:25.9566612Z ta

Updates #11874

Change-Id: I3432ed52d670743e532be4642f38dbd6e3763b1b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 727c0d6cfd ipn/ipnserver: close a small race in ipnserver, ~simplify code
There was a small window in ipnserver after we assigned a LocalBackend
to the ipnserver's atomic but before we Start'ed it where our
initalization Start could conflict with API calls from the LocalAPI.

Simplify that a bit and lay out the rules in the docs.

Updates #12028

Change-Id: Ic5f5e4861e26340599184e20e308e709edec68b1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Maisem Ali 32bc596062 ipn/ipnlocal: acquire b.mu once in Start
We used to Lock, Unlock, Lock, Unlock quite a few
times in Start resulting in all sorts of weird race
conditions. Simplify it all and only Lock/Unlock once.

Updates #11649

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Maisem Ali 9380e2dfc6 ipn/ipnlocal: use lockAndGetUnlock in Start
This removes one of the Lock,Unlock,Lock,Unlock at least in
the Start function. Still has 3 more of these.

Updates #11649

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Maisem Ali e1011f1387 ipn/ipnlocal: call SetNetInfoCallback from NewLocalBackend
Instead of calling it from Start everytime, call it from NewLocalBackend
once.

Updates #11649

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Maisem Ali 85b9a6c601 net/netcheck: do not add derps if IPv4/IPv6 is set to "none"
It was documented as such but seems to have been dropped in a
refactor, restore the behavior. This brings down the time it
takes to run a single integration test by 2s which adds up
quite a bit.

Updates tailscale/corp#19786

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Brad Fitzpatrick d7bdd8e2a7 go.toolchain.rev: update to Go 1.22.3
Updates #12044

Change-Id: I4ad16f2bfcec13735cb10713e028b2c5527501ed
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
kari-ts 3c4c9dc1d2
web: use EditPrefs instead of passing UpdatePrefs to starting (#12040)
Web version of https://github.com/tailscale/tailscale-android/pull/370
This allows us to update the prefs rather than creating new prefs

Updates tailscale/tailscale#11731

Signed-off-by: kari-ts <kari@tailscale.com>
2 months ago
Brad Fitzpatrick 80df8ffb85 control/controlclient: early return and outdent some code
I found this too hard to read before.

This is pulled out of #12033 as it's unrelated cleanup in retrospect.

Updates #12028

Change-Id: I727c47e573217e3d1973c5b66a76748139cf79ee
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Andrew Lytvynov 471731771c
ipn/ipnlocal: set default NoStatefulFiltering in ipn.NewPrefs (#12031)
This way the default gets populated on first start, when no existing
state exists to migrate. Also fix `ipn.PrefsFromBytes` to preserve empty
fields, rather than layering `NewPrefs` values on top.

Updates https://github.com/tailscale/corp/issues/19623

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Paul Scott 78fa698fe6 cmd/tailscale/cli/ffcomplete: remove fullstop from ShortHelp
Updates #cleanup

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Maisem Ali 482890b9ed tailcfg: bump capver for using NodeAttrUserDialUseRoutes for DNS
Missed in f62e678df8.

Updates tailscale/corp#18725
Updates #4529

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Maisem Ali af97e7a793 tailcfg,all: add/plumb Node.IsJailed
This adds a new bool that can be sent down from control
to do jailing on the client side. Previously this would
only be done from control by modifying the packet filter
we sent down to clients. This would result in a lot of
additional work/CPU on control, we could instead just
do this on the client. This has always been a TODO which
we keep putting off, might as well do it now.

Updates tailscale/corp#19623

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Maisem Ali e67069550b ipn/ipnlocal,net/tstun,wgengine: create and plumb jailed packet filter
This plumbs a packet filter for jailed nodes through to the
tstun.Wrapper; the filter for a jailed node is equivalent to a "shields
up" filter. Currently a no-op as there is no way for control to
tell the client whether a peer is jailed.

Updates tailscale/corp#19623

Co-authored-by: Andrew Dunham <andrew@du.nham.ca>
Signed-off-by: Maisem Ali <maisem@tailscale.com>
Change-Id: I5ccc5f00e197fde15dd567485b2a99d8254391ad
2 months ago
Nick Khyl f62e678df8 net/dns/resolver, control/controlknobs, tailcfg: use UserDial instead of SystemDial to dial DNS servers
Now that tsdial.Dialer.UserDial has been updated to honor the configured routes
and dial external network addresses without going through Tailscale, while also being
able to dial a node/subnet router on the tailnet, we can start using UserDial to forward
DNS requests. This is primarily needed for DNS over TCP when forwarding requests
to internal DNS servers, but we also update getKnownDoHClientForProvider to use it.

Updates tailscale/corp#18725

Signed-off-by: Nick Khyl <nickk@tailscale.com>
2 months ago
Andrew Lytvynov c28f5767bf
various: implement stateful firewalling on Linux (#12025)
Updates https://github.com/tailscale/corp/issues/19623


Change-Id: I7980e1fb736e234e66fa000d488066466c96ec85

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Co-authored-by: Andrew Dunham <andrew@du.nham.ca>
2 months ago
Maisem Ali 5ef178fdca net/tstun: refactor peerConfig to allow storing more details
This refactors the peerConfig struct to allow storing more
details about a peer and not just the masq addresses. To be
used in a follow up change.

As a side effect, this also makes the DNAT logic on the inbound
packet stricter. Previously it would only match against the packets
dst IP, not it also takes the src IP into consideration. The beahvior
is at parity with the SNAT case.

Updates tailscale/corp#19623

Co-authored-by: Andrew Dunham <andrew@du.nham.ca>
Signed-off-by: Maisem Ali <maisem@tailscale.com>
Change-Id: I5f40802bebbf0f055436eb8824e4511d0052772d
2 months ago
Brad Fitzpatrick f3d2fd22ef cmd/tailscale/cli: don't start WatchIPNBus until after up's initial Start
The CLI "up" command is a historical mess, both on the CLI side and
the LocalBackend side. We're getting closer to cleaning it up, but in
the meantime it was again implicated in flaky tests.

In this case, the background goroutine running WatchIPNBus was very
occasionally running enough to get to its StartLoginInteractive call
before the original goroutine did its Start call. That meant
integration tests were very rarely but sometimes logging in with the
default control plane URL out on the internet
(controlplane.tailscale.com) instead of the localhost control server
for tests.

This also might've affected new Headscale etc users on initial "up".

Fixes #11960
Fixes #11962

Change-Id: I36f8817b69267a99271b5ee78cb7dbf0fcc0bd34
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick aadb8d9d21 ipn/ipnlocal: don't send an empty BrowseToURL w/ WatchIPNBus NotifyInitialState
I noticed this while working on the following fix to #11962.

Updates #11962

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
Change-Id: I4c5894d8899d1ae8c42f54ecfd4d05a4a7ac598c
2 months ago
Brad Fitzpatrick e26f76a1c4 tstest/integration: add more debugging, logs to catch flaky test
Updates #11962

Change-Id: I1ab0db69bdf8d1d535aa2cef434c586311f0fe18
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Nick Khyl caa3d7594f ipn/ipnlocal, net/tsdial: plumb routes into tsdial and use them in UserDial
We'd like to use tsdial.Dialer.UserDial instead of SystemDial for DNS over TCP.
This is primarily necessary to properly dial internal DNS servers accessible
over Tailscale and subnet routes. However, to avoid issues when switching
between Wi-Fi and cellular, we need to ensure that we don't retain connections
to any external addresses on the old interface. Therefore, we need to determine
which dialer to use internally based on the configured routes.

This plumbs routes and localRoutes from router.Config to tsdial.Dialer,
and updates UserDial to use either the peer dialer or the system dialer,
depending on the network address and the configured routes.

Updates tailscale/corp#18725
Fixes #4529

Signed-off-by: Nick Khyl <nickk@tailscale.com>
2 months ago
Brad Fitzpatrick ce8969d82b net/portmapper: add envknob to disable portmapper in localhost integration tests
Updates #11962

Change-Id: I8212cd814985b455d96986de0d4c45f119516cb3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 7e0dd61e61 ipn/ipnlocal, tstest/integration: add panic to catch flaky test in the act
Updates #11962

Change-Id: Ifa24b82f9c76639bfd83278a7c2fe9cf42897bbb
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
License Updater 258b5042fe licenses: update license notices
Signed-off-by: License Updater <noreply+license-updater@tailscale.com>
2 months ago
Brad Fitzpatrick c3c18027c6 all: make more tests pass/skip in airplane mode
Updates tailscale/corp#19786

Change-Id: Iedc6730fe91c627b556bff5325bdbaf7bf79d8e6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Claire Wang 41f2195899
util/syspolicy: add auto exit node related keys (#11996)
Updates tailscale/corp#19681

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Brad Fitzpatrick 1a963342c7 util/set: add Of variant of SetOf that takes variadic parameter
set.Of(1, 2, 3) is prettier than set.SetOf([]int{1, 2, 3}).

I was going to change the signature of SetOf but then I noticed its
name has stutter anyway, so I kept it for compatibility. People can
prefer to use set.Of for new code or slowly migrate.

Also add a lazy Make method, which I often find myself wanting,
without having to resort to uglier mak.Set(&set, k, struct{}{}).

Updates #cleanup

Change-Id: Ic6f3870115334efcbd65e79c437de2ad3edb7625
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Will Norris 80decd83c1 tsweb: remove redundant bumpStartIfNeeded func
Updates #12001

Signed-off-by: Will Norris <will@tailscale.com>
2 months ago
Maisem Ali ed843e643f types/views: add AppendStrings util func
Updates tailscale/corp#19623

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Maisem Ali fd6ba43b97 types/views: remove duplicate SliceContainsFunc
We already have `(Slice[T]).ContainsFunc`.

Updates #cleanup

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Will Norris 46980c9664 tsweb: ensure in-flight requests are always marked as finished
The inflight request tracker only starts recording a new bucket after
the first non-error request. Unfortunately, it's written in such a way
that ONLY successful requests are ever marked as being finished. Once a
bucket has had at least one successful request and begun to be tracked,
all subsequent error cases are never marked finished and always appear
as in-flight.

This change ensures that if a request is recorded has having been
started, we also mark it as finished at the end.

Updates tailscale/corp#19767

Signed-off-by: Will Norris <will@tailscale.com>
2 months ago
Percy Wegmann 817badf9ca ipn/ipnlocal: reuse transport across Taildrive remotes
This prevents us from opening a new connection on each HTTP
request.

Updates #11967

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 2cf764e998 drive: actually cache results on statcache
Updates #11967

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Irbe Krumina 406293682c
cmd/k8s-operator: cleanup runReconciler signature (#11993)
Updates#cleanup

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Claire Wang 35872e86d2
ipnlocal, magicsock: store last suggested exit node id in local backend (#11959)
Updates tailscale/corp#19681

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Brad Fitzpatrick b62cfc430a tstest/integration/testcontrol: fix data race
Noticed in earlier GitHub actions failure.

Fixes #11994

Change-Id: Iba8d753caaa3dacbe2da9171d96c5f99b12e62d7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Andrew Dunham e9505e5432 ipn/ipnlocal: plumb health.Tracker into profileManager constructor
Setting the field after-the-fact wasn't working because we could migrate
prefs on creation, which would set health status for auto updates.

Updates #11986

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I41d79ebd61d64829a3a9e70586ce56f62d24ccfd
2 months ago
Brad Fitzpatrick e42c4396cf net/netcheck: don't spam on ICMP socket permission denied errors
While debugging a failing test in airplane mode on macOS, I noticed
netcheck logspam about ICMP socket creation permission denied errors.

Apparently macOS just can't do those, or at least not in airplane
mode. Not worth spamming about.

Updates #cleanup

Change-Id: I302620cfd3c8eabb25202d7eef040c01bd8a843c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 15fc6cd966 derp/derphttp: fix netcheck HTTPS probes
The netcheck client, when no UDP is available, probes distance using
HTTPS.

Several problems:

* It probes using /derp/latency-check.
* But cmd/derper serves the handler at /derp/probe
* Despite the difference, it work by accident until c8f4dfc8c0
  which made netcheck's probe require a 2xx status code.
* in tests, we only use derphttp.Handler, so the cmd/derper-installed
  mux routes aren't preesnt, so there's no probe. That breaks
  tests in airplane mode. netcheck.Client then reports "unexpected
  HTTP status 426" (Upgrade Required)

This makes derp handle both /derp/probe and /derp/latency-check
equivalently, and in both cmd/derper and derphttp.Handler standalone
modes.

I notice this when wgengine/magicsock TestActiveDiscovery was failing
in airplane mode (no wifi). It still doesn't pass, but it gets
further.

Fixes #11989

Change-Id: I45213d4bd137e0f29aac8bd4a9ac92091065113f
2 months ago
Brad Fitzpatrick 1fe0983f2d cmd/derper,tstest/nettest: skip network-needing test in airplane mode
Not buying wifi on a short flight is a good way to find tests
that require network. Whoops.

Updates #cleanup

Change-Id: Ibe678e9c755d27269ad7206413ffe9971f07d298
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 46f3feae96 ssh/tailssh: plumb health.Tracker in test
In prep for it being required in more places.

Updates #11874

Change-Id: Ib743205fc2a6c6ff3d2c4ed3a2b28cac79156539
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 4fa6cbec27 ssh/tailssh: use ptr.To in test
Updates #cleanup

Change-Id: Ic98ba1b63c8205084b30f59f0ca343788edea5b0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick ee3bd4dbda derp/derphttp, net/netcheck: plumb netmon.Monitor to derp netcheck client
Fixes #11981

Change-Id: I0e15a09f93aefb3cfddbc12d463c1c08b83e09fd
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Percy Wegmann a03cb866b4 drive: use secret token to authenticate access to file server on localhost
This prevents Mark-of-the-Web bypass attacks in case someone visits the
localhost WebDAV server directly.

Fixes tailscale/corp#19592

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 745fb31bd4 drive: use secret token to authenticate access to file server on localhost
This prevents Mark-of-the-Web bypass attacks in case someone visits the
localhost WebDAV server directly.

Fixes tailscale/corp#19592

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 07e783c7be drive: use secret token to authenticate access to file server on localhost
This prevents Mark-of-the-Web bypass attacks in case someone visits the
localhost WebDAV server directly.

Fixes tailscale/corp#19592

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 3349e86c0a drive: use secret token to authenticate access to file server on localhost
This prevents Mark-of-the-Web bypass attacks in case someone visits the
localhost WebDAV server directly.

Fixes tailscale/corp#19592

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 0c11fd978b drive: use secret token to authenticate access to file server on localhost
This prevents Mark-of-the-Web bypass attacks in case someone visits the
localhost WebDAV server directly.

Fixes tailscale/corp#19592

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 9d22ec0ba2 drive: use secret token to authenticate access to file server on localhost
This prevents Mark-of-the-Web bypass attacks in case someone visits the
localhost WebDAV server directly.

Fixes tailscale/corp#19592

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Irbe Krumina cd633a7252
cmd/k8s-operator/deploy,k8s-operator: document that metrics are unstable (#11979)
Updates#11292

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Andrew Dunham f97d0ac994 net/dns/resolver: add better error wrapping
To aid in debugging exactly what's going wrong, instead of the
not-particularly-useful "dns udp query: context deadline exceeded" error
that we currently get.

Updates #3786
Updates #10768
Updates #11620
(etc.)

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I76334bf0681a8a2c72c90700f636c4174931432c
2 months ago
Claire Wang e0287a4b33
wgengine: add exit destination logging enable for wgengine logger (#11952)
Updates tailscale/corp#18625
Co-authored-by: Kevin Liang <kevinliang@tailscale.com>
Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Irbe Krumina 19b31ac9a6
cmd/{k8s-operator,k8s-nameserver},k8s-operator: update nameserver config with records for ingress/egress proxies (#11019)
cmd/k8s-operator: optionally update dnsrecords Configmap with DNS records for proxies.

This commit adds functionality to automatically populate
DNS records for the in-cluster ts.net nameserver
to allow cluster workloads to resolve MagicDNS names
associated with operator's proxies.

The records are created as follows:
* For tailscale Ingress proxies there will be
a record mapping the MagicDNS name of the Ingress
device and each proxy Pod's IP address.
* For cluster egress proxies, configured via
tailscale.com/tailnet-fqdn annotation, there will be
a record for each proxy Pod, mapping
the MagicDNS name of the exposed
tailnet workload to the proxy Pod's IP.

No records will be created for any other proxy types.
Records will only be created if users have configured
the operator to deploy an in-cluster ts.net nameserver
by applying tailscale.com/v1alpha1.DNSConfig.

It is user's responsibility to add the ts.net nameserver
as a stub nameserver for ts.net DNS names.
https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#configuration-of-stub-domain-and-upstream-nameserver-using-coredns
https://cloud.google.com/kubernetes-engine/docs/how-to/kube-dns#upstream_nameservers

See also https://github.com/tailscale/tailscale/pull/11017

Updates tailscale/tailscale#10499

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Maisem Ali a49ed2e145 derp,ipn/ipnlocal: stop calling rand.Seed
It's deprecated and using it gets us the old slow behavior
according to https://go.dev/blog/randv2.

> Having eliminated repeatability of the global output stream, Go 1.20
> was also able to make the global generator scale better in programs
> that don’t call rand.Seed, replacing the Go 1 generator with a very
> cheap per-thread wyrand generator already used inside the Go
> runtime. This removed the global mutex and made the top-level
> functions scale much better. Programs that do call rand.Seed fall
> back to the mutex-protected Go 1 generator.

Updates #7123

Change-Id: Ia5452e66bd16b5457d4b1c290a59294545e13291
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 months ago
Brad Fitzpatrick 96712e10a7 health, ipn/ipnlocal: move more health warning code into health.Tracker
In prep for making health warnings rich objects with metadata rather
than a bunch of strings, start moving it all into the same place.

We'll still ultimately need the stringified form for the CLI and
LocalAPI for compatibility but we'll next convert all these warnings
into Warnables that have severity levels and such, and legacy
stringification will just be something each Warnable thing can do.

Updates #4136

Change-Id: I83e189435daae3664135ed53c98627c66e9e53da
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Andrew Dunham be663c84c1 net/tstun: rename natConfig to peerConfig
So that we can use this for additional, non-NAT configuration without it
being confusing.

Updates #cleanup

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I1658d59c9824217917a94ee76d2d08f0a682986f
2 months ago
Andrew Dunham 10497acc95 net/tstun: refactor natConfig to not be per-family
This was a holdover from the older, pre-BART days and is no longer
necessary.

Updates #cleanup

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I71b892bab1898077767b9ff51cef33d59c08faf8
2 months ago
Andrew Lytvynov 13e1355546
scripts/installer.sh: remove unnecessary escaping in grep (#11950)
Updates #11263

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Percy Wegmann 843afe7c53 ssh/tailssh: add integration test
Updates tailscale/corp#11854

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Jonathan Nobels 45b9aa0d83
net/netmon: remove spammy log statements (#11953)
Updates tailscale/corp#18960

Tests in corp called us using the wrong logging calls.  Removed.
This is logged downstream anyway.

Signed-off-by: Jonathan Nobels <jonathan@tailscale.com>
2 months ago
Paul Scott 4c08410011 cmd/tailscale/cli: set localClient.UseSocketOnly during flag parsing
This configures localClient correctly during flag parsing, so that the --socket
option is effective when generating tab-completion results. For example, the
following would not connect to the system Tailscale for tab-completion results:

    tailscale --socket=/tmp/tailscaled.socket switch <TAB>

Updates #3793

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Paul Scott ba34943133 cmd/tailscale/cli/ffcomplete: omit and clean completion results
Updates #3793

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Jonathan Nobels fa1303d632
net/netmon: swap to swift-derived defaultRoute on macos (#11936)
Updates tailscale/corp#18960

iOS uses Apple's NetworkMonitor to track the default interface and
there's no reason we shouldn't also use this on macOS, for the same
reasons noted in the comments for why this change was made on iOS.

This eliminates the need to load and parse the routing table when
querying the defaultRouter() in almost all cases.

A slight modification here (on both platforms) to fallback to the default
BSD logic in the unhappy-path rather than making assumptions that
may not hold.  If netmon is eventually parsing AF_ROUTE and able
to give a consistently correct answer for the  default interface index,
we can fall back to that and eliminate the Swift dependency.

Signed-off-by: Jonathan Nobels <jonathan@tailscale.com>
2 months ago
Gabe Gorelick de85610be0
cmd/k8s-operator/deploy/chart: allow users to configure additional labels for the operator's Pod via Helm chart values.
cmd/k8s-operator/deploy/chart: allow users to configure additional labels for the operator's Pod via Helm chart values.

Fixes #11947

Signed-off-by: Gabe Gorelick <gabe@hightouch.io>
2 months ago
Percy Wegmann 2648d475d7 drive: don't allow DELETE on read-only shares
Fixes tailscale/corp#19646

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Brad Fitzpatrick 7455e027e9 util/slicesx: add AppendMatching
We had this in a different repo, but moving it here, as this a more
fitting package.

Updates #cleanup

Change-Id: I5fb9b10e465932aeef5841c67deba4d77d473d57
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Andrew Dunham fe009c134e ipn/ipnlocal: reset the dialPlan only when the URL is unchanged
Also, reset it in a few more places (e.g. logout, new blank profiles,
etc.) to avoid a few more cases where a pre-existing dialPlan can cause
a new Headscale server take 10+ seconds to connect.

Updates #11938

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I3095173a5a3d9720507afe4452548491e9e45a3e
2 months ago
Brad Fitzpatrick c47f9303b0 types/views: use slices.Contains{,Func}
Updates #8419

Change-Id: Ib1a9cb3fb425284b7e02684072a4e7a35975f35c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Joe Tsai 5db80cf2d8
syncs: fix AtomicValue for interface kinds (#11943)
If AtomicValue[T] is used with a T that is an interface kind,
then Store may panic if different concret types are ever stored.

Fix this by always wrapping in a concrete type.
Technically, this is only needed if T is an interface kind,
but there is no harm in doing it also for non-interface kinds.

Updates #cleanup

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 months ago
Irbe Krumina 44aa809cb0
cmd/{k8s-nameserver,k8s-operator},k8s-operator: add a kube nameserver, make operator deploy it (#11919)
* cmd/k8s-nameserver,k8s-operator: add a nameserver that can resolve ts.net DNS names in cluster.

Adds a simple nameserver that can respond to A record queries for ts.net DNS names.
It can respond to queries from in-memory records, populated from a ConfigMap
mounted at /config. It dynamically updates its records as the ConfigMap
contents changes.
It will respond with NXDOMAIN to queries for any other record types
(AAAA to be implemented in the future).
It can respond to queries over UDP or TCP. It runs a miekg/dns
DNS server with a single registered handler for ts.net domain names.
Queries for other domain names will be refused.

The intended use of this is:
1) to allow non-tailnet cluster workloads to talk to HTTPS tailnet
services exposed via Tailscale operator egress over HTTPS
2) to allow non-tailnet cluster workloads to talk to workloads in
the same cluster that have been exposed to tailnet over their
MagicDNS names but on their cluster IPs.

DNSConfig CRD can be used to configure
the operator to deploy kube nameserver (./cmd/k8s-nameserver) to cluster.

Updates tailscale/tailscale#10499

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Shaw Drastin 1fe073098c
Reset dial plan when switching profile (#11933)
When switching profile, the server URL can change (e.g.
because of switching to a self-hosted headscale instance).

If it is not reset here, dial plans returned by old
server (e.g. tailscale control server) will be used to
connect to new server (e.g. self-hosted headscale server),
and the register request will be blocked by it until
timeout, leading to very slow profile switches.

Updates #11938 11938

Signed-off-by: Shaw Drastin <showier.drastic0a@icloud.com>
2 months ago
Jordan Whited a47ce618bd
net/tstun: implement env var for disabling UDP GRO on Linux (#11924)
Certain device drivers (e.g. vxlan, geneve) do not properly handle
coalesced UDP packets later in the stack, resulting in packet loss.

Updates #11026

Signed-off-by: Jordan Whited <jordan@tailscale.com>
2 months ago
Mario Minardi ec04c677c0
api.md: add documentation for new split DNS endpoints (#11922)
Add documentation for GET/PATCH/PUT `api/v2/tailnet/<ID>/dns/split-dns`.
These endpoints allow for reading, partially updating, and replacing the
split DNS settings for a given tailnet.

Updates https://github.com/tailscale/corp/issues/19483

Signed-off-by: Mario Minardi <mario@tailscale.com>
2 months ago
Andrew Lytvynov 7ba8f03936
ipn/ipnlocal: fix TestOnTailnetDefaultAutoUpdate on unsupported platforms (#11921)
Fixes #11894

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Irbe Krumina 7d9c3f9897
cmd/k8s-operator/deploy/manifests: check if IPv6 module is loaded before using it (#11867)
Before attempting to enable IPv6 forwarding in the proxy init container
check if the relevant module is found, else the container crashes
on hosts that don't have it.

Updates#11860

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Andrew Lytvynov d02f1be46a
scripts/installer.sh: enable Alpine community repo if needed (#11837)
The tailscale package is in the community Alpine repo. Check if it's
commented out in `/etc/apk/repositories` and run `setup-apkrepos -c -1`
if it's not.

Fixes #11263

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Claire Wang 5254f6de06
tailcfg: add suggest exit node UI node attribute (#11918)
Add node attribute to determine whether or not to show suggested exit
node in UI.
Updates tailscale/corp#19515

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Andrew Lytvynov ce5c80d0fe
clientupdate: exec systemctl instead of using dbus to restart (#11923)
Shell out to "systemctl", which lets us drop an extra dependency.

Updates https://github.com/tailscale/corp/issues/18935

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Fran Bull 6a0fbacc28 appc: setting AdvertiseRoutes explicitly discards app connector routes
This fixes bugs where after using the cli to set AdvertiseRoutes users
were finding that they had to restart tailscaled before the app
connector would advertise previously learned routes again. And seems
more in line with user expectations.

Fixes #11006
Signed-off-by: Fran Bull <fran@tailscale.com>
2 months ago
Fran Bull c27dc1ca31 appc: unadvertise routes when reconfiguring app connector
If the controlknob to persist app connector routes is enabled, when
reconfiguring an app connector unadvertise routes that are no longer
relevant.

Updates #11008
Signed-off-by: Fran Bull <fran@tailscale.com>
2 months ago
Fran Bull fea2e73bc1 appc: write discovered domains to StateStore
If the controlknob is on.
This will allow us to remove discovered routes associated with a
particular domain.

Updates #11008
Signed-off-by: Fran Bull <fran@tailscale.com>
2 months ago
Fran Bull 1bd1b387b2 appc: add flag shouldStoreRoutes and controlknob for it
When an app connector is reconfigured and domains to route are removed,
we would like to no longer advertise routes that were discovered for
those domains. In order to do this we plan to store which routes were
discovered for which domains.

Add a controlknob so that we can enable/disable the new behavior.

Updates #11008
Signed-off-by: Fran Bull <fran@tailscale.com>
2 months ago
Fran Bull 79836e7bfd appc: add RouteInfo struct and persist it to StateStore
Lays the groundwork for the ability to persist app connectors discovered
routes, which will allow us to stop advertising routes for a domain if
the app connector no longer monitors that domain.

Updates #11008
Signed-off-by: Fran Bull <fran@tailscale.com>
2 months ago
Andrew Dunham b2b49cb3d5 wgengine/wgcfg/nmcfg: skip expired peers
Updates tailscale/corp#19315

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I1ad0c8796efe3dd456280e51efaf81f6d2049772
2 months ago
Mario Minardi 74c399483c
api.md: explicitly set content-type headers in POST CURL examples (#11916)
Explicitly set `-H "Content-Type: application/json"` in CURL examples
for POST endpoints as the default content type used by CURL is otherwise
`application/x-www-form-urlencoded` and these endpoints expect JSON data.

Updates https://github.com/tailscale/tailscale/issues/11914

Signed-off-by: Mario Minardi <mario@tailscale.com>
2 months ago
Irbe Krumina 1452faf510
cmd/containerboot,kube,ipn/store/kubestore: allow interactive login on kube, check Secret create perms, allow empty state Secret (#11326)
cmd/containerboot,kube,ipn/store/kubestore: allow interactive login and empty state Secrets, check perms

* Allow users to pre-create empty state Secrets

* Add a fake internal kube client, test functionality that has dependencies on kube client operations.

* Fix an issue where interactive login was not allowed in an edge case where state Secret does not exist

* Make the CheckSecretPermissions method report whether we have permissions to create/patch a Secret if it's determined that these operations will be needed

Updates tailscale/tailscale#11170

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Kristoffer Dalby 1e6cdb7d86 api.md: fix missing links after move of device posture
Updates tailscale/corp#18572

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2 months ago
Brad Fitzpatrick b9adbe2002 net/{interfaces,netmon}, all: merge net/interfaces package into net/netmon
In prep for most of the package funcs in net/interfaces to become
methods in a long-lived netmon.Monitor that can cache things.  (Many
of the funcs are very heavy to call regularly, whereas the long-lived
netmon.Monitor can subscribe to things from the OS and remember
answers to questions it's asked regularly later)

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: Ie4e8dedb70136af2d611b990b865a822cd1797e5
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 6b95219e3a net/netmon, add: add netmon.State type alias of interfaces.State
... in prep for merging the net/interfaces package into net/netmon.

This is a no-op change that updates a bunch of the API signatures ahead of
a future change to actually move things (and remove the type alias)

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: I477613388f09389214db0d77ccf24a65bff2199c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Irbe Krumina 45f0721530
cmd/containerboot: wait on tailscaled process only (#11897)
Modifies containerboot to wait on tailscaled process
only, not on any child process of containerboot.
Waiting on any subprocess was racing with Go's
exec.Cmd.Run, used to run iptables commands and
that starts its own subprocesses and waits on them.

Containerboot itself does not run anything else
except for tailscaled, so there shouldn't be a need
to wait on anything else.

Updates tailscale/tailscale#11593

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Brad Fitzpatrick 3672f29a4e net/netns, net/dns/resolver, etc: make netmon required in most places
The goal is to move more network state accessors to netmon.Monitor
where they can be cheaper/cached. But first (this change and others)
we need to make sure the one netmon.Monitor is plumbed everywhere.

Some notable bits:

* tsdial.NewDialer is added, taking a now-required netmon

* because a tsdial.Dialer always has a netmon, anything taking both
  a Dialer and a NetMon is now redundant; take only the Dialer and
  get the NetMon from that if/when needed.

* netmon.NewStatic is added, primarily for tests

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: I877f9cb87618c4eb037cee098241d18da9c01691
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 4f73a26ea5 ipn/ipnlocal: skip TestOnTailnetDefaultAutoUpdate on macOS for now
While it's broken.

Updates #11894

Change-Id: I24698707ffe405471a14ab2683aea7e836531da8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 7a62dddeac net/netcheck, wgengine/magicsock: make netmon.Monitor required
This has been a TODO for ages. Time to do it.

The goal is to move more network state accessors to netmon.Monitor
where they can be cheaper/cached.

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: I60fc6508cd2d8d079260bda371fc08b6318bcaf1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 4dece0c359 net/netutil: remove a use of deprecated interfaces.GetState
I'm working on moving all network state queries to be on
netmon.Monitor, removing old APIs.

Updates tailscale/corp#10910
Updates tailscale/corp#18960
Updates #7967
Updates #3299

Change-Id: If0de137e0e2e145520f69e258597fb89cf39a2a3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 7f587d0321 health, wgengine/magicsock: remove last of health package globals
Fixes #11874
Updates #4136

Change-Id: Ib70e6831d4c19c32509fe3d7eee4aa0e9f233564
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Jonathan Nobels 71e9258ad9
ipn/ipnlocal: fix null dereference for early suggested exit node queries (#11885)
Fixes tailscale/corp#19558

A request for the suggested exit nodes that occurs too early in the
VPN lifecycle would result in a null deref of the netmap and/or
the netcheck report.  This checks both and errors out.

Signed-off-by: Jonathan Nobels <jonathan@tailscale.com>
2 months ago
Brad Fitzpatrick 745931415c health, all: remove health.Global, finish plumbing health.Tracker
Updates #11874
Updates #4136

Change-Id: I414470f71d90be9889d44c3afd53956d9f26cd61
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick a4a282cd49 control/controlclient: plumb health.Tracker
Updates #11874
Updates #4136

Change-Id: Ia941153bd83523f0c8b56852010f5231d774d91a
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 6d69fc137f ipn/{ipnlocal,localapi},wgengine{,/magicsock}: plumb health.Tracker
Down to 25 health.Global users. After this remains controlclient &
net/dns & wgengine/router.

Updates #11874
Updates #4136

Change-Id: I6dd1856e3d9bf523bdd44b60fb3b8f7501d5dc0d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Irbe Krumina df8f40905b
cmd/k8s-operator,k8s-operator: optionally serve tailscaled metrics on Pod IP (#11699)
Adds a new .spec.metrics field to ProxyClass to allow users to optionally serve
client metrics (tailscaled --debug) on <Pod-IP>:9001.
Metrics cannot currently be enabled for proxies that egress traffic to tailnet
and for Ingress proxies with tailscale.com/experimental-forward-cluster-traffic-via-ingress annotation
(because they currently forward all cluster traffic to their respective backends).

The assumption is that users will want to have these metrics enabled
continuously to be able to monitor proxy behaviour (as opposed to enabling
them temporarily for debugging). Hence we expose them on Pod IP to make it
easier to consume them i.e via Prometheus PodMonitor.

Updates tailscale/tailscale#11292

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Brad Fitzpatrick 723c775dbb tsd, ipnlocal, etc: add tsd.System.HealthTracker, start some plumbing
This adds a health.Tracker to tsd.System, accessible via
a new tsd.System.HealthTracker method.

In the future, that new method will return a tsd.System-specific
HealthTracker, so multiple tsnet.Servers in the same process are
isolated. For now, though, it just always returns the temporary
health.Global value. That permits incremental plumbing over a number
of changes. When the second to last health.Global reference is gone,
then the tsd.System.HealthTracker implementation can return a private
Tracker.

The primary plumbing this does is adding it to LocalBackend and its
dozen and change health calls. A few misc other callers are also
plumbed. Subsequent changes will flesh out other parts of the tree
(magicsock, controlclient, etc).

Updates #11874
Updates #4136

Change-Id: Id51e73cfc8a39110425b6dc19d18b3975eac75ce
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick cb66952a0d health: permit Tracker method calls on nil receiver
In prep for tsd.System Tracker plumbing throughout tailscaled,
defensively permit all methods on Tracker to accept a nil receiver
without crashing, lest I screw something up later. (A health tracking
system that itself causes crashes would be no good.) Methods on nil
receivers should not be called, so a future change will also collect
their stacks (and panic during dev/test), but we should at least not
crash in prod.

This also locks that in with a test using reflect to automatically
call all methods on a nil receiver and check they don't crash.

Updates #11874
Updates #4136

Change-Id: I8e955046ebf370ec8af0c1fb63e5123e6282a9d3
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Chris Palmer 7349b274bd
safeweb: handle mux pattern collisions more generally (#11801)
Fixes #11800

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
2 months ago
Brad Fitzpatrick 5b32264033 health: break Warnable into a global and per-Tracker value halves
Previously it was both metadata about the class of warnable item as
well as the value.

Now it's only metadata and the value is per-Tracker.

Updates #11874
Updates #4136

Change-Id: Ia1ed1b6c95d34bc5aae36cffdb04279e6ba77015
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick ebc552d2e0 health: add Tracker type, in prep for removing global variables
This moves most of the health package global variables to a new
`health.Tracker` type.

But then rather than plumbing the Tracker in tsd.System everywhere,
this only goes halfway and makes one new global Tracker
(`health.Global`) that all the existing callers now use.

A future change will eliminate that global.

Updates #11874
Updates #4136

Change-Id: I6ee27e0b2e35f68cb38fecdb3b2dc4c3f2e09d68
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Claire Wang d5fc52a0f5
tailcfg: add auto exit node attribute (#11871)
Updates tailscale/corp#19515

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Sonia Appasamy 18765cd4f9 release/dist/qnap: omit .qpkg.codesigning files
Updates tailscale/tailscale-qpkg#135

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
2 months ago
Percy Wegmann 955ad12489 ipn/ipnlocal: only show Taildrive peers to which ACLs grant us access
This improves convenience and security.

* Convenience - no need to see nodes that can't share anything with you.
* Security - malicious nodes can't expose shares to peers that aren't
             allowed to access their shares.

Updates tailscale/corp#19432

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Sonia Appasamy 5d4b4ffc3c release/dist/qnap: update perms for tmpDir files
Allows all users to read all files, and .sh/.cgi files to be
executable.

Updates tailscale/tailscale-qpkg#135

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
2 months ago
Lee Briggs 14ac41febc
cmd/k8s-operator,k8s-operator: proxyclass affinity (#11862)
add ability to set affinity rules to proxyclass

Updates#11861

Signed-off-by: Lee Briggs <lee@leebriggs.co.uk>
2 months ago
Anton Tolchanov 31e6bdbc82 ipn/ipnlocal: always stop the engine on auth when key has expired
If seamless key renewal is enabled, we typically do not stop the engine
(deconfigure networking). However, if the node key has expired there is
no point in keeping the connection up, and it might actually prevent
key renewal if auth relies on endpoints routed via app connectors.

Fixes tailscale/corp#5800

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 months ago
Andrea Gottardo 1d3e77f373
util/syspolicy: add ReadStringArray interface (#11857)
Fixes tailscale/corp#19459

This PR adds the ability for users of the syspolicy handler to read string arrays from the MDM solution configured on the system.

Signed-off-by: Andrea Gottardo <andrea@gottardo.me>
2 months ago
Sonia Appasamy 0cce456ee5 release/dist/qnap: use tmp file directory for qpkg building
This change allows for the release/dist/qnap package to be used
outside of the tailscale repo (notably, will be used from corp),
by using an embedded file system for build files which gets
temporarily written to a new folder during qnap build runs.

Without this change, when used from corp, the release/dist/qnap
folder will fail to be found within the corp repo, causing
various steps of the build to fail.

The file renames in this change are to combine the build files
into a /files folder, separated into /scripts and /Tailscale.

Updates tailscale/tailscale-qpkg#135

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
2 months ago
Percy Wegmann c8e912896e wgengine/router: consolidate routes before reconfiguring router for mobile clients
This helps reduce memory pressure on tailnets with large numbers
of routes.

Updates tailscale/corp#19332

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Irbe Krumina add62af7c6
util/linuxfw,go.{mod,sum}: don't log errors when deleting non-existant chains and rules (#11852)
This PR bumps iptables to a newer version that has a function to detect
'NotExists' errors and uses that function to determine whether errors
received on iptables rule and chain clean up are because the rule/chain
does not exist- if so don't log the error.

Updates corp#19336

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Irbe Krumina 3af0f526b8
cmd{containerboot,k8s-operator},util/linuxfw: support ExternalName Services (#11802)
* cmd/containerboot,util/linuxfw: support proxy backends specified by DNS name

Adds support for optionally configuring containerboot to proxy
traffic to backends configured by passing TS_EXPERIMENTAL_DEST_DNS_NAME env var
to containerboot.
Containerboot will periodically (every 10 minutes) attempt to resolve
the DNS name and ensure that all traffic sent to the node's
tailnet IP gets forwarded to the resolved backend IP addresses.

Currently:
- if the firewall mode is iptables, traffic will be load balanced
accross the backend IP addresses using round robin. There are
no health checks for whether the IPs are reachable.
- if the firewall mode is nftables traffic will only be forwarded
to the first IP address in the list. This is to be improved.

* cmd/k8s-operator: support ExternalName Services

 Adds support for exposing endpoints, accessible from within
a cluster to the tailnet via DNS names using ExternalName Services.
This can be done by annotating the ExternalName Service with
tailscale.com/expose: "true" annotation.
The operator will deploy a proxy configured to route tailnet
traffic to the backend IPs that service.spec.externalName
resolves to. The backend IPs must be reachable from the operator's
namespace.

Updates tailscale/tailscale#10606

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
License Updater bf46bff678 licenses: update license notices
Signed-off-by: License Updater <noreply+license-updater@tailscale.com>
2 months ago
Percy Wegmann b7e5122226 util/osuser: add unit test for parseGroupIds
Updates #11682

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Andrew Dunham e985c6e58f ssh/tailssh: try fetching group IDs for user with the 'id' command
Since the tailscaled binaries that we distribute are static and don't
link cgo, we previously wouldn't fetch group IDs that are returned via
NSS. Try shelling out to the 'id' command, similar to how we call
'getent', to detect such cases.

Updates #11682

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I9bdc938bd76c71bc130d44a97cc2233064d64799
2 months ago
Kristoffer Dalby 9779eb6dba api.md: move device posture api to api.md
Updates tailscale/corp#18572

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
2 months ago
Brad Fitzpatrick c07aa2cfed syncs: fix flaky test by deleting the code it tested (Watch)
Fixes #11766

Change-Id: Id5a875aab23eb1b48a57dc379d0cdd42412fd18b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Joe Tsai 63b3c82587
ipn/local: log OS-specific diagnostic information as JSON (#11700)
There is an undocumented 16KiB limit for text log messages.
However, the limit for JSON messages is 256KiB.
Even worse, logging JSON as text results in significant overhead
since each double quote needs to be escaped.

Instead, use logger.Logf.JSON to explicitly log the info as JSON.

We also modify osdiag to return the information as structured data
rather than implicitly have the package log on our behalf.
This gives more control to the caller on how to log.

Updates #7802

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 months ago
Andrew Lytvynov 06502b9048
ipn/ipnlocal: reset auto-updates if unsupported on profile load (#11838)
Prior to
1613b18f82 (diff-314ba0d799f70c8998940903efb541e511f352b39a9eeeae8d475c921d66c2ac),
nodes could set AutoUpdate.Apply=true on unsupported platforms via
`EditPrefs`. Specifically, this affects tailnets where default
auto-updates are on.

Fix up those invalid prefs on profile reload, as a migration.

Updates #11544

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Sonia Appasamy 0a84215036 release/dist/qnap: add qnap target builder
Creates new QNAP builder target, which builds go binaries then uses
docker to build into QNAP packages. Much of the docker/script code
here is pulled over from https://github.com/tailscale/tailscale-qpkg,
with adaptation into our builder structures.

The qnap/Tailscale folder contains static resources needed to build
Tailscale qpkg packages, and is an exact copy of the existing folder
in the tailscale-qpkg repo.

Builds can be run with:
```
sudo ./tool/go run ./cmd/dist build qnap
```

Updates tailscale/tailscale-qpkg#135

Signed-off-by: Sonia Appasamy <sonia@tailscale.com>
2 months ago
Andrew Lytvynov b743b85dad
ipn/ipnlocal,ssh/tailssh: reject c2n /update if SSH conns are active (#11820)
Since we already track active SSH connections, it's not hard to
proactively reject updates until those finish. We attempt to do the same
on the control side, but the detection latency for new connections is in
the minutes, which is not fast enough for common short sessions.

Handle a `force=true` query parameter to override this behavior, so that
control can still trigger an update on a server where some long-running
abandoned SSH session is open.

Updates https://github.com/tailscale/corp/issues/18556

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Brad Fitzpatrick 5100bdeba7 types/persist: remove unused field Persist.Provider
It was only obviously unused after the previous change, c39cde79d.

Updates #19334

Change-Id: I9896d5fa692cb4346c070b4a339d0d12340c18f7
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick c39cde79d2 tailcfg: remove some unused fields from RegisterResponseAuth
Fixes #19334

Change-Id: Id6463f28af23078a7bc25b9280c99d4491bd9651
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 05bfa022f2 tailcfg: pointerify RegisterRequest.Auth, omitemptify RegisterResponseAuth
We were storing server-side lots of:

    "Auth":{"Provider":"","LoginName":"","Oauth2Token":null,"AuthKey":""},

That was about 7% of our total storage of pending RegisterRequest
bodies.

Updates tailscale/corp#19327

Change-Id: Ib73842759a2b303ff5fe4c052a76baea0d68ae7d
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Andrew Dunham 375617c5c8 net/tsdial: assume all connections are affected if no default route is present
If this happens, it results in us pessimistically closing more
connections than might be necessary, but is more correct since we won't
"miss" a change to the default route interface and keep trying to send
data over a nonexistent interface, or one that can't reach the internet.

Updates tailscale/corp#19124

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ia0b8b04cb8cdcb0da0155fd08751c9dccba62c1a
2 months ago
Nick Khyl 9e1c86901b wgengine\router: fix the Tailscale-In firewall rule to work on domain networks
The Network Location Awareness service identifies networks authenticated against
an Active Directory domain and categorizes them as "Domain Authenticated".
This includes the Tailscale network if a Domain Controller is reachable through it.

If a network is categories as NLM_NETWORK_CATEGORY_DOMAIN_AUTHENTICATED,
it is not possible to override its category, and we shouldn't attempt to do so.
Additionally, our Windows Firewall rules should be compatible with both private
and domain networks.

This fixes both issues.

Fixes #11813

Signed-off-by: Nick Khyl <nickk@tailscale.com>
2 months ago
Andrew Lytvynov bff527622d
ipn/ipnlocal,clientupdate: disallow auto-updates in containers (#11814)
Containers are typically immutable and should be updated as a whole (and
not individual packages within). Deny enablement of auto-updates in
containers.

Also, add the missing check in EditPrefs in LocalAPI, to catch cases
like tailnet default auto-updates getting enabled for nodes that don't
support it.

Updates #11544

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Andrew Lytvynov b3fb3bf084
clientupdate: return OS-specific version from LatestTailscaleVersion (#11812)
We don't always have the same latest version for all platforms (like
with 1.64.2 is only Synology+Windows), so we should use the OS-specific
result from pkgs JSON response instead of the main Version field.

Updates #11795

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Irbe Krumina bbe194c80d
cmd/k8s-operator: correctly determine cluster domain (#11512)
Kubernetes cluster domain defaults to 'cluster.local', but can also be customized.
We need to determine cluster domain to set up in-cluster forwarding to our egress proxies.
This was previously hardcoded to 'cluster.local', so was the egress proxies were not usable in clusters with custom domains.
This PR ensures that we attempt to determine the cluster domain by parsing /etc/resolv.conf.
In case the cluster domain cannot be determined from /etc/resolv.conf, we fall back to 'cluster.local'.

Updates tailscale/tailscale#10399,tailscale/tailscale#11445

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Percy Wegmann d16c1293e9 ipn/ipnlocal: remove origin and referer headers from Taildrive requests
peerapi does not want these, but rclone includes them.
Removing them allows rclone to work with Taildrive configured
as a WebDAV remote.

Updates #cleanup

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 94c0403104 ipn/ipnlocal: strip origin and referer headers from Taildrive requests
peerapi does not want these, but rclone includes them.
Stripping them out allows rclone to work with Taildrive configured
as a WebDAV remote.

Updates #cleanup

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Percy Wegmann 787f8c08ec drive: rewrite Location headers
This ensures that MOVE, LOCK and any other verbs that use the Location
header work correctly.

Fixes #11758

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Claire Wang c24f2eee34
tailcfg: rename exit node destination network flow log node attribute (#11779)
Updates tailscale/corp#18625

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
kari-ts 048cb61dd0
interfaces: create android impl (#11784)
-Move Android impl into interfaces_android.go
-Instead of using ip route to get the interface name, use the one passed in by Android (ip route is restricted in Android 13+ per termux/termux-app#2993)

Follow-up will be to do the same for router

Fixes tailscale/corp#19215
Fixes tailscale/corp#19124

Signed-off-by: kari-ts <kari@tailscale.com>
2 months ago
Aaron Klotz 7132b782d4 hostinfo: use Distro field for distinguishing Windows Server builds
Some editions of Windows server share the same build number as their
client counterpart; we must use an additional field found in the OS
version information to distinguish between them.

Even though "Distro" has Linux connotations, it is the most appropriate
hostinfo field. What is Windows Server if not an alternate distribution
of Windows? This PR populates Distro with "Server" when applicable.

Fixes #11785

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
2 months ago
Percy Wegmann 02c6af2a69 cmd/tailscale: clarify Taildrive grants in help text
Fixes #cleanup

Signed-off-by: Percy Wegmann <percy@tailscale.com>
2 months ago
Chris Palmer bdfaef4879
safeweb: allow object-src: self in CSP (#11782)
This change is safe (self is still safe, by
definition), and makes the code match the comment.

Updates #cleanup

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
2 months ago
Andrew Lytvynov e775de3c63
go.mod: bump golang.org/x/net (#11775)
One more place to pick up a fix for
https://pkg.go.dev/vuln/GO-2024-2687.

Updates https://github.com/tailscale/corp/issues/18893

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Adrian Dewhurst c8b0adb382 docs/windows/policy: add missing key expiration warning interval
Fixes #11345

Change-Id: Ib53b639690b77d1b7d857304dca2119f197227ce
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
2 months ago
Brad Fitzpatrick 03d5d1f0f9 wgengine/magicsock: disable portmapper in tunchan-faked tests
Most of the magicsock tests fake the network, simulating packets going
out and coming in. There's no reason to actually hit your router to do
UPnP/NAT-PMP/PCP during in tests. But while debugging thousands of
iterations of tests to deflake some things, I saw it slamming my
router. This stops that.

Updates #11762

Change-Id: I59b9f48f8f5aff1fa16b4935753d786342e87744
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Andrew Lytvynov 22bd506129
ipn/ipnlocal: hold the mutex when in onTailnetDefaultAutoUpdate (#11786)
Turns out, profileManager is not safe for concurrent use and I missed
all the locking infrastructure in LocalBackend, oops.

I was not able to reproduce the race even with `go test -count 100`, but
this seems like an obvious fix.

Fixes #11773

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Chris Palmer 88a7767492
safeweb: set SameSite=Strict, with an option for Lax (#11781)
Fixes #11780

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
2 months ago
dependabot[bot] dd48cad89a build(deps-dev): bump vite from 5.1.4 to 5.1.7 in /client/web
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 5.1.4 to 5.1.7.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v5.1.7/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v5.1.7/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2 months ago
Andrew Dunham b85c2b2313 net/dns/resolver: use SystemDial in DoH forwarder
This ensures that we close the underlying connection(s) when a major
link change happens. If we don't do this, on mobile platforms switching
between WiFi and cellular can result in leftover connections in the
http.Client's connection pool which are bound to the "wrong" interface.

Updates #10821
Updates tailscale/corp#19124

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ibd51ce2efcaf4bd68e14f6fdeded61d4e99f9a01
2 months ago
Paul Scott 82394debb7 cmd/tailscale: add shell tab-completion
The approach is lifted from cobra: `tailscale completion bash` emits a bash
script for configuring the shell's autocomplete:

    . <( tailscale completion bash )

so that typing:

    tailscale st<TAB>

invokes:

    tailscale completion __complete -- st

RELNOTE=tailscale CLI now supports shell tab-completion

Fixes #3793

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Brad Fitzpatrick 21a0fe1b9b ipn/store: omit AWS & Kubernetes support on 'small' Linux GOARCHes
This removes AWS and Kubernetes support from Linux binaries by default
on GOARCH values where people don't typically run on AWS or use
Kubernetes, such as 32-bit mips CPUs.

It primarily focuses on optimizing for the static binaries we
distribute. But for people building it themselves, they can set
ts_kube or ts_aws (the opposite of ts_omit_kube or ts_omit_aws) to
force it back on.

Makes tailscaled binary ~2.3MB (~7%) smaller.

Updates #7272, #10627 etc

Change-Id: I42a8775119ce006fa321462cb2d28bc985d1c146
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
dependabot[bot] 449be38e03
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 (#11410)
* build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0

Bumps google.golang.org/protobuf from 1.32.0 to 1.33.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* cmd/{derper,stund}: update depaware.txt

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Lytvynov <awly@tailscale.com>
2 months ago
Irbe Krumina 3ef7f895c8
go.{mod,sum}: bump nftables to the latest commit (#11772)
Updates#deps

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Andrew Dunham 226486eb9a net/interfaces: handle removed interfaces in State.Equal
This wasn't previously handling the case where an interface in s2 was
removed and not present in s1, and would cause the Equal method to
incorrectly return that the states were equal.

Updates tailscale/corp#19124

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I3af22bc631015d1ddd0a1d01bfdf312161b9532d
2 months ago
Paul Scott 454a03a766 cmd/tailscale/cli: prepend "tailscale" to usage errors
Updates #11626

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Paul Scott d07ede461a cmd/tailscale/cli: fix "subcommand required" errors when typod
Fixes #11672

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Paul Scott 3ff3445e9d cmd/tailscale/cli: improve ShortHelp/ShortUsage unit test, fix new errors
Updates #11364

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Paul Scott eb34b8a173 cmd/tailscale/cli: remove explicit usageFunc - its default
Updates #cleanup

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Paul Scott a50e4e604e cmd/tailscale/cli: remove duplicate "tailscale " in drive subcmd usage
Updates #cleanup

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Paul Scott 62d4be873d cmd/tailscale/cli: fix drive --help usage identation
Updates #cleanup

Signed-off-by: Paul Scott <paul@tailscale.com>
2 months ago
Brad Fitzpatrick 7c1d6e35a5 all: use Go 1.22 range-over-int
Updates #11058

Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 068db1f972 net/interfaces: delete unused unexported function
It should've been deleted in 11ece02f52.

Updates #9040

Change-Id: If8a136bdb6c82804af658c9d2b0a8c63ce02d509
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Jonathan Nobels 7e2b4268d6
ipn/{localapi, ipnlocal}: forget the prior exit node when localAPI is used to zero the ExitNodeID (#11681)
Updates tailscale/corp#18724

When localAPI clients directly set ExitNodeID to "", the expected behaviour is that the prior exit node also gets zero'd - effectively setting the UI state back to 'no exit node was ever selected'

The IntenalExitNodePrior has been changed to be a non-opaque type, as it is read by the UI to render the users last selected exit node, and must be concrete. Future-us can either break this, or deprecate it and replace it with something more interesting.

Signed-off-by: Jonathan Nobels <jonathan@tailscale.com>
2 months ago
Brad Fitzpatrick 0fba9e7570 cmd/tailscale/cli: prevent concurrent Start calls in 'up'
Seems to deflake tstest/integration tests. I can't reproduce it
anymore on one of my VMs that was consistently flaking after a dozen
runs before. Now I can run hundreds of times.

Updates #11649
Fixes #7036

Change-Id: I2f7d4ae97500d507bdd78af9e92cd1242e8e44b8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Irbe Krumina 26f9bbc02b
cmd/k8s-operator,k8s-operator: document tailscale.com Custom Resource Definitions better. (#11665)
Updates tailscale/tailscale#10880

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Adrian Dewhurst ca5cb41b43 tailcfg: document use of CapMap for peers
Updates tailscale/corp#17516
Updates #11508

Change-Id: Iad2dafb38ffb9948bc2f3dfaf9c268f7d772cf56
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
2 months ago
Brad Fitzpatrick 3c1e2bba5b ipn/ipnlocal: remove outdated iOS hacky workaround in Start
We haven't needed this hack for quite some time Andrea says.

Updates #11649

Change-Id: Ie854b7edd0a01e92495669daa466c7c0d57e7438
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick dd6c76ea24 ipn: remove unused Options.LegacyMigrationPrefs
I'm on a mission to simplify LocalBackend.Start and its locking
and deflake some tests.

I noticed this hasn't been used since March 2023 when it was removed
from the Windows client in corp 66be796d33c.

So, delete.

Updates #11649

Change-Id: I40f2cb75fb3f43baf23558007655f65a8ec5e1b2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 7ec0dc3834 ipn/ipnlocal: make StartLoginInteractive take (yet unused) context
In prep for future fix to undermentioned issue.

Updates tailscale/tailscale#7036

Change-Id: Ide114db917dcba43719482ffded6a9a54630d99e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Claire Wang 9171b217ba
cmd/tailscale, ipn/ipnlocal: add suggest exit node CLI option (#11407)
Updates tailscale/corp#17516

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Charlotte Brandhorst-Satzkorn 449f46c207
wgengine/magicsock: rebind/restun if a syscall.EPERM error is returned (#11711)
We have seen in macOS client logs that the "operation not permitted", a
syscall.EPERM error, is being returned when traffic is attempted to be
sent. This may be caused by security software on the client.

This change will perform a rebind and restun if we receive a
syscall.EPERM error on clients running darwin. Rebinds will only be
called if we haven't performed one specifically for an EPERM error in
the past 5 seconds.

Updates #11710

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 months ago
Will Norris 14c8b674ea Revert "licenses: add gliderlabs/ssh license"
The gliderlabs/ssh license is actually already included in the standard
package listing.  I'm not sure why I thought it wasn't.

Updates tailscale/corp#5780

This reverts commit 11dca08e93.

Signed-off-by: Will Norris <will@tailscale.com>
2 months ago
Brad Fitzpatrick 952e06aa46 wgengine/router: don't attempt route cleanup on Synology
Trying to run iptables/nftables on Synology pauses for minutes with
lots of errors and ultimately does nothing as it's not used and we
lack permissions.

This fixes a regression from db760d0bac (#11601) that landed
between Synology testing on unstable 1.63.110 and 1.64.0 being cut.

Fixes #11737

Change-Id: Iaf9563363b8e45319a9b6fe94c8d5ffaecc9ccef
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Irbe Krumina 38fb23f120
cmd/k8s-operator,k8s-operator: allow users to configure proxy env vars via ProxyClass (#11743)
Adds new ProxyClass.spec.statefulSet.pod.{tailscaleContainer,tailscaleInitContainer}.Env field
that allow users to provide key, value pairs that will be set as env vars for the respective containers.
Allow overriding all containerboot env vars,
but warn that this is not supported and might break (in docs + a warning when validating ProxyClass).

Updates tailscale/tailscale#10709

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
2 months ago
Brad Fitzpatrick 9258bcc360 Makefile: fix default SYNO_ARCH in Makefile
It was broken with the move to dist in 32e0ba5e68 which doesn't accept
amd64 anymore.

Updates #cleanup

Change-Id: Iaaaba2d73c6a09a226934fe8e5c18b16731ee7a6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick b9aa7421d6 ipn/ipnlocal: remove some dead code (legacyBackend methods) from LocalBackend
Nothing used it.

Updates #11649

Change-Id: Ic1c331d947974cd7d4738ff3aafe9c498853689e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick a6739c49df paths: set default state path on AIX
Updates #11361

Change-Id: I196727a540be6b7c75303f9958490b1d76189fd6
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 271cfdb3d3 util/syspolicy: clean up doc grammar and consistency
Updates #cleanup

Change-Id: I912574cbd5ef4d8b7417b8b2a9b9a2ccfef88840
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick bad3159b62 ipn/ipnlocal: delete useless SetControlClientGetterForTesting use
Updates #11649

Change-Id: I56c069b9c97bd3e30ff87ec6655ec57e1698427c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 8186cd0349 ipn/ipnlocal: delete redundant TestStatusWithoutPeers
We have tstest/integration nowadays.

And this test was one of the lone holdouts using the to-be-nuked
SetControlClientGetterForTesting.

Updates #11649

Change-Id: Icf8a6a2e9b8ae1ac534754afa898c00dc0b7623b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 68043a17c2 ipn/ipnlocal: centralize assignments to cc + ccAuto in new method
cc vs ccAuto is a mess. It needs to go. But this is a baby step towards
getting there.

Updates #11649

Change-Id: I34f33934844e580bd823a7d8f2b945cf26c87b3b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 970b1e21d0 ipn/ipnlocal: inline assertClientLocked into its now sole caller
Updates #11649

Change-Id: I8e2a5e59125a0cad5c0a8c9ed8930585f1735d03
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Brad Fitzpatrick 170c618483 ipn/ipnlocal: remove dead code now that Android uses LocalAPI instead
The new Android app and its libtailscale don't use this anymore;
it uses LocalAPI like other clients now.

Updates #11649

Change-Id: Ic9f42b41e0e0280b82294329093dc6c275f41d50
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
Flakes Updater 65f215115f go.mod.sri: update SRI hash for go.mod changes
Signed-off-by: Flakes Updater <noreply+flakes-updater@tailscale.com>
2 months ago
Brad Fitzpatrick a1abd12f35 cmd/tailscaled, net/tstun: build for aix/ppc64
At least in userspace-networking mode.

Fixes #11361

Change-Id: I78d33f0f7e05fe9e9ee95b97c99b593f8fe498f2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 months ago
kari-ts 1cd51f95c7
ipnlocal: enable allow LAN for android (#11709)
Updates tailscale/corp#18984
Updates tailscale/corp#18202
2 months ago
Claire Wang 976d3c7b5f
tailcfg: add exit destination for network flow logs node attribute (#11698)
Updates tailscale/corp#18625

Signed-off-by: Claire Wang <claire@tailscale.com>
2 months ago
Joe Tsai 7a77a2edf1
logtail: optimize JSON processing (#11671)
Changes made:

* Avoid "encoding/json" for JSON processing, and instead use
"github.com/go-json-experiment/json/jsontext".
Use jsontext.Value.IsValid for validation, which is much faster.
Use jsontext.AppendQuote instead of our own JSON escaping.

* In drainPending, use a different maxLen depending on lowMem.
In lowMem mode, it is better to perform multiple uploads
than it is to construct a large body that OOMs the process.

* In drainPending, if an error is encountered draining,
construct an error message in the logtail JSON format
rather than something that is invalid JSON.

* In appendTextOrJSONLocked, use jsontext.Decoder to check
whether the input is a valid JSON object. This is faster than
the previous approach of unmarshaling into map[string]any and
then re-marshaling that data structure.
This is especially beneficial for network flow logging,
which produces relatively large JSON objects.

* In appendTextOrJSONLocked, enforce maxSize on the input.
If too large, then we may end up in a situation where the logs
can never be uploaded because it exceeds the maximum body size
that the Tailscale logs service accepts.

* Use "tailscale.com/util/truncate" to properly truncate a string
on valid UTF-8 boundaries.

* In general, remove unnecessary spaces in JSON output.

Performance:

    name       old time/op    new time/op    delta
    WriteText     776ns ± 2%     596ns ± 1%   -23.24%  (p=0.000 n=10+10)
    WriteJSON     110µs ± 0%       9µs ± 0%   -91.77%  (p=0.000 n=8+8)

    name       old alloc/op   new alloc/op   delta
    WriteText      448B ± 0%        0B       -100.00%  (p=0.000 n=10+10)
    WriteJSON    37.9kB ± 0%     0.0kB ± 0%   -99.87%  (p=0.000 n=10+10)

    name       old allocs/op  new allocs/op  delta
    WriteText      1.00 ± 0%      0.00       -100.00%  (p=0.000 n=10+10)
    WriteJSON     1.08k ± 0%     0.00k ± 0%   -99.91%  (p=0.000 n=10+10)

For text payloads, this is 1.30x faster.
For JSON payloads, this is 12.2x faster.

Updates #cleanup
Updates tailscale/corp#18514

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 months ago
Aaron Klotz 4d5d669cd5 net/dns: unconditionally write NRPT rules to local settings
We were being too aggressive when deciding whether to write our NRPT rules
to the local registry key or the group policy registry key.

After once again reviewing the document which calls itself a spec
(see issue), it is clear that the presence of the DnsPolicyConfig subkey
is the important part, not the presence of values set in the DNSClient
subkey. Furthermore, a footnote indicates that the presence of
DnsPolicyConfig in the GPO key will always override its counterpart in
the local key. The implication of this is important: we may unconditionally
write our NRPT rules to the local key. We copy our rules to the policy
key only when it contains NRPT rules belonging to somebody other than us.

Fixes https://github.com/tailscale/corp/issues/19071

Signed-off-by: Aaron Klotz <aaron@tailscale.com>
2 months ago
License Updater 9d021579e7 licenses: update license notices
Signed-off-by: License Updater <noreply+license-updater@tailscale.com>
2 months ago
Will Norris 11dca08e93 licenses: add gliderlabs/ssh license
This package is included in the tempfork directory, rather than as a go
module dependency, so is not included in the normal package list.

Updates tailscale/corp#5780

Signed-off-by: Will Norris <will@tailscale.com>
2 months ago
Jenny Zhang 2207643312 VERSION.txt: this is v1.65.0
Signed-off-by: Jenny Zhang <jz@tailscale.com>
2 months ago
Jenny Zhang 09524b58f3 VERSION.txt: this is v1.64.0
Signed-off-by: Jenny Zhang <jz@tailscale.com>
2 months ago

@ -32,7 +32,6 @@ jobs:
- "ubuntu:18.04"
- "ubuntu:20.04"
- "ubuntu:22.04"
- "ubuntu:22.10"
- "ubuntu:23.04"
- "elementary/docker:stable"
- "elementary/docker:unstable"
@ -91,7 +90,10 @@ jobs:
|| contains(matrix.image, 'parrotsec')
|| contains(matrix.image, 'kalilinux')
- name: checkout
uses: actions/checkout@v4
# We cannot use v4, as it requires a newer glibc version than some of the
# tested images provide. See
# https://github.com/actions/checkout/issues/1487
uses: actions/checkout@v3
- name: run installer
run: scripts/installer.sh
# Package installation can fail in docker because systemd is not running

@ -0,0 +1,23 @@
# Run the ssh integration tests with `make sshintegrationtest`.
# These tests can also be running locally.
name: "ssh-integrationtest"
concurrency:
group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
cancel-in-progress: true
on:
pull_request:
paths:
- "ssh/**"
- "tempfork/gliderlabs/ssh/**"
- ".github/workflows/ssh-integrationtest"
jobs:
ssh-integrationtest:
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Run SSH integration tests
run: |
make sshintegrationtest

@ -194,7 +194,7 @@ jobs:
- name: chown
run: chown -R $(id -u):$(id -g) $PWD
- name: privileged tests
run: ./tool/go test ./util/linuxfw
run: ./tool/go test ./util/linuxfw ./derp/xdp
vm:
runs-on: ["self-hosted", "linux", "vm"]
@ -254,9 +254,6 @@ jobs:
goarch: amd64
- goos: openbsd
goarch: amd64
# Plan9 (disabled until 3p dependencies are fixed)
# - goos: plan9
# goarch: amd64
runs-on: ubuntu-22.04
steps:
@ -305,6 +302,47 @@ jobs:
GOOS: ios
GOARCH: arm64
crossmin: # cross-compile for platforms where we only check cmd/tailscale{,d}
strategy:
fail-fast: false # don't abort the entire matrix if one element fails
matrix:
include:
# Plan9
- goos: plan9
goarch: amd64
# AIX
- goos: aix
goarch: ppc64
runs-on: ubuntu-22.04
steps:
- name: checkout
uses: actions/checkout@v4
- name: Restore Cache
uses: actions/cache@v3
with:
# Note: unlike the other setups, this is only grabbing the mod download
# cache, rather than the whole mod directory, as the download cache
# contains zips that can be unpacked in parallel faster than they can be
# fetched and extracted by tar
path: |
~/.cache/go-build
~/go/pkg/mod/cache
~\AppData\Local\go-build
# The -2- here should be incremented when the scheme of data to be
# cached changes (e.g. path above changes).
key: ${{ github.job }}-${{ runner.os }}-${{ matrix.goos }}-${{ matrix.goarch }}-go-2-${{ hashFiles('**/go.sum') }}-${{ github.run_id }}
restore-keys: |
${{ github.job }}-${{ runner.os }}-${{ matrix.goos }}-${{ matrix.goarch }}-go-2-${{ hashFiles('**/go.sum') }}
${{ github.job }}-${{ runner.os }}-${{ matrix.goos }}-${{ matrix.goarch }}-go-2-
- name: build core
run: ./tool/go build ./cmd/tailscale ./cmd/tailscaled
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
GOARM: ${{ matrix.goarm }}
CGO_ENABLED: "0"
android:
# similar to cross above, but android fails to build a few pieces of the
# repo. We should fix those pieces, they're small, but as a stepping stone,
@ -318,7 +356,7 @@ jobs:
# some Android breakages early.
# TODO(bradfitz): better; see https://github.com/tailscale/tailscale/issues/4482
- name: build some
run: ./tool/go install ./net/netns ./ipn/ipnlocal ./wgengine/magicsock/ ./wgengine/ ./wgengine/router/ ./wgengine/netstack ./util/dnsname/ ./ipn/ ./net/interfaces ./wgengine/router/ ./tailcfg/ ./types/logger/ ./net/dns ./hostinfo ./version
run: ./tool/go install ./net/netns ./ipn/ipnlocal ./wgengine/magicsock/ ./wgengine/ ./wgengine/router/ ./wgengine/netstack ./util/dnsname/ ./ipn/ ./net/netmon ./wgengine/router/ ./tailcfg/ ./types/logger/ ./net/dns ./hostinfo ./version
env:
GOOS: android
GOARCH: arm64
@ -442,7 +480,7 @@ jobs:
uses: actions/checkout@v4
- name: check that 'go generate' is clean
run: |
pkgs=$(./tool/go list ./... | grep -Ev 'dnsfallback|k8s-operator')
pkgs=$(./tool/go list ./... | grep -Ev 'dnsfallback|k8s-operator|xdp')
./tool/go generate $pkgs
echo
echo

1
.gitignore vendored

@ -9,6 +9,7 @@
cmd/tailscale/tailscale
cmd/tailscaled/tailscaled
ssh/tailssh/testcontainers/tailscaled
# Test binary, built with `go test -c`
*.test

@ -1,5 +1,5 @@
IMAGE_REPO ?= tailscale/tailscale
SYNO_ARCH ?= "amd64"
SYNO_ARCH ?= "x86_64"
SYNO_DSM ?= "7"
TAGS ?= "latest"
@ -100,6 +100,23 @@ publishdevoperator: ## Build and publish k8s-operator image to location specifie
@test "${REPO}" != "ghcr.io/tailscale/k8s-operator" || (echo "REPO=... must not be ghcr.io/tailscale/k8s-operator" && exit 1)
TAGS="${TAGS}" REPOS=${REPO} PLATFORM=${PLATFORM} PUSH=true TARGET=operator ./build_docker.sh
publishdevnameserver: ## Build and publish k8s-nameserver image to location specified by ${REPO}
@test -n "${REPO}" || (echo "REPO=... required; e.g. REPO=ghcr.io/${USER}/tailscale" && exit 1)
@test "${REPO}" != "tailscale/tailscale" || (echo "REPO=... must not be tailscale/tailscale" && exit 1)
@test "${REPO}" != "ghcr.io/tailscale/tailscale" || (echo "REPO=... must not be ghcr.io/tailscale/tailscale" && exit 1)
@test "${REPO}" != "tailscale/k8s-nameserver" || (echo "REPO=... must not be tailscale/k8s-nameserver" && exit 1)
@test "${REPO}" != "ghcr.io/tailscale/k8s-nameserver" || (echo "REPO=... must not be ghcr.io/tailscale/k8s-nameserver" && exit 1)
TAGS="${TAGS}" REPOS=${REPO} PLATFORM=${PLATFORM} PUSH=true TARGET=k8s-nameserver ./build_docker.sh
.PHONY: sshintegrationtest
sshintegrationtest: ## Run the SSH integration tests in various Docker containers
@GOOS=linux GOARCH=amd64 ./tool/go test -tags integrationtest -c ./ssh/tailssh -o ssh/tailssh/testcontainers/tailssh.test && \
GOOS=linux GOARCH=amd64 ./tool/go build -o ssh/tailssh/testcontainers/tailscaled ./cmd/tailscaled && \
echo "Testing on ubuntu:focal" && docker build --build-arg="BASE=ubuntu:focal" -t ssh-ubuntu-focal ssh/tailssh/testcontainers && \
echo "Testing on ubuntu:jammy" && docker build --build-arg="BASE=ubuntu:jammy" -t ssh-ubuntu-jammy ssh/tailssh/testcontainers && \
echo "Testing on ubuntu:mantic" && docker build --build-arg="BASE=ubuntu:mantic" -t ssh-ubuntu-mantic ssh/tailssh/testcontainers && \
echo "Testing on ubuntu:noble" && docker build --build-arg="BASE=ubuntu:noble" -t ssh-ubuntu-noble ssh/tailssh/testcontainers
help: ## Show this help
@echo "\nSpecify a command. The choices are:\n"
@grep -hE '^[0-9a-zA-Z_-]+:.*?## .*$$' ${MAKEFILE_LIST} | awk 'BEGIN {FS = ":.*?## "}; {printf " \033[0;36m%-20s\033[m %s\n", $$1, $$2}'

@ -1 +1 @@
1.63.0
1.69.0

1863
api.md

File diff suppressed because it is too large Load Diff

@ -15,6 +15,7 @@ import (
"slices"
"strings"
"sync"
"time"
xmaps "golang.org/x/exp/maps"
"golang.org/x/net/dns/dnsmessage"
@ -23,8 +24,49 @@ import (
"tailscale.com/util/dnsname"
"tailscale.com/util/execqueue"
"tailscale.com/util/mak"
"tailscale.com/util/slicesx"
)
// rateLogger responds to calls to update by adding a count for the current period and
// calling the callback if any previous period has finished since update was last called
type rateLogger struct {
interval time.Duration
start time.Time
periodStart time.Time
periodCount int64
now func() time.Time
callback func(int64, time.Time, int64)
}
func (rl *rateLogger) currentIntervalStart(now time.Time) time.Time {
millisSince := now.Sub(rl.start).Milliseconds() % rl.interval.Milliseconds()
return now.Add(-(time.Duration(millisSince)) * time.Millisecond)
}
func (rl *rateLogger) update(numRoutes int64) {
now := rl.now()
periodEnd := rl.periodStart.Add(rl.interval)
if periodEnd.Before(now) {
if rl.periodCount != 0 {
rl.callback(rl.periodCount, rl.periodStart, numRoutes)
}
rl.periodCount = 0
rl.periodStart = rl.currentIntervalStart(now)
}
rl.periodCount++
}
func newRateLogger(now func() time.Time, interval time.Duration, callback func(int64, time.Time, int64)) *rateLogger {
nowTime := now()
return &rateLogger{
callback: callback,
now: now,
interval: interval,
start: nowTime,
periodStart: nowTime,
}
}
// RouteAdvertiser is an interface that allows the AppConnector to advertise
// newly discovered routes that need to be served through the AppConnector.
type RouteAdvertiser interface {
@ -36,6 +78,19 @@ type RouteAdvertiser interface {
UnadvertiseRoute(...netip.Prefix) error
}
// RouteInfo is a data structure used to persist the in memory state of an AppConnector
// so that we can know, even after a restart, which routes came from ACLs and which were
// learned from domains.
type RouteInfo struct {
// Control is the routes from the 'routes' section of an app connector acl.
Control []netip.Prefix `json:",omitempty"`
// Domains are the routes discovered by observing DNS lookups for configured domains.
Domains map[string][]netip.Addr `json:",omitempty"`
// Wildcards are the configured DNS lookup domains to observe. When a DNS query matches Wildcards,
// its result is added to Domains.
Wildcards []string `json:",omitempty"`
}
// AppConnector is an implementation of an AppConnector that performs
// its function as a subsystem inside of a tailscale node. At the control plane
// side App Connector routing is configured in terms of domains rather than IP
@ -49,6 +104,9 @@ type AppConnector struct {
logf logger.Logf
routeAdvertiser RouteAdvertiser
// storeRoutesFunc will be called to persist routes if it is not nil.
storeRoutesFunc func(*RouteInfo) error
// mu guards the fields that follow
mu sync.Mutex
@ -64,14 +122,67 @@ type AppConnector struct {
// queue provides ordering for update operations
queue execqueue.ExecQueue
writeRateMinute *rateLogger
writeRateDay *rateLogger
}
// NewAppConnector creates a new AppConnector.
func NewAppConnector(logf logger.Logf, routeAdvertiser RouteAdvertiser) *AppConnector {
return &AppConnector{
func NewAppConnector(logf logger.Logf, routeAdvertiser RouteAdvertiser, routeInfo *RouteInfo, storeRoutesFunc func(*RouteInfo) error) *AppConnector {
ac := &AppConnector{
logf: logger.WithPrefix(logf, "appc: "),
routeAdvertiser: routeAdvertiser,
storeRoutesFunc: storeRoutesFunc,
}
if routeInfo != nil {
ac.domains = routeInfo.Domains
ac.wildcards = routeInfo.Wildcards
ac.controlRoutes = routeInfo.Control
}
ac.writeRateMinute = newRateLogger(time.Now, time.Minute, func(c int64, s time.Time, l int64) {
ac.logf("routeInfo write rate: %d in minute starting at %v (%d routes)", c, s, l)
})
ac.writeRateDay = newRateLogger(time.Now, 24*time.Hour, func(c int64, s time.Time, l int64) {
ac.logf("routeInfo write rate: %d in 24 hours starting at %v (%d routes)", c, s, l)
})
return ac
}
// ShouldStoreRoutes returns true if the appconnector was created with the controlknob on
// and is storing its discovered routes persistently.
func (e *AppConnector) ShouldStoreRoutes() bool {
return e.storeRoutesFunc != nil
}
// storeRoutesLocked takes the current state of the AppConnector and persists it
func (e *AppConnector) storeRoutesLocked() error {
if !e.ShouldStoreRoutes() {
return nil
}
// log write rate and write size
numRoutes := int64(len(e.controlRoutes))
for _, rs := range e.domains {
numRoutes += int64(len(rs))
}
e.writeRateMinute.update(numRoutes)
e.writeRateDay.update(numRoutes)
return e.storeRoutesFunc(&RouteInfo{
Control: e.controlRoutes,
Domains: e.domains,
Wildcards: e.wildcards,
})
}
// ClearRoutes removes all route state from the AppConnector.
func (e *AppConnector) ClearRoutes() error {
e.mu.Lock()
defer e.mu.Unlock()
e.controlRoutes = nil
e.domains = nil
e.wildcards = nil
return e.storeRoutesLocked()
}
// UpdateDomainsAndRoutes starts an asynchronous update of the configuration
@ -125,10 +236,26 @@ func (e *AppConnector) updateDomains(domains []string) {
for _, wc := range e.wildcards {
if dnsname.HasSuffix(d, wc) {
e.domains[d] = addrs
delete(oldDomains, d)
break
}
}
}
// Everything left in oldDomains is a domain we're no longer tracking
// and if we are storing route info we can unadvertise the routes
if e.ShouldStoreRoutes() {
toRemove := []netip.Prefix{}
for _, addrs := range oldDomains {
for _, a := range addrs {
toRemove = append(toRemove, netip.PrefixFrom(a, a.BitLen()))
}
}
if err := e.routeAdvertiser.UnadvertiseRoute(toRemove...); err != nil {
e.logf("failed to unadvertise routes on domain removal: %v: %v: %v", xmaps.Keys(oldDomains), toRemove, err)
}
}
e.logf("handling domains: %v and wildcards: %v", xmaps.Keys(e.domains), e.wildcards)
}
@ -152,6 +279,14 @@ func (e *AppConnector) updateRoutes(routes []netip.Prefix) {
var toRemove []netip.Prefix
// If we're storing routes and know e.controlRoutes is a good
// representation of what should be in AdvertisedRoutes we can stop
// advertising routes that used to be in e.controlRoutes but are not
// in routes.
if e.ShouldStoreRoutes() {
toRemove = routesWithout(e.controlRoutes, routes)
}
nextRoute:
for _, r := range routes {
for _, addr := range e.domains {
@ -170,6 +305,9 @@ nextRoute:
}
e.controlRoutes = routes
if err := e.storeRoutesLocked(); err != nil {
e.logf("failed to store route info: %v", err)
}
}
// Domains returns the currently configured domain list.
@ -380,6 +518,9 @@ func (e *AppConnector) scheduleAdvertisement(domain string, routes ...netip.Pref
e.logf("[v2] advertised route for %v: %v", domain, addr)
}
}
if err := e.storeRoutesLocked(); err != nil {
e.logf("failed to store route info: %v", err)
}
})
}
@ -400,3 +541,15 @@ func (e *AppConnector) addDomainAddrLocked(domain string, addr netip.Addr) {
func compareAddr(l, r netip.Addr) int {
return l.Compare(r)
}
// routesWithout returns a without b where a and b
// are unsorted slices of netip.Prefix
func routesWithout(a, b []netip.Prefix) []netip.Prefix {
m := make(map[netip.Prefix]bool, len(b))
for _, p := range b {
m[p] = true
}
return slicesx.Filter(make([]netip.Prefix, 0, len(a)), a, func(p netip.Prefix) bool {
return !m[p]
})
}

@ -9,17 +9,27 @@ import (
"reflect"
"slices"
"testing"
"time"
xmaps "golang.org/x/exp/maps"
"golang.org/x/net/dns/dnsmessage"
"tailscale.com/appc/appctest"
"tailscale.com/tstest"
"tailscale.com/util/mak"
"tailscale.com/util/must"
)
func fakeStoreRoutes(*RouteInfo) error { return nil }
func TestUpdateDomains(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
a := NewAppConnector(t.Logf, nil)
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, &appctest.RouteCollector{}, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, &appctest.RouteCollector{}, nil, nil)
}
a.UpdateDomains([]string{"example.com"})
a.Wait(ctx)
@ -43,11 +53,18 @@ func TestUpdateDomains(t *testing.T) {
t.Errorf("got %v; want %v", got, want)
}
}
}
func TestUpdateRoutes(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
rc := &appctest.RouteCollector{}
a := NewAppConnector(t.Logf, rc)
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
a.updateDomains([]string{"*.example.com"})
// This route should be collapsed into the range
@ -80,10 +97,17 @@ func TestUpdateRoutes(t *testing.T) {
t.Fatalf("unexpected removed routes: %v", rc.RemovedRoutes())
}
}
}
func TestUpdateRoutesUnadvertisesContainedRoutes(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
rc := &appctest.RouteCollector{}
a := NewAppConnector(t.Logf, rc)
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
mak.Set(&a.domains, "example.com", []netip.Addr{netip.MustParseAddr("192.0.2.1")})
rc.SetRoutes([]netip.Prefix{netip.MustParsePrefix("192.0.2.1/32")})
routes := []netip.Prefix{netip.MustParsePrefix("192.0.2.0/24")}
@ -93,10 +117,17 @@ func TestUpdateRoutesUnadvertisesContainedRoutes(t *testing.T) {
t.Fatalf("got %v, want %v", rc.Routes(), routes)
}
}
}
func TestDomainRoutes(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
rc := &appctest.RouteCollector{}
a := NewAppConnector(t.Logf, rc)
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
a.updateDomains([]string{"example.com"})
a.ObserveDNSResponse(dnsResponse("example.com.", "192.0.0.8"))
a.Wait(context.Background())
@ -109,11 +140,18 @@ func TestDomainRoutes(t *testing.T) {
t.Fatalf("DomainRoutes: got %v, want %v", got, want)
}
}
}
func TestObserveDNSResponse(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
rc := &appctest.RouteCollector{}
a := NewAppConnector(t.Logf, rc)
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
// a has no domains configured, so it should not advertise any routes
a.ObserveDNSResponse(dnsResponse("example.com.", "192.0.0.8"))
@ -177,11 +215,18 @@ func TestObserveDNSResponse(t *testing.T) {
t.Errorf("missing %v from %v", "192.0.2.1", a.domains["exmaple.com"])
}
}
}
func TestWildcardDomains(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
rc := &appctest.RouteCollector{}
a := NewAppConnector(t.Logf, rc)
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
a.updateDomains([]string{"*.example.com"})
a.ObserveDNSResponse(dnsResponse("foo.example.com.", "192.0.0.8"))
@ -207,6 +252,7 @@ func TestWildcardDomains(t *testing.T) {
t.Errorf("expected only one wildcard domain, got %v", a.wildcards)
}
}
}
// dnsResponse is a test helper that creates a DNS response buffer for the given domain and address
func dnsResponse(domain, address string) []byte {
@ -310,3 +356,216 @@ func prefixCompare(a, b netip.Prefix) int {
}
return a.Addr().Compare(b.Addr())
}
func prefixes(in ...string) []netip.Prefix {
toRet := make([]netip.Prefix, len(in))
for i, s := range in {
toRet[i] = netip.MustParsePrefix(s)
}
return toRet
}
func TestUpdateRouteRouteRemoval(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
rc := &appctest.RouteCollector{}
assertRoutes := func(prefix string, routes, removedRoutes []netip.Prefix) {
if !slices.Equal(routes, rc.Routes()) {
t.Fatalf("%s: (shouldStore=%t) routes want %v, got %v", prefix, shouldStore, routes, rc.Routes())
}
if !slices.Equal(removedRoutes, rc.RemovedRoutes()) {
t.Fatalf("%s: (shouldStore=%t) removedRoutes want %v, got %v", prefix, shouldStore, removedRoutes, rc.RemovedRoutes())
}
}
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
// nothing has yet been advertised
assertRoutes("appc init", []netip.Prefix{}, []netip.Prefix{})
a.UpdateDomainsAndRoutes([]string{}, prefixes("1.2.3.1/32", "1.2.3.2/32"))
a.Wait(ctx)
// the routes passed to UpdateDomainsAndRoutes have been advertised
assertRoutes("simple update", prefixes("1.2.3.1/32", "1.2.3.2/32"), []netip.Prefix{})
// one route the same, one different
a.UpdateDomainsAndRoutes([]string{}, prefixes("1.2.3.1/32", "1.2.3.3/32"))
a.Wait(ctx)
// old behavior: routes are not removed, resulting routes are both old and new
// (we have dupe 1.2.3.1 routes because the test RouteAdvertiser doesn't have the deduplication
// the real one does)
wantRoutes := prefixes("1.2.3.1/32", "1.2.3.2/32", "1.2.3.1/32", "1.2.3.3/32")
wantRemovedRoutes := []netip.Prefix{}
if shouldStore {
// new behavior: routes are removed, resulting routes are new only
wantRoutes = prefixes("1.2.3.1/32", "1.2.3.1/32", "1.2.3.3/32")
wantRemovedRoutes = prefixes("1.2.3.2/32")
}
assertRoutes("removal", wantRoutes, wantRemovedRoutes)
}
}
func TestUpdateDomainRouteRemoval(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
rc := &appctest.RouteCollector{}
assertRoutes := func(prefix string, routes, removedRoutes []netip.Prefix) {
if !slices.Equal(routes, rc.Routes()) {
t.Fatalf("%s: (shouldStore=%t) routes want %v, got %v", prefix, shouldStore, routes, rc.Routes())
}
if !slices.Equal(removedRoutes, rc.RemovedRoutes()) {
t.Fatalf("%s: (shouldStore=%t) removedRoutes want %v, got %v", prefix, shouldStore, removedRoutes, rc.RemovedRoutes())
}
}
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
assertRoutes("appc init", []netip.Prefix{}, []netip.Prefix{})
a.UpdateDomainsAndRoutes([]string{"a.example.com", "b.example.com"}, []netip.Prefix{})
a.Wait(ctx)
// adding domains doesn't immediately cause any routes to be advertised
assertRoutes("update domains", []netip.Prefix{}, []netip.Prefix{})
a.ObserveDNSResponse(dnsResponse("a.example.com.", "1.2.3.1"))
a.ObserveDNSResponse(dnsResponse("a.example.com.", "1.2.3.2"))
a.ObserveDNSResponse(dnsResponse("b.example.com.", "1.2.3.3"))
a.ObserveDNSResponse(dnsResponse("b.example.com.", "1.2.3.4"))
a.Wait(ctx)
// observing dns responses causes routes to be advertised
assertRoutes("observed dns", prefixes("1.2.3.1/32", "1.2.3.2/32", "1.2.3.3/32", "1.2.3.4/32"), []netip.Prefix{})
a.UpdateDomainsAndRoutes([]string{"a.example.com"}, []netip.Prefix{})
a.Wait(ctx)
// old behavior, routes are not removed
wantRoutes := prefixes("1.2.3.1/32", "1.2.3.2/32", "1.2.3.3/32", "1.2.3.4/32")
wantRemovedRoutes := []netip.Prefix{}
if shouldStore {
// new behavior, routes are removed for b.example.com
wantRoutes = prefixes("1.2.3.1/32", "1.2.3.2/32")
wantRemovedRoutes = prefixes("1.2.3.3/32", "1.2.3.4/32")
}
assertRoutes("removal", wantRoutes, wantRemovedRoutes)
}
}
func TestUpdateWildcardRouteRemoval(t *testing.T) {
for _, shouldStore := range []bool{false, true} {
ctx := context.Background()
rc := &appctest.RouteCollector{}
assertRoutes := func(prefix string, routes, removedRoutes []netip.Prefix) {
if !slices.Equal(routes, rc.Routes()) {
t.Fatalf("%s: (shouldStore=%t) routes want %v, got %v", prefix, shouldStore, routes, rc.Routes())
}
if !slices.Equal(removedRoutes, rc.RemovedRoutes()) {
t.Fatalf("%s: (shouldStore=%t) removedRoutes want %v, got %v", prefix, shouldStore, removedRoutes, rc.RemovedRoutes())
}
}
var a *AppConnector
if shouldStore {
a = NewAppConnector(t.Logf, rc, &RouteInfo{}, fakeStoreRoutes)
} else {
a = NewAppConnector(t.Logf, rc, nil, nil)
}
assertRoutes("appc init", []netip.Prefix{}, []netip.Prefix{})
a.UpdateDomainsAndRoutes([]string{"a.example.com", "*.b.example.com"}, []netip.Prefix{})
a.Wait(ctx)
// adding domains doesn't immediately cause any routes to be advertised
assertRoutes("update domains", []netip.Prefix{}, []netip.Prefix{})
a.ObserveDNSResponse(dnsResponse("a.example.com.", "1.2.3.1"))
a.ObserveDNSResponse(dnsResponse("a.example.com.", "1.2.3.2"))
a.ObserveDNSResponse(dnsResponse("1.b.example.com.", "1.2.3.3"))
a.ObserveDNSResponse(dnsResponse("2.b.example.com.", "1.2.3.4"))
a.Wait(ctx)
// observing dns responses causes routes to be advertised
assertRoutes("observed dns", prefixes("1.2.3.1/32", "1.2.3.2/32", "1.2.3.3/32", "1.2.3.4/32"), []netip.Prefix{})
a.UpdateDomainsAndRoutes([]string{"a.example.com"}, []netip.Prefix{})
a.Wait(ctx)
// old behavior, routes are not removed
wantRoutes := prefixes("1.2.3.1/32", "1.2.3.2/32", "1.2.3.3/32", "1.2.3.4/32")
wantRemovedRoutes := []netip.Prefix{}
if shouldStore {
// new behavior, routes are removed for *.b.example.com
wantRoutes = prefixes("1.2.3.1/32", "1.2.3.2/32")
wantRemovedRoutes = prefixes("1.2.3.3/32", "1.2.3.4/32")
}
assertRoutes("removal", wantRoutes, wantRemovedRoutes)
}
}
func TestRoutesWithout(t *testing.T) {
assert := func(msg string, got, want []netip.Prefix) {
if !slices.Equal(want, got) {
t.Errorf("%s: want %v, got %v", msg, want, got)
}
}
assert("empty routes", routesWithout([]netip.Prefix{}, []netip.Prefix{}), []netip.Prefix{})
assert("a empty", routesWithout([]netip.Prefix{}, prefixes("1.1.1.1/32", "1.1.1.2/32")), []netip.Prefix{})
assert("b empty", routesWithout(prefixes("1.1.1.1/32", "1.1.1.2/32"), []netip.Prefix{}), prefixes("1.1.1.1/32", "1.1.1.2/32"))
assert("no overlap", routesWithout(prefixes("1.1.1.1/32", "1.1.1.2/32"), prefixes("1.1.1.3/32", "1.1.1.4/32")), prefixes("1.1.1.1/32", "1.1.1.2/32"))
assert("a has fewer", routesWithout(prefixes("1.1.1.1/32", "1.1.1.2/32"), prefixes("1.1.1.1/32", "1.1.1.2/32", "1.1.1.3/32", "1.1.1.4/32")), []netip.Prefix{})
assert("a has more", routesWithout(prefixes("1.1.1.1/32", "1.1.1.2/32", "1.1.1.3/32", "1.1.1.4/32"), prefixes("1.1.1.1/32", "1.1.1.3/32")), prefixes("1.1.1.2/32", "1.1.1.4/32"))
}
func TestRateLogger(t *testing.T) {
clock := tstest.Clock{}
wasCalled := false
rl := newRateLogger(func() time.Time { return clock.Now() }, 1*time.Second, func(count int64, _ time.Time, _ int64) {
if count != 3 {
t.Fatalf("count for prev period: got %d, want 3", count)
}
wasCalled = true
})
for i := 0; i < 3; i++ {
clock.Advance(1 * time.Millisecond)
rl.update(0)
if wasCalled {
t.Fatalf("wasCalled: got true, want false")
}
}
clock.Advance(1 * time.Second)
rl.update(0)
if !wasCalled {
t.Fatalf("wasCalled: got false, want true")
}
wasCalled = false
rl = newRateLogger(func() time.Time { return clock.Now() }, 1*time.Hour, func(count int64, _ time.Time, _ int64) {
if count != 3 {
t.Fatalf("count for prev period: got %d, want 3", count)
}
wasCalled = true
})
for i := 0; i < 3; i++ {
clock.Advance(1 * time.Minute)
rl.update(0)
if wasCalled {
t.Fatalf("wasCalled: got true, want false")
}
}
clock.Advance(1 * time.Hour)
rl.update(0)
if !wasCalled {
t.Fatalf("wasCalled: got false, want true")
}
}

@ -37,7 +37,7 @@ while [ "$#" -gt 1 ]; do
--extra-small)
shift
ldflags="$ldflags -w -s"
tags="${tags:+$tags,}ts_omit_aws,ts_omit_bird,ts_omit_tap,ts_omit_kube"
tags="${tags:+$tags,}ts_omit_aws,ts_omit_bird,ts_omit_tap,ts_omit_kube,ts_omit_completion"
;;
--box)
shift

@ -49,6 +49,7 @@ case "$TARGET" in
-X tailscale.com/version.gitCommitStamp=${VERSION_GIT_HASH}" \
--base="${BASE}" \
--tags="${TAGS}" \
--gotags="ts_kube" \
--repos="${REPOS}" \
--push="${PUSH}" \
--target="${PLATFORM}" \
@ -70,6 +71,22 @@ case "$TARGET" in
--target="${PLATFORM}" \
/usr/local/bin/operator
;;
k8s-nameserver)
DEFAULT_REPOS="tailscale/k8s-nameserver"
REPOS="${REPOS:-${DEFAULT_REPOS}}"
go run github.com/tailscale/mkctr \
--gopaths="tailscale.com/cmd/k8s-nameserver:/usr/local/bin/k8s-nameserver" \
--ldflags=" \
-X tailscale.com/version.longStamp=${VERSION_LONG} \
-X tailscale.com/version.shortStamp=${VERSION_SHORT} \
-X tailscale.com/version.gitCommitStamp=${VERSION_GIT_HASH}" \
--base="${BASE}" \
--tags="${TAGS}" \
--repos="${REPOS}" \
--push="${PUSH}" \
--target="${PLATFORM}" \
/usr/local/bin/k8s-nameserver
;;
*)
echo "unknown target: $TARGET"
exit 1

@ -49,3 +49,11 @@ type ReloadConfigResponse struct {
Reloaded bool // whether the config was reloaded
Err string // any error message
}
// ExitNodeSuggestionResponse is the response to a LocalAPI suggest-exit-node GET request.
// It returns the StableNodeID, name, and location of a suggested exit node for the client making the request.
type ExitNodeSuggestionResponse struct {
ID tailcfg.StableNodeID
Name string
Location tailcfg.LocationView `json:",omitempty"`
}

@ -10,6 +10,7 @@ import (
"context"
"encoding/json"
"fmt"
"log"
"net/http"
"net/url"
@ -39,6 +40,7 @@ type Device struct {
// It's currently just 1 element, the 100.x.y.z Tailscale IP.
Addresses []string `json:"addresses"`
DeviceID string `json:"id"`
NodeID string `json:"nodeId"`
User string `json:"user"`
Name string `json:"name"`
Hostname string `json:"hostname"`
@ -213,6 +215,9 @@ func (c *Client) DeleteDevice(ctx context.Context, deviceID string) (err error)
if err != nil {
return err
}
log.Printf("RESP: %di, path: %s", resp.StatusCode, path)
// If status code was not successful, return the error.
// TODO: Change the check for the StatusCode to include other 2XX success codes.
if resp.StatusCode != http.StatusOK {

@ -253,11 +253,16 @@ func (lc *LocalClient) sendWithHeaders(
}
if res.StatusCode != wantStatus {
err = fmt.Errorf("%v: %s", res.Status, bytes.TrimSpace(slurp))
return nil, nil, bestError(err, slurp)
return nil, nil, httpStatusError{bestError(err, slurp), res.StatusCode}
}
return slurp, res.Header, nil
}
type httpStatusError struct {
error
HTTPStatus int
}
func (lc *LocalClient) get200(ctx context.Context, path string) ([]byte, error) {
return lc.send(ctx, "GET", path, 200, nil)
}
@ -278,9 +283,50 @@ func decodeJSON[T any](b []byte) (ret T, err error) {
}
// WhoIs returns the owner of the remoteAddr, which must be an IP or IP:port.
//
// If not found, the error is ErrPeerNotFound.
//
// For connections proxied by tailscaled, this looks up the owner of the given
// address as TCP first, falling back to UDP; if you want to only check a
// specific address family, use WhoIsProto.
func (lc *LocalClient) WhoIs(ctx context.Context, remoteAddr string) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(remoteAddr))
if err != nil {
if hs, ok := err.(httpStatusError); ok && hs.HTTPStatus == http.StatusNotFound {
return nil, ErrPeerNotFound
}
return nil, err
}
return decodeJSON[*apitype.WhoIsResponse](body)
}
// ErrPeerNotFound is returned by WhoIs and WhoIsNodeKey when a peer is not found.
var ErrPeerNotFound = errors.New("peer not found")
// WhoIsNodeKey returns the owner of the given wireguard public key.
//
// If not found, the error is ErrPeerNotFound.
func (lc *LocalClient) WhoIsNodeKey(ctx context.Context, key key.NodePublic) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?addr="+url.QueryEscape(key.String()))
if err != nil {
if hs, ok := err.(httpStatusError); ok && hs.HTTPStatus == http.StatusNotFound {
return nil, ErrPeerNotFound
}
return nil, err
}
return decodeJSON[*apitype.WhoIsResponse](body)
}
// WhoIsProto returns the owner of the remoteAddr, which must be an IP or
// IP:port, for the given protocol (tcp or udp).
//
// If not found, the error is ErrPeerNotFound.
func (lc *LocalClient) WhoIsProto(ctx context.Context, proto, remoteAddr string) (*apitype.WhoIsResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/whois?proto="+url.QueryEscape(proto)+"&addr="+url.QueryEscape(remoteAddr))
if err != nil {
if hs, ok := err.(httpStatusError); ok && hs.HTTPStatus == http.StatusNotFound {
return nil, ErrPeerNotFound
}
return nil, err
}
return decodeJSON[*apitype.WhoIsResponse](body)
@ -699,6 +745,27 @@ func (lc *LocalClient) CheckUDPGROForwarding(ctx context.Context) error {
return nil
}
// SetUDPGROForwarding enables UDP GRO forwarding for the main interface of this
// node. This can be done to improve performance of tailnet nodes acting as exit
// nodes or subnet routers.
// See https://tailscale.com/kb/1320/performance-best-practices#linux-optimizations-for-subnet-routers-and-exit-nodes
func (lc *LocalClient) SetUDPGROForwarding(ctx context.Context) error {
body, err := lc.get200(ctx, "/localapi/v0/set-udp-gro-forwarding")
if err != nil {
return err
}
var jres struct {
Warning string
}
if err := json.Unmarshal(body, &jres); err != nil {
return fmt.Errorf("invalid JSON from set-udp-gro-forwarding: %w", err)
}
if jres.Warning != "" {
return errors.New(jres.Warning)
}
return nil
}
// CheckPrefs validates the provided preferences, without making any changes.
//
// The CLI uses this before a Start call to fail fast if the preferences won't
@ -778,6 +845,17 @@ func (lc *LocalClient) SetDNS(ctx context.Context, name, value string) error {
//
// The ctx is only used for the duration of the call, not the lifetime of the net.Conn.
func (lc *LocalClient) DialTCP(ctx context.Context, host string, port uint16) (net.Conn, error) {
return lc.UserDial(ctx, "tcp", host, port)
}
// UserDial connects to the host's port via Tailscale for the given network.
//
// The host may be a base DNS name (resolved from the netmap inside tailscaled),
// a FQDN, or an IP address.
//
// The ctx is only used for the duration of the call, not the lifetime of the
// net.Conn.
func (lc *LocalClient) UserDial(ctx context.Context, network, host string, port uint16) (net.Conn, error) {
connCh := make(chan net.Conn, 1)
trace := httptrace.ClientTrace{
GotConn: func(info httptrace.GotConnInfo) {
@ -794,6 +872,7 @@ func (lc *LocalClient) DialTCP(ctx context.Context, host string, port uint16) (n
"Connection": []string{"upgrade"},
"Dial-Host": []string{host},
"Dial-Port": []string{fmt.Sprint(port)},
"Dial-Network": []string{network},
}
res, err := lc.DoLocalRequest(req)
if err != nil {
@ -1514,3 +1593,12 @@ func (w *IPNBusWatcher) Next() (ipn.Notify, error) {
}
return n, nil
}
// SuggestExitNode requests an exit node suggestion and returns the exit node's details.
func (lc *LocalClient) SuggestExitNode(ctx context.Context) (apitype.ExitNodeSuggestionResponse, error) {
body, err := lc.get200(ctx, "/localapi/v0/suggest-exit-node")
if err != nil {
return apitype.ExitNodeSuggestionResponse{}, err
}
return decodeJSON[apitype.ExitNodeSuggestionResponse](body)
}

@ -6,9 +6,14 @@
package tailscale
import (
"context"
"net"
"net/http"
"net/http/httptest"
"testing"
"tailscale.com/tstest/deptest"
"tailscale.com/types/key"
)
func TestGetServeConfigFromJSON(t *testing.T) {
@ -30,11 +35,38 @@ func TestGetServeConfigFromJSON(t *testing.T) {
}
}
func TestWhoIsPeerNotFound(t *testing.T) {
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(404)
}))
defer ts.Close()
lc := &LocalClient{
Dial: func(ctx context.Context, network, addr string) (net.Conn, error) {
var std net.Dialer
return std.DialContext(ctx, network, ts.Listener.Addr().(*net.TCPAddr).String())
},
}
var k key.NodePublic
if err := k.UnmarshalText([]byte("nodekey:5c8f86d5fc70d924e55f02446165a5dae8f822994ad26bcf4b08fd841f9bf261")); err != nil {
t.Fatal(err)
}
res, err := lc.WhoIsNodeKey(context.Background(), k)
if err != ErrPeerNotFound {
t.Errorf("got (%v, %v), want ErrPeerNotFound", res, err)
}
res, err = lc.WhoIs(context.Background(), "1.2.3.4:5678")
if err != ErrPeerNotFound {
t.Errorf("got (%v, %v), want ErrPeerNotFound", res, err)
}
}
func TestDeps(t *testing.T) {
deptest.DepChecker{
BadDeps: map[string]string{
// Make sure we don't again accidentally bring in a dependency on
// drive or its transitive dependencies
"testing": "do not use testing package in production code",
"tailscale.com/drive/driveimpl": "https://github.com/tailscale/tailscale/pull/10631",
"github.com/studio-b12/gowebdav": "https://github.com/tailscale/tailscale/pull/10631",
},

@ -223,7 +223,7 @@ func (s *Server) awaitUserAuth(ctx context.Context, session *browserSession) err
func (s *Server) newSessionID() (string, error) {
raw := make([]byte, 16)
for i := 0; i < 5; i++ {
for range 5 {
if _, err := rand.Read(raw); err != nil {
return "", err
}

@ -34,7 +34,7 @@
"prettier-plugin-organize-imports": "^3.2.2",
"tailwindcss": "^3.3.3",
"typescript": "^5.3.3",
"vite": "^5.1.4",
"vite": "^5.1.7",
"vite-plugin-svgr": "^4.2.0",
"vite-tsconfig-paths": "^3.5.0",
"vitest": "^1.3.1"

@ -1150,7 +1150,15 @@ func (s *Server) tailscaleUp(ctx context.Context, st *ipnstate.Status, opt tails
if !isRunning {
ipnOptions := ipn.Options{AuthKey: opt.AuthKey}
if opt.ControlURL != "" {
ipnOptions.UpdatePrefs = &ipn.Prefs{ControlURL: opt.ControlURL}
_, err := s.lc.EditPrefs(ctx, &ipn.MaskedPrefs{
Prefs: ipn.Prefs{
ControlURL: opt.ControlURL,
},
ControlURLSet: true,
})
if err != nil {
s.logf("edit prefs: %v", err)
}
}
if err := s.lc.Start(ctx, ipnOptions); err != nil {
s.logf("start: %v", err)

@ -20,7 +20,7 @@
"@jridgewell/gen-mapping" "^0.3.0"
"@jridgewell/trace-mapping" "^0.3.9"
"@babel/code-frame@^7.0.0", "@babel/code-frame@^7.22.10", "@babel/code-frame@^7.22.13", "@babel/code-frame@^7.22.5", "@babel/code-frame@^7.23.4":
"@babel/code-frame@^7.0.0", "@babel/code-frame@^7.22.13", "@babel/code-frame@^7.23.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/code-frame/-/code-frame-7.23.4.tgz#03ae5af150be94392cb5c7ccd97db5a19a5da6aa"
integrity sha512-r1IONyb6Ia+jYR2vvIDhdWdlTGhqbBoFqLTQidzZ4kepUFH15ejXvFHxCVbtl7BOXIudsIubf4E81xeA3h3IXA==
@ -63,7 +63,7 @@
eslint-visitor-keys "^2.1.0"
semver "^6.3.1"
"@babel/generator@^7.22.10", "@babel/generator@^7.23.0", "@babel/generator@^7.23.3", "@babel/generator@^7.23.4":
"@babel/generator@^7.23.3", "@babel/generator@^7.23.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/generator/-/generator-7.23.4.tgz#4a41377d8566ec18f807f42962a7f3551de83d1c"
integrity sha512-esuS49Cga3HcThFNebGhlgsrVLkvhqvYDTzgjfFFlHJcIfLe5jFmRRfCQ1KuBfc4Jrtn3ndLgKWAKjBE+IraYQ==
@ -87,7 +87,7 @@
dependencies:
"@babel/types" "^7.22.15"
"@babel/helper-compilation-targets@^7.22.10", "@babel/helper-compilation-targets@^7.22.15", "@babel/helper-compilation-targets@^7.22.6":
"@babel/helper-compilation-targets@^7.22.15", "@babel/helper-compilation-targets@^7.22.6":
version "7.22.15"
resolved "https://registry.yarnpkg.com/@babel/helper-compilation-targets/-/helper-compilation-targets-7.22.15.tgz#0698fc44551a26cf29f18d4662d5bf545a6cfc52"
integrity sha512-y6EEzULok0Qvz8yyLkCvVX+02ic+By2UdOhylwUOvOn9dvYc9mKICJuuU1n1XBI02YWsNsnrY1kc6DVbjcXbtw==
@ -160,14 +160,14 @@
dependencies:
"@babel/types" "^7.23.0"
"@babel/helper-module-imports@^7.22.15", "@babel/helper-module-imports@^7.22.5":
"@babel/helper-module-imports@^7.22.15":
version "7.22.15"
resolved "https://registry.yarnpkg.com/@babel/helper-module-imports/-/helper-module-imports-7.22.15.tgz#16146307acdc40cc00c3b2c647713076464bdbf0"
integrity sha512-0pYVBnDKZO2fnSPCrgM/6WMc7eS20Fbok+0r88fp+YtWVLZrp4CkafFGIp+W0VKw4a22sgebPT99y+FDNMdP4w==
dependencies:
"@babel/types" "^7.22.15"
"@babel/helper-module-transforms@^7.22.9", "@babel/helper-module-transforms@^7.23.3":
"@babel/helper-module-transforms@^7.23.3":
version "7.23.3"
resolved "https://registry.yarnpkg.com/@babel/helper-module-transforms/-/helper-module-transforms-7.23.3.tgz#d7d12c3c5d30af5b3c0fcab2a6d5217773e2d0f1"
integrity sha512-7bBs4ED9OmswdfDzpz4MpWgSrV7FXlc3zIagvLFjS5H+Mk7Snr21vQ6QwrsoCGMfNC4e4LQPdoULEt4ykz0SRQ==
@ -229,17 +229,17 @@
dependencies:
"@babel/types" "^7.22.5"
"@babel/helper-string-parser@^7.22.5", "@babel/helper-string-parser@^7.23.4":
"@babel/helper-string-parser@^7.23.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/helper-string-parser/-/helper-string-parser-7.23.4.tgz#9478c707febcbbe1ddb38a3d91a2e054ae622d83"
integrity sha512-803gmbQdqwdf4olxrX4AJyFBV/RTr3rSmOj0rKwesmzlfhYNDEs+/iOcznzpNWlJlIlTJC2QfPFcHB6DlzdVLQ==
"@babel/helper-validator-identifier@^7.22.20", "@babel/helper-validator-identifier@^7.22.5":
"@babel/helper-validator-identifier@^7.22.20":
version "7.22.20"
resolved "https://registry.yarnpkg.com/@babel/helper-validator-identifier/-/helper-validator-identifier-7.22.20.tgz#c4ae002c61d2879e724581d96665583dbc1dc0e0"
integrity sha512-Y4OZ+ytlatR8AI+8KZfKuL5urKp7qey08ha31L8b3BwewJAoJamTzyvxPR/5D+KkdJCGPq/+8TukHBlY10FX9A==
"@babel/helper-validator-option@^7.22.15", "@babel/helper-validator-option@^7.22.5":
"@babel/helper-validator-option@^7.22.15":
version "7.22.15"
resolved "https://registry.yarnpkg.com/@babel/helper-validator-option/-/helper-validator-option-7.22.15.tgz#694c30dfa1d09a6534cdfcafbe56789d36aba040"
integrity sha512-bMn7RmyFjY/mdECUbgn9eoSY4vqvacUnS9i9vGAGttgFWesO6B4CYWA7XlpbWgBt71iv/hfbPlynohStqnu5hA==
@ -253,7 +253,7 @@
"@babel/template" "^7.22.15"
"@babel/types" "^7.22.19"
"@babel/helpers@^7.22.10", "@babel/helpers@^7.23.2":
"@babel/helpers@^7.23.2":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/helpers/-/helpers-7.23.4.tgz#7d2cfb969aa43222032193accd7329851facf3c1"
integrity sha512-HfcMizYz10cr3h29VqyfGL6ZWIjTwWfvYBMsBVGwpcbhNGe3wQ1ZXZRPzZoAHhd9OqHadHqjQ89iVKINXnbzuw==
@ -262,7 +262,7 @@
"@babel/traverse" "^7.23.4"
"@babel/types" "^7.23.4"
"@babel/highlight@^7.22.10", "@babel/highlight@^7.22.13", "@babel/highlight@^7.23.4":
"@babel/highlight@^7.23.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/highlight/-/highlight-7.23.4.tgz#edaadf4d8232e1a961432db785091207ead0621b"
integrity sha512-acGdbYSfp2WheJoJm/EBBBLh/ID8KDc64ISZ9DYtBmC8/Q204PZJLHyzeB5qMzJ5trcOkybd78M4x2KWsUq++A==
@ -271,7 +271,7 @@
chalk "^2.4.2"
js-tokens "^4.0.0"
"@babel/parser@^7.22.10", "@babel/parser@^7.22.15", "@babel/parser@^7.22.5", "@babel/parser@^7.23.0", "@babel/parser@^7.23.3", "@babel/parser@^7.23.4":
"@babel/parser@^7.22.15", "@babel/parser@^7.23.3", "@babel/parser@^7.23.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/parser/-/parser-7.23.4.tgz#409fbe690c333bb70187e2de4021e1e47a026661"
integrity sha512-vf3Xna6UEprW+7t6EtOmFpHNAuxw3xqPZghy+brsnusscJRW5BMUzzHZc5ICjULee81WeUV2jjakG09MDglJXQ==
@ -1093,7 +1093,7 @@
dependencies:
regenerator-runtime "^0.14.0"
"@babel/template@^7.22.15", "@babel/template@^7.22.5":
"@babel/template@^7.22.15":
version "7.22.15"
resolved "https://registry.yarnpkg.com/@babel/template/-/template-7.22.15.tgz#09576efc3830f0430f4548ef971dde1350ef2f38"
integrity sha512-QPErUVm4uyJa60rkI73qneDacvdvzxshT3kksGqlGWYdOTIUOwJ7RDUL8sGqslY1uXWSL6xMFKEXDS3ox2uF0w==
@ -1102,7 +1102,7 @@
"@babel/parser" "^7.22.15"
"@babel/types" "^7.22.15"
"@babel/traverse@^7.22.10", "@babel/traverse@^7.23.3", "@babel/traverse@^7.23.4":
"@babel/traverse@^7.23.3", "@babel/traverse@^7.23.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/traverse/-/traverse-7.23.4.tgz#c2790f7edf106d059a0098770fe70801417f3f85"
integrity sha512-IYM8wSUwunWTB6tFC2dkKZhxbIjHoWemdK+3f8/wq8aKhbUscxD5MX72ubd90fxvFknaLPeGw5ycU84V1obHJg==
@ -1118,7 +1118,7 @@
debug "^4.1.0"
globals "^11.1.0"
"@babel/types@^7.21.3", "@babel/types@^7.22.10", "@babel/types@^7.22.15", "@babel/types@^7.22.19", "@babel/types@^7.22.5", "@babel/types@^7.23.0", "@babel/types@^7.23.3", "@babel/types@^7.23.4", "@babel/types@^7.4.4":
"@babel/types@^7.21.3", "@babel/types@^7.22.15", "@babel/types@^7.22.19", "@babel/types@^7.22.5", "@babel/types@^7.23.0", "@babel/types@^7.23.3", "@babel/types@^7.23.4", "@babel/types@^7.4.4":
version "7.23.4"
resolved "https://registry.yarnpkg.com/@babel/types/-/types-7.23.4.tgz#7206a1810fc512a7f7f7d4dace4cb4c1c9dbfb8e"
integrity sha512-7uIFwVYpoplT5jp/kVv6EF93VaJ8H+Yn5IczYiaAi98ajzjfoZfslet/e0sLh+wVBjb2qqIut1b0S26VSafsSQ==
@ -2474,7 +2474,7 @@ camelcase@^6.2.0:
resolved "https://registry.yarnpkg.com/camelcase/-/camelcase-6.3.0.tgz#5685b95eb209ac9c0c177467778c9c84df58ba9a"
integrity sha512-Gmy6FhYlCY7uOElZUSbxo2UCDH8owEk996gkbrpsgGtrJLM3J7jGxl9Ic7Qwwj4ivOE5AWZWRMecDdF7hqGjFA==
caniuse-lite@^1.0.30001517, caniuse-lite@^1.0.30001520, caniuse-lite@^1.0.30001541:
caniuse-lite@^1.0.30001520, caniuse-lite@^1.0.30001541:
version "1.0.30001565"
resolved "https://registry.yarnpkg.com/caniuse-lite/-/caniuse-lite-1.0.30001565.tgz#a528b253c8a2d95d2b415e11d8b9942acc100c4f"
integrity sha512-xrE//a3O7TP0vaJ8ikzkD2c2NgcVUvsEe2IvFTntV4Yd1Z9FVzh+gW+enX96L0psrbaFMcVcH2l90xNuGDWc8w==
@ -2587,11 +2587,6 @@ confusing-browser-globals@^1.0.11:
resolved "https://registry.yarnpkg.com/confusing-browser-globals/-/confusing-browser-globals-1.0.11.tgz#ae40e9b57cdd3915408a2805ebd3a5585608dc81"
integrity sha512-JsPKdmh8ZkmnHxDk55FZ1TqVLvEQTvoByJZRN9jzI0UjxK/QgAmsphz7PGtqgPieQZ/CQcHWXCR7ATDNhGe+YA==
convert-source-map@^1.7.0:
version "1.9.0"
resolved "https://registry.yarnpkg.com/convert-source-map/-/convert-source-map-1.9.0.tgz#7faae62353fb4213366d0ca98358d22e8368b05f"
integrity sha512-ASFBup0Mz1uyiIjANan1jzLQami9z1PoYSZCiiYW2FczPbenXc45FZdBZLzOT+r6+iciuEModtmCti+hjaAk0A==
convert-source-map@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/convert-source-map/-/convert-source-map-2.0.0.tgz#4b560f649fc4e918dd0ab75cf4961e8bc882d82a"
@ -2772,7 +2767,7 @@ dot-case@^3.0.4:
no-case "^3.0.4"
tslib "^2.0.3"
electron-to-chromium@^1.4.477, electron-to-chromium@^1.4.535:
electron-to-chromium@^1.4.535:
version "1.4.596"
resolved "https://registry.yarnpkg.com/electron-to-chromium/-/electron-to-chromium-1.4.596.tgz#6752d1aa795d942d49dfc5d3764d6ea283fab1d7"
integrity sha512-zW3zbZ40Icb2BCWjm47nxwcFGYlIgdXkAx85XDO7cyky9J4QQfq8t0W19/TLZqq3JPQXtlv8BPIGmfa9Jb4scg==
@ -3323,7 +3318,7 @@ gensync@^1.0.0-beta.2:
resolved "https://registry.yarnpkg.com/gensync/-/gensync-1.0.0-beta.2.tgz#32a6ee76c3d7f52d46b2b1ae5d93fea8580a25e0"
integrity sha512-3hN7NaskYvMDLQY55gnW3NQ+mesEAepTqlg+VEbj7zzqEMBVNhzcGYYeqFo/TlYz6eQiFcp1HcsCZO+nGgS8zg==
get-func-name@^2.0.0, get-func-name@^2.0.1, get-func-name@^2.0.2:
get-func-name@^2.0.1, get-func-name@^2.0.2:
version "2.0.2"
resolved "https://registry.yarnpkg.com/get-func-name/-/get-func-name-2.0.2.tgz#0d7cf20cd13fda808669ffa88f4ffc7a3943fc41"
integrity sha512-8vXOvuE167CtIc3OyItco7N/dpRtBbYOsPsXCz7X/PMnlGjYjSGuZJgM1Y7mmew7BKf9BqvLX2tnOVy1BBUsxQ==
@ -3486,13 +3481,6 @@ has-tostringtag@^1.0.0:
dependencies:
has-symbols "^1.0.2"
has@^1.0.3:
version "1.0.3"
resolved "https://registry.yarnpkg.com/has/-/has-1.0.3.tgz#722d7cbfc1f6aa8241f16dd814e011e1f41e8796"
integrity sha512-f2dvO0VU6Oej7RkWJGrehjbzMAjFp5/VKPp5tTpWIV4JHHZK1/BxbFRtf/siA2SWTe09caDmVtYYzWEIbBS4zw==
dependencies:
function-bind "^1.1.1"
hasown@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/hasown/-/hasown-2.0.0.tgz#f4c513d454a57b7c7e1650778de226b11700546c"
@ -4087,7 +4075,7 @@ mz@^2.7.0:
object-assign "^4.0.1"
thenify-all "^1.0.0"
nanoid@^3.3.6, nanoid@^3.3.7:
nanoid@^3.3.7:
version "3.3.7"
resolved "https://registry.yarnpkg.com/nanoid/-/nanoid-3.3.7.tgz#d0c301a691bc8d54efa0a2226ccf3fe2fd656bd8"
integrity sha512-eSRppjcPIatRIMC1U6UngP8XFcz8MQWGQdt1MTBQ7NaAmvXDfvNxbvWV3x2y6CdEUciCSsDHDQZbhYaB8QEo2g==
@ -5121,7 +5109,7 @@ typescript@^5.3.3:
resolved "https://registry.yarnpkg.com/typescript/-/typescript-5.3.3.tgz#b3ce6ba258e72e6305ba66f5c9b452aaee3ffe37"
integrity sha512-pXWcraxM0uxAS+tN0AG/BF2TyqmHO014Z070UsJ+pFvYuRSq8KH8DmWpnbXe0pEPDHXZV3FcAbJkijJ5oNEnWw==
ufo@^1.1.2, ufo@^1.3.2:
ufo@^1.3.2:
version "1.4.0"
resolved "https://registry.yarnpkg.com/ufo/-/ufo-1.4.0.tgz#39845b31be81b4f319ab1d99fd20c56cac528d32"
integrity sha512-Hhy+BhRBleFjpJ2vchUNN40qgkh0366FWJGqVLYBHev0vpHTrXSA0ryT+74UiW6KWsldNurQMKGqCm1M2zBciQ==
@ -5169,7 +5157,7 @@ universalify@^0.2.0:
resolved "https://registry.yarnpkg.com/universalify/-/universalify-0.2.0.tgz#6451760566fa857534745ab1dde952d1b1761be0"
integrity sha512-CJ1QgKmNg3CwvAv/kOFmtnEN05f0D/cn9QntgNOQlQF9dgvVTHj3t+8JPdjqawCHk7V/KA+fbUqzZ9XWhcqPUg==
update-browserslist-db@^1.0.11, update-browserslist-db@^1.0.13:
update-browserslist-db@^1.0.13:
version "1.0.13"
resolved "https://registry.yarnpkg.com/update-browserslist-db/-/update-browserslist-db-1.0.13.tgz#3c5e4f5c083661bd38ef64b6328c26ed6c8248c4"
integrity sha512-xebP81SNcPuNpPP3uzeW1NYXxI3rxyJzF3pD6sH4jE7o/IX+WtSpwnVU+qIsDPyk0d3hmFQ7mjqc6AtV604hbg==
@ -5247,10 +5235,10 @@ vite-tsconfig-paths@^3.5.0:
recrawl-sync "^2.0.3"
tsconfig-paths "^4.0.0"
vite@^5.0.0, vite@^5.1.4:
version "5.1.4"
resolved "https://registry.yarnpkg.com/vite/-/vite-5.1.4.tgz#14e9d3e7a6e488f36284ef13cebe149f060bcfb6"
integrity sha512-n+MPqzq+d9nMVTKyewqw6kSt+R3CkvF9QAKY8obiQn8g1fwTscKxyfaYnC632HtBXAQGc1Yjomphwn1dtwGAHg==
vite@^5.0.0, vite@^5.1.7:
version "5.1.7"
resolved "https://registry.yarnpkg.com/vite/-/vite-5.1.7.tgz#9f685a2c4c70707fef6d37341b0e809c366da619"
integrity sha512-sgnEEFTZYMui/sTlH1/XEnVNHMujOahPLGMxn1+5sIT45Xjng1Ec1K78jRP15dSmVgg5WBin9yO81j3o9OxofA==
dependencies:
esbuild "^0.19.3"
postcss "^8.4.35"

@ -37,11 +37,18 @@ import (
)
const (
CurrentTrack = ""
StableTrack = "stable"
UnstableTrack = "unstable"
)
var CurrentTrack = func() string {
if version.IsUnstableBuild() {
return UnstableTrack
} else {
return StableTrack
}
}()
func versionToTrack(v string) (string, error) {
_, rest, ok := strings.Cut(v, ".")
if !ok {
@ -106,7 +113,7 @@ func (args Arguments) validate() error {
return fmt.Errorf("only one of Version(%q) or Track(%q) can be set", args.Version, args.Track)
}
switch args.Track {
case StableTrack, UnstableTrack, CurrentTrack:
case StableTrack, UnstableTrack, "":
// All valid values.
default:
return fmt.Errorf("unsupported track %q", args.Track)
@ -119,11 +126,17 @@ type Updater struct {
// Update is a platform-specific method that updates the installation. May be
// nil (not all platforms support updates from within Tailscale).
Update func() error
// currentVersion is the short form of the current client version as
// returned by version.Short(), typically "x.y.z". Used for tests to
// override the actual current version.
currentVersion string
}
func NewUpdater(args Arguments) (*Updater, error) {
up := Updater{
Arguments: args,
currentVersion: version.Short(),
}
if up.Stdout == nil {
up.Stdout = os.Stdout
@ -139,18 +152,15 @@ func NewUpdater(args Arguments) (*Updater, error) {
if args.ForAutoUpdate && !canAutoUpdate {
return nil, errors.ErrUnsupported
}
if up.Track == CurrentTrack {
switch {
case up.Version != "":
if up.Track == "" {
if up.Version != "" {
var err error
up.Track, err = versionToTrack(args.Version)
if err != nil {
return nil, err
}
case version.IsUnstableBuild():
up.Track = UnstableTrack
default:
up.Track = StableTrack
} else {
up.Track = CurrentTrack
}
}
if up.Arguments.PkgsAddr == "" {
@ -259,14 +269,17 @@ func Update(args Arguments) error {
}
func (up *Updater) confirm(ver string) bool {
switch cmpver.Compare(version.Short(), ver) {
case 0:
// Only check version when we're not switching tracks.
if up.Track == "" || up.Track == CurrentTrack {
switch c := cmpver.Compare(up.currentVersion, ver); {
case c == 0:
up.Logf("already running %v version %v; no update needed", up.Track, ver)
return false
case 1:
up.Logf("installed %v version %v is newer than the latest available version %v; no update needed", up.Track, version.Short(), ver)
case c > 0:
up.Logf("installed %v version %v is newer than the latest available version %v; no update needed", up.Track, up.currentVersion, ver)
return false
}
}
if up.Confirm != nil {
return up.Confirm(ver)
}
@ -436,7 +449,7 @@ func (up *Updater) updateDebLike() error {
return fmt.Errorf("apt-get update failed: %w; output:\n%s", err, out)
}
for i := 0; i < 2; i++ {
for range 2 {
out, err := exec.Command("apt-get", "install", "--yes", "--allow-downgrades", "tailscale="+ver).CombinedOutput()
if err != nil {
if !bytes.Contains(out, []byte(`dpkg was interrupted`)) {
@ -651,6 +664,9 @@ func (up *Updater) updateAlpineLike() (err error) {
return fmt.Errorf(`failed to parse latest version from "apk info tailscale": %w`, err)
}
if !up.confirm(ver) {
if err := checkOutdatedAlpineRepo(up.Logf, ver, up.Track); err != nil {
up.Logf("failed to check whether Alpine release is outdated: %v", err)
}
return nil
}
@ -678,7 +694,7 @@ func parseAlpinePackageVersion(out []byte) (string, error) {
return "", fmt.Errorf("malformed info line: %q", line)
}
ver := parts[1]
if cmpver.Compare(ver, maxVer) == 1 {
if cmpver.Compare(ver, maxVer) > 0 {
maxVer = ver
}
}
@ -688,6 +704,37 @@ func parseAlpinePackageVersion(out []byte) (string, error) {
return "", errors.New("tailscale version not found in output")
}
var apkRepoVersionRE = regexp.MustCompile(`v[0-9]+\.[0-9]+`)
func checkOutdatedAlpineRepo(logf logger.Logf, apkVer, track string) error {
latest, err := LatestTailscaleVersion(track)
if err != nil {
return err
}
if latest == apkVer {
// Actually on latest release.
return nil
}
f, err := os.Open("/etc/apk/repositories")
if err != nil {
return err
}
defer f.Close()
// Read the first repo line. Typically, there are multiple repos that all
// contain the same version in the path, like:
// https://dl-cdn.alpinelinux.org/alpine/v3.20/main
// https://dl-cdn.alpinelinux.org/alpine/v3.20/community
s := bufio.NewScanner(f)
if !s.Scan() {
return s.Err()
}
alpineVer := apkRepoVersionRE.FindString(s.Text())
if alpineVer != "" {
logf("The latest Tailscale release for Linux is %q, but your apk repository only provides %q.\nYour Alpine version is %q, you may need to upgrade the system to get the latest Tailscale version: https://wiki.alpinelinux.org/wiki/Upgrading_Alpine", latest, apkVer, alpineVer)
}
return nil
}
func (up *Updater) updateMacSys() error {
return errors.New("NOTREACHED: On MacSys builds, `tailscale update` is handled in Swift to launch the GUI updater")
}
@ -846,7 +893,7 @@ func (up *Updater) installMSI(msi string) error {
break
}
up.Logf("Install attempt failed: %v", err)
uninstallVersion := version.Short()
uninstallVersion := up.currentVersion
if v := os.Getenv("TS_DEBUG_UNINSTALL_VERSION"); v != "" {
uninstallVersion = v
}
@ -1017,6 +1064,20 @@ func (up *Updater) updateLinuxBinary() error {
return nil
}
func restartSystemdUnit(ctx context.Context) error {
if _, err := exec.LookPath("systemctl"); err != nil {
// Likely not a systemd-managed distro.
return errors.ErrUnsupported
}
if out, err := exec.Command("systemctl", "daemon-reload").CombinedOutput(); err != nil {
return fmt.Errorf("systemctl daemon-reload failed: %w\noutput: %s", err, out)
}
if out, err := exec.Command("systemctl", "restart", "tailscaled.service").CombinedOutput(); err != nil {
return fmt.Errorf("systemctl restart failed: %w\noutput: %s", err, out)
}
return nil
}
func (up *Updater) downloadLinuxTarball(ver string) (string, error) {
dlDir, err := os.UserCacheDir()
if err != nil {
@ -1283,22 +1344,31 @@ func requestedTailscaleVersion(ver, track string) (string, error) {
// LatestTailscaleVersion returns the latest released version for the given
// track from pkgs.tailscale.com.
func LatestTailscaleVersion(track string) (string, error) {
if track == CurrentTrack {
if version.IsUnstableBuild() {
track = UnstableTrack
} else {
track = StableTrack
}
if track == "" {
track = CurrentTrack
}
latest, err := latestPackages(track)
if err != nil {
return "", err
}
if latest.Version == "" {
return "", fmt.Errorf("no latest version found for %q track", track)
ver := latest.Version
switch runtime.GOOS {
case "windows":
ver = latest.MSIsVersion
case "darwin":
ver = latest.MacZipsVersion
case "linux":
ver = latest.TarballsVersion
if distro.Get() == distro.Synology {
ver = latest.SPKsVersion
}
}
return latest.Version, nil
if ver == "" {
return "", fmt.Errorf("no latest version found for OS %q on %q track", runtime.GOOS, track)
}
return ver, nil
}
type trackPackages struct {

@ -663,7 +663,7 @@ func genTarball(t *testing.T, path string, files map[string]string) {
func TestWriteFileOverwrite(t *testing.T) {
path := filepath.Join(t.TempDir(), "test")
for i := 0; i < 2; i++ {
for i := range 2 {
content := fmt.Sprintf("content %d", i)
if err := writeFile(strings.NewReader(content), path, 0600); err != nil {
t.Fatal(err)
@ -846,3 +846,107 @@ func TestParseUnraidPluginVersion(t *testing.T) {
})
}
}
func TestConfirm(t *testing.T) {
curTrack := CurrentTrack
defer func() { CurrentTrack = curTrack }()
tests := []struct {
desc string
fromTrack string
toTrack string
fromVer string
toVer string
confirm func(string) bool
want bool
}{
{
desc: "on latest stable",
fromTrack: StableTrack,
toTrack: StableTrack,
fromVer: "1.66.0",
toVer: "1.66.0",
want: false,
},
{
desc: "stable upgrade",
fromTrack: StableTrack,
toTrack: StableTrack,
fromVer: "1.66.0",
toVer: "1.68.0",
want: true,
},
{
desc: "unstable upgrade",
fromTrack: UnstableTrack,
toTrack: UnstableTrack,
fromVer: "1.67.1",
toVer: "1.67.2",
want: true,
},
{
desc: "from stable to unstable",
fromTrack: StableTrack,
toTrack: UnstableTrack,
fromVer: "1.66.0",
toVer: "1.67.1",
want: true,
},
{
desc: "from unstable to stable",
fromTrack: UnstableTrack,
toTrack: StableTrack,
fromVer: "1.67.1",
toVer: "1.66.0",
want: true,
},
{
desc: "confirm callback rejects",
fromTrack: StableTrack,
toTrack: StableTrack,
fromVer: "1.66.0",
toVer: "1.66.1",
confirm: func(string) bool {
return false
},
want: false,
},
{
desc: "confirm callback allows",
fromTrack: StableTrack,
toTrack: StableTrack,
fromVer: "1.66.0",
toVer: "1.66.1",
confirm: func(string) bool {
return true
},
want: true,
},
{
desc: "downgrade",
fromTrack: StableTrack,
toTrack: StableTrack,
fromVer: "1.66.1",
toVer: "1.66.0",
want: false,
},
}
for _, tt := range tests {
t.Run(tt.desc, func(t *testing.T) {
CurrentTrack = tt.fromTrack
up := Updater{
currentVersion: tt.fromVer,
Arguments: Arguments{
Track: tt.toTrack,
Confirm: tt.confirm,
Logf: t.Logf,
},
}
if got := up.confirm(tt.toVer); got != tt.want {
t.Errorf("got %v, want %v", got, tt.want)
}
})
}
}

@ -445,7 +445,7 @@ type testServer struct {
func newTestServer(t *testing.T) *testServer {
var roots []rootKeyPair
for i := 0; i < 3; i++ {
for range 3 {
roots = append(roots, newRootKeyPair(t))
}

@ -1,37 +0,0 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
package clientupdate
import (
"context"
"errors"
"fmt"
"github.com/coreos/go-systemd/v22/dbus"
)
func restartSystemdUnit(ctx context.Context) error {
c, err := dbus.NewWithContext(ctx)
if err != nil {
// Likely not a systemd-managed distro.
return errors.ErrUnsupported
}
defer c.Close()
if err := c.ReloadContext(ctx); err != nil {
return fmt.Errorf("failed to reload tailscaled.service: %w", err)
}
ch := make(chan string, 1)
if _, err := c.RestartUnitContext(ctx, "tailscaled.service", "replace", ch); err != nil {
return fmt.Errorf("failed to restart tailscaled.service: %w", err)
}
select {
case res := <-ch:
if res != "done" {
return fmt.Errorf("systemd service restart failed with result %q", res)
}
case <-ctx.Done():
return ctx.Err()
}
return nil
}

@ -1,15 +0,0 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !linux
package clientupdate
import (
"context"
"errors"
)
func restartSystemdUnit(ctx context.Context) error {
return errors.ErrUnsupported
}

@ -102,7 +102,7 @@ func gen(buf *bytes.Buffer, it *codegen.ImportTracker, typ *types.Named) {
writef("}")
writef("dst := new(%s)", name)
writef("*dst = *src")
for i := 0; i < t.NumFields(); i++ {
for i := range t.NumFields() {
fname := t.Field(i).Name()
ft := t.Field(i).Type()
if !codegen.ContainsPointers(ft) || codegen.HasNoClone(t.Tag(i)) {

@ -8,6 +8,7 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
@ -18,36 +19,20 @@ import (
"tailscale.com/tailcfg"
)
// findKeyInKubeSecret inspects the kube secret secretName for a data
// field called "authkey", and returns its value if present.
func findKeyInKubeSecret(ctx context.Context, secretName string) (string, error) {
s, err := kc.GetSecret(ctx, secretName)
if err != nil {
return "", err
}
ak, ok := s.Data["authkey"]
if !ok {
return "", nil
}
return string(ak), nil
}
// storeDeviceInfo writes deviceID into the "device_id" data field of the kube
// secret secretName.
func storeDeviceInfo(ctx context.Context, secretName string, deviceID tailcfg.StableNodeID, fqdn string, addresses []netip.Prefix) error {
// First check if the secret exists at all. Even if running on
// kubernetes, we do not necessarily store state in a k8s secret.
if _, err := kc.GetSecret(ctx, secretName); err != nil {
if s, ok := err.(*kube.Status); ok {
if s.Code >= 400 && s.Code <= 499 {
// Assume the secret doesn't exist, or we don't have
// permission to access it.
return nil
}
// storeDeviceID writes deviceID to 'device_id' data field of the named
// Kubernetes Secret.
func storeDeviceID(ctx context.Context, secretName string, deviceID tailcfg.StableNodeID) error {
s := &kube.Secret{
Data: map[string][]byte{
"device_id": []byte(deviceID),
},
}
return err
return kc.StrategicMergePatchSecret(ctx, secretName, s, "tailscale-container")
}
// storeDeviceEndpoints writes device's tailnet IPs and MagicDNS name to fields
// 'device_ips', 'device_fqdn' of the named Kubernetes Secret.
func storeDeviceEndpoints(ctx context.Context, secretName string, fqdn string, addresses []netip.Prefix) error {
var ips []string
for _, addr := range addresses {
ips = append(ips, addr.Addr().String())
@ -57,14 +42,13 @@ func storeDeviceInfo(ctx context.Context, secretName string, deviceID tailcfg.St
return err
}
m := &kube.Secret{
s := &kube.Secret{
Data: map[string][]byte{
"device_id": []byte(deviceID),
"device_fqdn": []byte(fqdn),
"device_ips": deviceIPs,
},
}
return kc.StrategicMergePatchSecret(ctx, secretName, m, "tailscale-container")
return kc.StrategicMergePatchSecret(ctx, secretName, s, "tailscale-container")
}
// deleteAuthKey deletes the 'authkey' field of the given kube
@ -88,9 +72,59 @@ func deleteAuthKey(ctx context.Context, secretName string) error {
return nil
}
var kc *kube.Client
var kc kube.Client
// setupKube is responsible for doing any necessary configuration and checks to
// ensure that tailscale state storage and authentication mechanism will work on
// Kubernetes.
func (cfg *settings) setupKube(ctx context.Context) error {
if cfg.KubeSecret == "" {
return nil
}
canPatch, canCreate, err := kc.CheckSecretPermissions(ctx, cfg.KubeSecret)
if err != nil {
return fmt.Errorf("Some Kubernetes permissions are missing, please check your RBAC configuration: %v", err)
}
cfg.KubernetesCanPatch = canPatch
s, err := kc.GetSecret(ctx, cfg.KubeSecret)
if err != nil && kube.IsNotFoundErr(err) && !canCreate {
return fmt.Errorf("Tailscale state Secret %s does not exist and we don't have permissions to create it. "+
"If you intend to store tailscale state elsewhere than a Kubernetes Secret, "+
"you can explicitly set TS_KUBE_SECRET env var to an empty string. "+
"Else ensure that RBAC is set up that allows the service account associated with this installation to create Secrets.", cfg.KubeSecret)
} else if err != nil && !kube.IsNotFoundErr(err) {
return fmt.Errorf("Getting Tailscale state Secret %s: %v", cfg.KubeSecret, err)
}
if cfg.AuthKey == "" && !isOneStepConfig(cfg) {
if s == nil {
log.Print("TS_AUTHKEY not provided and kube secret does not exist, login will be interactive if needed.")
return nil
}
keyBytes, _ := s.Data["authkey"]
key := string(keyBytes)
if key != "" {
// This behavior of pulling authkeys from kube secrets was added
// at the same time as the patch permission, so we can enforce
// that we must be able to patch out the authkey after
// authenticating if you want to use this feature. This avoids
// us having to deal with the case where we might leave behind
// an unnecessary reusable authkey in a secret, like a rake in
// the grass.
if !cfg.KubernetesCanPatch {
return errors.New("authkey found in TS_KUBE_SECRET, but the pod doesn't have patch permissions on the secret to manage the authkey.")
}
cfg.AuthKey = key
} else {
log.Print("No authkey found in kube secret and TS_AUTHKEY not provided, login will be interactive if needed.")
}
}
return nil
}
func initKube(root string) {
func initKubeClient(root string) {
if root != "/" {
// If we are running in a test, we need to set the root path to the fake
// service account directory.
@ -101,9 +135,9 @@ func initKube(root string) {
if err != nil {
log.Fatalf("Error creating kube client: %v", err)
}
if root != "/" {
// If we are running in a test, we need to set the URL to the
// httptest server.
if (root != "/") || os.Getenv("TS_KUBERNETES_READ_API_SERVER_ADDRESS_FROM_ENV") == "true" {
// Derive the API server address from the environment variables
// Used to set http server in tests, or optionally enabled by flag
kc.SetURL(fmt.Sprintf("https://%s:%s", os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT_HTTPS")))
}
}

@ -0,0 +1,206 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build linux
package main
import (
"context"
"errors"
"testing"
"github.com/google/go-cmp/cmp"
"tailscale.com/kube"
)
func TestSetupKube(t *testing.T) {
tests := []struct {
name string
cfg *settings
wantErr bool
wantCfg *settings
kc kube.Client
}{
{
name: "TS_AUTHKEY set, state Secret exists",
cfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, false, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return nil, nil
},
},
wantCfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
},
{
name: "TS_AUTHKEY set, state Secret does not exist, we have permissions to create it",
cfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, true, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return nil, &kube.Status{Code: 404}
},
},
wantCfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
},
{
name: "TS_AUTHKEY set, state Secret does not exist, we do not have permissions to create it",
cfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, false, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return nil, &kube.Status{Code: 404}
},
},
wantCfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
wantErr: true,
},
{
name: "TS_AUTHKEY set, we encounter a non-404 error when trying to retrieve the state Secret",
cfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, false, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return nil, &kube.Status{Code: 403}
},
},
wantCfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
wantErr: true,
},
{
name: "TS_AUTHKEY set, we encounter a non-404 error when trying to check Secret permissions",
cfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
wantCfg: &settings{
AuthKey: "foo",
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, false, errors.New("broken")
},
},
wantErr: true,
},
{
// Interactive login using URL in Pod logs
name: "TS_AUTHKEY not set, state Secret does not exist, we have permissions to create it",
cfg: &settings{
KubeSecret: "foo",
},
wantCfg: &settings{
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, true, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return nil, &kube.Status{Code: 404}
},
},
},
{
// Interactive login using URL in Pod logs
name: "TS_AUTHKEY not set, state Secret exists, but does not contain auth key",
cfg: &settings{
KubeSecret: "foo",
},
wantCfg: &settings{
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, false, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return &kube.Secret{}, nil
},
},
},
{
name: "TS_AUTHKEY not set, state Secret contains auth key, we do not have RBAC to patch it",
cfg: &settings{
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return false, false, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return &kube.Secret{Data: map[string][]byte{"authkey": []byte("foo")}}, nil
},
},
wantCfg: &settings{
KubeSecret: "foo",
},
wantErr: true,
},
{
name: "TS_AUTHKEY not set, state Secret contains auth key, we have RBAC to patch it",
cfg: &settings{
KubeSecret: "foo",
},
kc: &kube.FakeClient{
CheckSecretPermissionsImpl: func(context.Context, string) (bool, bool, error) {
return true, false, nil
},
GetSecretImpl: func(context.Context, string) (*kube.Secret, error) {
return &kube.Secret{Data: map[string][]byte{"authkey": []byte("foo")}}, nil
},
},
wantCfg: &settings{
KubeSecret: "foo",
AuthKey: "foo",
KubernetesCanPatch: true,
},
},
}
for _, tt := range tests {
kc = tt.kc
t.Run(tt.name, func(t *testing.T) {
if err := tt.cfg.setupKube(context.Background()); (err != nil) != tt.wantErr {
t.Errorf("settings.setupKube() error = %v, wantErr %v", err, tt.wantErr)
}
if diff := cmp.Diff(*tt.cfg, *tt.wantCfg); diff != "" {
t.Errorf("unexpected contents of settings after running settings.setupKube()\n(-got +want):\n%s", diff)
}
})
}
}

@ -18,7 +18,11 @@
// previously advertised routes. To accept routes, use TS_EXTRA_ARGS to pass
// in --accept-routes.
// - TS_DEST_IP: proxy all incoming Tailscale traffic to the given
// destination.
// destination defined by an IP address.
// - TS_EXPERIMENTAL_DEST_DNS_NAME: proxy all incoming Tailscale traffic to the given
// destination defined by a DNS name. The DNS name will be periodically resolved and firewall rules updated accordingly.
// This is currently intended to be used by the Kubernetes operator (ExternalName Services).
// This is an experimental env var and will likely change in the future.
// - TS_TAILNET_TARGET_IP: proxy all incoming non-Tailscale traffic to the given
// destination defined by an IP.
// - TS_TAILNET_TARGET_FQDN: proxy all incoming non-Tailscale traffic to the given
@ -48,13 +52,20 @@
// ${TS_CERT_DOMAIN}, it will be replaced with the value of the available FQDN.
// It cannot be used in conjunction with TS_DEST_IP. The file is watched for changes,
// and will be re-applied when it changes.
// - EXPERIMENTAL_TS_CONFIGFILE_PATH: if specified, a path to tailscaled
// config. If this is set, TS_HOSTNAME, TS_EXTRA_ARGS, TS_AUTHKEY,
// - TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR: if specified, a path to a
// directory that containers tailscaled config in file. The config file needs to be
// named cap-<current-tailscaled-cap>.hujson. If this is set, TS_HOSTNAME,
// TS_EXTRA_ARGS, TS_AUTHKEY,
// TS_ROUTES, TS_ACCEPT_DNS env vars must not be set. If this is set,
// containerboot only runs `tailscaled --config <path-to-this-configfile>`
// and not `tailscale up` or `tailscale set`.
// The config file contents are currently read once on container start.
// NB: This env var is currently experimental and the logic will likely change!
// TS_EXPERIMENTAL_ENABLE_FORWARDING_OPTIMIZATIONS: set to true to
// autoconfigure the default network interface for optimal performance for
// Tailscale subnet router/exit node.
// https://tailscale.com/kb/1320/performance-best-practices#linux-optimizations-for-subnet-routers-and-exit-nodes
// NB: This env var is currently experimental and the logic will likely change!
// - EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS: if set to true
// and if this containerboot instance is an L7 ingress proxy (created by
// the Kubernetes operator), set up rules to allow proxying cluster traffic,
@ -82,12 +93,16 @@ import (
"fmt"
"io/fs"
"log"
"math"
"net"
"net/netip"
"os"
"os/exec"
"os/signal"
"path"
"path/filepath"
"reflect"
"slices"
"strconv"
"strings"
"sync"
@ -100,6 +115,7 @@ import (
"tailscale.com/client/tailscale"
"tailscale.com/ipn"
"tailscale.com/ipn/conffile"
kubeutils "tailscale.com/k8s-operator"
"tailscale.com/tailcfg"
"tailscale.com/types/logger"
"tailscale.com/types/ptr"
@ -122,7 +138,8 @@ func main() {
Hostname: defaultEnv("TS_HOSTNAME", ""),
Routes: defaultEnvStringPointer("TS_ROUTES"),
ServeConfigPath: defaultEnv("TS_SERVE_CONFIG", ""),
ProxyTo: defaultEnv("TS_DEST_IP", ""),
ProxyTargetIP: defaultEnv("TS_DEST_IP", ""),
ProxyTargetDNSName: defaultEnv("TS_EXPERIMENTAL_DEST_DNS_NAME", ""),
TailnetTargetIP: defaultEnv("TS_TAILNET_TARGET_IP", ""),
TailnetTargetFQDN: defaultEnv("TS_TAILNET_TARGET_FQDN", ""),
DaemonExtraArgs: defaultEnv("TS_TAILSCALED_EXTRA_ARGS", ""),
@ -137,9 +154,10 @@ func main() {
Socket: defaultEnv("TS_SOCKET", "/tmp/tailscaled.sock"),
AuthOnce: defaultBool("TS_AUTH_ONCE", false),
Root: defaultEnv("TS_TEST_ONLY_ROOT", "/"),
TailscaledConfigFilePath: defaultEnv("EXPERIMENTAL_TS_CONFIGFILE_PATH", ""),
TailscaledConfigFilePath: tailscaledConfigFilePath(),
AllowProxyingClusterTrafficViaIngress: defaultBool("EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS", false),
PodIP: defaultEnv("POD_IP", ""),
EnableForwardingOptimizations: defaultBool("TS_EXPERIMENTAL_ENABLE_FORWARDING_OPTIMIZATIONS", false),
}
if err := cfg.validate(); err != nil {
@ -150,8 +168,8 @@ func main() {
if err := ensureTunFile(cfg.Root); err != nil {
log.Fatalf("Unable to create tuntap device file: %v", err)
}
if cfg.ProxyTo != "" || cfg.Routes != nil || cfg.TailnetTargetIP != "" || cfg.TailnetTargetFQDN != "" {
if err := ensureIPForwarding(cfg.Root, cfg.ProxyTo, cfg.TailnetTargetIP, cfg.TailnetTargetFQDN, cfg.Routes); err != nil {
if cfg.ProxyTargetIP != "" || cfg.ProxyTargetDNSName != "" || cfg.Routes != nil || cfg.TailnetTargetIP != "" || cfg.TailnetTargetFQDN != "" {
if err := ensureIPForwarding(cfg.Root, cfg.ProxyTargetIP, cfg.TailnetTargetIP, cfg.TailnetTargetFQDN, cfg.Routes); err != nil {
log.Printf("Failed to enable IP forwarding: %v", err)
log.Printf("To run tailscale as a proxy or router container, IP forwarding must be enabled.")
if cfg.InKubernetes {
@ -163,44 +181,16 @@ func main() {
}
}
if cfg.InKubernetes {
initKube(cfg.Root)
}
// Context is used for all setup stuff until we're in steady
// state, so that if something is hanging we eventually time out
// and crashloop the container.
bootCtx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
if cfg.InKubernetes && cfg.KubeSecret != "" {
canPatch, err := kc.CheckSecretPermissions(bootCtx, cfg.KubeSecret)
if err != nil {
log.Fatalf("Some Kubernetes permissions are missing, please check your RBAC configuration: %v", err)
}
cfg.KubernetesCanPatch = canPatch
if cfg.AuthKey == "" && !isOneStepConfig(cfg) {
key, err := findKeyInKubeSecret(bootCtx, cfg.KubeSecret)
if err != nil {
log.Fatalf("Getting authkey from kube secret: %v", err)
}
if key != "" {
// This behavior of pulling authkeys from kube secrets was added
// at the same time as the patch permission, so we can enforce
// that we must be able to patch out the authkey after
// authenticating if you want to use this feature. This avoids
// us having to deal with the case where we might leave behind
// an unnecessary reusable authkey in a secret, like a rake in
// the grass.
if !cfg.KubernetesCanPatch {
log.Fatalf("authkey found in TS_KUBE_SECRET, but the pod doesn't have patch permissions on the secret to manage the authkey.")
}
log.Print("Using authkey found in kube secret")
cfg.AuthKey = key
} else {
log.Print("No authkey found in kube secret and TS_AUTHKEY not provided, login will be interactive if needed.")
}
if cfg.InKubernetes {
initKubeClient(cfg.Root)
if err := cfg.setupKube(bootCtx); err != nil {
log.Fatalf("error setting up for running on Kubernetes: %v", err)
}
}
@ -215,6 +205,12 @@ func main() {
}
defer killTailscaled()
if cfg.EnableForwardingOptimizations {
if err := client.SetUDPGROForwarding(bootCtx); err != nil {
log.Printf("[unexpected] error enabling UDP GRO forwarding: %v", err)
}
}
w, err := client.WatchIPNBus(bootCtx, ipn.NotifyInitialNetMap|ipn.NotifyInitialPrefs|ipn.NotifyInitialState)
if err != nil {
log.Fatalf("failed to watch tailscaled for updates: %v", err)
@ -325,7 +321,7 @@ authLoop:
}
}
if cfg.InKubernetes && cfg.KubeSecret != "" && cfg.KubernetesCanPatch && isTwoStepConfigAuthOnce(cfg) {
if hasKubeStateStore(cfg) && isTwoStepConfigAuthOnce(cfg) {
// We were told to only auth once, so any secret-bound
// authkey is no longer needed. We don't strictly need to
// wipe it, but it's good hygiene.
@ -341,14 +337,16 @@ authLoop:
}
var (
wantProxy = cfg.ProxyTo != "" || cfg.TailnetTargetIP != "" || cfg.TailnetTargetFQDN != "" || cfg.AllowProxyingClusterTrafficViaIngress
wantDeviceInfo = cfg.InKubernetes && cfg.KubeSecret != "" && cfg.KubernetesCanPatch
startupTasksDone = false
currentIPs deephash.Sum // tailscale IPs assigned to device
currentDeviceInfo deephash.Sum // device ID and fqdn
currentDeviceID deephash.Sum // device ID
currentDeviceEndpoints deephash.Sum // device FQDN and IPs
currentEgressIPs deephash.Sum
addrs []netip.Prefix
backendAddrs []net.IP
certDomain = new(atomic.Pointer[string])
certDomainChanged = make(chan bool, 1)
)
@ -356,12 +354,50 @@ authLoop:
go watchServeConfigChanges(ctx, cfg.ServeConfigPath, certDomainChanged, certDomain, client)
}
var nfr linuxfw.NetfilterRunner
if wantProxy {
if isL3Proxy(cfg) {
nfr, err = newNetfilterRunner(log.Printf)
if err != nil {
log.Fatalf("error creating new netfilter runner: %v", err)
}
}
// Setup for proxies that are configured to proxy to a target specified
// by a DNS name (TS_EXPERIMENTAL_DEST_DNS_NAME).
const defaultCheckPeriod = time.Minute * 10 // how often to check what IPs the DNS name resolves to
var (
tc = make(chan string, 1)
failedResolveAttempts int
t *time.Timer = time.AfterFunc(defaultCheckPeriod, func() {
if cfg.ProxyTargetDNSName != "" {
tc <- "recheck"
}
})
)
defer t.Stop()
// resetTimer resets timer for when to next attempt to resolve the DNS
// name for the proxy configured with TS_EXPERIMENTAL_DEST_DNS_NAME. The
// timer gets reset to 10 minutes from now unless the last resolution
// attempt failed. If one or more consecutive previous resolution
// attempts failed, the next resolution attempt will happen after the smallest
// of (10 minutes, 2 ^ number-of-consecutive-failed-resolution-attempts
// seconds) i.e 2s, 4s, 8s ... 10 minutes.
resetTimer := func(lastResolveFailed bool) {
if !lastResolveFailed {
log.Printf("reconfigureTimer: next DNS resolution attempt in %s", defaultCheckPeriod)
t.Reset(defaultCheckPeriod)
failedResolveAttempts = 0
return
}
minDelay := 2 // 2 seconds
nextTick := time.Second * time.Duration(math.Pow(float64(minDelay), float64(failedResolveAttempts)))
if nextTick > defaultCheckPeriod {
nextTick = defaultCheckPeriod // cap at 10 minutes
}
log.Printf("reconfigureTimer: last DNS resolution attempt failed, next DNS resolution attempt in %v", nextTick)
t.Reset(nextTick)
failedResolveAttempts++
}
notifyChan := make(chan ipn.Notify)
errChan := make(chan error)
go func() {
@ -399,10 +435,24 @@ runLoop:
log.Fatalf("tailscaled left running state (now in state %q), exiting", *n.State)
}
if n.NetMap != nil {
addrs := n.NetMap.SelfNode.Addresses().AsSlice()
addrs = n.NetMap.SelfNode.Addresses().AsSlice()
newCurrentIPs := deephash.Hash(&addrs)
ipsHaveChanged := newCurrentIPs != currentIPs
// Store device ID in a Kubernetes Secret before
// setting up any routing rules. This ensures
// that, for containerboot instances that are
// Kubernetes operator proxies, the operator is
// able to retrieve the device ID from the
// Kubernetes Secret to clean up tailnet nodes
// for proxies whose route setup continuously
// fails.
deviceID := n.NetMap.SelfNode.StableID()
if hasKubeStateStore(cfg) && deephash.Update(&currentDeviceID, &deviceID) {
if err := storeDeviceID(ctx, cfg.KubeSecret, n.NetMap.SelfNode.StableID()); err != nil {
log.Fatalf("storing device ID in Kubernetes Secret: %v", err)
}
}
if cfg.TailnetTargetFQDN != "" {
var (
egressAddrs []netip.Prefix
@ -425,7 +475,7 @@ runLoop:
egressAddrs = node.Addresses().AsSlice()
newCurentEgressIPs = deephash.Hash(&egressAddrs)
egressIPsHaveChanged = newCurentEgressIPs != currentEgressIPs
if egressIPsHaveChanged && len(egressAddrs) > 0 {
if egressIPsHaveChanged && len(egressAddrs) != 0 {
for _, egressAddr := range egressAddrs {
ea := egressAddr.Addr()
// TODO (irbekrm): make it work for IPv6 too.
@ -441,13 +491,32 @@ runLoop:
}
currentEgressIPs = newCurentEgressIPs
}
if cfg.ProxyTo != "" && len(addrs) > 0 && ipsHaveChanged {
if cfg.ProxyTargetIP != "" && len(addrs) != 0 && ipsHaveChanged {
log.Printf("Installing proxy rules")
if err := installIngressForwardingRule(ctx, cfg.ProxyTo, addrs, nfr); err != nil {
if err := installIngressForwardingRule(ctx, cfg.ProxyTargetIP, addrs, nfr); err != nil {
log.Fatalf("installing ingress proxy rules: %v", err)
}
}
if cfg.ServeConfigPath != "" && len(n.NetMap.DNS.CertDomains) > 0 {
if cfg.ProxyTargetDNSName != "" && len(addrs) != 0 && ipsHaveChanged {
newBackendAddrs, err := resolveDNS(ctx, cfg.ProxyTargetDNSName)
if err != nil {
log.Printf("[unexpected] error resolving DNS name %s: %v", cfg.ProxyTargetDNSName, err)
resetTimer(true)
continue
}
backendsHaveChanged := !(slices.EqualFunc(backendAddrs, newBackendAddrs, func(ip1 net.IP, ip2 net.IP) bool {
return slices.ContainsFunc(newBackendAddrs, func(ip net.IP) bool { return ip.Equal(ip1) })
}))
if backendsHaveChanged {
log.Printf("installing ingress proxy rules for backends %v", newBackendAddrs)
if err := installIngressForwardingRuleForDNSTarget(ctx, newBackendAddrs, addrs, nfr); err != nil {
log.Fatalf("error installing ingress proxy rules: %v", err)
}
}
resetTimer(false)
backendAddrs = newBackendAddrs
}
if cfg.ServeConfigPath != "" && len(n.NetMap.DNS.CertDomains) != 0 {
cd := n.NetMap.DNS.CertDomains[0]
prev := certDomain.Swap(ptr.To(cd))
if prev == nil || *prev != cd {
@ -457,7 +526,7 @@ runLoop:
}
}
}
if cfg.TailnetTargetIP != "" && ipsHaveChanged && len(addrs) > 0 {
if cfg.TailnetTargetIP != "" && ipsHaveChanged && len(addrs) != 0 {
log.Printf("Installing forwarding rules for destination %v", cfg.TailnetTargetIP)
if err := installEgressForwardingRule(ctx, cfg.TailnetTargetIP, addrs, nfr); err != nil {
log.Fatalf("installing egress proxy rules: %v", err)
@ -469,7 +538,7 @@ runLoop:
// enabled, set up proxy rule each time the
// tailnet IPs of this node change (including
// the first time they become available).
if cfg.AllowProxyingClusterTrafficViaIngress && cfg.ServeConfigPath != "" && ipsHaveChanged && len(addrs) > 0 {
if cfg.AllowProxyingClusterTrafficViaIngress && cfg.ServeConfigPath != "" && ipsHaveChanged && len(addrs) != 0 {
log.Printf("installing rules to forward traffic for %s to node's tailnet IP", cfg.PodIP)
if err := installTSForwardingRuleForDestination(ctx, cfg.PodIP, addrs, nfr); err != nil {
log.Fatalf("installing rules to forward traffic to node's tailnet IP: %v", err)
@ -477,46 +546,85 @@ runLoop:
}
currentIPs = newCurrentIPs
deviceInfo := []any{n.NetMap.SelfNode.StableID(), n.NetMap.SelfNode.Name()}
if cfg.InKubernetes && cfg.KubernetesCanPatch && cfg.KubeSecret != "" && deephash.Update(&currentDeviceInfo, &deviceInfo) {
if err := storeDeviceInfo(ctx, cfg.KubeSecret, n.NetMap.SelfNode.StableID(), n.NetMap.SelfNode.Name(), n.NetMap.SelfNode.Addresses().AsSlice()); err != nil {
log.Fatalf("storing device ID in kube secret: %v", err)
// Only store device FQDN and IP addresses to
// Kubernetes Secret when any required proxy
// route setup has succeeded. IPs and FQDN are
// read from the Secret by the Tailscale
// Kubernetes operator and, for some proxy
// types, such as Tailscale Ingress, advertized
// on the Ingress status. Writing them to the
// Secret only after the proxy routing has been
// set up ensures that the operator does not
// advertize endpoints of broken proxies.
// TODO (irbekrm): instead of using the IP and FQDN, have some other mechanism for the proxy signal that it is 'Ready'.
deviceEndpoints := []any{n.NetMap.SelfNode.Name(), n.NetMap.SelfNode.Addresses()}
if hasKubeStateStore(cfg) && deephash.Update(&currentDeviceEndpoints, &deviceEndpoints) {
if err := storeDeviceEndpoints(ctx, cfg.KubeSecret, n.NetMap.SelfNode.Name(), n.NetMap.SelfNode.Addresses().AsSlice()); err != nil {
log.Fatalf("storing device IPs and FQDN in Kubernetes Secret: %v", err)
}
}
}
if !startupTasksDone {
if (!wantProxy || currentIPs != deephash.Sum{}) && (!wantDeviceInfo || currentDeviceInfo != deephash.Sum{}) {
// For containerboot instances that act as TCP
// proxies (proxying traffic to an endpoint
// passed via one of the env vars that
// containerbot reads) and store state in a
// Kubernetes Secret, we consider startup tasks
// done at the point when device info has been
// successfully stored to state Secret.
// For all other containerboot instances, if we
// just get to this point the startup tasks can
// be considered done.
if !isL3Proxy(cfg) || !hasKubeStateStore(cfg) || (currentDeviceEndpoints != deephash.Sum{} && currentDeviceID != deephash.Sum{}) {
// This log message is used in tests to detect when all
// post-auth configuration is done.
log.Println("Startup complete, waiting for shutdown signal")
startupTasksDone = true
// Reap all processes, since we are PID1 and need to collect zombies. We can
// only start doing this once we've stopped shelling out to things
// `tailscale up`, otherwise this goroutine can reap the CLI subprocesses
// and wedge bringup.
// Wait on tailscaled process. It won't
// be cleaned up by default when the
// container exits as it is not PID1.
// TODO (irbekrm): perhaps we can
// replace the reaper by a running
// cmd.Wait in a goroutine immediately
// after starting tailscaled?
reaper := func() {
defer wg.Done()
for {
var status unix.WaitStatus
pid, err := unix.Wait4(-1, &status, 0, nil)
_, err := unix.Wait4(daemonProcess.Pid, &status, 0, nil)
if errors.Is(err, unix.EINTR) {
continue
}
if err != nil {
log.Fatalf("Waiting for exited processes: %v", err)
log.Fatalf("Waiting for tailscaled to exit: %v", err)
}
if pid == daemonProcess.Pid {
log.Printf("Tailscaled exited")
log.Print("tailscaled exited")
os.Exit(0)
}
}
}
wg.Add(1)
go reaper()
}
}
case <-tc:
newBackendAddrs, err := resolveDNS(ctx, cfg.ProxyTargetDNSName)
if err != nil {
log.Printf("[unexpected] error resolving DNS name %s: %v", cfg.ProxyTargetDNSName, err)
resetTimer(true)
continue
}
backendsHaveChanged := !(slices.EqualFunc(backendAddrs, newBackendAddrs, func(ip1 net.IP, ip2 net.IP) bool {
return slices.ContainsFunc(newBackendAddrs, func(ip net.IP) bool { return ip.Equal(ip1) })
}))
if backendsHaveChanged && len(addrs) != 0 {
log.Printf("Backend address change detected, installing proxy rules for backends %v", newBackendAddrs)
if err := installIngressForwardingRuleForDNSTarget(ctx, newBackendAddrs, addrs, nfr); err != nil {
log.Fatalf("installing ingress proxy rules for DNS target %s: %v", cfg.ProxyTargetDNSName, err)
}
}
backendAddrs = newBackendAddrs
resetTimer(false)
}
}
wg.Wait()
@ -757,12 +865,12 @@ func ensureTunFile(root string) error {
}
// ensureIPForwarding enables IPv4/IPv6 forwarding for the container.
func ensureIPForwarding(root, clusterProxyTarget, tailnetTargetiP, tailnetTargetFQDN string, routes *string) error {
func ensureIPForwarding(root, clusterProxyTargetIP, tailnetTargetIP, tailnetTargetFQDN string, routes *string) error {
var (
v4Forwarding, v6Forwarding bool
)
if clusterProxyTarget != "" {
proxyIP, err := netip.ParseAddr(clusterProxyTarget)
if clusterProxyTargetIP != "" {
proxyIP, err := netip.ParseAddr(clusterProxyTargetIP)
if err != nil {
return fmt.Errorf("invalid cluster destination IP: %v", err)
}
@ -772,8 +880,8 @@ func ensureIPForwarding(root, clusterProxyTarget, tailnetTargetiP, tailnetTarget
v6Forwarding = true
}
}
if tailnetTargetiP != "" {
proxyIP, err := netip.ParseAddr(tailnetTargetiP)
if tailnetTargetIP != "" {
proxyIP, err := netip.ParseAddr(tailnetTargetIP)
if err != nil {
return fmt.Errorf("invalid tailnet destination IP: %v", err)
}
@ -801,7 +909,10 @@ func ensureIPForwarding(root, clusterProxyTarget, tailnetTargetiP, tailnetTarget
}
}
}
return enableIPForwarding(v4Forwarding, v6Forwarding, root)
}
func enableIPForwarding(v4Forwarding, v6Forwarding bool, root string) error {
var paths []string
if v4Forwarding {
paths = append(paths, filepath.Join(root, "proc/sys/net/ipv4/ip_forward"))
@ -896,16 +1007,23 @@ func installIngressForwardingRule(ctx context.Context, dstStr string, tsIPs []ne
return err
}
var local netip.Addr
proxyHasIPv4Address := false
for _, pfx := range tsIPs {
if !pfx.IsSingleIP() {
continue
}
if pfx.Addr().Is4() {
proxyHasIPv4Address = true
}
if pfx.Addr().Is4() != dst.Is4() {
continue
}
local = pfx.Addr()
break
}
if proxyHasIPv4Address && dst.Is6() {
log.Printf("Warning: proxy backend ClusterIP is an IPv6 address and the proxy has a IPv4 tailnet address. You might need to disable IPv4 address allocation for the proxy for forwarding to work. See https://github.com/tailscale/tailscale/issues/12156")
}
if !local.IsValid() {
return fmt.Errorf("no tailscale IP matching family of %s found in %v", dstStr, tsIPs)
}
@ -918,15 +1036,89 @@ func installIngressForwardingRule(ctx context.Context, dstStr string, tsIPs []ne
return nil
}
func installIngressForwardingRuleForDNSTarget(ctx context.Context, backendAddrs []net.IP, tsIPs []netip.Prefix, nfr linuxfw.NetfilterRunner) error {
var (
tsv4 netip.Addr
tsv6 netip.Addr
v4Backends []netip.Addr
v6Backends []netip.Addr
)
for _, pfx := range tsIPs {
if pfx.IsSingleIP() && pfx.Addr().Is4() {
tsv4 = pfx.Addr()
continue
}
if pfx.IsSingleIP() && pfx.Addr().Is6() {
tsv6 = pfx.Addr()
continue
}
}
// TODO: log if more than one backend address is found and firewall is
// in nftables mode that only the first IP will be used.
for _, ip := range backendAddrs {
if ip.To4() != nil {
v4Backends = append(v4Backends, netip.AddrFrom4([4]byte(ip.To4())))
}
if ip.To16() != nil {
v6Backends = append(v6Backends, netip.AddrFrom16([16]byte(ip.To16())))
}
}
// Enable IP forwarding here as opposed to at the start of containerboot
// as the IPv4/IPv6 requirements might have changed.
// For Kubernetes operator proxies, forwarding for both IPv4 and IPv6 is
// enabled by an init container, so in practice enabling forwarding here
// is only needed if this proxy has been configured by manually setting
// TS_EXPERIMENTAL_DEST_DNS_NAME env var for a containerboot instance.
if err := enableIPForwarding(len(v4Backends) != 0, len(v6Backends) != 0, ""); err != nil {
log.Printf("[unexpected] failed to ensure IP forwarding: %v", err)
}
updateFirewall := func(dst netip.Addr, backendTargets []netip.Addr) error {
if err := nfr.DNATWithLoadBalancer(dst, backendTargets); err != nil {
return fmt.Errorf("installing DNAT rules for ingress backends %+#v: %w", backendTargets, err)
}
// The backend might advertize MSS higher than that of the
// tailscale interfaces. Clamp MSS of packets going out via
// tailscale0 interface to its MTU to prevent broken connections
// in environments where path MTU discovery is not working.
if err := nfr.ClampMSSToPMTU("tailscale0", dst); err != nil {
return fmt.Errorf("adding rule to clamp traffic via tailscale0: %v", err)
}
return nil
}
if len(v4Backends) != 0 {
if !tsv4.IsValid() {
log.Printf("backend targets %v contain at least one IPv4 address, but this node's Tailscale IPs do not contain a valid IPv4 address: %v", backendAddrs, tsIPs)
} else if err := updateFirewall(tsv4, v4Backends); err != nil {
return fmt.Errorf("Installing IPv4 firewall rules: %w", err)
}
}
if len(v6Backends) != 0 && !tsv6.IsValid() {
if !tsv6.IsValid() {
log.Printf("backend targets %v contain at least one IPv6 address, but this node's Tailscale IPs do not contain a valid IPv6 address: %v", backendAddrs, tsIPs)
} else if !nfr.HasIPV6NAT() {
log.Printf("backend targets %v contain at least one IPv6 address, but the chosen firewall mode does not support IPv6 NAT", backendAddrs)
} else if err := updateFirewall(tsv6, v6Backends); err != nil {
return fmt.Errorf("Installing IPv6 firewall rules: %w", err)
}
}
return nil
}
// settings is all the configuration for containerboot.
type settings struct {
AuthKey string
Hostname string
Routes *string
// ProxyTo is the destination IP to which all incoming
// ProxyTargetIP is the destination IP to which all incoming
// Tailscale traffic should be proxied. If empty, no proxying
// is done. This is typically a locally reachable IP.
ProxyTo string
ProxyTargetIP string
// ProxyTargetDNSName is a DNS name to whose backing IP addresses all
// incoming Tailscale traffic should be proxied.
ProxyTargetDNSName string
// TailnetTargetIP is the destination IP to which all incoming
// non-Tailscale traffic should be proxied. This is typically a
// Tailscale IP.
@ -950,6 +1142,7 @@ type settings struct {
Root string
KubernetesCanPatch bool
TailscaledConfigFilePath string
EnableForwardingOptimizations bool
// If set to true and, if this containerboot instance is a Kubernetes
// ingress proxy, set up rules to forward incoming cluster traffic to be
// forwarded to the ingress target in cluster.
@ -962,13 +1155,26 @@ type settings struct {
func (s *settings) validate() error {
if s.TailscaledConfigFilePath != "" {
dir, file := path.Split(s.TailscaledConfigFilePath)
if _, err := os.Stat(dir); err != nil {
return fmt.Errorf("error validating whether directory with tailscaled config file %s exists: %w", dir, err)
}
if _, err := os.Stat(s.TailscaledConfigFilePath); err != nil {
return fmt.Errorf("error validating whether tailscaled config directory %q contains tailscaled config for current capability version %q: %w. If this is a Tailscale Kubernetes operator proxy, please ensure that the version of the operator is not older than the version of the proxy", dir, file, err)
}
if _, err := conffile.Load(s.TailscaledConfigFilePath); err != nil {
return fmt.Errorf("error validating tailscaled configfile contents: %w", err)
}
}
if s.ProxyTo != "" && s.UserspaceMode {
if s.ProxyTargetIP != "" && s.UserspaceMode {
return errors.New("TS_DEST_IP is not supported with TS_USERSPACE")
}
if s.ProxyTargetDNSName != "" && s.UserspaceMode {
return errors.New("TS_EXPERIMENTAL_DEST_DNS_NAME is not supported with TS_USERSPACE")
}
if s.ProxyTargetDNSName != "" && s.ProxyTargetIP != "" {
return errors.New("TS_EXPERIMENTAL_DEST_DNS_NAME and TS_DEST_IP cannot both be set")
}
if s.TailnetTargetIP != "" && s.UserspaceMode {
return errors.New("TS_TAILNET_TARGET_IP is not supported with TS_USERSPACE")
}
@ -979,7 +1185,7 @@ func (s *settings) validate() error {
return errors.New("Both TS_TAILNET_TARGET_IP and TS_TAILNET_FQDN cannot be set")
}
if s.TailscaledConfigFilePath != "" && (s.AcceptDNS != nil || s.AuthKey != "" || s.Routes != nil || s.ExtraArgs != "" || s.Hostname != "") {
return errors.New("EXPERIMENTAL_TS_CONFIGFILE_PATH cannot be set in combination with TS_HOSTNAME, TS_EXTRA_ARGS, TS_AUTHKEY, TS_ROUTES, TS_ACCEPT_DNS.")
return errors.New("TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR cannot be set in combination with TS_HOSTNAME, TS_EXTRA_ARGS, TS_AUTHKEY, TS_ROUTES, TS_ACCEPT_DNS.")
}
if s.AllowProxyingClusterTrafficViaIngress && s.UserspaceMode {
return errors.New("EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS is not supported in userspace mode")
@ -990,9 +1196,34 @@ func (s *settings) validate() error {
if s.AllowProxyingClusterTrafficViaIngress && s.PodIP == "" {
return errors.New("EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS is set but POD_IP is not set")
}
if s.EnableForwardingOptimizations && s.UserspaceMode {
return errors.New("TS_EXPERIMENTAL_ENABLE_FORWARDING_OPTIMIZATIONS is not supported in userspace mode")
}
return nil
}
func resolveDNS(ctx context.Context, name string) ([]net.IP, error) {
// TODO (irbekrm): look at using recursive.Resolver instead to resolve
// the DNS names as well as retrieve TTLs. It looks though that this
// seems to return very short TTLs (shorter than on the actual records).
ip4s, err := net.DefaultResolver.LookupIP(ctx, "ip4", name)
if err != nil {
if e, ok := err.(*net.DNSError); !(ok && e.IsNotFound) {
return nil, fmt.Errorf("error looking up IPv4 addresses: %v", err)
}
}
ip6s, err := net.DefaultResolver.LookupIP(ctx, "ip6", name)
if err != nil {
if e, ok := err.(*net.DNSError); !(ok && e.IsNotFound) {
return nil, fmt.Errorf("error looking up IPv6 addresses: %v", err)
}
}
if len(ip4s) == 0 && len(ip6s) == 0 {
return nil, fmt.Errorf("no IPv4 or IPv6 addresses found for host: %s", name)
}
return append(ip4s, ip6s...), nil
}
// defaultEnv returns the value of the given envvar name, or defVal if
// unset.
func defaultEnv(name, defVal string) string {
@ -1089,3 +1320,55 @@ func isTwoStepConfigAlwaysAuth(cfg *settings) bool {
func isOneStepConfig(cfg *settings) bool {
return cfg.TailscaledConfigFilePath != ""
}
// isL3Proxy returns true if the Tailscale node needs to be configured to act
// as an L3 proxy, proxying to an endpoint provided via one of the config env
// vars.
func isL3Proxy(cfg *settings) bool {
return cfg.ProxyTargetIP != "" || cfg.ProxyTargetDNSName != "" || cfg.TailnetTargetIP != "" || cfg.TailnetTargetFQDN != "" || cfg.AllowProxyingClusterTrafficViaIngress
}
// hasKubeStateStore returns true if the state must be stored in a Kubernetes
// Secret.
func hasKubeStateStore(cfg *settings) bool {
return cfg.InKubernetes && cfg.KubernetesCanPatch && cfg.KubeSecret != ""
}
// tailscaledConfigFilePath returns the path to the tailscaled config file that
// should be used for the current capability version. It is determined by the
// TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR environment variable and looks for a
// file named cap-<capability_version>.hujson in the directory. It searches for
// the highest capability version that is less than or equal to the current
// capability version.
func tailscaledConfigFilePath() string {
dir := os.Getenv("TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR")
if dir == "" {
return ""
}
fe, err := os.ReadDir(dir)
if err != nil {
log.Fatalf("error reading tailscaled config directory %q: %v", dir, err)
}
maxCompatVer := tailcfg.CapabilityVersion(-1)
for _, e := range fe {
// We don't check if type if file as in most cases this will
// come from a mounted kube Secret, where the directory contents
// will be various symlinks.
if e.Type().IsDir() {
continue
}
cv, err := kubeutils.CapVerFromFileName(e.Name())
if err != nil {
log.Printf("skipping file %q in tailscaled config directory %q: %v", e.Name(), dir, err)
continue
}
if cv > maxCompatVer && cv <= tailcfg.CurrentCapabilityVersion {
maxCompatVer = cv
}
}
if maxCompatVer == -1 {
log.Fatalf("no tailscaled config file found in %q for current capability version %q", dir, tailcfg.CurrentCapabilityVersion)
}
log.Printf("Using tailscaled config file %q for capability version %q", maxCompatVer, tailcfg.CurrentCapabilityVersion)
return path.Join(dir, kubeutils.TailscaledConfigFileNameForCap(maxCompatVer))
}

@ -65,7 +65,7 @@ func TestContainerBoot(t *testing.T) {
"dev/net",
"proc/sys/net/ipv4",
"proc/sys/net/ipv6/conf/all",
"etc",
"etc/tailscaled",
}
for _, path := range dirs {
if err := os.MkdirAll(filepath.Join(d, path), 0700); err != nil {
@ -80,7 +80,7 @@ func TestContainerBoot(t *testing.T) {
"dev/net/tun": []byte(""),
"proc/sys/net/ipv4/ip_forward": []byte("0"),
"proc/sys/net/ipv6/conf/all/forwarding": []byte("0"),
"etc/tailscaled": tailscaledConfBytes,
"etc/tailscaled/cap-95.hujson": tailscaledConfBytes,
}
resetFiles := func() {
for path, content := range files {
@ -638,14 +638,14 @@ func TestContainerBoot(t *testing.T) {
},
},
{
Name: "experimental tailscaled configfile",
Name: "experimental tailscaled config path",
Env: map[string]string{
"EXPERIMENTAL_TS_CONFIGFILE_PATH": filepath.Join(d, "etc/tailscaled"),
"TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR": filepath.Join(d, "etc/tailscaled/"),
},
Phases: []phase{
{
WantCmds: []string{
"/usr/bin/tailscaled --socket=/tmp/tailscaled.sock --state=mem: --statedir=/tmp --tun=userspace-networking --config=/etc/tailscaled",
"/usr/bin/tailscaled --socket=/tmp/tailscaled.sock --state=mem: --statedir=/tmp --tun=userspace-networking --config=/etc/tailscaled/cap-95.hujson",
},
}, {
Notify: runningNotify,

@ -0,0 +1,74 @@
# DERP
This is the code for the [Tailscale DERP server](https://tailscale.com/kb/1232/derp-servers).
In general, you should not need to nor want to run this code. The overwhelming majority of Tailscale users (both individuals and companies) do not.
In the happy path, Tailscale establishes direct connections between peers and
data plane traffic flows directly between them, without using DERP for more than
acting as a low bandwidth side channel to bootstrap the NAT traversal. If you
find yourself wanting DERP for more bandwidth, the real problem is usually the
network configuration of your Tailscale node(s), making sure that Tailscale can
get direction connections via some mechanism.
But if you've decided or been advised to run your own `derper`, then read on.
## Caveats
* Node sharing and other cross-Tailnet features don't work when using custom
DERP servers.
* DERP servers only see encrypted WireGuard packets and thus are not useful for
network-level debugging.
* The Tailscale control plane does certain geo-level steering features and
optimizations that are not available when using custom DERP servers.
## Guide to running `cmd/derper`
* You must build and update the `cmd/derper` binary yourself. There are no
packages. Use `go install tailscale.com/cmd/derper@latest` with the latest
version of Go.
* The DERP protocol does a protocol switch inside TLS from HTTP to a custom
bidirectional binary protocol. It is thus incompatible with many HTTP proxies.
Do not put `derper` behind another HTTP proxy.
* The `tailscaled` client does its own selection of the fastest/nearest DERP
server based on latency measurements. Do not put `derper` behind a global load
balancer.
* DERP servers should ideally have both a static IPv4 and static IPv6 address.
Both of those should be listed in the DERP map so the client doesn't need to
rely on its DNS which might be broken and dependent on DERP to get back up.
* A DERP server should not share an IP address with any other DERP server.
* Avoid having multiple DERP nodes in a region. If you must, they all need to be
meshed with each other and monitored. Having two one-node "regions" in the
same datacenter is usually easier and more reliable than meshing, at the cost
of more required connections from clients in some cases. If your clients
aren't mobile (battery constrained), one node regions are definitely
preferred. If you really need multiple nodes in a region for HA reasons, two
is sufficient.
* Monitor your DERP servers with [`cmd/derpprobe`](../derpprobe/).
* If using `--verify-clients`, a `tailscaled` must be running alongside the
`derper`.
* If using `--verify-clients`, a `tailscaled` must also be running alongside
your `derpprobe`, and `derpprobe` needs to use `--derp-map=local`.
* The firewall on the `derper` should permit TCP ports 80 and 443 and UDP port
3478.
* Only LetsEncrypt certs are rotated automatically. Other cert updates require a
restart.
* Don't use a firewall in front of `derper` that suppresses `RST`s upon
receiving traffic to a dead or unknown connection.
* Don't rate-limit UDP STUN packets.
* Don't rate-limit outbound TCP traffic (only inbound).

@ -5,26 +5,35 @@ package main
import (
"context"
"encoding/binary"
"encoding/json"
"expvar"
"log"
"math/rand/v2"
"net"
"net/http"
"net/netip"
"strconv"
"strings"
"sync/atomic"
"time"
"tailscale.com/syncs"
"tailscale.com/util/mak"
"tailscale.com/util/slicesx"
)
const refreshTimeout = time.Minute
type dnsEntryMap map[string][]net.IP
type dnsEntryMap struct {
IPs map[string][]net.IP
Percent map[string]float64 // "foo.com" => 0.5 for 50%
}
var (
dnsCache syncs.AtomicValue[dnsEntryMap]
dnsCache atomic.Pointer[dnsEntryMap]
dnsCacheBytes syncs.AtomicValue[[]byte] // of JSON
unpublishedDNSCache syncs.AtomicValue[dnsEntryMap]
unpublishedDNSCache atomic.Pointer[dnsEntryMap]
bootstrapLookupMap syncs.Map[string, bool]
)
@ -34,6 +43,7 @@ var (
publishedDNSMisses = expvar.NewInt("counter_bootstrap_dns_published_misses")
unpublishedDNSHits = expvar.NewInt("counter_bootstrap_dns_unpublished_hits")
unpublishedDNSMisses = expvar.NewInt("counter_bootstrap_dns_unpublished_misses")
unpublishedDNSPercentMisses = expvar.NewInt("counter_bootstrap_dns_unpublished_percent_misses")
)
func init() {
@ -59,15 +69,13 @@ func refreshBootstrapDNS() {
}
ctx, cancel := context.WithTimeout(context.Background(), refreshTimeout)
defer cancel()
dnsEntries := resolveList(ctx, strings.Split(*bootstrapDNS, ","))
dnsEntries := resolveList(ctx, *bootstrapDNS)
// Randomize the order of the IPs for each name to avoid the client biasing
// to IPv6
for k := range dnsEntries {
ips := dnsEntries[k]
slicesx.Shuffle(ips)
dnsEntries[k] = ips
for _, vv := range dnsEntries.IPs {
slicesx.Shuffle(vv)
}
j, err := json.MarshalIndent(dnsEntries, "", "\t")
j, err := json.MarshalIndent(dnsEntries.IPs, "", "\t")
if err != nil {
// leave the old values in place
return
@ -81,27 +89,50 @@ func refreshUnpublishedDNS() {
if *unpublishedDNS == "" {
return
}
ctx, cancel := context.WithTimeout(context.Background(), refreshTimeout)
defer cancel()
dnsEntries := resolveList(ctx, strings.Split(*unpublishedDNS, ","))
dnsEntries := resolveList(ctx, *unpublishedDNS)
unpublishedDNSCache.Store(dnsEntries)
}
func resolveList(ctx context.Context, names []string) dnsEntryMap {
dnsEntries := make(dnsEntryMap)
// resolveList takes a comma-separated list of DNS names to resolve.
//
// If an entry contains a slash, it's two DNS names: the first is the one to
// resolve and the second is that of a TXT recording containing the rollout
// percentage in range "0".."100". If the TXT record doesn't exist or is
// malformed, the percentage is 0. If the TXT record is not provided (there's no
// slash), then the percentage is 100.
func resolveList(ctx context.Context, list string) *dnsEntryMap {
ents := strings.Split(list, ",")
ret := &dnsEntryMap{}
var r net.Resolver
for _, name := range names {
for _, ent := range ents {
name, txtName, _ := strings.Cut(ent, "/")
addrs, err := r.LookupIP(ctx, "ip", name)
if err != nil {
log.Printf("bootstrap DNS lookup %q: %v", name, err)
continue
}
dnsEntries[name] = addrs
mak.Set(&ret.IPs, name, addrs)
if txtName == "" {
mak.Set(&ret.Percent, name, 1.0)
continue
}
vals, err := r.LookupTXT(ctx, txtName)
if err != nil {
log.Printf("bootstrap DNS lookup %q: %v", txtName, err)
continue
}
for _, v := range vals {
if v, err := strconv.Atoi(v); err == nil && v >= 0 && v <= 100 {
mak.Set(&ret.Percent, name, float64(v)/100)
}
}
}
return dnsEntries
return ret
}
func handleBootstrapDNS(w http.ResponseWriter, r *http.Request) {
@ -115,22 +146,36 @@ func handleBootstrapDNS(w http.ResponseWriter, r *http.Request) {
// Try answering a query from our hidden map first
if q := r.URL.Query().Get("q"); q != "" {
bootstrapLookupMap.Store(q, true)
if ips, ok := unpublishedDNSCache.Load()[q]; ok && len(ips) > 0 {
if bootstrapLookupMap.Len() > 500 { // defensive
bootstrapLookupMap.Clear()
}
if m := unpublishedDNSCache.Load(); m != nil && len(m.IPs[q]) > 0 {
unpublishedDNSHits.Add(1)
percent := m.Percent[q]
if remoteAddrMatchesPercent(r.RemoteAddr, percent) {
// Only return the specific query, not everything.
m := dnsEntryMap{q: ips}
m := map[string][]net.IP{q: m.IPs[q]}
j, err := json.MarshalIndent(m, "", "\t")
if err == nil {
w.Write(j)
return
}
} else {
unpublishedDNSPercentMisses.Add(1)
}
}
// If we have a "q" query for a name in the published cache
// list, then track whether that's a hit/miss.
if m, ok := dnsCache.Load()[q]; ok {
if len(m) > 0 {
m := dnsCache.Load()
var inPub bool
var ips []net.IP
if m != nil {
ips, inPub = m.IPs[q]
}
if inPub {
if len(ips) > 0 {
publishedDNSHits.Add(1)
} else {
publishedDNSMisses.Add(1)
@ -146,3 +191,29 @@ func handleBootstrapDNS(w http.ResponseWriter, r *http.Request) {
j := dnsCacheBytes.Load()
w.Write(j)
}
// percent is [0.0, 1.0].
func remoteAddrMatchesPercent(remoteAddr string, percent float64) bool {
if percent == 0 {
return false
}
if percent == 1 {
return true
}
reqIPStr, _, err := net.SplitHostPort(remoteAddr)
if err != nil {
return false
}
reqIP, err := netip.ParseAddr(reqIPStr)
if err != nil {
return false
}
if reqIP.IsLoopback() {
// For local testing.
return rand.Float64() < 0.5
}
reqIP16 := reqIP.As16()
rndSrc := rand.NewPCG(binary.LittleEndian.Uint64(reqIP16[:8]), binary.LittleEndian.Uint64(reqIP16[8:]))
rnd := rand.New(rndSrc)
return percent > rnd.Float64()
}

@ -4,15 +4,19 @@
package main
import (
"bytes"
"encoding/json"
"io"
"net"
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"reflect"
"testing"
"tailscale.com/tstest"
"tailscale.com/tstest/nettest"
)
func BenchmarkHandleBootstrapDNS(b *testing.B) {
@ -37,7 +41,7 @@ func (b *bitbucketResponseWriter) Write(p []byte) (int, error) { return len(p),
func (b *bitbucketResponseWriter) WriteHeader(statusCode int) {}
func getBootstrapDNS(t *testing.T, q string) dnsEntryMap {
func getBootstrapDNS(t *testing.T, q string) map[string][]net.IP {
t.Helper()
req, _ := http.NewRequest("GET", "https://localhost/bootstrap-dns?q="+url.QueryEscape(q), nil)
w := httptest.NewRecorder()
@ -47,14 +51,17 @@ func getBootstrapDNS(t *testing.T, q string) dnsEntryMap {
if res.StatusCode != 200 {
t.Fatalf("got status=%d; want %d", res.StatusCode, 200)
}
var ips dnsEntryMap
if err := json.NewDecoder(res.Body).Decode(&ips); err != nil {
t.Fatalf("error decoding response body: %v", err)
var m map[string][]net.IP
var buf bytes.Buffer
if err := json.NewDecoder(io.TeeReader(res.Body, &buf)).Decode(&m); err != nil {
t.Fatalf("error decoding response body %q: %v", buf.Bytes(), err)
}
return ips
return m
}
func TestUnpublishedDNS(t *testing.T) {
nettest.SkipIfNoNetwork(t)
const published = "login.tailscale.com"
const unpublished = "log.tailscale.io"
@ -104,15 +111,21 @@ func resetMetrics() {
// Verify that we don't count an empty list in the unpublishedDNSCache as a
// cache hit in our metrics.
func TestUnpublishedDNSEmptyList(t *testing.T) {
pub := dnsEntryMap{
"tailscale.com": {net.IPv4(10, 10, 10, 10)},
pub := &dnsEntryMap{
IPs: map[string][]net.IP{"tailscale.com": {net.IPv4(10, 10, 10, 10)}},
}
dnsCache.Store(pub)
dnsCacheBytes.Store([]byte(`{"tailscale.com":["10.10.10.10"]}`))
unpublishedDNSCache.Store(dnsEntryMap{
unpublishedDNSCache.Store(&dnsEntryMap{
IPs: map[string][]net.IP{
"log.tailscale.io": {},
"controlplane.tailscale.com": {net.IPv4(1, 2, 3, 4)},
},
Percent: map[string]float64{
"log.tailscale.io": 1.0,
"controlplane.tailscale.com": 1.0,
},
})
t.Run("CacheMiss", func(t *testing.T) {
@ -122,8 +135,8 @@ func TestUnpublishedDNSEmptyList(t *testing.T) {
ips := getBootstrapDNS(t, q)
// Expected our public map to be returned on a cache miss
if !reflect.DeepEqual(ips, pub) {
t.Errorf("got ips=%+v; want %+v", ips, pub)
if !reflect.DeepEqual(ips, pub.IPs) {
t.Errorf("got ips=%+v; want %+v", ips, pub.IPs)
}
if v := unpublishedDNSHits.Value(); v != 0 {
t.Errorf("got hits=%d; want 0", v)
@ -138,7 +151,7 @@ func TestUnpublishedDNSEmptyList(t *testing.T) {
t.Run("CacheHit", func(t *testing.T) {
resetMetrics()
ips := getBootstrapDNS(t, "controlplane.tailscale.com")
want := dnsEntryMap{"controlplane.tailscale.com": {net.IPv4(1, 2, 3, 4)}}
want := map[string][]net.IP{"controlplane.tailscale.com": {net.IPv4(1, 2, 3, 4)}}
if !reflect.DeepEqual(ips, want) {
t.Errorf("got ips=%+v; want %+v", ips, want)
}
@ -163,3 +176,54 @@ func TestLookupMetric(t *testing.T) {
t.Errorf("bootstrapLookupMap.Len() want=5, got %v", bootstrapLookupMap.Len())
}
}
func TestRemoteAddrMatchesPercent(t *testing.T) {
tests := []struct {
remoteAddr string
percent float64
want bool
}{
// 0% and 100%.
{"10.0.0.1:1234", 0.0, false},
{"10.0.0.1:1234", 1.0, true},
// Invalid IP.
{"", 1.0, true},
{"", 0.0, false},
{"", 0.5, false},
// Small manual sample at 50%. The func uses a deterministic PRNG seed.
{"1.2.3.4:567", 0.5, true},
{"1.2.3.5:567", 0.5, true},
{"1.2.3.6:567", 0.5, false},
{"1.2.3.7:567", 0.5, true},
{"1.2.3.8:567", 0.5, false},
{"1.2.3.9:567", 0.5, true},
{"1.2.3.10:567", 0.5, true},
}
for _, tt := range tests {
got := remoteAddrMatchesPercent(tt.remoteAddr, tt.percent)
if got != tt.want {
t.Errorf("remoteAddrMatchesPercent(%q, %v) = %v; want %v", tt.remoteAddr, tt.percent, got, tt.want)
}
}
var match, all int
const wantPercent = 0.5
for a := range 256 {
for b := range 256 {
all++
if remoteAddrMatchesPercent(
netip.AddrPortFrom(netip.AddrFrom4([4]byte{1, 2, byte(a), byte(b)}), 12345).String(),
wantPercent) {
match++
}
}
}
gotPercent := float64(match) / float64(all)
const tolerance = 0.005
t.Logf("got percent %v (goal %v)", gotPercent, wantPercent)
if gotPercent < wantPercent-tolerance || gotPercent > wantPercent+tolerance {
t.Errorf("got %v; want %v ± %v", gotPercent, wantPercent, tolerance)
}
}

@ -20,7 +20,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
github.com/google/uuid from tailscale.com/util/fastuuid
github.com/hdevalence/ed25519consensus from tailscale.com/tka
L github.com/josharian/native from github.com/mdlayher/netlink+
L 💣 github.com/jsimonetti/rtnetlink from tailscale.com/net/interfaces+
L 💣 github.com/jsimonetti/rtnetlink from tailscale.com/net/netmon
L github.com/jsimonetti/rtnetlink/internal/unix from github.com/jsimonetti/rtnetlink
L 💣 github.com/mdlayher/netlink from github.com/google/nftables+
L 💣 github.com/mdlayher/netlink/nlenc from github.com/jsimonetti/rtnetlink+
@ -46,14 +46,15 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
L github.com/vishvananda/netns from github.com/tailscale/netlink+
github.com/x448/float16 from github.com/fxamacker/cbor/v2
💣 go4.org/mem from tailscale.com/client/tailscale+
go4.org/netipx from tailscale.com/net/tsaddr+
W 💣 golang.zx2c4.com/wireguard/windows/tunnel/winipcfg from tailscale.com/net/interfaces+
go4.org/netipx from tailscale.com/net/tsaddr
W 💣 golang.zx2c4.com/wireguard/windows/tunnel/winipcfg from tailscale.com/net/netmon+
google.golang.org/protobuf/encoding/protodelim from github.com/prometheus/common/expfmt
google.golang.org/protobuf/encoding/prototext from github.com/prometheus/common/expfmt+
google.golang.org/protobuf/encoding/protowire from google.golang.org/protobuf/encoding/protodelim+
google.golang.org/protobuf/internal/descfmt from google.golang.org/protobuf/internal/filedesc
google.golang.org/protobuf/internal/descopts from google.golang.org/protobuf/internal/filedesc+
google.golang.org/protobuf/internal/detrand from google.golang.org/protobuf/internal/descfmt+
google.golang.org/protobuf/internal/editiondefaults from google.golang.org/protobuf/internal/filedesc
google.golang.org/protobuf/internal/encoding/defval from google.golang.org/protobuf/internal/encoding/tag+
google.golang.org/protobuf/internal/encoding/messageset from google.golang.org/protobuf/encoding/prototext+
google.golang.org/protobuf/internal/encoding/tag from google.golang.org/protobuf/internal/impl
@ -88,21 +89,18 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/disco from tailscale.com/derp
tailscale.com/drive from tailscale.com/client/tailscale+
tailscale.com/envknob from tailscale.com/client/tailscale+
tailscale.com/health from tailscale.com/net/tlsdial
tailscale.com/hostinfo from tailscale.com/net/interfaces+
tailscale.com/health from tailscale.com/net/tlsdial+
tailscale.com/hostinfo from tailscale.com/net/netmon+
tailscale.com/ipn from tailscale.com/client/tailscale
tailscale.com/ipn/ipnstate from tailscale.com/client/tailscale+
tailscale.com/metrics from tailscale.com/cmd/derper+
tailscale.com/net/dnscache from tailscale.com/derp/derphttp
tailscale.com/net/flowtrack from tailscale.com/net/packet+
💣 tailscale.com/net/interfaces from tailscale.com/net/netmon+
tailscale.com/net/ktimeout from tailscale.com/cmd/derper
tailscale.com/net/netaddr from tailscale.com/ipn+
tailscale.com/net/netknob from tailscale.com/net/netns
tailscale.com/net/netmon from tailscale.com/derp/derphttp+
💣 tailscale.com/net/netmon from tailscale.com/derp/derphttp+
tailscale.com/net/netns from tailscale.com/derp/derphttp
tailscale.com/net/netutil from tailscale.com/client/tailscale
tailscale.com/net/packet from tailscale.com/wgengine/filter
tailscale.com/net/sockstats from tailscale.com/derp/derphttp
tailscale.com/net/stun from tailscale.com/net/stunserver
tailscale.com/net/stunserver from tailscale.com/cmd/derper
@ -116,16 +114,16 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/syncs from tailscale.com/cmd/derper+
tailscale.com/tailcfg from tailscale.com/client/tailscale+
tailscale.com/tka from tailscale.com/client/tailscale+
W tailscale.com/tsconst from tailscale.com/net/interfaces
W tailscale.com/tsconst from tailscale.com/net/netmon
tailscale.com/tstime from tailscale.com/derp+
tailscale.com/tstime/mono from tailscale.com/tstime/rate
tailscale.com/tstime/rate from tailscale.com/derp+
tailscale.com/tstime/rate from tailscale.com/derp
tailscale.com/tsweb from tailscale.com/cmd/derper
tailscale.com/tsweb/promvarz from tailscale.com/tsweb
tailscale.com/tsweb/varz from tailscale.com/tsweb+
tailscale.com/types/dnstype from tailscale.com/tailcfg
tailscale.com/types/empty from tailscale.com/ipn
tailscale.com/types/ipproto from tailscale.com/net/flowtrack+
tailscale.com/types/ipproto from tailscale.com/tailcfg+
tailscale.com/types/key from tailscale.com/client/tailscale+
tailscale.com/types/lazy from tailscale.com/version+
tailscale.com/types/logger from tailscale.com/cmd/derper+
@ -137,6 +135,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/types/structs from tailscale.com/ipn+
tailscale.com/types/tkatype from tailscale.com/client/tailscale+
tailscale.com/types/views from tailscale.com/ipn+
tailscale.com/util/cibuild from tailscale.com/health
tailscale.com/util/clientmetric from tailscale.com/net/netmon+
tailscale.com/util/cloudenv from tailscale.com/hostinfo+
W tailscale.com/util/cmpver from tailscale.com/net/tshttpproxy
@ -147,7 +146,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/util/httpm from tailscale.com/client/tailscale
tailscale.com/util/lineread from tailscale.com/hostinfo+
L tailscale.com/util/linuxfw from tailscale.com/net/netns
tailscale.com/util/mak from tailscale.com/net/interfaces+
tailscale.com/util/mak from tailscale.com/health+
tailscale.com/util/multierr from tailscale.com/health+
tailscale.com/util/nocasemaps from tailscale.com/types/ipproto
tailscale.com/util/set from tailscale.com/derp+
@ -156,9 +155,10 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
tailscale.com/util/syspolicy from tailscale.com/ipn
tailscale.com/util/vizerror from tailscale.com/tailcfg+
W 💣 tailscale.com/util/winutil from tailscale.com/hostinfo+
W 💣 tailscale.com/util/winutil/winenv from tailscale.com/hostinfo+
tailscale.com/version from tailscale.com/derp+
tailscale.com/version/distro from tailscale.com/envknob+
tailscale.com/wgengine/filter from tailscale.com/types/netmap
tailscale.com/wgengine/filter/filtertype from tailscale.com/types/netmap
golang.org/x/crypto/acme from golang.org/x/crypto/acme/autocert
golang.org/x/crypto/acme/autocert from tailscale.com/cmd/derper
golang.org/x/crypto/argon2 from tailscale.com/tka
@ -173,6 +173,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
golang.org/x/crypto/nacl/box from tailscale.com/types/key
golang.org/x/crypto/nacl/secretbox from golang.org/x/crypto/nacl/box
golang.org/x/crypto/salsa20/salsa from golang.org/x/crypto/nacl/box+
W golang.org/x/exp/constraints from tailscale.com/util/winutil
L golang.org/x/net/bpf from github.com/mdlayher/netlink+
golang.org/x/net/dns/dnsmessage from net+
golang.org/x/net/http/httpguts from net/http
@ -233,7 +234,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
encoding/pem from crypto/tls+
errors from bufio+
expvar from github.com/prometheus/client_golang/prometheus+
flag from tailscale.com/cmd/derper+
flag from tailscale.com/cmd/derper
fmt from compress/flate+
go/token from google.golang.org/protobuf/internal/strs
hash from crypto+
@ -251,7 +252,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
math/big from crypto/dsa+
math/bits from compress/flate+
math/rand from github.com/mdlayher/netlink+
math/rand/v2 from tailscale.com/util/fastuuid
math/rand/v2 from tailscale.com/util/fastuuid+
mime from github.com/prometheus/common/expfmt+
mime/multipart from net/http
mime/quotedprintable from mime/multipart
@ -275,7 +276,7 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
runtime/debug from github.com/prometheus/client_golang/prometheus+
runtime/metrics from github.com/prometheus/client_golang/prometheus+
runtime/pprof from net/http/pprof
runtime/trace from net/http/pprof+
runtime/trace from net/http/pprof
slices from tailscale.com/ipn/ipnstate+
sort from compress/flate+
strconv from compress/flate+
@ -283,7 +284,6 @@ tailscale.com/cmd/derper dependencies: (generated by github.com/tailscale/depawa
sync from compress/flate+
sync/atomic from context+
syscall from crypto/rand+
testing from tailscale.com/util/syspolicy
text/tabwriter from runtime/pprof
time from compress/gzip+
unicode from bytes+

@ -55,7 +55,7 @@ var (
meshPSKFile = flag.String("mesh-psk-file", defaultMeshPSKFile(), "if non-empty, path to file containing the mesh pre-shared key file. It should contain some hex string; whitespace is trimmed.")
meshWith = flag.String("mesh-with", "", "optional comma-separated list of hostnames to mesh with; the server's own hostname can be in the list")
bootstrapDNS = flag.String("bootstrap-dns-names", "", "optional comma-separated list of hostnames to make available at /bootstrap-dns")
unpublishedDNS = flag.String("unpublished-bootstrap-dns-names", "", "optional comma-separated list of hostnames to make available at /bootstrap-dns and not publish in the list")
unpublishedDNS = flag.String("unpublished-bootstrap-dns-names", "", "optional comma-separated list of hostnames to make available at /bootstrap-dns and not publish in the list. If an entry contains a slash, the second part names a DNS record to poll for its TXT record with a `0` to `100` value for rollout percentage.")
verifyClients = flag.Bool("verify-clients", false, "verify clients to this DERP server through a local tailscaled instance.")
verifyClientURL = flag.String("verify-client-url", "", "if non-empty, an admission controller URL for permitting client connections; see tailcfg.DERPAdmitClientRequest")
verifyFailOpen = flag.Bool("verify-client-url-fail-open", true, "whether we fail open if --verify-client-url is unreachable")
@ -191,7 +191,12 @@ func main() {
http.Error(w, "derp server disabled", http.StatusNotFound)
}))
}
mux.HandleFunc("/derp/probe", probeHandler)
// These two endpoints are the same. Different versions of the clients
// have assumes different paths over time so we support both.
mux.HandleFunc("/derp/probe", derphttp.ProbeHandler)
mux.HandleFunc("/derp/latency-check", derphttp.ProbeHandler)
go refreshBootstrapDNSLoop()
mux.HandleFunc("/bootstrap-dns", tsweb.BrowserHeaderHandlerFunc(handleBootstrapDNS))
mux.Handle("/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@ -370,17 +375,6 @@ func isChallengeChar(c rune) bool {
c == '.' || c == '-' || c == '_'
}
// probeHandler is the endpoint that js/wasm clients hit to measure
// DERP latency, since they can't do UDP STUN queries.
func probeHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "HEAD", "GET":
w.Header().Set("Access-Control-Allow-Origin", "*")
default:
http.Error(w, "bogus probe method", http.StatusMethodNotAllowed)
}
}
var validProdHostname = regexp.MustCompile(`^derp([^.]*)\.tailscale\.com\.?$`)
func prodAutocertHostPolicy(_ context.Context, host string) error {

@ -99,10 +99,13 @@ func TestNoContent(t *testing.T) {
func TestDeps(t *testing.T) {
deptest.DepChecker{
BadDeps: map[string]string{
"testing": "do not use testing package in production code",
"gvisor.dev/gvisor/pkg/buffer": "https://github.com/tailscale/tailscale/issues/9756",
"gvisor.dev/gvisor/pkg/cpuid": "https://github.com/tailscale/tailscale/issues/9756",
"gvisor.dev/gvisor/pkg/tcpip": "https://github.com/tailscale/tailscale/issues/9756",
"gvisor.dev/gvisor/pkg/tcpip/header": "https://github.com/tailscale/tailscale/issues/9756",
"tailscale.com/net/packet": "not needed in derper",
"github.com/gaissmai/bart": "not needed in derper",
},
}.Check(t)
}

@ -15,6 +15,7 @@ import (
"tailscale.com/derp"
"tailscale.com/derp/derphttp"
"tailscale.com/net/netmon"
"tailscale.com/types/key"
"tailscale.com/types/logger"
)
@ -36,7 +37,8 @@ func startMesh(s *derp.Server) error {
func startMeshWithHost(s *derp.Server, host string) error {
logf := logger.WithPrefix(log.Printf, fmt.Sprintf("mesh(%q): ", host))
c, err := derphttp.NewClient(s.PrivateKey(), "https://"+host+"/derp", logf)
netMon := netmon.NewStatic() // good enough for cmd/derper; no need for netns fanciness
c, err := derphttp.NewClient(s.PrivateKey(), "https://"+host+"/derp", logf, netMon)
if err != nil {
return err
}

@ -20,7 +20,7 @@ import (
)
var (
derpMapURL = flag.String("derp-map", "https://login.tailscale.com/derpmap/default", "URL to DERP map (https:// or file://)")
derpMapURL = flag.String("derp-map", "https://login.tailscale.com/derpmap/default", "URL to DERP map (https:// or file://) or 'local' to use the local tailscaled's DERP map")
versionFlag = flag.Bool("version", false, "print version and exit")
listen = flag.String("listen", ":8030", "HTTP listen address")
probeOnce = flag.Bool("once", false, "probe once and print results, then exit; ignores the listen flag")

13
cmd/dist/dist.go vendored

@ -13,11 +13,16 @@ import (
"tailscale.com/release/dist"
"tailscale.com/release/dist/cli"
"tailscale.com/release/dist/qnap"
"tailscale.com/release/dist/synology"
"tailscale.com/release/dist/unixpkgs"
)
var synologyPackageCenter bool
var (
synologyPackageCenter bool
qnapPrivateKeyPath string
qnapCertificatePath string
)
func getTargets() ([]dist.Target, error) {
var ret []dist.Target
@ -37,6 +42,10 @@ func getTargets() ([]dist.Target, error) {
// To build for package center, run
// ./tool/go run ./cmd/dist build --synology-package-center synology
ret = append(ret, synology.Targets(synologyPackageCenter, nil)...)
if (qnapPrivateKeyPath == "") != (qnapCertificatePath == "") {
return nil, errors.New("both --qnap-private-key-path and --qnap-certificate-path must be set")
}
ret = append(ret, qnap.Targets(qnapPrivateKeyPath, qnapCertificatePath)...)
return ret, nil
}
@ -45,6 +54,8 @@ func main() {
for _, subcmd := range cmd.Subcommands {
if subcmd.Name == "build" {
subcmd.FlagSet.BoolVar(&synologyPackageCenter, "synology-package-center", false, "build synology packages with extra metadata for the official package center")
subcmd.FlagSet.StringVar(&qnapPrivateKeyPath, "qnap-private-key-path", "", "sign qnap packages with given key (must also provide --qnap-certificate-path)")
subcmd.FlagSet.StringVar(&qnapCertificatePath, "qnap-certificate-path", "", "sign qnap packages with given certificate (must also provide --qnap-private-key-path)")
}
}

@ -0,0 +1,379 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !plan9
// k8s-nameserver is a simple nameserver implementation meant to be used with
// k8s-operator to allow to resolve magicDNS names associated with tailnet
// proxies in cluster.
package main
import (
"context"
"encoding/json"
"fmt"
"log"
"net"
"os"
"os/signal"
"path/filepath"
"sync"
"syscall"
"github.com/fsnotify/fsnotify"
"github.com/miekg/dns"
operatorutils "tailscale.com/k8s-operator"
"tailscale.com/util/dnsname"
)
const (
// tsNetDomain is the domain that this DNS nameserver has registered a handler for.
tsNetDomain = "ts.net"
// addr is the the address that the UDP and TCP listeners will listen on.
addr = ":1053"
// The following constants are specific to the nameserver configuration
// provided by a mounted Kubernetes Configmap. The Configmap mounted at
// /config is the only supported way for configuring this nameserver.
defaultDNSConfigDir = "/config"
kubeletMountedConfigLn = "..data"
)
// nameserver is a simple nameserver that responds to DNS queries for A records
// for ts.net domain names over UDP or TCP. It serves DNS responses from
// in-memory IPv4 host records. It is intended to be deployed on Kubernetes with
// a ConfigMap mounted at /config that should contain the host records. It
// dynamically reconfigures its in-memory mappings as the contents of the
// mounted ConfigMap changes.
type nameserver struct {
// configReader returns the latest desired configuration (host records)
// for the nameserver. By default it gets set to a reader that reads
// from a Kubernetes ConfigMap mounted at /config, but this can be
// overridden in tests.
configReader configReaderFunc
// configWatcher is a watcher that returns an event when the desired
// configuration has changed and the nameserver should update the
// in-memory records.
configWatcher <-chan string
mu sync.Mutex // protects following
// ip4 are the in-memory hostname -> IP4 mappings that the nameserver
// uses to respond to A record queries.
ip4 map[dnsname.FQDN][]net.IP
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
// Ensure that we watch the kube Configmap mounted at /config for
// nameserver configuration updates and send events when updates happen.
c := ensureWatcherForKubeConfigMap(ctx)
ns := &nameserver{
configReader: configMapConfigReader,
configWatcher: c,
}
// Ensure that in-memory records get set up to date now and will get
// reset when the configuration changes.
ns.runRecordsReconciler(ctx)
// Register a DNS server handle for ts.net domain names. Not having a
// handle registered for any other domain names is how we enforce that
// this nameserver can only be used for ts.net domains - querying any
// other domain names returns Rcode Refused.
dns.HandleFunc(tsNetDomain, ns.handleFunc())
// Listen for DNS queries over UDP and TCP.
udpSig := make(chan os.Signal)
tcpSig := make(chan os.Signal)
go listenAndServe("udp", addr, udpSig)
go listenAndServe("tcp", addr, tcpSig)
sig := make(chan os.Signal, 1)
signal.Notify(sig, syscall.SIGINT, syscall.SIGTERM)
s := <-sig
log.Printf("OS signal (%s) received, shutting down", s)
cancel() // exit the records reconciler and configmap watcher goroutines
udpSig <- s // stop the UDP listener
tcpSig <- s // stop the TCP listener
}
// handleFunc is a DNS query handler that can respond to A record queries from
// the nameserver's in-memory records.
// - If an A record query is received and the
// nameserver's in-memory records contain records for the queried domain name,
// return a success response.
// - If an A record query is received, but the
// nameserver's in-memory records do not contain records for the queried domain name,
// return NXDOMAIN.
// - If an A record query is received, but the queried domain name is not valid, return Format Error.
// - If a query is received for any other record type than A, return Not Implemented.
func (n *nameserver) handleFunc() func(w dns.ResponseWriter, r *dns.Msg) {
h := func(w dns.ResponseWriter, r *dns.Msg) {
m := new(dns.Msg)
defer func() {
w.WriteMsg(m)
}()
if len(r.Question) < 1 {
log.Print("[unexpected] nameserver received a request with no questions")
m = r.SetRcodeFormatError(r)
return
}
// TODO (irbekrm): maybe set message compression
switch r.Question[0].Qtype {
case dns.TypeA:
q := r.Question[0].Name
fqdn, err := dnsname.ToFQDN(q)
if err != nil {
m = r.SetRcodeFormatError(r)
return
}
// The only supported use of this nameserver is as a
// single source of truth for MagicDNS names by
// non-tailnet Kubernetes workloads.
m.Authoritative = true
m.RecursionAvailable = false
ips := n.lookupIP4(fqdn)
if ips == nil || len(ips) == 0 {
// As we are the authoritative nameserver for MagicDNS
// names, if we do not have a record for this MagicDNS
// name, it does not exist.
m = m.SetRcode(r, dns.RcodeNameError)
return
}
// TODO (irbekrm): TTL is currently set to 0, meaning
// that cluster workloads will not cache the DNS
// records. Revisit this in future when we understand
// the usage patterns better- is it putting too much
// load on kube DNS server or is this fine?
for _, ip := range ips {
rr := &dns.A{Hdr: dns.RR_Header{Name: q, Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 0}, A: ip}
m.SetRcode(r, dns.RcodeSuccess)
m.Answer = append(m.Answer, rr)
}
case dns.TypeAAAA:
// TODO (irbekrm): add IPv6 support.
// The nameserver currently does not support IPv6
// (records are not being created for IPv6 Pod addresses).
// However, we can expect that some callers will
// nevertheless send AAAA queries.
// We have to return NOERROR if a query is received for
// an AAAA record for a DNS name that we have an A
// record for- else the caller might not follow with an
// A record query.
// https://github.com/tailscale/tailscale/issues/12321
// https://datatracker.ietf.org/doc/html/rfc4074
q := r.Question[0].Name
fqdn, err := dnsname.ToFQDN(q)
if err != nil {
m = r.SetRcodeFormatError(r)
return
}
// The only supported use of this nameserver is as a
// single source of truth for MagicDNS names by
// non-tailnet Kubernetes workloads.
m.Authoritative = true
ips := n.lookupIP4(fqdn)
if len(ips) == 0 {
// As we are the authoritative nameserver for MagicDNS
// names, if we do not have a record for this MagicDNS
// name, it does not exist.
m = m.SetRcode(r, dns.RcodeNameError)
return
}
m.SetRcode(r, dns.RcodeSuccess)
default:
log.Printf("[unexpected] nameserver received a query for an unsupported record type: %s", r.Question[0].String())
m.SetRcode(r, dns.RcodeNotImplemented)
}
}
return h
}
// runRecordsReconciler ensures that nameserver's in-memory records are
// reset when the provided configuration changes.
func (n *nameserver) runRecordsReconciler(ctx context.Context) {
log.Print("updating nameserver's records from the provided configuration...")
if err := n.resetRecords(); err != nil { // ensure records are up to date before the nameserver starts
log.Fatalf("error setting nameserver's records: %v", err)
}
log.Print("nameserver's records were updated")
go func() {
for {
select {
case <-ctx.Done():
log.Printf("context cancelled, exiting records reconciler")
return
case <-n.configWatcher:
log.Print("configuration update detected, resetting records")
if err := n.resetRecords(); err != nil {
// TODO (irbekrm): this runs in a
// container that will be thrown away,
// so this should be ok. But maybe still
// need to ensure that the DNS server
// terminates connections more
// gracefully.
log.Fatalf("error resetting records: %v", err)
}
log.Print("nameserver records were reset")
}
}
}()
}
// resetRecords sets the in-memory DNS records of this nameserver from the
// provided configuration. It does not check for the diff, so the caller is
// expected to ensure that this is only called when reset is needed.
func (n *nameserver) resetRecords() error {
dnsCfgBytes, err := n.configReader()
if err != nil {
log.Printf("error reading nameserver's configuration: %v", err)
return err
}
if dnsCfgBytes == nil || len(dnsCfgBytes) < 1 {
log.Print("nameserver's configuration is empty, any in-memory records will be unset")
n.mu.Lock()
n.ip4 = make(map[dnsname.FQDN][]net.IP)
n.mu.Unlock()
return nil
}
dnsCfg := &operatorutils.Records{}
err = json.Unmarshal(dnsCfgBytes, dnsCfg)
if err != nil {
return fmt.Errorf("error unmarshalling nameserver configuration: %v\n", err)
}
if dnsCfg.Version != operatorutils.Alpha1Version {
return fmt.Errorf("unsupported configuration version %s, supported versions are %s\n", dnsCfg.Version, operatorutils.Alpha1Version)
}
ip4 := make(map[dnsname.FQDN][]net.IP)
defer func() {
n.mu.Lock()
defer n.mu.Unlock()
n.ip4 = ip4
}()
if len(dnsCfg.IP4) == 0 {
log.Print("nameserver's configuration contains no records, any in-memory records will be unset")
return nil
}
for fqdn, ips := range dnsCfg.IP4 {
fqdn, err := dnsname.ToFQDN(fqdn)
if err != nil {
log.Printf("invalid nameserver's configuration: %s is not a valid FQDN: %v; skipping this record", fqdn, err)
continue // one invalid hostname should not break the whole nameserver
}
for _, ipS := range ips {
ip := net.ParseIP(ipS).To4()
if ip == nil { // To4 returns nil if IP is not a IPv4 address
log.Printf("invalid nameserver's configuration: %v does not appear to be an IPv4 address; skipping this record", ipS)
continue // one invalid IP address should not break the whole nameserver
}
ip4[fqdn] = []net.IP{ip}
}
}
return nil
}
// listenAndServe starts a DNS server for the provided network and address.
func listenAndServe(net, addr string, shutdown chan os.Signal) {
s := &dns.Server{Addr: addr, Net: net}
go func() {
<-shutdown
log.Printf("shutting down server for %s", net)
s.Shutdown()
}()
log.Printf("listening for %s queries on %s", net, addr)
if err := s.ListenAndServe(); err != nil {
log.Fatalf("error running %s server: %v", net, err)
}
}
// ensureWatcherForKubeConfigMap sets up a new file watcher for the ConfigMap
// that's expected to be mounted at /config. Returns a channel that receives an
// event every time the contents get updated.
func ensureWatcherForKubeConfigMap(ctx context.Context) chan string {
c := make(chan string)
watcher, err := fsnotify.NewWatcher()
if err != nil {
log.Fatalf("error creating a new watcher for the mounted ConfigMap: %v", err)
}
// kubelet mounts configmap to a Pod using a series of symlinks, one of
// which is <mount-dir>/..data that Kubernetes recommends consumers to
// use if they need to monitor changes
// https://github.com/kubernetes/kubernetes/blob/v1.28.1/pkg/volume/util/atomic_writer.go#L39-L61
toWatch := filepath.Join(defaultDNSConfigDir, kubeletMountedConfigLn)
go func() {
defer watcher.Close()
log.Printf("starting file watch for %s", defaultDNSConfigDir)
for {
select {
case <-ctx.Done():
log.Print("context cancelled, exiting ConfigMap watcher")
return
case event, ok := <-watcher.Events:
if !ok {
log.Fatal("watcher finished; exiting")
}
if event.Name == toWatch {
msg := fmt.Sprintf("ConfigMap update received: %s", event)
log.Print(msg)
c <- msg
}
case err, ok := <-watcher.Errors:
if err != nil {
// TODO (irbekrm): this runs in a
// container that will be thrown away,
// so this should be ok. But maybe still
// need to ensure that the DNS server
// terminates connections more
// gracefully.
log.Fatalf("[unexpected] error watching configuration: %v", err)
}
if !ok {
// TODO (irbekrm): this runs in a
// container that will be thrown away,
// so this should be ok. But maybe still
// need to ensure that the DNS server
// terminates connections more
// gracefully.
log.Fatalf("[unexpected] errors watcher exited")
}
}
}
}()
if err = watcher.Add(defaultDNSConfigDir); err != nil {
log.Fatalf("failed setting up a watcher for the mounted ConfigMap: %v", err)
}
return c
}
// configReaderFunc is a function that returns the desired nameserver configuration.
type configReaderFunc func() ([]byte, error)
// configMapConfigReader reads the desired nameserver configuration from a
// records.json file in a ConfigMap mounted at /config.
var configMapConfigReader configReaderFunc = func() ([]byte, error) {
if contents, err := os.ReadFile(filepath.Join(defaultDNSConfigDir, operatorutils.DNSRecordsCMKey)); err == nil {
return contents, nil
} else if os.IsNotExist(err) {
return nil, nil
} else {
return nil, err
}
}
// lookupIP4 returns any IPv4 addresses for the given FQDN from nameserver's
// in-memory records.
func (n *nameserver) lookupIP4(fqdn dnsname.FQDN) []net.IP {
if n.ip4 == nil {
return nil
}
n.mu.Lock()
defer n.mu.Unlock()
f := n.ip4[fqdn]
return f
}

@ -0,0 +1,229 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !plan9
package main
import (
"net"
"testing"
"github.com/google/go-cmp/cmp"
"github.com/miekg/dns"
"tailscale.com/util/dnsname"
)
func TestNameserver(t *testing.T) {
tests := []struct {
name string
ip4 map[dnsname.FQDN][]net.IP
query *dns.Msg
wantResp *dns.Msg
}{
{
name: "A record query, record exists",
ip4: map[dnsname.FQDN][]net.IP{dnsname.FQDN("foo.bar.com."): {{1, 2, 3, 4}}},
query: &dns.Msg{
Question: []dns.Question{{Name: "foo.bar.com", Qtype: dns.TypeA}},
MsgHdr: dns.MsgHdr{Id: 1, RecursionDesired: true},
},
wantResp: &dns.Msg{
Answer: []dns.RR{&dns.A{Hdr: dns.RR_Header{
Name: "foo.bar.com", Rrtype: dns.TypeA, Class: dns.ClassINET, Ttl: 0},
A: net.IP{1, 2, 3, 4}}},
Question: []dns.Question{{Name: "foo.bar.com", Qtype: dns.TypeA}},
MsgHdr: dns.MsgHdr{
Id: 1,
Rcode: dns.RcodeSuccess,
RecursionAvailable: false,
RecursionDesired: true,
Response: true,
Opcode: dns.OpcodeQuery,
Authoritative: true,
}},
},
{
name: "A record query, record does not exist",
ip4: map[dnsname.FQDN][]net.IP{dnsname.FQDN("foo.bar.com."): {{1, 2, 3, 4}}},
query: &dns.Msg{
Question: []dns.Question{{Name: "baz.bar.com", Qtype: dns.TypeA}},
MsgHdr: dns.MsgHdr{Id: 1},
},
wantResp: &dns.Msg{
Question: []dns.Question{{Name: "baz.bar.com", Qtype: dns.TypeA}},
MsgHdr: dns.MsgHdr{
Id: 1,
Rcode: dns.RcodeNameError,
RecursionAvailable: false,
Response: true,
Opcode: dns.OpcodeQuery,
Authoritative: true,
}},
},
{
name: "A record query, but the name is not a valid FQDN",
ip4: map[dnsname.FQDN][]net.IP{dnsname.FQDN("foo.bar.com."): {{1, 2, 3, 4}}},
query: &dns.Msg{
Question: []dns.Question{{Name: "foo..bar.com", Qtype: dns.TypeA}},
MsgHdr: dns.MsgHdr{Id: 1},
},
wantResp: &dns.Msg{
Question: []dns.Question{{Name: "foo..bar.com", Qtype: dns.TypeA}},
MsgHdr: dns.MsgHdr{
Id: 1,
Rcode: dns.RcodeFormatError,
Response: true,
Opcode: dns.OpcodeQuery,
}},
},
{
name: "AAAA record query, A record exists",
ip4: map[dnsname.FQDN][]net.IP{dnsname.FQDN("foo.bar.com."): {{1, 2, 3, 4}}},
query: &dns.Msg{
Question: []dns.Question{{Name: "foo.bar.com", Qtype: dns.TypeAAAA}},
MsgHdr: dns.MsgHdr{Id: 1},
},
wantResp: &dns.Msg{
Question: []dns.Question{{Name: "foo.bar.com", Qtype: dns.TypeAAAA}},
MsgHdr: dns.MsgHdr{
Id: 1,
Rcode: dns.RcodeSuccess,
Response: true,
Opcode: dns.OpcodeQuery,
Authoritative: true,
}},
},
{
name: "AAAA record query, A record does not exist",
ip4: map[dnsname.FQDN][]net.IP{dnsname.FQDN("foo.bar.com."): {{1, 2, 3, 4}}},
query: &dns.Msg{
Question: []dns.Question{{Name: "baz.bar.com", Qtype: dns.TypeAAAA}},
MsgHdr: dns.MsgHdr{Id: 1},
},
wantResp: &dns.Msg{
Question: []dns.Question{{Name: "baz.bar.com", Qtype: dns.TypeAAAA}},
MsgHdr: dns.MsgHdr{
Id: 1,
Rcode: dns.RcodeNameError,
Response: true,
Opcode: dns.OpcodeQuery,
Authoritative: true,
}},
},
{
name: "CNAME record query",
ip4: map[dnsname.FQDN][]net.IP{dnsname.FQDN("foo.bar.com."): {{1, 2, 3, 4}}},
query: &dns.Msg{
Question: []dns.Question{{Name: "foo.bar.com", Qtype: dns.TypeCNAME}},
MsgHdr: dns.MsgHdr{Id: 1},
},
wantResp: &dns.Msg{
Question: []dns.Question{{Name: "foo.bar.com", Qtype: dns.TypeCNAME}},
MsgHdr: dns.MsgHdr{
Id: 1,
Rcode: dns.RcodeNotImplemented,
Response: true,
Opcode: dns.OpcodeQuery,
}},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ns := &nameserver{
ip4: tt.ip4,
}
handler := ns.handleFunc()
fakeRespW := &fakeResponseWriter{}
handler(fakeRespW, tt.query)
if diff := cmp.Diff(*fakeRespW.msg, *tt.wantResp); diff != "" {
t.Fatalf("unexpected response (-got +want): \n%s", diff)
}
})
}
}
func TestResetRecords(t *testing.T) {
tests := []struct {
name string
config []byte
hasIp4 map[dnsname.FQDN][]net.IP
wantsIp4 map[dnsname.FQDN][]net.IP
wantsErr bool
}{
{
name: "previously empty nameserver.ip4 gets set",
config: []byte(`{"version": "v1alpha1", "ip4": {"foo.bar.com": ["1.2.3.4"]}}`),
wantsIp4: map[dnsname.FQDN][]net.IP{"foo.bar.com.": {{1, 2, 3, 4}}},
},
{
name: "nameserver.ip4 gets reset",
hasIp4: map[dnsname.FQDN][]net.IP{"baz.bar.com.": {{1, 1, 3, 3}}},
config: []byte(`{"version": "v1alpha1", "ip4": {"foo.bar.com": ["1.2.3.4"]}}`),
wantsIp4: map[dnsname.FQDN][]net.IP{"foo.bar.com.": {{1, 2, 3, 4}}},
},
{
name: "configuration with incompatible version",
hasIp4: map[dnsname.FQDN][]net.IP{"baz.bar.com.": {{1, 1, 3, 3}}},
config: []byte(`{"version": "v1beta1", "ip4": {"foo.bar.com": ["1.2.3.4"]}}`),
wantsIp4: map[dnsname.FQDN][]net.IP{"baz.bar.com.": {{1, 1, 3, 3}}},
wantsErr: true,
},
{
name: "nameserver.ip4 gets reset to empty config when no configuration is provided",
hasIp4: map[dnsname.FQDN][]net.IP{"baz.bar.com.": {{1, 1, 3, 3}}},
wantsIp4: make(map[dnsname.FQDN][]net.IP),
},
{
name: "nameserver.ip4 gets reset to empty config when the provided configuration is empty",
hasIp4: map[dnsname.FQDN][]net.IP{"baz.bar.com.": {{1, 1, 3, 3}}},
config: []byte(`{"version": "v1alpha1", "ip4": {}}`),
wantsIp4: make(map[dnsname.FQDN][]net.IP),
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
ns := &nameserver{
ip4: tt.hasIp4,
configReader: func() ([]byte, error) { return tt.config, nil },
}
if err := ns.resetRecords(); err == nil == tt.wantsErr {
t.Errorf("resetRecords() returned err: %v, wantsErr: %v", err, tt.wantsErr)
}
if diff := cmp.Diff(ns.ip4, tt.wantsIp4); diff != "" {
t.Fatalf("unexpected nameserver.ip4 contents (-got +want): \n%s", diff)
}
})
}
}
// fakeResponseWriter is a faked out dns.ResponseWriter that can be used in
// tests that need to read the response message that was written.
type fakeResponseWriter struct {
msg *dns.Msg
}
var _ dns.ResponseWriter = &fakeResponseWriter{}
func (fr *fakeResponseWriter) WriteMsg(msg *dns.Msg) error {
fr.msg = msg
return nil
}
func (fr *fakeResponseWriter) LocalAddr() net.Addr {
return nil
}
func (fr *fakeResponseWriter) RemoteAddr() net.Addr {
return nil
}
func (fr *fakeResponseWriter) Write([]byte) (int, error) {
return 0, nil
}
func (fr *fakeResponseWriter) Close() error {
return nil
}
func (fr *fakeResponseWriter) TsigStatus() error {
return nil
}
func (fr *fakeResponseWriter) TsigTimersOnly(bool) {}
func (fr *fakeResponseWriter) Hijack() {}

@ -33,10 +33,7 @@ import (
const (
reasonConnectorCreationFailed = "ConnectorCreationFailed"
reasonConnectorCreated = "ConnectorCreated"
reasonConnectorCleanupFailed = "ConnectorCleanupFailed"
reasonConnectorCleanupInProgress = "ConnectorCleanupInProgress"
reasonConnectorInvalid = "ConnectorInvalid"
messageConnectorCreationFailed = "Failed creating Connector: %v"
@ -108,7 +105,7 @@ func (a *ConnectorReconciler) Reconcile(ctx context.Context, req reconcile.Reque
}
oldCnStatus := cn.Status.DeepCopy()
setStatus := func(cn *tsapi.Connector, conditionType tsapi.ConnectorConditionType, status metav1.ConditionStatus, reason, message string) (reconcile.Result, error) {
setStatus := func(cn *tsapi.Connector, _ tsapi.ConditionType, status metav1.ConditionStatus, reason, message string) (reconcile.Result, error) {
tsoperator.SetConnectorCondition(cn, tsapi.ConnectorReady, status, reason, message, cn.Generation, a.clock, logger)
if !apiequality.Semantic.DeepEqual(oldCnStatus, cn.Status) {
// An error encountered here should get returned by the Reconcile function.
@ -184,7 +181,7 @@ func (a *ConnectorReconciler) maybeProvisionConnector(ctx context.Context, logge
Connector: &connector{
isExitNode: cn.Spec.ExitNode,
},
ProxyClass: proxyClass,
ProxyClassName: proxyClass,
}
if cn.Spec.SubnetRouter != nil && len(cn.Spec.SubnetRouter.AdvertiseRoutes) > 0 {
@ -211,9 +208,29 @@ func (a *ConnectorReconciler) maybeProvisionConnector(ctx context.Context, logge
gaugeConnectorResources.Set(int64(connectors.Len()))
_, err := a.ssr.Provision(ctx, logger, sts)
if err != nil {
return err
}
_, tsHost, ips, err := a.ssr.DeviceInfo(ctx, crl)
if err != nil {
return err
}
if tsHost == "" {
logger.Debugf("no Tailscale hostname known yet, waiting for connector pod to finish auth")
// No hostname yet. Wait for the connector pod to auth.
cn.Status.TailnetIPs = nil
cn.Status.Hostname = ""
return nil
}
cn.Status.TailnetIPs = ips
cn.Status.Hostname = tsHost
return nil
}
func (a *ConnectorReconciler) maybeCleanupConnector(ctx context.Context, logger *zap.SugaredLogger, cn *tsapi.Connector) (bool, error) {
if done, err := a.ssr.Cleanup(ctx, logger, childResourceLabels(cn.Name, a.tsnamespace, "connector")); err != nil {
return false, fmt.Errorf("failed to cleanup Connector resources: %w", err)

@ -17,6 +17,7 @@ import (
"sigs.k8s.io/controller-runtime/pkg/client/fake"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/tstest"
"tailscale.com/util/mak"
)
func TestConnector(t *testing.T) {
@ -29,7 +30,7 @@ func TestConnector(t *testing.T) {
},
TypeMeta: metav1.TypeMeta{
Kind: tsapi.ConnectorKind,
APIVersion: "tailscale.io/v1alpha1",
APIVersion: "tailscale.com/v1alpha1",
},
Spec: tsapi.ConnectorSpec{
SubnetRouter: &tsapi.SubnetRouter{
@ -74,9 +75,26 @@ func TestConnector(t *testing.T) {
isExitNode: true,
subnetRoutes: "10.40.0.0/14",
}
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
// Connector status should get updated with the IP/hostname info when available.
const hostname = "foo.tailnetxyz.ts.net"
mustUpdate(t, fc, "operator-ns", opts.secretName, func(secret *corev1.Secret) {
mak.Set(&secret.Data, "device_id", []byte("1234"))
mak.Set(&secret.Data, "device_fqdn", []byte(hostname))
mak.Set(&secret.Data, "device_ips", []byte(`["127.0.0.1", "::1"]`))
})
expectReconciled(t, cr, "", "test")
cn.Finalizers = append(cn.Finalizers, "tailscale.com/finalizer")
cn.Status.IsExitNode = cn.Spec.ExitNode
cn.Status.SubnetRoutes = cn.Spec.SubnetRouter.AdvertiseRoutes.Stringify()
cn.Status.Hostname = hostname
cn.Status.TailnetIPs = []string{"127.0.0.1", "::1"}
expectEqual(t, fc, cn, func(o *tsapi.Connector) {
o.Status.Conditions = nil
})
// Add another route to be advertised.
mustUpdate[tsapi.Connector](t, fc, "", "test", func(conn *tsapi.Connector) {
conn.Spec.SubnetRouter.AdvertiseRoutes = []tsapi.Route{"10.40.0.0/14", "10.44.0.0/20"}
@ -152,7 +170,7 @@ func TestConnector(t *testing.T) {
subnetRoutes: "10.40.0.0/14",
hostname: "test-connector",
}
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
// Add an exit node.
@ -237,7 +255,7 @@ func TestConnectorWithProxyClass(t *testing.T) {
isExitNode: true,
subnetRoutes: "10.40.0.0/14",
}
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
// 2. Update Connector to specify a ProxyClass. ProxyClass is not yet
@ -254,9 +272,9 @@ func TestConnectorWithProxyClass(t *testing.T) {
// its resources.
mustUpdateStatus(t, fc, "", "custom-metadata", func(pc *tsapi.ProxyClass) {
pc.Status = tsapi.ProxyClassStatus{
Conditions: []tsapi.ConnectorCondition{{
Conditions: []metav1.Condition{{
Status: metav1.ConditionTrue,
Type: tsapi.ProxyClassready,
Type: string(tsapi.ProxyClassready),
ObservedGeneration: pc.Generation,
}}}
})

@ -21,6 +21,9 @@ spec:
{{- end }}
labels:
app: operator
{{- with .Values.operatorConfig.podLabels }}
{{- toYaml . | nindent 8 }}
{{- end }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
@ -46,7 +49,7 @@ spec:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- $operatorTag:= printf ":%s" ( .Values.operatorConfig.image.tag | default .Chart.AppVersion )}}
image: {{ .Values.operatorConfig.image.repo }}{{- if .Values.operatorConfig.image.digest -}}{{ printf "@%s" .Values.operatorConfig.image.digest}}{{- else -}}{{ printf "%s" $operatorTag }}{{- end }}
image: {{ coalesce .Values.operatorConfig.image.repo .Values.operatorConfig.image.repository }}{{- if .Values.operatorConfig.image.digest -}}{{ printf "@%s" .Values.operatorConfig.image.digest}}{{- else -}}{{ printf "%s" $operatorTag }}{{- end }}
imagePullPolicy: {{ .Values.operatorConfig.image.pullPolicy }}
env:
- name: OPERATOR_INITIAL_TAGS
@ -67,7 +70,7 @@ spec:
value: /oauth/client_secret
{{- $proxyTag := printf ":%s" ( .Values.proxyConfig.image.tag | default .Chart.AppVersion )}}
- name: PROXY_IMAGE
value: {{ .Values.proxyConfig.image.repo }}{{- if .Values.proxyConfig.image.digest -}}{{ printf "@%s" .Values.proxyConfig.image.digest}}{{- else -}}{{ printf "%s" $proxyTag }}{{- end }}
value: {{ coalesce .Values.proxyConfig.image.repo .Values.proxyConfig.image.repository }}{{- if .Values.proxyConfig.image.digest -}}{{ printf "@%s" .Values.proxyConfig.image.digest}}{{- else -}}{{ printf "%s" $proxyTag }}{{- end }}
- name: PROXY_TAGS
value: {{ .Values.proxyConfig.defaultTags }}
- name: APISERVER_PROXY

@ -24,6 +24,9 @@ rules:
- apiGroups: ["tailscale.com"]
resources: ["connectors", "connectors/status", "proxyclasses", "proxyclasses/status"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["tailscale.com"]
resources: ["dnsconfigs", "dnsconfigs/status"]
verbs: ["get", "list", "watch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
@ -45,11 +48,14 @@ metadata:
namespace: {{ .Release.Namespace }}
rules:
- apiGroups: [""]
resources: ["secrets"]
resources: ["secrets", "serviceaccounts", "configmaps"]
verbs: ["*"]
- apiGroups: ["apps"]
resources: ["statefulsets"]
resources: ["statefulsets", "deployments"]
verbs: ["*"]
- apiGroups: ["discovery.k8s.io"]
resources: ["endpointslices"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding

@ -23,7 +23,8 @@ operatorConfig:
- "tag:k8s-operator"
image:
repo: tailscale/k8s-operator
# Repository defaults to DockerHub, but images are also synced to ghcr.io/tailscale/k8s-operator.
repository: tailscale/k8s-operator
# Digest will be prioritized over tag. If neither are set appVersion will be
# used.
tag: ""
@ -37,6 +38,7 @@ operatorConfig:
resources: {}
podAnnotations: {}
podLabels: {}
tolerations: []
@ -50,9 +52,14 @@ operatorConfig:
# proxies created by the operator.
# https://tailscale.com/kb/1236/kubernetes-operator/#cluster-ingress
# https://tailscale.com/kb/1236/kubernetes-operator/#cluster-egress
# Note that this section contains only a few global configuration options and
# will not be updated with more configuration options in the future.
# If you need more configuration options, take a look at ProxyClass:
# https://tailscale.com/kb/1236/kubernetes-operator#cluster-resource-customization-using-proxyclass-custom-resource
proxyConfig:
image:
repo: tailscale/tailscale
# Repository defaults to DockerHub, but images are also synced to ghcr.io/tailscale/tailscale.
repository: tailscale/tailscale
# Digest will be prioritized over tag. If neither are set appVersion will be
# used.
tag: ""

@ -2,7 +2,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.13.0
controller-gen.kubebuilder.io/version: v0.15.1-0.20240618033008-7824932b0cab
name: connectors.tailscale.com
spec:
group: tailscale.com
@ -31,47 +31,95 @@ spec:
name: v1alpha1
schema:
openAPIV3Schema:
description: |-
Connector defines a Tailscale node that will be deployed in the cluster. The
node can be configured to act as a Tailscale subnet router and/or a Tailscale
exit node.
Connector is a cluster-scoped resource.
More info:
https://tailscale.com/kb/1236/kubernetes-operator#deploying-exit-nodes-and-subnet-routers-on-kubernetes-using-connector-custom-resource
type: object
required:
- spec
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: ConnectorSpec describes the desired Tailscale component.
description: |-
ConnectorSpec describes the desired Tailscale component.
More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
type: object
properties:
exitNode:
description: ExitNode defines whether the Connector node should act as a Tailscale exit node. Defaults to false. https://tailscale.com/kb/1103/exit-nodes
description: |-
ExitNode defines whether the Connector node should act as a
Tailscale exit node. Defaults to false.
https://tailscale.com/kb/1103/exit-nodes
type: boolean
hostname:
description: Hostname is the tailnet hostname that should be assigned to the Connector node. If unset, hostname defaults to <connector name>-connector. Hostname can contain lower case letters, numbers and dashes, it must not start or end with a dash and must be between 2 and 63 characters long.
description: |-
Hostname is the tailnet hostname that should be assigned to the
Connector node. If unset, hostname defaults to <connector
name>-connector. Hostname can contain lower case letters, numbers and
dashes, it must not start or end with a dash and must be between 2
and 63 characters long.
type: string
pattern: ^[a-z0-9][a-z0-9-]{0,61}[a-z0-9]$
proxyClass:
description: ProxyClass is the name of the ProxyClass custom resource that contains configuration options that should be applied to the resources created for this Connector. If unset, the operator will create resources with the default configuration.
description: |-
ProxyClass is the name of the ProxyClass custom resource that
contains configuration options that should be applied to the
resources created for this Connector. If unset, the operator will
create resources with the default configuration.
type: string
subnetRouter:
description: SubnetRouter defines subnet routes that the Connector node should expose to tailnet. If unset, none are exposed. https://tailscale.com/kb/1019/subnets/
description: |-
SubnetRouter defines subnet routes that the Connector node should
expose to tailnet. If unset, none are exposed.
https://tailscale.com/kb/1019/subnets/
type: object
required:
- advertiseRoutes
properties:
advertiseRoutes:
description: AdvertiseRoutes refer to CIDRs that the subnet router should make available. Route values must be strings that represent a valid IPv4 or IPv6 CIDR range. Values can be Tailscale 4via6 subnet routes. https://tailscale.com/kb/1201/4via6-subnets/
description: |-
AdvertiseRoutes refer to CIDRs that the subnet router should make
available. Route values must be strings that represent a valid IPv4
or IPv6 CIDR range. Values can be Tailscale 4via6 subnet routes.
https://tailscale.com/kb/1201/4via6-subnets/
type: array
minItems: 1
items:
type: string
format: cidr
tags:
description: Tags that the Tailscale node will be tagged with. Defaults to [tag:k8s]. To autoapprove the subnet routes or exit node defined by a Connector, you can configure Tailscale ACLs to give these tags the necessary permissions. See https://tailscale.com/kb/1018/acls/#auto-approvers-for-routes-and-exit-nodes. If you specify custom tags here, you must also make the operator an owner of these tags. See https://tailscale.com/kb/1236/kubernetes-operator/#setting-up-the-kubernetes-operator. Tags cannot be changed once a Connector node has been created. Tag values must be in form ^tag:[a-zA-Z][a-zA-Z0-9-]*$.
description: |-
Tags that the Tailscale node will be tagged with.
Defaults to [tag:k8s].
To autoapprove the subnet routes or exit node defined by a Connector,
you can configure Tailscale ACLs to give these tags the necessary
permissions.
See https://tailscale.com/kb/1018/acls/#auto-approvers-for-routes-and-exit-nodes.
If you specify custom tags here, you must also make the operator an owner of these tags.
See https://tailscale.com/kb/1236/kubernetes-operator/#setting-up-the-kubernetes-operator.
Tags cannot be changed once a Connector node has been created.
Tag values must be in form ^tag:[a-zA-Z][a-zA-Z0-9-]*$.
type: array
items:
type: string
@ -80,47 +128,92 @@ spec:
- rule: has(self.subnetRouter) || self.exitNode == true
message: A Connector needs to be either an exit node or a subnet router, or both.
status:
description: ConnectorStatus describes the status of the Connector. This is set and managed by the Tailscale operator.
description: |-
ConnectorStatus describes the status of the Connector. This is set
and managed by the Tailscale operator.
type: object
properties:
conditions:
description: List of status conditions to indicate the status of the Connector. Known condition types are `ConnectorReady`.
description: |-
List of status conditions to indicate the status of the Connector.
Known condition types are `ConnectorReady`.
type: array
items:
description: ConnectorCondition contains condition information for a Connector.
description: Condition contains details for one aspect of the current state of this API Resource.
type: object
required:
- lastTransitionTime
- message
- reason
- status
- type
properties:
lastTransitionTime:
description: LastTransitionTime is the timestamp corresponding to the last status change of this condition.
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
type: string
format: date-time
message:
description: Message is a human readable description of the details of the last transition, complementing reason.
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
type: string
maxLength: 32768
observedGeneration:
description: If set, this represents the .metadata.generation that the condition was set based upon. For instance, if .metadata.generation is currently 12, but the .status.condition[x].observedGeneration is 9, the condition is out of date with respect to the current state of the Connector.
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
type: integer
format: int64
minimum: 0
reason:
description: Reason is a brief machine readable explanation for the condition's last transition.
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
type: string
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
status:
description: Status of the condition, one of ('True', 'False', 'Unknown').
description: status of the condition, one of True, False, Unknown.
type: string
enum:
- "True"
- "False"
- Unknown
type:
description: Type of the condition, known values are (`SubnetRouterReady`).
description: type of condition in CamelCase or in foo.example.com/CamelCase.
type: string
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
hostname:
description: |-
Hostname is the fully qualified domain name of the Connector node.
If MagicDNS is enabled in your tailnet, it is the MagicDNS name of the
node.
type: string
isExitNode:
description: IsExitNode is set to true if the Connector acts as an exit node.
type: boolean
subnetRoutes:
description: SubnetRoutes are the routes currently exposed to tailnet via this Connector instance.
description: |-
SubnetRoutes are the routes currently exposed to tailnet via this
Connector instance.
type: string
tailnetIPs:
description: |-
TailnetIPs is the set of tailnet IP addresses (both IPv4 and IPv6)
assigned to the Connector node.
type: array
items:
type: string
served: true
storage: true

@ -0,0 +1,181 @@
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.15.1-0.20240618033008-7824932b0cab
name: dnsconfigs.tailscale.com
spec:
group: tailscale.com
names:
kind: DNSConfig
listKind: DNSConfigList
plural: dnsconfigs
shortNames:
- dc
singular: dnsconfig
scope: Cluster
versions:
- additionalPrinterColumns:
- description: Service IP address of the nameserver
jsonPath: .status.nameserver.ip
name: NameserverIP
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: |-
DNSConfig can be deployed to cluster to make a subset of Tailscale MagicDNS
names resolvable by cluster workloads. Use this if: A) you need to refer to
tailnet services, exposed to cluster via Tailscale Kubernetes operator egress
proxies by the MagicDNS names of those tailnet services (usually because the
services run over HTTPS)
B) you have exposed a cluster workload to the tailnet using Tailscale Ingress
and you also want to refer to the workload from within the cluster over the
Ingress's MagicDNS name (usually because you have some callback component
that needs to use the same URL as that used by a non-cluster client on
tailnet).
When a DNSConfig is applied to a cluster, Tailscale Kubernetes operator will
deploy a nameserver for ts.net DNS names and automatically populate it with records
for any Tailscale egress or Ingress proxies deployed to that cluster.
Currently you must manually update your cluster DNS configuration to add the
IP address of the deployed nameserver as a ts.net stub nameserver.
Instructions for how to do it:
https://kubernetes.io/docs/tasks/administer-cluster/dns-custom-nameservers/#configuration-of-stub-domain-and-upstream-nameserver-using-coredns (for CoreDNS),
https://cloud.google.com/kubernetes-engine/docs/how-to/kube-dns (for kube-dns).
Tailscale Kubernetes operator will write the address of a Service fronting
the nameserver to dsnconfig.status.nameserver.ip.
DNSConfig is a singleton - you must not create more than one.
NB: if you want cluster workloads to be able to refer to Tailscale Ingress
using its MagicDNS name, you must also annotate the Ingress resource with
tailscale.com/experimental-forward-cluster-traffic-via-ingress annotation to
ensure that the proxy created for the Ingress listens on its Pod IP address.
NB: Clusters where Pods get assigned IPv6 addresses only are currently not supported.
type: object
required:
- spec
properties:
apiVersion:
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
spec:
description: |-
Spec describes the desired DNS configuration.
More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
type: object
required:
- nameserver
properties:
nameserver:
description: |-
Configuration for a nameserver that can resolve ts.net DNS names
associated with in-cluster proxies for Tailscale egress Services and
Tailscale Ingresses. The operator will always deploy this nameserver
when a DNSConfig is applied.
type: object
properties:
image:
description: Nameserver image.
type: object
properties:
repo:
description: Repo defaults to tailscale/k8s-nameserver.
type: string
tag:
description: Tag defaults to operator's own tag.
type: string
status:
description: |-
Status describes the status of the DNSConfig. This is set
and managed by the Tailscale operator.
type: object
properties:
conditions:
type: array
items:
description: Condition contains details for one aspect of the current state of this API Resource.
type: object
required:
- lastTransitionTime
- message
- reason
- status
- type
properties:
lastTransitionTime:
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
type: string
format: date-time
message:
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
type: string
maxLength: 32768
observedGeneration:
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
type: integer
format: int64
minimum: 0
reason:
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
type: string
maxLength: 1024
minLength: 1
pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$
status:
description: status of the condition, one of True, False, Unknown.
type: string
enum:
- "True"
- "False"
- Unknown
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
type: string
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
x-kubernetes-list-map-keys:
- type
x-kubernetes-list-type: map
nameserver:
description: Nameserver describes the status of nameserver cluster resources.
type: object
properties:
ip:
description: |-
IP is the ClusterIP of the Service fronting the deployed ts.net nameserver.
Currently you must manually update your cluster DNS config to add
this address as a stub nameserver for ts.net for cluster workloads to be
able to resolve MagicDNS names associated with egress or Ingress
proxies.
The IP address will change if you delete and recreate the DNSConfig.
type: string
served: true
storage: true
subresources:
status: {}

File diff suppressed because it is too large Load Diff

@ -0,0 +1,6 @@
apiVersion: tailscale.com/v1alpha1
kind: DNSConfig
metadata:
name: ts-dns
spec:
nameserver: {}

@ -3,6 +3,8 @@ kind: ProxyClass
metadata:
name: prod
spec:
metrics:
enable: true
statefulSet:
annotations:
platform-component: infra
@ -10,6 +12,12 @@ spec:
labels:
team: eng
nodeSelector:
beta.kubernetes.io/os: "linux"
kubernetes.io/os: "linux"
imagePullSecrets:
- name: "foo"
tailscaleContainer:
image: "ghcr.io/tailscale/tailscale:v1.64"
imagePullPolicy: IfNotPresent
tailscaleInitContainer:
image: "ghcr.io/tailscale/tailscale:v1.64"
imagePullPolicy: IfNotPresent

@ -0,0 +1,4 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: dnsrecords

@ -0,0 +1,37 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: nameserver
spec:
replicas: 1
revisionHistoryLimit: 5
selector:
matchLabels:
app: nameserver
strategy:
type: Recreate
template:
metadata:
labels:
app: nameserver
spec:
containers:
- imagePullPolicy: IfNotPresent
name: nameserver
ports:
- name: tcp
protocol: TCP
containerPort: 1053
- name: udp
protocol: UDP
containerPort: 1053
volumeMounts:
- name: dnsrecords
mountPath: /config
restartPolicy: Always
serviceAccount: nameserver
serviceAccountName: nameserver
volumes:
- name: dnsrecords
configMap:
name: dnsrecords

@ -0,0 +1,4 @@
apiVersion: v1
kind: ServiceAccount
metadata:
name: nameserver

@ -0,0 +1,16 @@
apiVersion: v1
kind: Service
metadata:
name: nameserver
spec:
selector:
app: nameserver
ports:
- name: udp
targetPort: 1053
port: 53
protocol: UDP
- name: tcp
targetPort: 1053
port: 53
protocol: TCP

File diff suppressed because it is too large Load Diff

@ -14,10 +14,8 @@ spec:
- name: sysctler
securityContext:
privileged: true
command: ["/bin/sh"]
args:
- -c
- sysctl -w net.ipv4.ip_forward=1 net.ipv6.conf.all.forwarding=1
command: ["/bin/sh", "-c"]
args: [sysctl -w net.ipv4.ip_forward=1 && if sysctl net.ipv6.conf.all.forwarding; then sysctl -w net.ipv6.conf.all.forwarding=1; fi]
resources:
requests:
cpu: 1m

@ -20,3 +20,7 @@ spec:
env:
- name: TS_USERSPACE
value: "true"
- name: POD_IP
valueFrom:
fieldRef:
fieldPath: status.podIP

@ -0,0 +1,337 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !plan9
// tailscale-operator provides a way to expose services running in a Kubernetes
// cluster to your Tailnet and to make Tailscale nodes available to cluster
// workloads
package main
import (
"context"
"encoding/json"
"fmt"
"slices"
"go.uber.org/zap"
corev1 "k8s.io/api/core/v1"
discoveryv1 "k8s.io/api/discovery/v1"
networkingv1 "k8s.io/api/networking/v1"
apiequality "k8s.io/apimachinery/pkg/api/equality"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/types"
"k8s.io/utils/net"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
operatorutils "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/util/mak"
)
const (
dnsRecordsRecocilerFinalizer = "tailscale.com/dns-records-reconciler"
annotationTSMagicDNSName = "tailscale.com/magic-dnsname"
)
// dnsRecordsReconciler knows how to update dnsrecords ConfigMap with DNS
// records.
// The records that it creates are:
// - For tailscale Ingress, a mapping of the Ingress's MagicDNSName to the IP address of
// the ingress proxy Pod.
// - For egress proxies configured via tailscale.com/tailnet-fqdn annotation, a
// mapping of the tailnet FQDN to the IP address of the egress proxy Pod.
//
// Records will only be created if there is exactly one ready
// tailscale.com/v1alpha1.DNSConfig instance in the cluster (so that we know
// that there is a ts.net nameserver deployed in the cluster).
type dnsRecordsReconciler struct {
client.Client
tsNamespace string // namespace in which we provision tailscale resources
logger *zap.SugaredLogger
isDefaultLoadBalancer bool // true if operator is the default ingress controller in this cluster
}
// Reconcile takes a reconcile.Request for a headless Service fronting a
// tailscale proxy and updates DNS Records in dnsrecords ConfigMap for the
// in-cluster ts.net nameserver if required.
func (dnsRR *dnsRecordsReconciler) Reconcile(ctx context.Context, req reconcile.Request) (res reconcile.Result, err error) {
logger := dnsRR.logger.With("Service", req.NamespacedName)
logger.Debugf("starting reconcile")
defer logger.Debugf("reconcile finished")
headlessSvc := new(corev1.Service)
err = dnsRR.Client.Get(ctx, req.NamespacedName, headlessSvc)
if apierrors.IsNotFound(err) {
logger.Debugf("Service not found")
return reconcile.Result{}, nil
}
if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get Service: %w", err)
}
if !(isManagedByType(headlessSvc, "svc") || isManagedByType(headlessSvc, "ingress")) {
logger.Debugf("Service is not a headless Service for a tailscale ingress or egress proxy; do nothing")
return reconcile.Result{}, nil
}
if !headlessSvc.DeletionTimestamp.IsZero() {
logger.Debug("Service is being deleted, clean up resources")
return reconcile.Result{}, dnsRR.maybeCleanup(ctx, headlessSvc, logger)
}
// Check that there is a ts.net nameserver deployed to the cluster by
// checking that there is tailscale.com/v1alpha1.DNSConfig resource in a
// Ready state.
dnsCfgLst := new(tsapi.DNSConfigList)
if err = dnsRR.List(ctx, dnsCfgLst); err != nil {
return reconcile.Result{}, fmt.Errorf("error listing DNSConfigs: %w", err)
}
if len(dnsCfgLst.Items) == 0 {
logger.Debugf("DNSConfig does not exist, not creating DNS records")
return reconcile.Result{}, nil
}
if len(dnsCfgLst.Items) > 1 {
logger.Errorf("Invalid cluster state - more than one DNSConfig found in cluster. Please ensure no more than one exists")
return reconcile.Result{}, nil
}
dnsCfg := dnsCfgLst.Items[0]
if !operatorutils.DNSCfgIsReady(&dnsCfg) {
logger.Info("DNSConfig is not ready yet, waiting...")
return reconcile.Result{}, nil
}
return reconcile.Result{}, dnsRR.maybeProvision(ctx, headlessSvc, logger)
}
// maybeProvision ensures that dnsrecords ConfigMap contains a record for the
// proxy associated with the headless Service.
// The record is only provisioned if the proxy is for a tailscale Ingress or
// egress configured via tailscale.com/tailnet-fqdn annotation.
//
// For Ingress, the record is a mapping between the MagicDNSName of the Ingress, retrieved from
// ingress.status.loadBalancer.ingress.hostname field and the proxy Pod IP addresses
// retrieved from the EndpoinSlice associated with this headless Service, i.e
// Records{IP4: <MagicDNS name of the Ingress>: <[IPs of the ingress proxy Pods]>}
//
// For egress, the record is a mapping between tailscale.com/tailnet-fqdn
// annotation and the proxy Pod IP addresses, retrieved from the EndpointSlice
// associated with this headless Service, i.e
// Records{IP4: {<tailscale.com/tailnet-fqdn>: <[IPs of the egress proxy Pods]>}
//
// If records need to be created for this proxy, maybeProvision will also:
// - update the headless Service with a tailscale.com/magic-dnsname annotation
// - update the headless Service with a finalizer
func (dnsRR *dnsRecordsReconciler) maybeProvision(ctx context.Context, headlessSvc *corev1.Service, logger *zap.SugaredLogger) error {
if headlessSvc == nil {
logger.Info("[unexpected] maybeProvision called with a nil Service")
return nil
}
isEgressFQDNSvc, err := dnsRR.isSvcForFQDNEgressProxy(ctx, headlessSvc)
if err != nil {
return fmt.Errorf("error checking whether the Service is for an egress proxy: %w", err)
}
if !(isEgressFQDNSvc || isManagedByType(headlessSvc, "ingress")) {
logger.Debug("Service is not fronting a proxy that we create DNS records for; do nothing")
return nil
}
fqdn, err := dnsRR.fqdnForDNSRecord(ctx, headlessSvc, logger)
if err != nil {
return fmt.Errorf("error determining DNS name for record: %w", err)
}
if fqdn == "" {
logger.Debugf("MagicDNS name does not (yet) exist, not provisioning DNS record")
return nil // a new reconcile will be triggered once it's added
}
oldHeadlessSvc := headlessSvc.DeepCopy()
// Ensure that headless Service is annotated with a finalizer to help
// with records cleanup when proxy resources are deleted.
if !slices.Contains(headlessSvc.Finalizers, dnsRecordsRecocilerFinalizer) {
headlessSvc.Finalizers = append(headlessSvc.Finalizers, dnsRecordsRecocilerFinalizer)
}
// Ensure that headless Service is annotated with the current MagicDNS
// name to help with records cleanup when proxy resources are deleted or
// MagicDNS name changes.
oldFqdn := headlessSvc.Annotations[annotationTSMagicDNSName]
if oldFqdn != "" && oldFqdn != fqdn { // i.e user has changed the value of tailscale.com/tailnet-fqdn annotation
logger.Debugf("MagicDNS name has changed, remvoving record for %s", oldFqdn)
updateFunc := func(rec *operatorutils.Records) {
delete(rec.IP4, oldFqdn)
}
if err = dnsRR.updateDNSConfig(ctx, updateFunc); err != nil {
return fmt.Errorf("error removing record for %s: %w", oldFqdn, err)
}
}
mak.Set(&headlessSvc.Annotations, annotationTSMagicDNSName, fqdn)
if !apiequality.Semantic.DeepEqual(oldHeadlessSvc, headlessSvc) {
logger.Infof("provisioning DNS record for MagicDNS name: %s", fqdn) // this will be printed exactly once
if err := dnsRR.Update(ctx, headlessSvc); err != nil {
return fmt.Errorf("error updating proxy headless Service metadata: %w", err)
}
}
// Get the Pod IP addresses for the proxy from the EndpointSlice for the
// headless Service.
labels := map[string]string{discoveryv1.LabelServiceName: headlessSvc.Name} // https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/#ownership
eps, err := getSingleObject[discoveryv1.EndpointSlice](ctx, dnsRR.Client, dnsRR.tsNamespace, labels)
if err != nil {
return fmt.Errorf("error getting the EndpointSlice for the proxy's headless Service: %w", err)
}
if eps == nil {
logger.Debugf("proxy's headless Service EndpointSlice does not yet exist. We will reconcile again once it's created")
return nil
}
// An EndpointSlice for a Service can have a list of endpoints that each
// can have multiple addresses - these are the IP addresses of any Pods
// selected by that Service. Pick all the IPv4 addresses.
ips := make([]string, 0)
for _, ep := range eps.Endpoints {
for _, ip := range ep.Addresses {
if !net.IsIPv4String(ip) {
logger.Infof("EndpointSlice contains IP address %q that is not IPv4, ignoring. Currently only IPv4 is supported", ip)
} else {
ips = append(ips, ip)
}
}
}
if len(ips) == 0 {
logger.Debugf("EndpointSlice for the Service contains no IPv4 addresses. We will reconcile again once they are created.")
return nil
}
updateFunc := func(rec *operatorutils.Records) {
mak.Set(&rec.IP4, fqdn, ips)
}
if err = dnsRR.updateDNSConfig(ctx, updateFunc); err != nil {
return fmt.Errorf("error updating DNS records: %w", err)
}
return nil
}
// maybeCleanup ensures that the DNS record for the proxy has been removed from
// dnsrecords ConfigMap and the tailscale.com/dns-records-reconciler finalizer
// has been removed from the Service. If the record is not found in the
// ConfigMap, the ConfigMap does not exist, or the Service does not have
// tailscale.com/magic-dnsname annotation, just remove the finalizer.
func (h *dnsRecordsReconciler) maybeCleanup(ctx context.Context, headlessSvc *corev1.Service, logger *zap.SugaredLogger) error {
ix := slices.Index(headlessSvc.Finalizers, dnsRecordsRecocilerFinalizer)
if ix == -1 {
logger.Debugf("no finalizer, nothing to do")
return nil
}
cm := &corev1.ConfigMap{}
err := h.Client.Get(ctx, types.NamespacedName{Name: operatorutils.DNSRecordsCMName, Namespace: h.tsNamespace}, cm)
if apierrors.IsNotFound(err) {
logger.Debug("'dsnrecords' ConfigMap not found")
return h.removeHeadlessSvcFinalizer(ctx, headlessSvc)
}
if err != nil {
return fmt.Errorf("error retrieving 'dnsrecords' ConfigMap: %w", err)
}
if cm.Data == nil {
logger.Debug("'dnsrecords' ConfigMap contains no records")
return h.removeHeadlessSvcFinalizer(ctx, headlessSvc)
}
_, ok := cm.Data[operatorutils.DNSRecordsCMKey]
if !ok {
logger.Debug("'dnsrecords' ConfigMap contains no records")
return h.removeHeadlessSvcFinalizer(ctx, headlessSvc)
}
fqdn, _ := headlessSvc.GetAnnotations()[annotationTSMagicDNSName]
if fqdn == "" {
return h.removeHeadlessSvcFinalizer(ctx, headlessSvc)
}
logger.Infof("removing DNS record for MagicDNS name %s", fqdn)
updateFunc := func(rec *operatorutils.Records) {
delete(rec.IP4, fqdn)
}
if err = h.updateDNSConfig(ctx, updateFunc); err != nil {
return fmt.Errorf("error updating DNS config: %w", err)
}
return h.removeHeadlessSvcFinalizer(ctx, headlessSvc)
}
func (dnsRR *dnsRecordsReconciler) removeHeadlessSvcFinalizer(ctx context.Context, headlessSvc *corev1.Service) error {
idx := slices.Index(headlessSvc.Finalizers, dnsRecordsRecocilerFinalizer)
if idx == -1 {
return nil
}
headlessSvc.Finalizers = append(headlessSvc.Finalizers[:idx], headlessSvc.Finalizers[idx+1:]...)
return dnsRR.Update(ctx, headlessSvc)
}
// fqdnForDNSRecord returns MagicDNS name associated with a given headless Service.
// If the headless Service is for a tailscale Ingress proxy, returns ingress.status.loadBalancer.ingress.hostname.
// If the headless Service is for an tailscale egress proxy configured via tailscale.com/tailnet-fqdn annotation, returns the annotation value.
// This function is not expected to be called with headless Services for other
// proxy types, or any other Services, but it just returns an empty string if
// that happens.
func (dnsRR *dnsRecordsReconciler) fqdnForDNSRecord(ctx context.Context, headlessSvc *corev1.Service, logger *zap.SugaredLogger) (string, error) {
parentName := parentFromObjectLabels(headlessSvc)
if isManagedByType(headlessSvc, "ingress") {
ing := new(networkingv1.Ingress)
if err := dnsRR.Get(ctx, parentName, ing); err != nil {
return "", err
}
if len(ing.Status.LoadBalancer.Ingress) == 0 {
return "", nil
}
return ing.Status.LoadBalancer.Ingress[0].Hostname, nil
}
if isManagedByType(headlessSvc, "svc") {
svc := new(corev1.Service)
if err := dnsRR.Get(ctx, parentName, svc); apierrors.IsNotFound(err) {
logger.Info("[unexpected] parent Service for egress proxy %s not found", headlessSvc.Name)
return "", nil
} else if err != nil {
return "", err
}
return svc.Annotations[AnnotationTailnetTargetFQDN], nil
}
return "", nil
}
// updateDNSConfig runs the provided update function against dnsrecords
// ConfigMap. At this point the in-cluster ts.net nameserver is expected to be
// successfully created together with the ConfigMap.
func (dnsRR *dnsRecordsReconciler) updateDNSConfig(ctx context.Context, update func(*operatorutils.Records)) error {
cm := &corev1.ConfigMap{}
err := dnsRR.Get(ctx, types.NamespacedName{Name: operatorutils.DNSRecordsCMName, Namespace: dnsRR.tsNamespace}, cm)
if apierrors.IsNotFound(err) {
dnsRR.logger.Info("[unexpected] dnsrecords ConfigMap not found in cluster. Not updating DNS records. Please open an isue and attach operator logs.")
return nil
}
if err != nil {
return fmt.Errorf("error retrieving dnsrecords ConfigMap: %w", err)
}
dnsRecords := operatorutils.Records{Version: operatorutils.Alpha1Version, IP4: map[string][]string{}}
if cm.Data != nil && cm.Data[operatorutils.DNSRecordsCMKey] != "" {
if err := json.Unmarshal([]byte(cm.Data[operatorutils.DNSRecordsCMKey]), &dnsRecords); err != nil {
return err
}
}
update(&dnsRecords)
dnsRecordsBs, err := json.Marshal(dnsRecords)
if err != nil {
return fmt.Errorf("error marshalling DNS records: %w", err)
}
mak.Set(&cm.Data, operatorutils.DNSRecordsCMKey, string(dnsRecordsBs))
return dnsRR.Update(ctx, cm)
}
// isSvcForFQDNEgressProxy returns true if the Service is a headless Service
// created for a proxy for a tailscale egress Service configured via
// tailscale.com/tailnet-fqdn annotation.
func (dnsRR *dnsRecordsReconciler) isSvcForFQDNEgressProxy(ctx context.Context, svc *corev1.Service) (bool, error) {
if !isManagedByType(svc, "svc") {
return false, nil
}
parentName := parentFromObjectLabels(svc)
parentSvc := new(corev1.Service)
if err := dnsRR.Get(ctx, parentName, parentSvc); apierrors.IsNotFound(err) {
return false, nil
} else if err != nil {
return false, err
}
annots := parentSvc.Annotations
return annots != nil && annots[AnnotationTailnetTargetFQDN] != "", nil
}

@ -0,0 +1,198 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !plan9
package main
import (
"context"
"encoding/json"
"testing"
"github.com/google/go-cmp/cmp"
"go.uber.org/zap"
corev1 "k8s.io/api/core/v1"
discoveryv1 "k8s.io/api/discovery/v1"
networkingv1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
operatorutils "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/tstest"
"tailscale.com/types/ptr"
)
func TestDNSRecordsReconciler(t *testing.T) {
// Preconfigure a cluster with a DNSConfig
dnsConfig := &tsapi.DNSConfig{
ObjectMeta: metav1.ObjectMeta{
Name: "test",
},
TypeMeta: metav1.TypeMeta{Kind: "DNSConfig"},
Spec: tsapi.DNSConfigSpec{
Nameserver: &tsapi.Nameserver{},
}}
ing := &networkingv1.Ingress{
ObjectMeta: metav1.ObjectMeta{
Name: "ts-ingress",
Namespace: "test",
},
Spec: networkingv1.IngressSpec{
IngressClassName: ptr.To("tailscale"),
},
Status: networkingv1.IngressStatus{
LoadBalancer: networkingv1.IngressLoadBalancerStatus{
Ingress: []networkingv1.IngressLoadBalancerIngress{{
Hostname: "cluster.ingress.ts.net"}},
},
},
}
cm := &corev1.ConfigMap{ObjectMeta: metav1.ObjectMeta{Name: "dnsrecords", Namespace: "tailscale"}}
fc := fake.NewClientBuilder().
WithScheme(tsapi.GlobalScheme).
WithObjects(cm).
WithObjects(dnsConfig).
WithObjects(ing).
WithStatusSubresource(dnsConfig, ing).
Build()
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
cl := tstest.NewClock(tstest.ClockOpts{})
// Set the ready condition of the DNSConfig
mustUpdateStatus[tsapi.DNSConfig](t, fc, "", "test", func(c *tsapi.DNSConfig) {
operatorutils.SetDNSConfigCondition(c, tsapi.NameserverReady, metav1.ConditionTrue, reasonNameserverCreated, reasonNameserverCreated, 0, cl, zl.Sugar())
})
dnsRR := &dnsRecordsReconciler{
Client: fc,
logger: zl.Sugar(),
tsNamespace: "tailscale",
}
// 1. DNS record is created for an egress proxy configured via
// tailscale.com/tailnet-fqdn annotation
egressSvcFQDN := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "egress-fqdn",
Namespace: "test",
Annotations: map[string]string{"tailscale.com/tailnet-fqdn": "foo.bar.ts.net"},
},
Spec: corev1.ServiceSpec{
ExternalName: "unused",
Type: corev1.ServiceTypeExternalName,
},
}
headlessForEgressSvcFQDN := headlessSvcForParent(egressSvcFQDN, "svc") // create the proxy headless Service
ep := endpointSliceForService(headlessForEgressSvcFQDN, "10.9.8.7")
mustCreate(t, fc, egressSvcFQDN)
mustCreate(t, fc, headlessForEgressSvcFQDN)
mustCreate(t, fc, ep)
expectReconciled(t, dnsRR, "tailscale", "egress-fqdn") // dns-records-reconciler reconcile the headless Service
// ConfigMap should now have a record for foo.bar.ts.net -> 10.8.8.7
wantHosts := map[string][]string{"foo.bar.ts.net": {"10.9.8.7"}}
expectHostsRecords(t, fc, wantHosts)
// 2. DNS record is updated if tailscale.com/tailnet-fqdn annotation's
// value changes
mustUpdate(t, fc, "test", "egress-fqdn", func(svc *corev1.Service) {
svc.Annotations["tailscale.com/tailnet-fqdn"] = "baz.bar.ts.net"
})
expectReconciled(t, dnsRR, "tailscale", "egress-fqdn") // dns-records-reconciler reconcile the headless Service
wantHosts = map[string][]string{"baz.bar.ts.net": {"10.9.8.7"}}
expectHostsRecords(t, fc, wantHosts)
// 3. DNS record is updated if the IP address of the proxy Pod changes.
ep = endpointSliceForService(headlessForEgressSvcFQDN, "10.6.5.4")
mustUpdate(t, fc, ep.Namespace, ep.Name, func(ep *discoveryv1.EndpointSlice) {
ep.Endpoints[0].Addresses = []string{"10.6.5.4"}
})
expectReconciled(t, dnsRR, "tailscale", "egress-fqdn") // dns-records-reconciler reconcile the headless Service
wantHosts = map[string][]string{"baz.bar.ts.net": {"10.6.5.4"}}
expectHostsRecords(t, fc, wantHosts)
// 4. DNS record is created for an ingress proxy configured via Ingress
headlessForIngress := headlessSvcForParent(ing, "ingress")
ep = endpointSliceForService(headlessForIngress, "10.9.8.7")
mustCreate(t, fc, headlessForIngress)
mustCreate(t, fc, ep)
expectReconciled(t, dnsRR, "tailscale", "ts-ingress") // dns-records-reconciler should reconcile the headless Service
wantHosts["cluster.ingress.ts.net"] = []string{"10.9.8.7"}
expectHostsRecords(t, fc, wantHosts)
// 5. DNS records are updated if Ingress's MagicDNS name changes (i.e users changed spec.tls.hosts[0])
t.Log("test case 5")
mustUpdateStatus(t, fc, "test", "ts-ingress", func(ing *networkingv1.Ingress) {
ing.Status.LoadBalancer.Ingress[0].Hostname = "another.ingress.ts.net"
})
expectReconciled(t, dnsRR, "tailscale", "ts-ingress") // dns-records-reconciler should reconcile the headless Service
delete(wantHosts, "cluster.ingress.ts.net")
wantHosts["another.ingress.ts.net"] = []string{"10.9.8.7"}
expectHostsRecords(t, fc, wantHosts)
// 6. DNS records are updated if Ingress proxy's Pod IP changes
mustUpdate(t, fc, ep.Namespace, ep.Name, func(ep *discoveryv1.EndpointSlice) {
ep.Endpoints[0].Addresses = []string{"7.8.9.10"}
})
expectReconciled(t, dnsRR, "tailscale", "ts-ingress")
wantHosts["another.ingress.ts.net"] = []string{"7.8.9.10"}
expectHostsRecords(t, fc, wantHosts)
}
func headlessSvcForParent(o client.Object, typ string) *corev1.Service {
return &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: o.GetName(),
Namespace: "tailscale",
Labels: map[string]string{
LabelManaged: "true",
LabelParentName: o.GetName(),
LabelParentNamespace: o.GetNamespace(),
LabelParentType: typ,
},
},
Spec: corev1.ServiceSpec{
ClusterIP: "None",
Type: corev1.ServiceTypeClusterIP,
Selector: map[string]string{"foo": "bar"},
},
}
}
func endpointSliceForService(svc *corev1.Service, ip string) *discoveryv1.EndpointSlice {
return &discoveryv1.EndpointSlice{
ObjectMeta: metav1.ObjectMeta{
Name: svc.Name,
Namespace: svc.Namespace,
Labels: map[string]string{discoveryv1.LabelServiceName: svc.Name},
},
Endpoints: []discoveryv1.Endpoint{{
Addresses: []string{ip},
}},
}
}
func expectHostsRecords(t *testing.T, cl client.Client, wantsHosts map[string][]string) {
t.Helper()
cm := new(corev1.ConfigMap)
if err := cl.Get(context.Background(), types.NamespacedName{Name: "dnsrecords", Namespace: "tailscale"}, cm); err != nil {
t.Fatalf("getting dnsconfig ConfigMap: %v", err)
}
if cm.Data == nil {
t.Fatal("dnsconfig ConfigMap has no data")
}
dnsConfigString, ok := cm.Data[operatorutils.DNSRecordsCMKey]
if !ok {
t.Fatal("dnsconfig ConfigMap does not contain dnsconfig")
}
dnsConfig := &operatorutils.Records{}
if err := json.Unmarshal([]byte(dnsConfigString), dnsConfig); err != nil {
t.Fatalf("unmarshaling dnsconfig: %v", err)
}
if diff := cmp.Diff(dnsConfig.IP4, wantsHosts); diff != "" {
t.Fatalf("unexpected dns config (-got +want):\n%s", diff)
}
}

@ -22,9 +22,11 @@ const (
operatorDeploymentFilesPath = "cmd/k8s-operator/deploy"
connectorCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_connectors.yaml"
proxyClassCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_proxyclasses.yaml"
dnsConfigCRDPath = operatorDeploymentFilesPath + "/crds/tailscale.com_dnsconfigs.yaml"
helmTemplatesPath = operatorDeploymentFilesPath + "/chart/templates"
connectorCRDHelmTemplatePath = helmTemplatesPath + "/connector.yaml"
proxyClassCRDHelmTemplatePath = helmTemplatesPath + "/proxyclass.yaml"
dnsConfigCRDHelmTemplatePath = helmTemplatesPath + "/dnsconfig.yaml"
helmConditionalStart = "{{ if .Values.installCRDs -}}\n"
helmConditionalEnd = "{{- end -}}"
@ -36,10 +38,10 @@ func main() {
}
repoRoot := "../../"
switch os.Args[1] {
case "helmcrd": // insert CRD to Helm templates behind a installCRDs=true conditional check
log.Print("Adding Connector CRD to Helm templates")
case "helmcrd": // insert CRDs to Helm templates behind a installCRDs=true conditional check
log.Print("Adding CRDs to Helm templates")
if err := generate("./"); err != nil {
log.Fatalf("error adding Connector CRD to Helm templates: %v", err)
log.Fatalf("error adding CRDs to Helm templates: %v", err)
}
return
case "staticmanifests": // generate static manifests from Helm templates (including the CRD)
@ -108,7 +110,7 @@ func main() {
}
}
// generate places tailscale.com CRDs (currently Connector and ProxyClass) into
// generate places tailscale.com CRDs (currently Connector, ProxyClass and DNSConfig) into
// the Helm chart templates behind .Values.installCRDs=true condition (true by
// default).
func generate(baseDir string) error {
@ -140,6 +142,9 @@ func generate(baseDir string) error {
if err := addCRDToHelm(proxyClassCRDPath, proxyClassCRDHelmTemplatePath); err != nil {
return fmt.Errorf("error adding ProxyClass CRD to Helm templates: %w", err)
}
if err := addCRDToHelm(dnsConfigCRDPath, dnsConfigCRDHelmTemplatePath); err != nil {
return fmt.Errorf("error adding DNSConfig CRD to Helm templates: %w", err)
}
return nil
}
@ -151,5 +156,8 @@ func cleanup(baseDir string) error {
if err := os.Remove(filepath.Join(baseDir, proxyClassCRDHelmTemplatePath)); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("error cleaning up ProxyClass CRD template: %w", err)
}
if err := os.Remove(filepath.Join(baseDir, dnsConfigCRDHelmTemplatePath)); err != nil && !os.IsNotExist(err) {
return fmt.Errorf("error cleaning up DNSConfig CRD template: %w", err)
}
return nil
}

@ -56,6 +56,9 @@ func Test_generate(t *testing.T) {
if !strings.Contains(installContentsWithCRD.String(), "name: proxyclasses.tailscale.com") {
t.Errorf("ProxyClass CRD not found in default chart install")
}
if !strings.Contains(installContentsWithCRD.String(), "name: dnsconfigs.tailscale.com") {
t.Errorf("DNSConfig CRD not found in default chart install")
}
// Test that CRDs can be excluded from Helm chart install
installContentsWithoutCRD := bytes.NewBuffer([]byte{})
@ -71,4 +74,7 @@ func Test_generate(t *testing.T) {
if strings.Contains(installContentsWithoutCRD.String(), "name: connectors.tailscale.com") {
t.Errorf("ProxyClass CRD found in chart install that should not contain a CRD")
}
if strings.Contains(installContentsWithoutCRD.String(), "name: dnsconfigs.tailscale.com") {
t.Errorf("DNSConfig CRD found in chart install that should not contain a CRD")
}
}

@ -264,7 +264,7 @@ func (a *IngressReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
ServeConfig: sc,
Tags: tags,
ChildResourceLabels: crl,
ProxyClass: proxyClass,
ProxyClassName: proxyClass,
}
if val := ing.GetAnnotations()[AnnotationExperimentalForwardClusterTrafficViaL7IngresProxy]; val == "true" {

@ -100,7 +100,7 @@ func TestTailscaleIngress(t *testing.T) {
}
opts.serveConfig = serveConfig
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "ingress"), nil)
expectEqual(t, fc, expectedSTSUserspace(t, fc, opts), removeHashAnnotation)
@ -231,7 +231,7 @@ func TestTailscaleIngressWithProxyClass(t *testing.T) {
}
opts.serveConfig = serveConfig
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "ingress"), nil)
expectEqual(t, fc, expectedSTSUserspace(t, fc, opts), removeHashAnnotation)
@ -248,9 +248,9 @@ func TestTailscaleIngressWithProxyClass(t *testing.T) {
// created proxy resources.
mustUpdateStatus(t, fc, "", "custom-metadata", func(pc *tsapi.ProxyClass) {
pc.Status = tsapi.ProxyClassStatus{
Conditions: []tsapi.ConnectorCondition{{
Conditions: []metav1.Condition{{
Status: metav1.ConditionTrue,
Type: tsapi.ProxyClassready,
Type: string(tsapi.ProxyClassready),
ObservedGeneration: pc.Generation,
}}}
})

@ -0,0 +1,283 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !plan9
package main
import (
"context"
"fmt"
"slices"
"sync"
_ "embed"
"github.com/pkg/errors"
"go.uber.org/zap"
xslices "golang.org/x/exp/slices"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
apiequality "k8s.io/apimachinery/pkg/api/equality"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/tools/record"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"sigs.k8s.io/yaml"
tsoperator "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/tstime"
"tailscale.com/util/clientmetric"
"tailscale.com/util/set"
)
const (
reasonNameserverCreationFailed = "NameserverCreationFailed"
reasonMultipleDNSConfigsPresent = "MultipleDNSConfigsPresent"
reasonNameserverCreated = "NameserverCreated"
messageNameserverCreationFailed = "Failed creating nameserver resources: %v"
messageMultipleDNSConfigsPresent = "Multiple DNSConfig resources found in cluster. Please ensure no more than one is present."
defaultNameserverImageRepo = "tailscale/k8s-nameserver"
// TODO (irbekrm): once we start publishing nameserver images for stable
// track, replace 'unstable' here with the version of this operator
// instance.
defaultNameserverImageTag = "unstable"
)
// NameserverReconciler knows how to create nameserver resources in cluster in
// response to users applying DNSConfig.
type NameserverReconciler struct {
client.Client
logger *zap.SugaredLogger
recorder record.EventRecorder
clock tstime.Clock
tsNamespace string
mu sync.Mutex // protects following
managedNameservers set.Slice[types.UID] // one or none
}
var (
gaugeNameserverResources = clientmetric.NewGauge("k8s_nameserver_resources")
)
func (a *NameserverReconciler) Reconcile(ctx context.Context, req reconcile.Request) (res reconcile.Result, err error) {
logger := a.logger.With("dnsConfig", req.Name)
logger.Debugf("starting reconcile")
defer logger.Debugf("reconcile finished")
var dnsCfg tsapi.DNSConfig
err = a.Get(ctx, req.NamespacedName, &dnsCfg)
if apierrors.IsNotFound(err) {
// Request object not found, could have been deleted after reconcile request.
logger.Debugf("dnsconfig not found, assuming it was deleted")
return reconcile.Result{}, nil
} else if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get dnsconfig: %w", err)
}
if !dnsCfg.DeletionTimestamp.IsZero() {
ix := xslices.Index(dnsCfg.Finalizers, FinalizerName)
if ix < 0 {
logger.Debugf("no finalizer, nothing to do")
return reconcile.Result{}, nil
}
logger.Info("Cleaning up DNSConfig resources")
if err := a.maybeCleanup(ctx, &dnsCfg, logger); err != nil {
logger.Errorf("error cleaning up reconciler resource: %v", err)
return res, err
}
dnsCfg.Finalizers = append(dnsCfg.Finalizers[:ix], dnsCfg.Finalizers[ix+1:]...)
if err := a.Update(ctx, &dnsCfg); err != nil {
logger.Errorf("error removing finalizer: %v", err)
return reconcile.Result{}, err
}
logger.Infof("Nameserver resources cleaned up")
return reconcile.Result{}, nil
}
oldCnStatus := dnsCfg.Status.DeepCopy()
setStatus := func(dnsCfg *tsapi.DNSConfig, conditionType tsapi.ConditionType, status metav1.ConditionStatus, reason, message string) (reconcile.Result, error) {
tsoperator.SetDNSConfigCondition(dnsCfg, tsapi.NameserverReady, status, reason, message, dnsCfg.Generation, a.clock, logger)
if !apiequality.Semantic.DeepEqual(oldCnStatus, dnsCfg.Status) {
// An error encountered here should get returned by the Reconcile function.
if updateErr := a.Client.Status().Update(ctx, dnsCfg); updateErr != nil {
err = errors.Wrap(err, updateErr.Error())
}
}
return res, err
}
var dnsCfgs tsapi.DNSConfigList
if err := a.List(ctx, &dnsCfgs); err != nil {
return res, fmt.Errorf("error listing DNSConfigs: %w", err)
}
if len(dnsCfgs.Items) > 1 { // enforce DNSConfig to be a singleton
msg := "invalid cluster configuration: more than one tailscale.com/dnsconfigs found. Please ensure that no more than one is created."
logger.Error(msg)
a.recorder.Event(&dnsCfg, corev1.EventTypeWarning, reasonMultipleDNSConfigsPresent, messageMultipleDNSConfigsPresent)
setStatus(&dnsCfg, tsapi.NameserverReady, metav1.ConditionFalse, reasonMultipleDNSConfigsPresent, messageMultipleDNSConfigsPresent)
}
if !slices.Contains(dnsCfg.Finalizers, FinalizerName) {
logger.Infof("ensuring nameserver resources")
dnsCfg.Finalizers = append(dnsCfg.Finalizers, FinalizerName)
if err := a.Update(ctx, &dnsCfg); err != nil {
msg := fmt.Sprintf(messageNameserverCreationFailed, err)
logger.Error(msg)
return setStatus(&dnsCfg, tsapi.NameserverReady, metav1.ConditionFalse, reasonNameserverCreationFailed, msg)
}
}
if err := a.maybeProvision(ctx, &dnsCfg, logger); err != nil {
return reconcile.Result{}, fmt.Errorf("error provisioning nameserver resources: %w", err)
}
a.mu.Lock()
a.managedNameservers.Add(dnsCfg.UID)
a.mu.Unlock()
gaugeNameserverResources.Set(int64(a.managedNameservers.Len()))
svc := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{Name: "nameserver", Namespace: a.tsNamespace},
}
if err := a.Client.Get(ctx, client.ObjectKeyFromObject(svc), svc); err != nil {
return res, fmt.Errorf("error getting Service: %w", err)
}
if ip := svc.Spec.ClusterIP; ip != "" && ip != "None" {
dnsCfg.Status.Nameserver = &tsapi.NameserverStatus{
IP: ip,
}
return setStatus(&dnsCfg, tsapi.NameserverReady, metav1.ConditionTrue, reasonNameserverCreated, reasonNameserverCreated)
}
logger.Info("nameserver Service does not have an IP address allocated, waiting...")
return reconcile.Result{}, nil
}
func nameserverResourceLabels(name, namespace string) map[string]string {
labels := childResourceLabels(name, namespace, "nameserver")
labels["app.kubernetes.io/name"] = "tailscale"
labels["app.kubernetes.io/component"] = "nameserver"
return labels
}
func (a *NameserverReconciler) maybeProvision(ctx context.Context, tsDNSCfg *tsapi.DNSConfig, logger *zap.SugaredLogger) error {
labels := nameserverResourceLabels(tsDNSCfg.Name, a.tsNamespace)
dCfg := &deployConfig{
ownerRefs: []metav1.OwnerReference{*metav1.NewControllerRef(tsDNSCfg, tsapi.SchemeGroupVersion.WithKind("DNSConfig"))},
namespace: a.tsNamespace,
labels: labels,
imageRepo: defaultNameserverImageRepo,
imageTag: defaultNameserverImageTag,
}
if tsDNSCfg.Spec.Nameserver.Image != nil && tsDNSCfg.Spec.Nameserver.Image.Repo != "" {
dCfg.imageRepo = tsDNSCfg.Spec.Nameserver.Image.Repo
}
if tsDNSCfg.Spec.Nameserver.Image != nil && tsDNSCfg.Spec.Nameserver.Image.Tag != "" {
dCfg.imageTag = tsDNSCfg.Spec.Nameserver.Image.Tag
}
for _, deployable := range []deployable{saDeployable, deployDeployable, svcDeployable, cmDeployable} {
if err := deployable.updateObj(ctx, dCfg, a.Client); err != nil {
return fmt.Errorf("error reconciling %s: %w", deployable.kind, err)
}
}
return nil
}
// maybeCleanup removes DNSConfig from being tracked. The cluster resources
// created, will be automatically garbage collected as they are owned by the
// DNSConfig.
func (a *NameserverReconciler) maybeCleanup(ctx context.Context, dnsCfg *tsapi.DNSConfig, logger *zap.SugaredLogger) error {
a.mu.Lock()
a.managedNameservers.Remove(dnsCfg.UID)
a.mu.Unlock()
gaugeNameserverResources.Set(int64(a.managedNameservers.Len()))
return nil
}
type deployable struct {
kind string
updateObj func(context.Context, *deployConfig, client.Client) error
}
type deployConfig struct {
imageRepo string
imageTag string
labels map[string]string
ownerRefs []metav1.OwnerReference
namespace string
}
var (
//go:embed deploy/manifests/nameserver/cm.yaml
cmYaml []byte
//go:embed deploy/manifests/nameserver/deploy.yaml
deployYaml []byte
//go:embed deploy/manifests/nameserver/sa.yaml
saYaml []byte
//go:embed deploy/manifests/nameserver/svc.yaml
svcYaml []byte
deployDeployable = deployable{
kind: "Deployment",
updateObj: func(ctx context.Context, cfg *deployConfig, kubeClient client.Client) error {
d := new(appsv1.Deployment)
if err := yaml.Unmarshal(deployYaml, &d); err != nil {
return fmt.Errorf("error unmarshalling Deployment yaml: %w", err)
}
d.Spec.Template.Spec.Containers[0].Image = fmt.Sprintf("%s:%s", cfg.imageRepo, cfg.imageTag)
d.ObjectMeta.Namespace = cfg.namespace
d.ObjectMeta.Labels = cfg.labels
d.ObjectMeta.OwnerReferences = cfg.ownerRefs
updateF := func(oldD *appsv1.Deployment) {
oldD.Spec = d.Spec
}
_, err := createOrUpdate[appsv1.Deployment](ctx, kubeClient, cfg.namespace, d, updateF)
return err
},
}
saDeployable = deployable{
kind: "ServiceAccount",
updateObj: func(ctx context.Context, cfg *deployConfig, kubeClient client.Client) error {
sa := new(corev1.ServiceAccount)
if err := yaml.Unmarshal(saYaml, &sa); err != nil {
return fmt.Errorf("error unmarshalling ServiceAccount yaml: %w", err)
}
sa.ObjectMeta.Labels = cfg.labels
sa.ObjectMeta.OwnerReferences = cfg.ownerRefs
sa.ObjectMeta.Namespace = cfg.namespace
_, err := createOrUpdate(ctx, kubeClient, cfg.namespace, sa, func(*corev1.ServiceAccount) {})
return err
},
}
svcDeployable = deployable{
kind: "Service",
updateObj: func(ctx context.Context, cfg *deployConfig, kubeClient client.Client) error {
svc := new(corev1.Service)
if err := yaml.Unmarshal(svcYaml, &svc); err != nil {
return fmt.Errorf("error unmarshalling Service yaml: %w", err)
}
svc.ObjectMeta.Labels = cfg.labels
svc.ObjectMeta.OwnerReferences = cfg.ownerRefs
svc.ObjectMeta.Namespace = cfg.namespace
_, err := createOrUpdate[corev1.Service](ctx, kubeClient, cfg.namespace, svc, func(*corev1.Service) {})
return err
},
}
cmDeployable = deployable{
kind: "ConfigMap",
updateObj: func(ctx context.Context, cfg *deployConfig, kubeClient client.Client) error {
cm := new(corev1.ConfigMap)
if err := yaml.Unmarshal(cmYaml, &cm); err != nil {
return fmt.Errorf("error unmarshalling ConfigMap yaml: %w", err)
}
cm.ObjectMeta.Labels = cfg.labels
cm.ObjectMeta.OwnerReferences = cfg.ownerRefs
cm.ObjectMeta.Namespace = cfg.namespace
_, err := createOrUpdate[corev1.ConfigMap](ctx, kubeClient, cfg.namespace, cm, func(cm *corev1.ConfigMap) {})
return err
},
}
)

@ -0,0 +1,127 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !plan9
// tailscale-operator provides a way to expose services running in a Kubernetes
// cluster to your Tailnet and to make Tailscale nodes available to cluster
// workloads
package main
import (
"encoding/json"
"testing"
"time"
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"sigs.k8s.io/yaml"
operatorutils "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/tstest"
"tailscale.com/util/mak"
)
func TestNameserverReconciler(t *testing.T) {
dnsCfg := &tsapi.DNSConfig{
TypeMeta: metav1.TypeMeta{Kind: "DNSConfig", APIVersion: "tailscale.com/v1alpha1"},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
},
Spec: tsapi.DNSConfigSpec{
Nameserver: &tsapi.Nameserver{
Image: &tsapi.Image{
Repo: "test",
Tag: "v0.0.1",
},
},
},
}
fc := fake.NewClientBuilder().
WithScheme(tsapi.GlobalScheme).
WithObjects(dnsCfg).
WithStatusSubresource(dnsCfg).
Build()
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
cl := tstest.NewClock(tstest.ClockOpts{})
nr := &NameserverReconciler{
Client: fc,
clock: cl,
logger: zl.Sugar(),
tsNamespace: "tailscale",
}
expectReconciled(t, nr, "", "test")
// Verify that nameserver Deployment has been created and has the expected fields.
wantsDeploy := &appsv1.Deployment{ObjectMeta: metav1.ObjectMeta{Name: "nameserver", Namespace: "tailscale"}, TypeMeta: metav1.TypeMeta{Kind: "Deployment", APIVersion: appsv1.SchemeGroupVersion.Identifier()}}
if err := yaml.Unmarshal(deployYaml, wantsDeploy); err != nil {
t.Fatalf("unmarshalling yaml: %v", err)
}
dnsCfgOwnerRef := metav1.NewControllerRef(dnsCfg, tsapi.SchemeGroupVersion.WithKind("DNSConfig"))
wantsDeploy.OwnerReferences = []metav1.OwnerReference{*dnsCfgOwnerRef}
wantsDeploy.Spec.Template.Spec.Containers[0].Image = "test:v0.0.1"
wantsDeploy.Namespace = "tailscale"
labels := nameserverResourceLabels("test", "tailscale")
wantsDeploy.ObjectMeta.Labels = labels
expectEqual(t, fc, wantsDeploy, nil)
// Verify that DNSConfig advertizes the nameserver's Service IP address,
// has the ready status condition and tailscale finalizer.
mustUpdate(t, fc, "tailscale", "nameserver", func(svc *corev1.Service) {
svc.Spec.ClusterIP = "1.2.3.4"
})
expectReconciled(t, nr, "", "test")
dnsCfg.Status.Nameserver = &tsapi.NameserverStatus{
IP: "1.2.3.4",
}
dnsCfg.Finalizers = []string{FinalizerName}
dnsCfg.Status.Conditions = append(dnsCfg.Status.Conditions, metav1.Condition{
Type: string(tsapi.NameserverReady),
Status: metav1.ConditionTrue,
Reason: reasonNameserverCreated,
Message: reasonNameserverCreated,
LastTransitionTime: metav1.Time{Time: cl.Now().Truncate(time.Second)},
})
expectEqual(t, fc, dnsCfg, nil)
// // Verify that nameserver image gets updated to match DNSConfig spec.
mustUpdate(t, fc, "", "test", func(dnsCfg *tsapi.DNSConfig) {
dnsCfg.Spec.Nameserver.Image.Tag = "v0.0.2"
})
expectReconciled(t, nr, "", "test")
wantsDeploy.Spec.Template.Spec.Containers[0].Image = "test:v0.0.2"
expectEqual(t, fc, wantsDeploy, nil)
// Verify that when another actor sets ConfigMap data, it does not get
// overwritten by nameserver reconciler.
dnsRecords := &operatorutils.Records{Version: "v1alpha1", IP4: map[string][]string{"foo.ts.net": {"1.2.3.4"}}}
bs, err := json.Marshal(dnsRecords)
if err != nil {
t.Fatalf("error marshalling ConfigMap contents: %v", err)
}
mustUpdate(t, fc, "tailscale", "dnsrecords", func(cm *corev1.ConfigMap) {
mak.Set(&cm.Data, "records.json", string(bs))
})
expectReconciled(t, nr, "", "test")
wantCm := &corev1.ConfigMap{ObjectMeta: metav1.ObjectMeta{Name: "dnsrecords",
Namespace: "tailscale", Labels: labels, OwnerReferences: []metav1.OwnerReference{*dnsCfgOwnerRef}},
TypeMeta: metav1.TypeMeta{Kind: "ConfigMap", APIVersion: "v1"},
Data: map[string]string{"records.json": string(bs)},
}
expectEqual(t, fc, wantCm, nil)
// Verify that if dnsconfig.spec.nameserver.image.{repo,tag} are unset,
// the nameserver image defaults to tailscale/k8s-nameserver:unstable.
mustUpdate(t, fc, "", "test", func(dnsCfg *tsapi.DNSConfig) {
dnsCfg.Spec.Nameserver.Image = nil
})
expectReconciled(t, nr, "", "test")
wantsDeploy.Spec.Template.Spec.Containers[0].Image = "tailscale/k8s-nameserver:unstable"
expectEqual(t, fc, wantsDeploy, nil)
}

@ -20,6 +20,7 @@ import (
"golang.org/x/oauth2/clientcredentials"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
discoveryv1 "k8s.io/api/discovery/v1"
networkingv1 "k8s.io/api/networking/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/rest"
@ -44,12 +45,12 @@ import (
"tailscale.com/version"
)
// Generate static manifests for deploying Tailscale operator on Kubernetes from the operator's Helm chart.
//go:generate go run tailscale.com/cmd/k8s-operator/generate staticmanifests
// Generate Connector and ProxyClass CustomResourceDefinition yamls from their Go types.
//go:generate go run sigs.k8s.io/controller-tools/cmd/controller-gen crd schemapatch:manifests=./deploy/crds output:dir=./deploy/crds paths=../../k8s-operator/apis/...
// Generate static manifests for deploying Tailscale operator on Kubernetes from the operator's Helm chart.
//go:generate go run tailscale.com/cmd/k8s-operator/generate staticmanifests
// Generate CRD docs from the yamls
//go:generate go run fybrik.io/crdoc --resources=./deploy/crds --output=../../k8s-operator/api.md
@ -65,6 +66,7 @@ func main() {
priorityClassName = defaultEnv("PROXY_PRIORITY_CLASS_NAME", "")
tags = defaultEnv("PROXY_TAGS", "tag:k8s")
tsFirewallMode = defaultEnv("PROXY_FIREWALL_MODE", "")
isDefaultLoadBalancer = defaultBool("OPERATOR_DEFAULT_LOAD_BALANCER", false)
)
var opts []kzap.Opts
@ -93,9 +95,19 @@ func main() {
defer s.Close()
restConfig := config.GetConfigOrDie()
maybeLaunchAPIServerProxy(zlog, restConfig, s, mode)
// TODO (irbekrm): gather the reconciler options into an opts struct
// rather than passing a million of them in one by one.
runReconcilers(zlog, s, tsNamespace, restConfig, tsClient, image, priorityClassName, tags, tsFirewallMode)
rOpts := reconcilerOpts{
log: zlog,
tsServer: s,
tsClient: tsClient,
tailscaleNamespace: tsNamespace,
restConfig: restConfig,
proxyImage: image,
proxyPriorityClassName: priorityClassName,
proxyActAsDefaultLoadBalancer: isDefaultLoadBalancer,
proxyTags: tags,
proxyFirewallMode: tsFirewallMode,
}
runReconcilers(rOpts)
}
// initTSNet initializes the tsnet.Server and logs in to Tailscale. It uses the
@ -203,11 +215,8 @@ waitOnline:
// runReconcilers starts the controller-runtime manager and registers the
// ServiceReconciler. It blocks forever.
func runReconcilers(zlog *zap.SugaredLogger, s *tsnet.Server, tsNamespace string, restConfig *rest.Config, tsClient *tailscale.Client, image, priorityClassName, tags, tsFirewallMode string) {
var (
isDefaultLoadBalancer = defaultBool("OPERATOR_DEFAULT_LOAD_BALANCER", false)
)
startlog := zlog.Named("startReconcilers")
func runReconcilers(opts reconcilerOpts) {
startlog := opts.log.Named("startReconcilers")
// For secrets and statefulsets, we only get permission to touch the objects
// in the controller's own namespace. This cannot be expressed by
// .Watches(...) below, instead you have to add a per-type field selector to
@ -215,7 +224,7 @@ func runReconcilers(zlog *zap.SugaredLogger, s *tsnet.Server, tsNamespace string
// implicitly filter what parts of the world the builder code gets to see at
// all.
nsFilter := cache.ByObject{
Field: client.InNamespace(tsNamespace).AsSelector(),
Field: client.InNamespace(opts.tailscaleNamespace).AsSelector(),
}
mgrOpts := manager.Options{
// TODO (irbekrm): stricter filtering what we watch/cache/call
@ -224,32 +233,36 @@ func runReconcilers(zlog *zap.SugaredLogger, s *tsnet.Server, tsNamespace string
Cache: cache.Options{
ByObject: map[client.Object]cache.ByObject{
&corev1.Secret{}: nsFilter,
&corev1.ServiceAccount{}: nsFilter,
&corev1.ConfigMap{}: nsFilter,
&appsv1.StatefulSet{}: nsFilter,
&appsv1.Deployment{}: nsFilter,
&discoveryv1.EndpointSlice{}: nsFilter,
},
},
Scheme: tsapi.GlobalScheme,
}
mgr, err := manager.New(restConfig, mgrOpts)
mgr, err := manager.New(opts.restConfig, mgrOpts)
if err != nil {
startlog.Fatalf("could not create manager: %v", err)
}
svcFilter := handler.EnqueueRequestsFromMapFunc(serviceHandler)
svcChildFilter := handler.EnqueueRequestsFromMapFunc(managedResourceHandlerForType("svc"))
// If a ProxyClassChanges, enqueue all Services labeled with that
// If a ProxyClass changes, enqueue all Services labeled with that
// ProxyClass's name.
proxyClassFilterForSvc := handler.EnqueueRequestsFromMapFunc(proxyClassHandlerForSvc(mgr.GetClient(), startlog))
eventRecorder := mgr.GetEventRecorderFor("tailscale-operator")
ssr := &tailscaleSTSReconciler{
Client: mgr.GetClient(),
tsnetServer: s,
tsClient: tsClient,
defaultTags: strings.Split(tags, ","),
operatorNamespace: tsNamespace,
proxyImage: image,
proxyPriorityClassName: priorityClassName,
tsFirewallMode: tsFirewallMode,
tsnetServer: opts.tsServer,
tsClient: opts.tsClient,
defaultTags: strings.Split(opts.proxyTags, ","),
operatorNamespace: opts.tailscaleNamespace,
proxyImage: opts.proxyImage,
proxyPriorityClassName: opts.proxyPriorityClassName,
tsFirewallMode: opts.proxyFirewallMode,
}
err = builder.
ControllerManagedBy(mgr).
@ -261,9 +274,11 @@ func runReconcilers(zlog *zap.SugaredLogger, s *tsnet.Server, tsNamespace string
Complete(&ServiceReconciler{
ssr: ssr,
Client: mgr.GetClient(),
logger: zlog.Named("service-reconciler"),
isDefaultLoadBalancer: isDefaultLoadBalancer,
logger: opts.log.Named("service-reconciler"),
isDefaultLoadBalancer: opts.proxyActAsDefaultLoadBalancer,
recorder: eventRecorder,
tsNamespace: opts.tailscaleNamespace,
clock: tstime.DefaultClock{},
})
if err != nil {
startlog.Fatalf("could not create service reconciler: %v", err)
@ -285,7 +300,7 @@ func runReconcilers(zlog *zap.SugaredLogger, s *tsnet.Server, tsNamespace string
ssr: ssr,
recorder: eventRecorder,
Client: mgr.GetClient(),
logger: zlog.Named("ingress-reconciler"),
logger: opts.log.Named("ingress-reconciler"),
})
if err != nil {
startlog.Fatalf("could not create ingress reconciler: %v", err)
@ -304,29 +319,201 @@ func runReconcilers(zlog *zap.SugaredLogger, s *tsnet.Server, tsNamespace string
ssr: ssr,
recorder: eventRecorder,
Client: mgr.GetClient(),
logger: zlog.Named("connector-reconciler"),
logger: opts.log.Named("connector-reconciler"),
clock: tstime.DefaultClock{},
})
if err != nil {
startlog.Fatal("could not create connector reconciler: %v", err)
startlog.Fatalf("could not create connector reconciler: %v", err)
}
// TODO (irbekrm): switch to metadata-only watches for resources whose
// spec we don't need to inspect to reduce memory consumption.
// https://github.com/kubernetes-sigs/controller-runtime/issues/1159
nameserverFilter := handler.EnqueueRequestsFromMapFunc(managedResourceHandlerForType("nameserver"))
err = builder.ControllerManagedBy(mgr).
For(&tsapi.DNSConfig{}).
Watches(&appsv1.Deployment{}, nameserverFilter).
Watches(&corev1.ConfigMap{}, nameserverFilter).
Watches(&corev1.Service{}, nameserverFilter).
Watches(&corev1.ServiceAccount{}, nameserverFilter).
Complete(&NameserverReconciler{
recorder: eventRecorder,
tsNamespace: opts.tailscaleNamespace,
Client: mgr.GetClient(),
logger: opts.log.Named("nameserver-reconciler"),
clock: tstime.DefaultClock{},
})
if err != nil {
startlog.Fatalf("could not create nameserver reconciler: %v", err)
}
err = builder.ControllerManagedBy(mgr).
For(&tsapi.ProxyClass{}).
Complete(&ProxyClassReconciler{
Client: mgr.GetClient(),
recorder: eventRecorder,
logger: zlog.Named("proxyclass-reconciler"),
logger: opts.log.Named("proxyclass-reconciler"),
clock: tstime.DefaultClock{},
})
if err != nil {
startlog.Fatal("could not create proxyclass reconciler: %v", err)
}
logger := startlog.Named("dns-records-reconciler-event-handlers")
// On EndpointSlice events, if it is an EndpointSlice for an
// ingress/egress proxy headless Service, reconcile the headless
// Service.
dnsRREpsOpts := handler.EnqueueRequestsFromMapFunc(dnsRecordsReconcilerEndpointSliceHandler)
// On DNSConfig changes, reconcile all headless Services for
// ingress/egress proxies in operator namespace.
dnsRRDNSConfigOpts := handler.EnqueueRequestsFromMapFunc(enqueueAllIngressEgressProxySvcsInNS(opts.tailscaleNamespace, mgr.GetClient(), logger))
// On Service events, if it is an ingress/egress proxy headless Service, reconcile it.
dnsRRServiceOpts := handler.EnqueueRequestsFromMapFunc(dnsRecordsReconcilerServiceHandler)
// On Ingress events, if it is a tailscale Ingress or if tailscale is the default ingress controller, reconcile the proxy
// headless Service.
dnsRRIngressOpts := handler.EnqueueRequestsFromMapFunc(dnsRecordsReconcilerIngressHandler(opts.tailscaleNamespace, opts.proxyActAsDefaultLoadBalancer, mgr.GetClient(), logger))
err = builder.ControllerManagedBy(mgr).
Named("dns-records-reconciler").
Watches(&corev1.Service{}, dnsRRServiceOpts).
Watches(&networkingv1.Ingress{}, dnsRRIngressOpts).
Watches(&discoveryv1.EndpointSlice{}, dnsRREpsOpts).
Watches(&tsapi.DNSConfig{}, dnsRRDNSConfigOpts).
Complete(&dnsRecordsReconciler{
Client: mgr.GetClient(),
tsNamespace: opts.tailscaleNamespace,
logger: opts.log.Named("dns-records-reconciler"),
isDefaultLoadBalancer: opts.proxyActAsDefaultLoadBalancer,
})
if err != nil {
startlog.Fatalf("could not create DNS records reconciler: %v", err)
}
startlog.Infof("Startup complete, operator running, version: %s", version.Long())
if err := mgr.Start(signals.SetupSignalHandler()); err != nil {
startlog.Fatalf("could not start manager: %v", err)
}
}
type reconcilerOpts struct {
log *zap.SugaredLogger
tsServer *tsnet.Server
tsClient *tailscale.Client
tailscaleNamespace string // namespace in which operator resources will be deployed
restConfig *rest.Config // config for connecting to the kube API server
proxyImage string // <proxy-image-repo>:<proxy-image-tag>
// proxyPriorityClassName isPriorityClass to be set for proxy Pods. This
// is a legacy mechanism for cluster resource configuration options -
// going forward use ProxyClass.
// https://kubernetes.io/docs/concepts/scheduling-eviction/pod-priority-preemption/#priorityclass
proxyPriorityClassName string
// proxyTags are ACL tags to tag proxy auth keys. Multiple tags should
// be provided as a string with comma-separated tag values. Proxy tags
// default to tag:k8s.
// https://tailscale.com/kb/1085/auth-keys
proxyTags string
// proxyActAsDefaultLoadBalancer determines whether this operator
// instance should act as the default ingress controller when looking at
// Ingress resources with unset ingress.spec.ingressClassName.
// TODO (irbekrm): this setting does not respect the default
// IngressClass.
// https://kubernetes.io/docs/concepts/services-networking/ingress/#default-ingress-class
// We should fix that and preferably integrate with that mechanism as
// well - perhaps make the operator itself create the default
// IngressClass if this is set to true.
proxyActAsDefaultLoadBalancer bool
// proxyFirewallMode determines whether non-userspace proxies should use
// iptables or nftables for firewall configuration. Accepted values are
// iptables, nftables and auto. If set to auto, proxy will automatically
// determine which mode is supported for a given host (prefer nftables).
// Auto is usually the best choice, unless you want to explicitly set
// specific mode for debugging purposes.
proxyFirewallMode string
}
// enqueueAllIngressEgressProxySvcsinNS returns a reconcile request for each
// ingress/egress proxy headless Service found in the provided namespace.
func enqueueAllIngressEgressProxySvcsInNS(ns string, cl client.Client, logger *zap.SugaredLogger) handler.MapFunc {
return func(ctx context.Context, _ client.Object) []reconcile.Request {
reqs := make([]reconcile.Request, 0)
// Get all headless Services for proxies configured using Service.
svcProxyLabels := map[string]string{
LabelManaged: "true",
LabelParentType: "svc",
}
svcHeadlessSvcList := &corev1.ServiceList{}
if err := cl.List(ctx, svcHeadlessSvcList, client.InNamespace(ns), client.MatchingLabels(svcProxyLabels)); err != nil {
logger.Errorf("error listing headless Services for tailscale ingress/egress Services in operator namespace: %v", err)
return nil
}
for _, svc := range svcHeadlessSvcList.Items {
reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{Namespace: svc.Namespace, Name: svc.Name}})
}
// Get all headless Services for proxies configured using Ingress.
ingProxyLabels := map[string]string{
LabelManaged: "true",
LabelParentType: "ingress",
}
ingHeadlessSvcList := &corev1.ServiceList{}
if err := cl.List(ctx, ingHeadlessSvcList, client.InNamespace(ns), client.MatchingLabels(ingProxyLabels)); err != nil {
logger.Errorf("error listing headless Services for tailscale Ingresses in operator namespace: %v", err)
return nil
}
for _, svc := range ingHeadlessSvcList.Items {
reqs = append(reqs, reconcile.Request{NamespacedName: types.NamespacedName{Namespace: svc.Namespace, Name: svc.Name}})
}
return reqs
}
}
// dnsRecordsReconciler filters EndpointSlice events for which
// dns-records-reconciler should reconcile a headless Service. The only events
// it should reconcile are those for EndpointSlices associated with proxy
// headless Services.
func dnsRecordsReconcilerEndpointSliceHandler(ctx context.Context, o client.Object) []reconcile.Request {
if !isManagedByType(o, "svc") && !isManagedByType(o, "ingress") {
return nil
}
headlessSvcName, ok := o.GetLabels()[discoveryv1.LabelServiceName] // https://kubernetes.io/docs/concepts/services-networking/endpoint-slices/#ownership
if !ok {
return nil
}
return []reconcile.Request{{NamespacedName: types.NamespacedName{Namespace: o.GetNamespace(), Name: headlessSvcName}}}
}
// dnsRecordsReconcilerServiceHandler filters Service events for which
// dns-records-reconciler should reconcile. If the event is for a cluster
// ingress/cluster egress proxy's headless Service, returns the Service for
// reconcile.
func dnsRecordsReconcilerServiceHandler(ctx context.Context, o client.Object) []reconcile.Request {
if isManagedByType(o, "svc") || isManagedByType(o, "ingress") {
return []reconcile.Request{{NamespacedName: types.NamespacedName{Namespace: o.GetNamespace(), Name: o.GetName()}}}
}
return nil
}
// dnsRecordsReconcilerIngressHandler filters Ingress events to ensure that
// dns-records-reconciler only reconciles on tailscale Ingress events. When an
// event is observed on a tailscale Ingress, reconcile the proxy headless Service.
func dnsRecordsReconcilerIngressHandler(ns string, isDefaultLoadBalancer bool, cl client.Client, logger *zap.SugaredLogger) handler.MapFunc {
return func(ctx context.Context, o client.Object) []reconcile.Request {
ing, ok := o.(*networkingv1.Ingress)
if !ok {
return nil
}
if !isDefaultLoadBalancer && (ing.Spec.IngressClassName == nil || *ing.Spec.IngressClassName != "tailscale") {
return nil
}
proxyResourceLabels := childResourceLabels(ing.Name, ing.Namespace, "ingress")
headlessSvc, err := getSingleObject[corev1.Service](ctx, cl, ns, proxyResourceLabels)
if err != nil {
logger.Errorf("error getting headless Service from parent labels: %v", err)
return nil
}
if headlessSvc == nil {
return nil
}
return []reconcile.Request{{NamespacedName: types.NamespacedName{Namespace: headlessSvc.Namespace, Name: headlessSvc.Name}}}
}
}
type tsClient interface {
CreateKey(ctx context.Context, caps tailscale.KeyCapabilities) (string, *tailscale.Key, error)
DeleteDevice(ctx context.Context, nodeStableID string) error

@ -9,6 +9,7 @@ import (
"context"
"fmt"
"testing"
"time"
"github.com/google/go-cmp/cmp"
"go.uber.org/zap"
@ -17,10 +18,15 @@ import (
networkingv1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/tools/record"
"sigs.k8s.io/controller-runtime/pkg/client/fake"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/net/dns/resolvconffile"
"tailscale.com/tstest"
"tailscale.com/tstime"
"tailscale.com/types/ptr"
"tailscale.com/util/dnsname"
"tailscale.com/util/mak"
)
@ -31,6 +37,7 @@ func TestLoadBalancerClass(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -41,10 +48,12 @@ func TestLoadBalancerClass(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
recorder: record.NewFakeRecorder(100),
}
// Create a service that we should manage, and check that the initial round
// of objects looks right.
// Create a service that we should manage, but start with a miconfiguration
// in the annotations.
mustCreate(t, fc, &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "test",
@ -53,6 +62,9 @@ func TestLoadBalancerClass(t *testing.T) {
// doesn't. So, set it explicitly because other code later depends
// on it being set.
UID: types.UID("1234-UID"),
Annotations: map[string]string{
AnnotationTailnetTargetFQDN: "invalid.example.com",
},
},
Spec: corev1.ServiceSpec{
ClusterIP: "10.20.30.40",
@ -63,6 +75,46 @@ func TestLoadBalancerClass(t *testing.T) {
expectReconciled(t, sr, "default", "test")
// The expected value of .status.conditions[0].LastTransitionTime until the
// proxy becomes ready.
t0 := conditionTime(clock)
// Should have an error about invalid config.
want := &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
UID: types.UID("1234-UID"),
Annotations: map[string]string{
AnnotationTailnetTargetFQDN: "invalid.example.com",
},
},
Spec: corev1.ServiceSpec{
ClusterIP: "10.20.30.40",
Type: corev1.ServiceTypeLoadBalancer,
LoadBalancerClass: ptr.To("tailscale"),
},
Status: corev1.ServiceStatus{
Conditions: []metav1.Condition{{
Type: string(tsapi.ProxyReady),
Status: metav1.ConditionFalse,
LastTransitionTime: t0,
Reason: reasonProxyInvalid,
Message: `unable to provision proxy resources: invalid Service: invalid value of annotation tailscale.com/tailnet-fqdn: "invalid.example.com" does not appear to be a valid MagicDNS name`,
}},
},
}
expectEqual(t, fc, want, nil)
// Delete the misconfiguration so the proxy starts getting created on the
// next reconcile.
mustUpdate(t, fc, "default", "test", func(s *corev1.Service) {
s.ObjectMeta.Annotations = nil
})
clock.Advance(time.Second)
expectReconciled(t, sr, "default", "test")
fullName, shortName := findGenName(t, fc, "default", "test", "svc")
opts := configOpts{
stsName: shortName,
@ -73,10 +125,23 @@ func TestLoadBalancerClass(t *testing.T) {
clusterTargetIP: "10.20.30.40",
}
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
want.Annotations = nil
want.ObjectMeta.Finalizers = []string{"tailscale.com/finalizer"}
want.Status = corev1.ServiceStatus{
Conditions: []metav1.Condition{{
Type: string(tsapi.ProxyReady),
Status: metav1.ConditionFalse,
LastTransitionTime: t0, // Status is still false, no update to transition time
Reason: reasonProxyPending,
Message: "no Tailscale hostname known yet, waiting for proxy pod to finish auth",
}},
}
expectEqual(t, fc, want, nil)
// Normally the Tailscale proxy pod would come up here and write its info
// into the secret. Simulate that, then verify reconcile again and verify
// that we get to the end.
@ -88,25 +153,10 @@ func TestLoadBalancerClass(t *testing.T) {
s.Data["device_fqdn"] = []byte("tailscale.device.name.")
s.Data["device_ips"] = []byte(`["100.99.98.97", "2c0a:8083:94d4:2012:3165:34a5:3616:5fdf"]`)
})
clock.Advance(time.Second)
expectReconciled(t, sr, "default", "test")
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
Finalizers: []string{"tailscale.com/finalizer"},
UID: types.UID("1234-UID"),
},
Spec: corev1.ServiceSpec{
ClusterIP: "10.20.30.40",
Type: corev1.ServiceTypeLoadBalancer,
LoadBalancerClass: ptr.To("tailscale"),
},
Status: corev1.ServiceStatus{
LoadBalancer: corev1.LoadBalancerStatus{
want.Status.Conditions = proxyCreatedCondition(clock)
want.Status.LoadBalancer = corev1.LoadBalancerStatus{
Ingress: []corev1.LoadBalancerIngress{
{
Hostname: "tailscale.device.name",
@ -115,8 +165,6 @@ func TestLoadBalancerClass(t *testing.T) {
IP: "100.99.98.97",
},
},
},
},
}
expectEqual(t, fc, want, nil)
@ -142,11 +190,9 @@ func TestLoadBalancerClass(t *testing.T) {
expectMissing[appsv1.StatefulSet](t, fc, "operator-ns", shortName)
expectMissing[corev1.Service](t, fc, "operator-ns", shortName)
expectMissing[corev1.Secret](t, fc, "operator-ns", fullName)
// Note that the Tailscale-specific condition status should be gone now.
want = &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -168,6 +214,7 @@ func TestTailnetTargetFQDNAnnotation(t *testing.T) {
t.Fatal(err)
}
tailnetTargetFQDN := "foo.bar.ts.net."
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -178,6 +225,7 @@ func TestTailnetTargetFQDNAnnotation(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -214,14 +262,10 @@ func TestTailnetTargetFQDNAnnotation(t *testing.T) {
hostname: "default-test",
}
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -236,9 +280,12 @@ func TestTailnetTargetFQDNAnnotation(t *testing.T) {
Type: corev1.ServiceTypeExternalName,
Selector: nil,
},
Status: corev1.ServiceStatus{
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
@ -278,6 +325,7 @@ func TestTailnetTargetIPAnnotation(t *testing.T) {
t.Fatal(err)
}
tailnetTargetIP := "100.66.66.66"
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -288,6 +336,7 @@ func TestTailnetTargetIPAnnotation(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -324,14 +373,10 @@ func TestTailnetTargetIPAnnotation(t *testing.T) {
hostname: "default-test",
}
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -346,9 +391,12 @@ func TestTailnetTargetIPAnnotation(t *testing.T) {
Type: corev1.ServiceTypeExternalName,
Selector: nil,
},
Status: corev1.ServiceStatus{
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
@ -387,6 +435,7 @@ func TestAnnotations(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -397,6 +446,7 @@ func TestAnnotations(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -431,14 +481,10 @@ func TestAnnotations(t *testing.T) {
clusterTargetIP: "10.20.30.40",
}
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -452,6 +498,9 @@ func TestAnnotations(t *testing.T) {
ClusterIP: "10.20.30.40",
Type: corev1.ServiceTypeClusterIP,
},
Status: corev1.ServiceStatus{
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
@ -471,10 +520,6 @@ func TestAnnotations(t *testing.T) {
expectMissing[corev1.Service](t, fc, "operator-ns", shortName)
expectMissing[corev1.Secret](t, fc, "operator-ns", fullName)
want = &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -495,6 +540,7 @@ func TestAnnotationIntoLB(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -505,6 +551,7 @@ func TestAnnotationIntoLB(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -539,7 +586,7 @@ func TestAnnotationIntoLB(t *testing.T) {
clusterTargetIP: "10.20.30.40",
}
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
@ -556,10 +603,6 @@ func TestAnnotationIntoLB(t *testing.T) {
})
expectReconciled(t, sr, "default", "test")
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -573,6 +616,9 @@ func TestAnnotationIntoLB(t *testing.T) {
ClusterIP: "10.20.30.40",
Type: corev1.ServiceTypeClusterIP,
},
Status: corev1.ServiceStatus{
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
@ -590,10 +636,6 @@ func TestAnnotationIntoLB(t *testing.T) {
// ... but the service should have a LoadBalancer status.
want = &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -616,6 +658,7 @@ func TestAnnotationIntoLB(t *testing.T) {
},
},
},
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
@ -628,6 +671,7 @@ func TestLBIntoAnnotation(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -638,6 +682,7 @@ func TestLBIntoAnnotation(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -670,7 +715,7 @@ func TestLBIntoAnnotation(t *testing.T) {
clusterTargetIP: "10.20.30.40",
}
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
@ -687,10 +732,6 @@ func TestLBIntoAnnotation(t *testing.T) {
})
expectReconciled(t, sr, "default", "test")
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -713,6 +754,7 @@ func TestLBIntoAnnotation(t *testing.T) {
},
},
},
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
@ -738,10 +780,6 @@ func TestLBIntoAnnotation(t *testing.T) {
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
want = &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -755,6 +793,9 @@ func TestLBIntoAnnotation(t *testing.T) {
ClusterIP: "10.20.30.40",
Type: corev1.ServiceTypeClusterIP,
},
Status: corev1.ServiceStatus{
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
}
@ -766,6 +807,7 @@ func TestCustomHostname(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -776,6 +818,7 @@ func TestCustomHostname(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -811,14 +854,10 @@ func TestCustomHostname(t *testing.T) {
clusterTargetIP: "10.20.30.40",
}
expectEqual(t, fc, expectedSecret(t, o), nil)
expectEqual(t, fc, expectedSecret(t, fc, o), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, o), removeHashAnnotation)
want := &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -833,6 +872,9 @@ func TestCustomHostname(t *testing.T) {
ClusterIP: "10.20.30.40",
Type: corev1.ServiceTypeClusterIP,
},
Status: corev1.ServiceStatus{
Conditions: proxyCreatedCondition(clock),
},
}
expectEqual(t, fc, want, nil)
@ -852,10 +894,6 @@ func TestCustomHostname(t *testing.T) {
expectMissing[corev1.Service](t, fc, "operator-ns", shortName)
expectMissing[corev1.Secret](t, fc, "operator-ns", fullName)
want = &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
@ -879,6 +917,7 @@ func TestCustomPriorityClassName(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -890,6 +929,7 @@ func TestCustomPriorityClassName(t *testing.T) {
proxyPriorityClassName: "custom-priority-class-name",
},
logger: zl.Sugar(),
clock: clock,
}
// Create a service that we should manage, and check that the initial round
@ -933,7 +973,11 @@ func TestProxyClassForService(t *testing.T) {
// Setup
pc := &tsapi.ProxyClass{
ObjectMeta: metav1.ObjectMeta{Name: "custom-metadata"},
Spec: tsapi.ProxyClassSpec{StatefulSet: &tsapi.StatefulSet{
Spec: tsapi.ProxyClassSpec{
TailscaleConfig: &tsapi.TailscaleConfig{
AcceptRoutes: true,
},
StatefulSet: &tsapi.StatefulSet{
Labels: map[string]string{"foo": "bar"},
Annotations: map[string]string{"bar.io/foo": "some-val"},
Pod: &tsapi.Pod{Annotations: map[string]string{"foo.io/bar": "some-val"}}}},
@ -948,6 +992,7 @@ func TestProxyClassForService(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -958,6 +1003,7 @@ func TestProxyClassForService(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// 1. A new tailscale LoadBalancer Service is created without any
@ -987,7 +1033,7 @@ func TestProxyClassForService(t *testing.T) {
hostname: "default-test",
clusterTargetIP: "10.20.30.40",
}
expectEqual(t, fc, expectedSecret(t, opts), nil)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
@ -999,21 +1045,23 @@ func TestProxyClassForService(t *testing.T) {
})
expectReconciled(t, sr, "default", "test")
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
// 3. ProxyClass is set to Ready, the Service gets reconciled by the
// services-reconciler and the customization from the ProxyClass is
// applied to the proxy resources.
mustUpdateStatus(t, fc, "", "custom-metadata", func(pc *tsapi.ProxyClass) {
pc.Status = tsapi.ProxyClassStatus{
Conditions: []tsapi.ConnectorCondition{{
Conditions: []metav1.Condition{{
Status: metav1.ConditionTrue,
Type: tsapi.ProxyClassready,
Type: string(tsapi.ProxyClassready),
ObservedGeneration: pc.Generation,
}}}
})
opts.proxyClass = pc.Name
expectReconciled(t, sr, "default", "test")
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
expectEqual(t, fc, expectedSecret(t, fc, opts), removeAuthKeyIfExistsModifier(t))
// 4. tailscale.com/proxy-class label is removed from the Service, the
// configuration from the ProxyClass is removed from the cluster
@ -1033,6 +1081,7 @@ func TestDefaultLoadBalancer(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -1043,6 +1092,7 @@ func TestDefaultLoadBalancer(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
isDefaultLoadBalancer: true,
}
@ -1087,6 +1137,7 @@ func TestProxyFirewallMode(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -1098,6 +1149,7 @@ func TestProxyFirewallMode(t *testing.T) {
tsFirewallMode: "nftables",
},
logger: zl.Sugar(),
clock: clock,
isDefaultLoadBalancer: true,
}
@ -1140,6 +1192,7 @@ func TestTailscaledConfigfileHash(t *testing.T) {
if err != nil {
t.Fatal(err)
}
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
@ -1150,6 +1203,7 @@ func TestTailscaledConfigfileHash(t *testing.T) {
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
isDefaultLoadBalancer: true,
}
@ -1180,7 +1234,7 @@ func TestTailscaledConfigfileHash(t *testing.T) {
parentType: "svc",
hostname: "default-test",
clusterTargetIP: "10.20.30.40",
confFileHash: "705e5ffd0bd5326237efdf542c850a65a54101284d5daa30775420fcc64d89c1",
confFileHash: "e09bededa0379920141cbd0b0dbdf9b8b66545877f9e8397423f5ce3e1ba439e",
}
expectEqual(t, fc, expectedSTS(t, fc, o), nil)
@ -1190,11 +1244,10 @@ func TestTailscaledConfigfileHash(t *testing.T) {
mak.Set(&svc.Annotations, AnnotationHostname, "another-test")
})
o.hostname = "another-test"
o.confFileHash = "1a087f887825d2b75d3673c7c2b0131f8ec1f0b1cb761d33e236dd28350dfe23"
o.confFileHash = "5d754cf55463135ee34aa9821f2fd8483b53eb0570c3740c84a086304f427684"
expectReconciled(t, sr, "default", "test")
expectEqual(t, fc, expectedSTS(t, fc, o), nil)
}
func Test_isMagicDNSName(t *testing.T) {
tests := []struct {
in string
@ -1352,3 +1405,165 @@ func Test_serviceHandlerForIngress(t *testing.T) {
t.Errorf("unexpected reconcile request for a Service that does not belong to any Ingress: %#+v\n", gotReqs)
}
}
func Test_clusterDomainFromResolverConf(t *testing.T) {
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
tests := []struct {
name string
conf *resolvconffile.Config
namespace string
want string
}{
{
name: "success- custom domain",
conf: &resolvconffile.Config{
SearchDomains: []dnsname.FQDN{toFQDN(t, "foo.svc.department.org.io"), toFQDN(t, "svc.department.org.io"), toFQDN(t, "department.org.io")},
},
namespace: "foo",
want: "department.org.io",
},
{
name: "success- default domain",
conf: &resolvconffile.Config{
SearchDomains: []dnsname.FQDN{toFQDN(t, "foo.svc.cluster.local."), toFQDN(t, "svc.cluster.local."), toFQDN(t, "cluster.local.")},
},
namespace: "foo",
want: "cluster.local",
},
{
name: "only two search domains found",
conf: &resolvconffile.Config{
SearchDomains: []dnsname.FQDN{toFQDN(t, "svc.department.org.io"), toFQDN(t, "department.org.io")},
},
namespace: "foo",
want: "cluster.local",
},
{
name: "first search domain does not match the expected structure",
conf: &resolvconffile.Config{
SearchDomains: []dnsname.FQDN{toFQDN(t, "foo.bar.department.org.io"), toFQDN(t, "svc.department.org.io"), toFQDN(t, "some.other.fqdn")},
},
namespace: "foo",
want: "cluster.local",
},
{
name: "second search domain does not match the expected structure",
conf: &resolvconffile.Config{
SearchDomains: []dnsname.FQDN{toFQDN(t, "foo.svc.department.org.io"), toFQDN(t, "foo.department.org.io"), toFQDN(t, "some.other.fqdn")},
},
namespace: "foo",
want: "cluster.local",
},
{
name: "third search domain does not match the expected structure",
conf: &resolvconffile.Config{
SearchDomains: []dnsname.FQDN{toFQDN(t, "foo.svc.department.org.io"), toFQDN(t, "svc.department.org.io"), toFQDN(t, "some.other.fqdn")},
},
namespace: "foo",
want: "cluster.local",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := clusterDomainFromResolverConf(tt.conf, tt.namespace, zl.Sugar()); got != tt.want {
t.Errorf("clusterDomainFromResolverConf() = %v, want %v", got, tt.want)
}
})
}
}
func Test_externalNameService(t *testing.T) {
fc := fake.NewFakeClient()
ft := &fakeTSClient{}
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
// 1. A External name Service that should be exposed via Tailscale gets
// created.
clock := tstest.NewClock(tstest.ClockOpts{})
sr := &ServiceReconciler{
Client: fc,
ssr: &tailscaleSTSReconciler{
Client: fc,
tsClient: ft,
defaultTags: []string{"tag:k8s"},
operatorNamespace: "operator-ns",
proxyImage: "tailscale/tailscale",
},
logger: zl.Sugar(),
clock: clock,
}
// 1. Create an ExternalName Service that we should manage, and check that the initial round
// of objects looks right.
mustCreate(t, fc, &corev1.Service{
ObjectMeta: metav1.ObjectMeta{
Name: "test",
Namespace: "default",
// The apiserver is supposed to set the UID, but the fake client
// doesn't. So, set it explicitly because other code later depends
// on it being set.
UID: types.UID("1234-UID"),
Annotations: map[string]string{
AnnotationExpose: "true",
},
},
Spec: corev1.ServiceSpec{
Type: corev1.ServiceTypeExternalName,
ExternalName: "foo.com",
},
})
expectReconciled(t, sr, "default", "test")
fullName, shortName := findGenName(t, fc, "default", "test", "svc")
opts := configOpts{
stsName: shortName,
secretName: fullName,
namespace: "default",
parentType: "svc",
hostname: "default-test",
clusterTargetDNS: "foo.com",
}
expectEqual(t, fc, expectedSecret(t, fc, opts), nil)
expectEqual(t, fc, expectedHeadlessService(shortName, "svc"), nil)
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
// 2. Change the ExternalName and verify that changes get propagated.
mustUpdate(t, sr, "default", "test", func(s *corev1.Service) {
s.Spec.ExternalName = "bar.com"
})
expectReconciled(t, sr, "default", "test")
opts.clusterTargetDNS = "bar.com"
expectEqual(t, fc, expectedSTS(t, fc, opts), removeHashAnnotation)
}
func toFQDN(t *testing.T, s string) dnsname.FQDN {
t.Helper()
fqdn, err := dnsname.ToFQDN(s)
if err != nil {
t.Fatalf("error coverting %q to dnsname.FQDN: %v", s, err)
}
return fqdn
}
func proxyCreatedCondition(clock tstime.Clock) []metav1.Condition {
return []metav1.Condition{{
Type: string(tsapi.ProxyReady),
Status: metav1.ConditionTrue,
ObservedGeneration: 0,
LastTransitionTime: conditionTime(clock),
Reason: reasonProxyCreated,
Message: reasonProxyCreated,
}}
}
func conditionTime(clock tstime.Clock) metav1.Time {
return metav1.NewTime(clock.Now().Truncate(time.Second))
}

@ -20,6 +20,7 @@ import (
"k8s.io/client-go/transport"
"tailscale.com/client/tailscale"
"tailscale.com/client/tailscale/apitype"
tskube "tailscale.com/kube"
"tailscale.com/tailcfg"
"tailscale.com/tsnet"
"tailscale.com/util/clientmetric"
@ -207,32 +208,24 @@ func runAPIServerProxy(s *tsnet.Server, rt http.RoundTripper, log *zap.SugaredLo
}
const (
capabilityName = "tailscale.com/cap/kubernetes"
oldCapabilityName = "https://" + capabilityName
// oldCapabilityName is a legacy form of
// tailfcg.PeerCapabilityKubernetes capability. The only capability rule
// that is respected for this form is group impersonation - for
// backwards compatibility reasons.
// TODO (irbekrm): determine if anyone uses this and remove if possible.
oldCapabilityName = "https://" + tailcfg.PeerCapabilityKubernetes
)
type capRule struct {
// Impersonate is a list of rules that specify how to impersonate the caller
// when proxying to the Kubernetes API.
Impersonate *impersonateRule `json:"impersonate,omitempty"`
}
// TODO(maisem): move this to some well-known location so that it can be shared
// with control.
type impersonateRule struct {
Groups []string `json:"groups,omitempty"`
}
// addImpersonationHeaders adds the appropriate headers to r to impersonate the
// caller when proxying to the Kubernetes API. It uses the WhoIsResponse stashed
// in the context by the apiserverProxy.
func addImpersonationHeaders(r *http.Request, log *zap.SugaredLogger) error {
log = log.With("remote", r.RemoteAddr)
who := whoIsKey.Value(r.Context())
rules, err := tailcfg.UnmarshalCapJSON[capRule](who.CapMap, capabilityName)
rules, err := tailcfg.UnmarshalCapJSON[tskube.KubernetesCapRule](who.CapMap, tailcfg.PeerCapabilityKubernetes)
if len(rules) == 0 && err == nil {
// Try the old capability name for backwards compatibility.
rules, err = tailcfg.UnmarshalCapJSON[capRule](who.CapMap, oldCapabilityName)
rules, err = tailcfg.UnmarshalCapJSON[tskube.KubernetesCapRule](who.CapMap, oldCapabilityName)
}
if err != nil {
return fmt.Errorf("failed to unmarshal capability: %v", err)

@ -49,7 +49,7 @@ func TestImpersonationHeaders(t *testing.T) {
name: "user-with-cap",
emailish: "foo@example.com",
capMap: tailcfg.PeerCapMap{
capabilityName: {
tailcfg.PeerCapabilityKubernetes: {
tailcfg.RawMessage(`{"impersonate":{"groups":["group1","group2"]}}`),
tailcfg.RawMessage(`{"impersonate":{"groups":["group1","group3"]}}`), // One group is duplicated.
tailcfg.RawMessage(`{"impersonate":{"groups":["group4"]}}`),
@ -71,7 +71,7 @@ func TestImpersonationHeaders(t *testing.T) {
emailish: "tagged-device",
tags: []string{"tag:foo", "tag:bar"},
capMap: tailcfg.PeerCapMap{
capabilityName: {
tailcfg.PeerCapabilityKubernetes: {
tailcfg.RawMessage(`{"impersonate":{"groups":["group1"]}}`),
},
},
@ -80,12 +80,26 @@ func TestImpersonationHeaders(t *testing.T) {
"Impersonate-User": {"node.ts.net"},
},
},
{
name: "mix-of-caps",
emailish: "tagged-device",
tags: []string{"tag:foo", "tag:bar"},
capMap: tailcfg.PeerCapMap{
tailcfg.PeerCapabilityKubernetes: {
tailcfg.RawMessage(`{"impersonate":{"groups":["group1"]},"recorder":["tag:foo"],"enforceRecorder":true}`),
},
},
wantHeaders: http.Header{
"Impersonate-Group": {"group1"},
"Impersonate-User": {"node.ts.net"},
},
},
{
name: "bad-cap",
emailish: "tagged-device",
tags: []string{"tag:foo", "tag:bar"},
capMap: tailcfg.PeerCapMap{
capabilityName: {
tailcfg.PeerCapabilityKubernetes: {
tailcfg.RawMessage(`[]`),
},
},

@ -3,14 +3,14 @@
//go:build !plan9
// tailscale-operator provides a way to expose services running in a Kubernetes
// cluster to your Tailnet.
package main
import (
"context"
"fmt"
"strings"
dockerref "github.com/distribution/reference"
"go.uber.org/zap"
corev1 "k8s.io/api/core/v1"
apiequality "k8s.io/apimachinery/pkg/api/equality"
@ -30,7 +30,9 @@ import (
const (
reasonProxyClassInvalid = "ProxyClassInvalid"
reasonProxyClassValid = "ProxyClassValid"
reasonCustomTSEnvVar = "CustomTSEnvVar"
messageProxyClassInvalid = "ProxyClass is not valid: %v"
messageCustomTSEnvVar = "ProxyClass overrides the default value for %s env var for %s container. Running with custom values for Tailscale env vars is not recommended and might break in the future."
)
type ProxyClassReconciler struct {
@ -98,6 +100,33 @@ func (a *ProxyClassReconciler) validate(pc *tsapi.ProxyClass) (violations field.
violations = append(violations, errs...)
}
}
if tc := pod.TailscaleContainer; tc != nil {
for _, e := range tc.Env {
if strings.HasPrefix(string(e.Name), "TS_") {
a.recorder.Event(pc, corev1.EventTypeWarning, reasonCustomTSEnvVar, fmt.Sprintf(messageCustomTSEnvVar, string(e.Name), "tailscale"))
}
if strings.EqualFold(string(e.Name), "EXPERIMENTAL_TS_CONFIGFILE_PATH") {
a.recorder.Event(pc, corev1.EventTypeWarning, reasonCustomTSEnvVar, fmt.Sprintf(messageCustomTSEnvVar, string(e.Name), "tailscale"))
}
if strings.EqualFold(string(e.Name), "EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS") {
a.recorder.Event(pc, corev1.EventTypeWarning, reasonCustomTSEnvVar, fmt.Sprintf(messageCustomTSEnvVar, string(e.Name), "tailscale"))
}
}
if tc.Image != "" {
// Same validation as used by kubelet https://github.com/kubernetes/kubernetes/blob/release-1.30/pkg/kubelet/images/image_manager.go#L212
if _, err := dockerref.ParseNormalizedNamed(tc.Image); err != nil {
violations = append(violations, field.TypeInvalid(field.NewPath("spec", "statefulSet", "pod", "tailscaleContainer", "image"), tc.Image, err.Error()))
}
}
}
if tc := pod.TailscaleInitContainer; tc != nil {
if tc.Image != "" {
// Same validation as used by kubelet https://github.com/kubernetes/kubernetes/blob/release-1.30/pkg/kubelet/images/image_manager.go#L212
if _, err := dockerref.ParseNormalizedNamed(tc.Image); err != nil {
violations = append(violations, field.TypeInvalid(field.NewPath("spec", "statefulSet", "pod", "tailscaleInitContainer", "image"), tc.Image, err.Error()))
}
}
}
}
}
// We do not validate embedded fields (security context, resource

@ -38,6 +38,11 @@ func TestProxyClass(t *testing.T) {
Pod: &tsapi.Pod{
Labels: map[string]string{"foo": "bar", "xyz1234": "abc567"},
Annotations: map[string]string{"foo.io/bar": "{'key': 'val1232'}"},
TailscaleContainer: &tsapi.Container{
Env: []tsapi.Env{{Name: "FOO", Value: "BAR"}},
ImagePullPolicy: "IfNotPresent",
Image: "ghcr.my-repo/tailscale:v0.01testsomething",
},
},
},
},
@ -51,27 +56,28 @@ func TestProxyClass(t *testing.T) {
if err != nil {
t.Fatal(err)
}
fr := record.NewFakeRecorder(3) // bump this if you expect a test case to throw more events
cl := tstest.NewClock(tstest.ClockOpts{})
pcr := &ProxyClassReconciler{
Client: fc,
logger: zl.Sugar(),
clock: cl,
recorder: record.NewFakeRecorder(1),
recorder: fr,
}
expectReconciled(t, pcr, "", "test")
// 1. A valid ProxyClass resource gets its status updated to Ready.
pc.Status.Conditions = append(pc.Status.Conditions, tsapi.ConnectorCondition{
Type: tsapi.ProxyClassready,
expectReconciled(t, pcr, "", "test")
pc.Status.Conditions = append(pc.Status.Conditions, metav1.Condition{
Type: string(tsapi.ProxyClassready),
Status: metav1.ConditionTrue,
Reason: reasonProxyClassValid,
Message: reasonProxyClassValid,
LastTransitionTime: &metav1.Time{Time: cl.Now().Truncate(time.Second)},
LastTransitionTime: metav1.Time{Time: cl.Now().Truncate(time.Second)},
})
expectEqual(t, fc, pc, nil)
// 2. An invalid ProxyClass resource gets its status updated to Invalid.
// 2. A ProxyClass resource with invalid labels gets its status updated to Invalid with an error message.
pc.Spec.StatefulSet.Labels["foo"] = "?!someVal"
mustUpdate(t, fc, "", "test", func(proxyClass *tsapi.ProxyClass) {
proxyClass.Spec.StatefulSet.Labels = pc.Spec.StatefulSet.Labels
@ -80,4 +86,51 @@ func TestProxyClass(t *testing.T) {
msg := `ProxyClass is not valid: .spec.statefulSet.labels: Invalid value: "?!someVal": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')`
tsoperator.SetProxyClassCondition(pc, tsapi.ProxyClassready, metav1.ConditionFalse, reasonProxyClassInvalid, msg, 0, cl, zl.Sugar())
expectEqual(t, fc, pc, nil)
expectedEvent := "Warning ProxyClassInvalid ProxyClass is not valid: .spec.statefulSet.labels: Invalid value: \"?!someVal\": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')"
expectEvents(t, fr, []string{expectedEvent})
// 3. A ProxyClass resource with invalid image reference gets it status updated to Invalid with an error message.
pc.Spec.StatefulSet.Labels = nil
pc.Spec.StatefulSet.Pod.TailscaleContainer.Image = "FOO bar"
mustUpdate(t, fc, "", "test", func(proxyClass *tsapi.ProxyClass) {
proxyClass.Spec.StatefulSet.Labels = nil
proxyClass.Spec.StatefulSet.Pod.TailscaleContainer.Image = pc.Spec.StatefulSet.Pod.TailscaleContainer.Image
})
expectReconciled(t, pcr, "", "test")
msg = `ProxyClass is not valid: spec.statefulSet.pod.tailscaleContainer.image: Invalid value: "FOO bar": invalid reference format: repository name (library/FOO bar) must be lowercase`
tsoperator.SetProxyClassCondition(pc, tsapi.ProxyClassready, metav1.ConditionFalse, reasonProxyClassInvalid, msg, 0, cl, zl.Sugar())
expectEqual(t, fc, pc, nil)
expectedEvent = `Warning ProxyClassInvalid ProxyClass is not valid: spec.statefulSet.pod.tailscaleContainer.image: Invalid value: "FOO bar": invalid reference format: repository name (library/FOO bar) must be lowercase`
expectEvents(t, fr, []string{expectedEvent})
// 4. A ProxyClass resource with invalid init container image reference gets it status updated to Invalid with an error message.
pc.Spec.StatefulSet.Labels = nil
pc.Spec.StatefulSet.Pod.TailscaleContainer.Image = ""
pc.Spec.StatefulSet.Pod.TailscaleInitContainer = &tsapi.Container{
Image: "FOO bar",
}
mustUpdate(t, fc, "", "test", func(proxyClass *tsapi.ProxyClass) {
proxyClass.Spec.StatefulSet.Pod.TailscaleContainer.Image = pc.Spec.StatefulSet.Pod.TailscaleContainer.Image
proxyClass.Spec.StatefulSet.Pod.TailscaleInitContainer = &tsapi.Container{
Image: pc.Spec.StatefulSet.Pod.TailscaleInitContainer.Image,
}
})
expectReconciled(t, pcr, "", "test")
msg = `ProxyClass is not valid: spec.statefulSet.pod.tailscaleInitContainer.image: Invalid value: "FOO bar": invalid reference format: repository name (library/FOO bar) must be lowercase`
tsoperator.SetProxyClassCondition(pc, tsapi.ProxyClassready, metav1.ConditionFalse, reasonProxyClassInvalid, msg, 0, cl, zl.Sugar())
expectEqual(t, fc, pc, nil)
expectedEvent = `Warning ProxyClassInvalid ProxyClass is not valid: spec.statefulSet.pod.tailscaleInitContainer.image: Invalid value: "FOO bar": invalid reference format: repository name (library/FOO bar) must be lowercase`
expectEvents(t, fr, []string{expectedEvent})
// 5. An valid ProxyClass but with a Tailscale env vars set results in warning events.
pc.Spec.StatefulSet.Pod.TailscaleInitContainer.Image = "" // unset previous test
mustUpdate(t, fc, "", "test", func(proxyClass *tsapi.ProxyClass) {
proxyClass.Spec.StatefulSet.Pod.TailscaleInitContainer.Image = pc.Spec.StatefulSet.Pod.TailscaleInitContainer.Image
proxyClass.Spec.StatefulSet.Pod.TailscaleContainer.Env = []tsapi.Env{{Name: "TS_USERSPACE", Value: "true"}, {Name: "EXPERIMENTAL_TS_CONFIGFILE_PATH"}, {Name: "EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS"}}
})
expectedEvents := []string{"Warning CustomTSEnvVar ProxyClass overrides the default value for TS_USERSPACE env var for tailscale container. Running with custom values for Tailscale env vars is not recommended and might break in the future.",
"Warning CustomTSEnvVar ProxyClass overrides the default value for EXPERIMENTAL_TS_CONFIGFILE_PATH env var for tailscale container. Running with custom values for Tailscale env vars is not recommended and might break in the future.",
"Warning CustomTSEnvVar ProxyClass overrides the default value for EXPERIMENTAL_ALLOW_PROXYING_CLUSTER_TRAFFIC_VIA_INGRESS env var for tailscale container. Running with custom values for Tailscale env vars is not recommended and might break in the future."}
expectReconciled(t, pcr, "", "test")
expectEvents(t, fr, expectedEvents)
}

@ -35,7 +35,6 @@ import (
"tailscale.com/tailcfg"
"tailscale.com/types/opt"
"tailscale.com/types/ptr"
"tailscale.com/util/dnsname"
"tailscale.com/util/mak"
)
@ -87,14 +86,11 @@ const (
// ensure that it does not get removed when a ProxyClass configuration
// is applied.
podAnnotationLastSetClusterIP = "tailscale.com/operator-last-set-cluster-ip"
podAnnotationLastSetClusterDNSName = "tailscale.com/operator-last-set-cluster-dns-name"
podAnnotationLastSetTailnetTargetIP = "tailscale.com/operator-last-set-ts-tailnet-target-ip"
podAnnotationLastSetTailnetTargetFQDN = "tailscale.com/operator-last-set-ts-tailnet-target-fqdn"
// podAnnotationLastSetConfigFileHash is sha256 hash of the current tailscaled configuration contents.
podAnnotationLastSetConfigFileHash = "tailscale.com/operator-last-set-config-file-hash"
// tailscaledConfigKey is the name of the key in proxy Secret Data that
// holds the tailscaled config contents.
tailscaledConfigKey = "tailscaled"
)
var (
@ -110,7 +106,8 @@ type tailscaleSTSConfig struct {
ChildResourceLabels map[string]string
ServeConfig *ipn.ServeConfig // if serve config is set, this is a proxy for Ingress
ClusterTargetIP string // ingress target
ClusterTargetIP string // ingress target IP
ClusterTargetDNSName string // ingress target DNS name
// If set to true, operator should configure containerboot to forward
// cluster traffic via the proxy set up for Kubernetes Ingress.
ForwardClusterTrafficViaL7IngressProxy bool
@ -126,7 +123,9 @@ type tailscaleSTSConfig struct {
// what this StatefulSet should be created for.
Connector *connector
ProxyClass string
ProxyClassName string // name of ProxyClass if one needs to be applied to the proxy
ProxyClass *tsapi.ProxyClass // ProxyClass that needs to be applied to the proxy (if there is one)
}
type connector struct {
@ -172,11 +171,23 @@ func (a *tailscaleSTSReconciler) Provision(ctx context.Context, logger *zap.Suga
return nil, fmt.Errorf("failed to reconcile headless service: %w", err)
}
secretName, tsConfigHash, err := a.createOrGetSecret(ctx, logger, sts, hsvc)
proxyClass := new(tsapi.ProxyClass)
if sts.ProxyClassName != "" {
if err := a.Get(ctx, types.NamespacedName{Name: sts.ProxyClassName}, proxyClass); err != nil {
return nil, fmt.Errorf("failed to get ProxyClass: %w", err)
}
if !tsoperator.ProxyClassIsReady(proxyClass) {
logger.Infof("ProxyClass %s specified for the proxy, but it is not (yet) in a ready state, waiting..")
return nil, nil
}
}
sts.ProxyClass = proxyClass
secretName, tsConfigHash, configs, err := a.createOrGetSecret(ctx, logger, sts, hsvc)
if err != nil {
return nil, fmt.Errorf("failed to create or get API key secret: %w", err)
}
_, err = a.reconcileSTS(ctx, logger, sts, hsvc, secretName, tsConfigHash)
_, err = a.reconcileSTS(ctx, logger, sts, hsvc, secretName, tsConfigHash, configs)
if err != nil {
return nil, fmt.Errorf("failed to reconcile statefulset: %w", err)
}
@ -289,7 +300,7 @@ func (a *tailscaleSTSReconciler) reconcileHeadlessService(ctx context.Context, l
return createOrUpdate(ctx, a.Client, a.operatorNamespace, hsvc, func(svc *corev1.Service) { svc.Spec = hsvc.Spec })
}
func (a *tailscaleSTSReconciler) createOrGetSecret(ctx context.Context, logger *zap.SugaredLogger, stsC *tailscaleSTSConfig, hsvc *corev1.Service) (string, string, error) {
func (a *tailscaleSTSReconciler) createOrGetSecret(ctx context.Context, logger *zap.SugaredLogger, stsC *tailscaleSTSConfig, hsvc *corev1.Service) (secretName, hash string, configs tailscaleConfigs, _ error) {
secret := &corev1.Secret{
ObjectMeta: metav1.ObjectMeta{
// Hardcode a -0 suffix so that in future, if we support
@ -305,25 +316,23 @@ func (a *tailscaleSTSReconciler) createOrGetSecret(ctx context.Context, logger *
logger.Debugf("secret %s/%s already exists", secret.GetNamespace(), secret.GetName())
orig = secret.DeepCopy()
} else if !apierrors.IsNotFound(err) {
return "", "", err
return "", "", nil, err
}
var (
authKey, hash string
)
var authKey string
if orig == nil {
// Initially it contains only tailscaled config, but when the
// proxy starts, it will also store there the state, certs and
// ACME account key.
sts, err := getSingleObject[appsv1.StatefulSet](ctx, a.Client, a.operatorNamespace, stsC.ChildResourceLabels)
if err != nil {
return "", "", err
return "", "", nil, err
}
if sts != nil {
// StatefulSet exists, so we have already created the secret.
// If the secret is missing, they should delete the StatefulSet.
logger.Errorf("Tailscale proxy secret doesn't exist, but the corresponding StatefulSet %s/%s already does. Something is wrong, please delete the StatefulSet.", sts.GetNamespace(), sts.GetName())
return "", "", nil
return "", "", nil, nil
}
// Create API Key secret which is going to be used by the statefulset
// to authenticate with Tailscale.
@ -334,45 +343,58 @@ func (a *tailscaleSTSReconciler) createOrGetSecret(ctx context.Context, logger *
}
authKey, err = a.newAuthKey(ctx, tags)
if err != nil {
return "", "", err
return "", "", nil, err
}
}
configs, err := tailscaledConfig(stsC, authKey, orig)
if err != nil {
return "", "", nil, fmt.Errorf("error creating tailscaled config: %w", err)
}
hash, err = tailscaledConfigHash(configs)
if err != nil {
return "", "", nil, fmt.Errorf("error calculating hash of tailscaled configs: %w", err)
}
confFileBytes, h, err := tailscaledConfig(stsC, authKey, orig)
latest := tailcfg.CapabilityVersion(-1)
var latestConfig ipn.ConfigVAlpha
for key, val := range configs {
fn := tsoperator.TailscaledConfigFileNameForCap(key)
b, err := json.Marshal(val)
if err != nil {
return "", "", fmt.Errorf("error creating tailscaled config: %w", err)
return "", "", nil, fmt.Errorf("error marshalling tailscaled config: %w", err)
}
mak.Set(&secret.StringData, fn, string(b))
if key > latest {
latest = key
latestConfig = val
}
}
hash = h
mak.Set(&secret.StringData, tailscaledConfigKey, string(confFileBytes))
if stsC.ServeConfig != nil {
j, err := json.Marshal(stsC.ServeConfig)
if err != nil {
return "", "", err
return "", "", nil, err
}
mak.Set(&secret.StringData, "serve-config", string(j))
}
if orig != nil {
logger.Debugf("patching the existing proxy Secret with tailscaled config %s", sanitizeConfigBytes(secret.Data[tailscaledConfigKey]))
logger.Debugf("patching the existing proxy Secret with tailscaled config %s", sanitizeConfigBytes(latestConfig))
if err := a.Patch(ctx, secret, client.MergeFrom(orig)); err != nil {
return "", "", err
return "", "", nil, err
}
} else {
logger.Debugf("creating a new Secret for the proxy with tailscaled config %s", sanitizeConfigBytes([]byte(secret.StringData[tailscaledConfigKey])))
logger.Debugf("creating a new Secret for the proxy with tailscaled config %s", sanitizeConfigBytes(latestConfig))
if err := a.Create(ctx, secret); err != nil {
return "", "", err
return "", "", nil, err
}
}
return secret.Name, hash, nil
return secret.Name, hash, configs, nil
}
// sanitizeConfigBytes returns ipn.ConfigVAlpha in string form with redacted
// auth key.
func sanitizeConfigBytes(bs []byte) string {
c := &ipn.ConfigVAlpha{}
if err := json.Unmarshal(bs, c); err != nil {
return "invalid config"
}
func sanitizeConfigBytes(c ipn.ConfigVAlpha) string {
if c.AuthKey != nil {
c.AuthKey = ptr.To("**redacted**")
}
@ -383,8 +405,10 @@ func sanitizeConfigBytes(bs []byte) string {
return string(sanitizedBytes)
}
// DeviceInfo returns the device ID and hostname for the Tailscale device
// associated with the given labels.
// DeviceInfo returns the device ID, hostname and IPs for the Tailscale device
// that acts as an operator proxy. It retrieves info from a Kubernetes Secret
// labeled with the provided labels.
// Either of device ID, hostname and IPs can be empty string if not found in the Secret.
func (a *tailscaleSTSReconciler) DeviceInfo(ctx context.Context, childLabels map[string]string) (id tailcfg.StableNodeID, hostname string, ips []string, err error) {
sec, err := getSingleObject[corev1.Secret](ctx, a.Client, a.operatorNamespace, childLabels)
if err != nil {
@ -401,7 +425,12 @@ func (a *tailscaleSTSReconciler) DeviceInfo(ctx context.Context, childLabels map
// to remove it.
hostname = strings.TrimSuffix(string(sec.Data["device_fqdn"]), ".")
if hostname == "" {
return "", "", nil, nil
// Device ID gets stored and retrieved in a different flow than
// FQDN and IPs. A device that acts as Kubernetes operator
// proxy, but whose route setup has failed might have an device
// ID, but no FQDN/IPs. If so, return the ID, to allow the
// operator to clean up such devices.
return id, "", nil, nil
}
if rawDeviceIPs, ok := sec.Data["device_ips"]; ok {
if err := json.Unmarshal(rawDeviceIPs, &ips); err != nil {
@ -435,7 +464,7 @@ var proxyYaml []byte
//go:embed deploy/manifests/userspace-proxy.yaml
var userspaceProxyYaml []byte
func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.SugaredLogger, sts *tailscaleSTSConfig, headlessSvc *corev1.Service, proxySecret, tsConfigHash string) (*appsv1.StatefulSet, error) {
func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.SugaredLogger, sts *tailscaleSTSConfig, headlessSvc *corev1.Service, proxySecret, tsConfigHash string, configs map[tailcfg.CapabilityVersion]ipn.ConfigVAlpha) (*appsv1.StatefulSet, error) {
ss := new(appsv1.StatefulSet)
if sts.ServeConfig != nil && sts.ForwardClusterTrafficViaL7IngressProxy != true { // If forwarding cluster traffic via is required we need non-userspace + NET_ADMIN + forwarding
if err := yaml.Unmarshal(userspaceProxyYaml, &ss); err != nil {
@ -455,16 +484,6 @@ func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.S
}
pod := &ss.Spec.Template
container := &pod.Spec.Containers[0]
proxyClass := new(tsapi.ProxyClass)
if sts.ProxyClass != "" {
if err := a.Get(ctx, types.NamespacedName{Name: sts.ProxyClass}, proxyClass); err != nil {
return nil, fmt.Errorf("failed to get ProxyClass: %w", err)
}
if !tsoperator.ProxyClassIsReady(proxyClass) {
logger.Infof("ProxyClass %s specified for the proxy, but it is not (yet) in a ready state, waiting..")
return nil, nil
}
}
container.Image = a.proxyImage
ss.ObjectMeta = metav1.ObjectMeta{
Name: headlessSvc.Name,
@ -491,9 +510,15 @@ func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.S
Value: proxySecret,
},
corev1.EnvVar{
// Old tailscaled config key is still used for backwards compatibility.
Name: "EXPERIMENTAL_TS_CONFIGFILE_PATH",
Value: "/etc/tsconfig/tailscaled",
},
corev1.EnvVar{
// New style is in the form of cap-<capability-version>.hujson.
Name: "TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR",
Value: "/etc/tsconfig",
},
)
if sts.ForwardClusterTrafficViaL7IngressProxy {
container.Env = append(container.Env, corev1.EnvVar{
@ -503,18 +528,16 @@ func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.S
}
// Configure containeboot to run tailscaled with a configfile read from the state Secret.
mak.Set(&ss.Spec.Template.Annotations, podAnnotationLastSetConfigFileHash, tsConfigHash)
pod.Spec.Volumes = append(ss.Spec.Template.Spec.Volumes, corev1.Volume{
configVolume := corev1.Volume{
Name: "tailscaledconfig",
VolumeSource: corev1.VolumeSource{
Secret: &corev1.SecretVolumeSource{
SecretName: proxySecret,
Items: []corev1.KeyToPath{{
Key: tailscaledConfigKey,
Path: tailscaledConfigKey,
}},
},
},
})
}
pod.Spec.Volumes = append(ss.Spec.Template.Spec.Volumes, configVolume)
container.VolumeMounts = append(container.VolumeMounts, corev1.VolumeMount{
Name: "tailscaledconfig",
ReadOnly: true,
@ -536,6 +559,12 @@ func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.S
Value: sts.ClusterTargetIP,
})
mak.Set(&ss.Spec.Template.Annotations, podAnnotationLastSetClusterIP, sts.ClusterTargetIP)
} else if sts.ClusterTargetDNSName != "" {
container.Env = append(container.Env, corev1.EnvVar{
Name: "TS_EXPERIMENTAL_DEST_DNS_NAME",
Value: sts.ClusterTargetDNSName,
})
mak.Set(&ss.Spec.Template.Annotations, podAnnotationLastSetClusterDNSName, sts.ClusterTargetDNSName)
} else if sts.TailnetTargetIP != "" {
container.Env = append(container.Env, corev1.EnvVar{
Name: "TS_TAILNET_TARGET_IP",
@ -563,18 +592,15 @@ func (a *tailscaleSTSReconciler) reconcileSTS(ctx context.Context, logger *zap.S
VolumeSource: corev1.VolumeSource{
Secret: &corev1.SecretVolumeSource{
SecretName: proxySecret,
Items: []corev1.KeyToPath{{
Key: "serve-config",
Path: "serve-config",
}},
Items: []corev1.KeyToPath{{Key: "serve-config", Path: "serve-config"}},
},
},
})
}
logger.Debugf("reconciling statefulset %s/%s", ss.GetNamespace(), ss.GetName())
if sts.ProxyClass != "" {
logger.Debugf("configuring proxy resources with ProxyClass %s", sts.ProxyClass)
ss = applyProxyClassToStatefulSet(proxyClass, ss)
if sts.ProxyClassName != "" {
logger.Debugf("configuring proxy resources with ProxyClass %s", sts.ProxyClassName)
ss = applyProxyClassToStatefulSet(sts.ProxyClass, ss, sts, logger)
}
updateSS := func(s *appsv1.StatefulSet) {
s.Spec = ss.Spec
@ -605,8 +631,28 @@ func mergeStatefulSetLabelsOrAnnots(current, custom map[string]string, managed [
return custom
}
func applyProxyClassToStatefulSet(pc *tsapi.ProxyClass, ss *appsv1.StatefulSet) *appsv1.StatefulSet {
if pc == nil || ss == nil || pc.Spec.StatefulSet == nil {
func applyProxyClassToStatefulSet(pc *tsapi.ProxyClass, ss *appsv1.StatefulSet, stsCfg *tailscaleSTSConfig, logger *zap.SugaredLogger) *appsv1.StatefulSet {
if pc == nil || ss == nil {
return ss
}
if pc.Spec.Metrics != nil && pc.Spec.Metrics.Enable {
if stsCfg.TailnetTargetFQDN == "" && stsCfg.TailnetTargetIP == "" && !stsCfg.ForwardClusterTrafficViaL7IngressProxy {
enableMetrics(ss, pc)
} else if stsCfg.ForwardClusterTrafficViaL7IngressProxy {
// TODO (irbekrm): fix this
// For Ingress proxies that have been configured with
// tailscale.com/experimental-forward-cluster-traffic-via-ingress
// annotation, all cluster traffic is forwarded to the
// Ingress backend(s).
logger.Info("ProxyClass specifies that metrics should be enabled, but this is currently not supported for Ingress proxies that accept cluster traffic.")
} else {
// TODO (irbekrm): fix this
// For egress proxies, currently all cluster traffic is forwarded to the tailnet target.
logger.Info("ProxyClass specifies that metrics should be enabled, but this is currently not supported for Ingress proxies that accept cluster traffic.")
}
}
if pc.Spec.StatefulSet == nil {
return ss
}
@ -633,6 +679,7 @@ func applyProxyClassToStatefulSet(pc *tsapi.ProxyClass, ss *appsv1.StatefulSet)
ss.Spec.Template.Spec.ImagePullSecrets = wantsPod.ImagePullSecrets
ss.Spec.Template.Spec.NodeName = wantsPod.NodeName
ss.Spec.Template.Spec.NodeSelector = wantsPod.NodeSelector
ss.Spec.Template.Spec.Affinity = wantsPod.Affinity
ss.Spec.Template.Spec.Tolerations = wantsPod.Tolerations
// Update containers.
@ -644,6 +691,21 @@ func applyProxyClassToStatefulSet(pc *tsapi.ProxyClass, ss *appsv1.StatefulSet)
base.SecurityContext = overlay.SecurityContext
}
base.Resources = overlay.Resources
for _, e := range overlay.Env {
// Env vars configured via ProxyClass might override env
// vars that have been specified by the operator, i.e
// TS_USERSPACE. The intended behaviour is to allow this
// and in practice it works without explicitly removing
// the operator configured value here as a later value
// in the env var list overrides an earlier one.
base.Env = append(base.Env, corev1.EnvVar{Name: string(e.Name), Value: e.Value})
}
if overlay.Image != "" {
base.Image = overlay.Image
}
if overlay.ImagePullPolicy != "" {
base.ImagePullPolicy = overlay.ImagePullPolicy
}
return base
}
for i, c := range ss.Spec.Template.Spec.Containers {
@ -663,42 +725,105 @@ func applyProxyClassToStatefulSet(pc *tsapi.ProxyClass, ss *appsv1.StatefulSet)
return ss
}
func enableMetrics(ss *appsv1.StatefulSet, pc *tsapi.ProxyClass) {
for i, c := range ss.Spec.Template.Spec.Containers {
if c.Name == "tailscale" {
// Serve metrics on on <pod-ip>:9001/debug/metrics. If
// we didn't specify Pod IP here, the proxy would, in
// some cases, also listen to its Tailscale IP- we don't
// want folks to start relying on this side-effect as a
// feature.
ss.Spec.Template.Spec.Containers[i].Env = append(ss.Spec.Template.Spec.Containers[i].Env, corev1.EnvVar{Name: "TS_TAILSCALED_EXTRA_ARGS", Value: "--debug=$(POD_IP):9001"})
ss.Spec.Template.Spec.Containers[i].Ports = append(ss.Spec.Template.Spec.Containers[i].Ports, corev1.ContainerPort{Name: "metrics", Protocol: "TCP", HostPort: 9001, ContainerPort: 9001})
break
}
}
}
func readAuthKey(secret *corev1.Secret, key string) (*string, error) {
origConf := &ipn.ConfigVAlpha{}
if err := json.Unmarshal([]byte(secret.Data[key]), origConf); err != nil {
return nil, fmt.Errorf("error unmarshaling previous tailscaled config in %q: %w", key, err)
}
return origConf.AuthKey, nil
}
// tailscaledConfig takes a proxy config, a newly generated auth key if
// generated and a Secret with the previous proxy state and auth key and
// produces returns tailscaled configuration and a hash of that configuration.
func tailscaledConfig(stsC *tailscaleSTSConfig, newAuthkey string, oldSecret *corev1.Secret) ([]byte, string, error) {
conf := ipn.ConfigVAlpha{
// returns tailscaled configuration and a hash of that configuration.
//
// As of 2024-05-09 it also returns legacy tailscaled config without the
// later added NoStatefulFilter field to support proxies older than cap95.
// TODO (irbekrm): remove the legacy config once we no longer need to support
// versions older than cap94,
// https://tailscale.com/kb/1236/kubernetes-operator#operator-and-proxies
func tailscaledConfig(stsC *tailscaleSTSConfig, newAuthkey string, oldSecret *corev1.Secret) (tailscaleConfigs, error) {
conf := &ipn.ConfigVAlpha{
Version: "alpha0",
AcceptDNS: "false",
AcceptRoutes: "false", // AcceptRoutes defaults to true
Locked: "false",
Hostname: &stsC.Hostname,
NoStatefulFiltering: "false",
}
// For egress proxies only, we need to ensure that stateful filtering is
// not in place so that traffic from cluster can be forwarded via
// Tailscale IPs.
if stsC.TailnetTargetFQDN != "" || stsC.TailnetTargetIP != "" {
conf.NoStatefulFiltering = "true"
}
if stsC.Connector != nil {
routes, err := netutil.CalcAdvertiseRoutes(stsC.Connector.routes, stsC.Connector.isExitNode)
if err != nil {
return nil, "", fmt.Errorf("error calculating routes: %w", err)
return nil, fmt.Errorf("error calculating routes: %w", err)
}
conf.AdvertiseRoutes = routes
}
if shouldAcceptRoutes(stsC.ProxyClass) {
conf.AcceptRoutes = "true"
}
if newAuthkey != "" {
conf.AuthKey = &newAuthkey
} else if oldSecret != nil && len(oldSecret.Data[tailscaledConfigKey]) > 0 { // write to StringData, read from Data as StringData is write-only
origConf := &ipn.ConfigVAlpha{}
if err := json.Unmarshal([]byte(oldSecret.Data[tailscaledConfigKey]), origConf); err != nil {
return nil, "", fmt.Errorf("error unmarshaling previous tailscaled config: %w", err)
} else if oldSecret != nil {
var err error
latest := tailcfg.CapabilityVersion(-1)
latestStr := ""
for k, data := range oldSecret.Data {
// write to StringData, read from Data as StringData is write-only
if len(data) == 0 {
continue
}
v, err := tsoperator.CapVerFromFileName(k)
if err != nil {
continue
}
conf.AuthKey = origConf.AuthKey
if v > latest {
latestStr = k
latest = v
}
confFileBytes, err := json.Marshal(conf)
if err != nil {
return nil, "", fmt.Errorf("error marshaling tailscaled config : %w", err)
}
hash, err := hashBytes(confFileBytes)
// Allow for configs that don't contain an auth key. Perhaps
// users have some mechanisms to delete them. Auth key is
// normally not needed after the initial login.
if latestStr != "" {
conf.AuthKey, err = readAuthKey(oldSecret, latestStr)
if err != nil {
return nil, "", fmt.Errorf("error calculating config hash: %w", err)
return nil, err
}
}
}
return confFileBytes, hash, nil
capVerConfigs := make(map[tailcfg.CapabilityVersion]ipn.ConfigVAlpha)
capVerConfigs[95] = *conf
// legacy config should not contain NoStatefulFiltering field.
conf.NoStatefulFiltering.Clear()
capVerConfigs[94] = *conf
return capVerConfigs, nil
}
func shouldAcceptRoutes(pc *tsapi.ProxyClass) bool {
return pc != nil && pc.Spec.TailscaleConfig != nil && pc.Spec.TailscaleConfig.AcceptRoutes
}
// ptrObject is a type constraint for pointer types that implement
@ -708,7 +833,9 @@ type ptrObject[T any] interface {
*T
}
// hashBytes produces a hash for the provided bytes that is the same across
type tailscaleConfigs map[tailcfg.CapabilityVersion]ipn.ConfigVAlpha
// hashBytes produces a hash for the provided tailscaled config that is the same across
// different invocations of this code. We do not use the
// tailscale.com/deephash.Hash here because that produces a different hash for
// the same value in different tailscale builds. The hash we are producing here
@ -717,10 +844,13 @@ type ptrObject[T any] interface {
// thing that changed is operator version (the hash is also exposed to users via
// an annotation and might be confusing if it changes without the config having
// changed).
func hashBytes(b []byte) (string, error) {
h := sha256.New()
_, err := h.Write(b)
func tailscaledConfigHash(c tailscaleConfigs) (string, error) {
b, err := json.Marshal(c)
if err != nil {
return "", fmt.Errorf("error marshalling tailscaled configs: %w", err)
}
h := sha256.New()
if _, err = h.Write(b); err != nil {
return "", fmt.Errorf("error calculating hash: %w", err)
}
return fmt.Sprintf("%x", h.Sum(nil)), nil
@ -816,14 +946,11 @@ func defaultEnv(envName, defVal string) string {
return v
}
func nameForService(svc *corev1.Service) (string, error) {
func nameForService(svc *corev1.Service) string {
if h, ok := svc.Annotations[AnnotationHostname]; ok {
if err := dnsname.ValidLabel(h); err != nil {
return "", fmt.Errorf("invalid Tailscale hostname %q: %w", h, err)
}
return h, nil
return h
}
return svc.Namespace + "-" + svc.Name, nil
return svc.Namespace + "-" + svc.Name
}
func isValidFirewallMode(m string) bool {

@ -14,6 +14,7 @@ import (
"testing"
"github.com/google/go-cmp/cmp"
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
@ -51,6 +52,10 @@ func Test_statefulSetNameBase(t *testing.T) {
}
func Test_applyProxyClassToStatefulSet(t *testing.T) {
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
// Setup
proxyClassAllOpts := &tsapi.ProxyClass{
Spec: tsapi.ProxyClassSpec{
@ -66,6 +71,7 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
ImagePullSecrets: []corev1.LocalObjectReference{{Name: "docker-creds"}},
NodeName: "some-node",
NodeSelector: map[string]string{"beta.kubernetes.io/os": "linux"},
Affinity: &corev1.Affinity{NodeAffinity: &corev1.NodeAffinity{RequiredDuringSchedulingIgnoredDuringExecution: &corev1.NodeSelector{}}},
Tolerations: []corev1.Toleration{{Key: "", Operator: "Exists"}},
TailscaleContainer: &tsapi.Container{
SecurityContext: &corev1.SecurityContext{
@ -75,6 +81,9 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
Limits: corev1.ResourceList{corev1.ResourceCPU: resource.MustParse("1000m"), corev1.ResourceMemory: resource.MustParse("128Mi")},
Requests: corev1.ResourceList{corev1.ResourceCPU: resource.MustParse("500m"), corev1.ResourceMemory: resource.MustParse("64Mi")},
},
Env: []tsapi.Env{{Name: "foo", Value: "bar"}, {Name: "TS_USERSPACE", Value: "true"}, {Name: "bar"}},
ImagePullPolicy: "IfNotPresent",
Image: "ghcr.io/my-repo/tailscale:v0.01testsomething",
},
TailscaleInitContainer: &tsapi.Container{
SecurityContext: &corev1.SecurityContext{
@ -85,6 +94,9 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
Limits: corev1.ResourceList{corev1.ResourceCPU: resource.MustParse("1000m"), corev1.ResourceMemory: resource.MustParse("128Mi")},
Requests: corev1.ResourceList{corev1.ResourceCPU: resource.MustParse("500m"), corev1.ResourceMemory: resource.MustParse("64Mi")},
},
Env: []tsapi.Env{{Name: "foo", Value: "bar"}, {Name: "TS_USERSPACE", Value: "true"}, {Name: "bar"}},
ImagePullPolicy: "IfNotPresent",
Image: "ghcr.io/my-repo/tailscale:v0.01testsomething",
},
},
},
@ -102,6 +114,12 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
},
},
}
proxyClassMetrics := &tsapi.ProxyClass{
Spec: tsapi.ProxyClassSpec{
Metrics: &tsapi.Metrics{Enable: true},
},
}
var userspaceProxySS, nonUserspaceProxySS appsv1.StatefulSet
if err := yaml.Unmarshal(userspaceProxyYaml, &userspaceProxySS); err != nil {
t.Fatalf("unmarshaling userspace proxy template: %v", err)
@ -121,10 +139,12 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
env := []corev1.EnvVar{{Name: "TS_HOSTNAME", Value: "nginx"}}
userspaceProxySS.Labels = labels
userspaceProxySS.Annotations = annots
userspaceProxySS.Spec.Template.Spec.Containers[0].Image = "tailscale/tailscale:v0.0.1"
userspaceProxySS.Spec.Template.Spec.Containers[0].Env = env
nonUserspaceProxySS.ObjectMeta.Labels = labels
nonUserspaceProxySS.ObjectMeta.Annotations = annots
nonUserspaceProxySS.Spec.Template.Spec.Containers[0].Env = env
nonUserspaceProxySS.Spec.Template.Spec.InitContainers[0].Image = "tailscale/tailscale:v0.0.1"
// 1. Test that a ProxyClass with all fields set gets correctly applied
// to a Statefulset built from non-userspace proxy template.
@ -137,13 +157,20 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
wantSS.Spec.Template.Spec.ImagePullSecrets = proxyClassAllOpts.Spec.StatefulSet.Pod.ImagePullSecrets
wantSS.Spec.Template.Spec.NodeName = proxyClassAllOpts.Spec.StatefulSet.Pod.NodeName
wantSS.Spec.Template.Spec.NodeSelector = proxyClassAllOpts.Spec.StatefulSet.Pod.NodeSelector
wantSS.Spec.Template.Spec.Affinity = proxyClassAllOpts.Spec.StatefulSet.Pod.Affinity
wantSS.Spec.Template.Spec.Tolerations = proxyClassAllOpts.Spec.StatefulSet.Pod.Tolerations
wantSS.Spec.Template.Spec.Containers[0].SecurityContext = proxyClassAllOpts.Spec.StatefulSet.Pod.TailscaleContainer.SecurityContext
wantSS.Spec.Template.Spec.InitContainers[0].SecurityContext = proxyClassAllOpts.Spec.StatefulSet.Pod.TailscaleInitContainer.SecurityContext
wantSS.Spec.Template.Spec.Containers[0].Resources = proxyClassAllOpts.Spec.StatefulSet.Pod.TailscaleContainer.Resources
wantSS.Spec.Template.Spec.InitContainers[0].Resources = proxyClassAllOpts.Spec.StatefulSet.Pod.TailscaleInitContainer.Resources
wantSS.Spec.Template.Spec.InitContainers[0].Env = append(wantSS.Spec.Template.Spec.InitContainers[0].Env, []corev1.EnvVar{{Name: "foo", Value: "bar"}, {Name: "TS_USERSPACE", Value: "true"}, {Name: "bar"}}...)
wantSS.Spec.Template.Spec.Containers[0].Env = append(wantSS.Spec.Template.Spec.Containers[0].Env, []corev1.EnvVar{{Name: "foo", Value: "bar"}, {Name: "TS_USERSPACE", Value: "true"}, {Name: "bar"}}...)
wantSS.Spec.Template.Spec.Containers[0].Image = "ghcr.io/my-repo/tailscale:v0.01testsomething"
wantSS.Spec.Template.Spec.Containers[0].ImagePullPolicy = "IfNotPresent"
wantSS.Spec.Template.Spec.InitContainers[0].Image = "ghcr.io/my-repo/tailscale:v0.01testsomething"
wantSS.Spec.Template.Spec.InitContainers[0].ImagePullPolicy = "IfNotPresent"
gotSS := applyProxyClassToStatefulSet(proxyClassAllOpts, nonUserspaceProxySS.DeepCopy())
gotSS := applyProxyClassToStatefulSet(proxyClassAllOpts, nonUserspaceProxySS.DeepCopy(), new(tailscaleSTSConfig), zl.Sugar())
if diff := cmp.Diff(gotSS, wantSS); diff != "" {
t.Fatalf("Unexpected result applying ProxyClass with all fields set to a StatefulSet for non-userspace proxy (-got +want):\n%s", diff)
}
@ -156,7 +183,7 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
wantSS.ObjectMeta.Annotations = mergeMapKeys(wantSS.ObjectMeta.Annotations, proxyClassJustLabels.Spec.StatefulSet.Annotations)
wantSS.Spec.Template.Labels = proxyClassJustLabels.Spec.StatefulSet.Pod.Labels
wantSS.Spec.Template.Annotations = proxyClassJustLabels.Spec.StatefulSet.Pod.Annotations
gotSS = applyProxyClassToStatefulSet(proxyClassJustLabels, nonUserspaceProxySS.DeepCopy())
gotSS = applyProxyClassToStatefulSet(proxyClassJustLabels, nonUserspaceProxySS.DeepCopy(), new(tailscaleSTSConfig), zl.Sugar())
if diff := cmp.Diff(gotSS, wantSS); diff != "" {
t.Fatalf("Unexpected result applying ProxyClass with custom labels and annotations to a StatefulSet for non-userspace proxy (-got +want):\n%s", diff)
}
@ -172,12 +199,16 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
wantSS.Spec.Template.Spec.ImagePullSecrets = proxyClassAllOpts.Spec.StatefulSet.Pod.ImagePullSecrets
wantSS.Spec.Template.Spec.NodeName = proxyClassAllOpts.Spec.StatefulSet.Pod.NodeName
wantSS.Spec.Template.Spec.NodeSelector = proxyClassAllOpts.Spec.StatefulSet.Pod.NodeSelector
wantSS.Spec.Template.Spec.Affinity = proxyClassAllOpts.Spec.StatefulSet.Pod.Affinity
wantSS.Spec.Template.Spec.Tolerations = proxyClassAllOpts.Spec.StatefulSet.Pod.Tolerations
wantSS.Spec.Template.Spec.Containers[0].SecurityContext = proxyClassAllOpts.Spec.StatefulSet.Pod.TailscaleContainer.SecurityContext
wantSS.Spec.Template.Spec.Containers[0].Resources = proxyClassAllOpts.Spec.StatefulSet.Pod.TailscaleContainer.Resources
gotSS = applyProxyClassToStatefulSet(proxyClassAllOpts, userspaceProxySS.DeepCopy())
wantSS.Spec.Template.Spec.Containers[0].Env = append(wantSS.Spec.Template.Spec.Containers[0].Env, []corev1.EnvVar{{Name: "foo", Value: "bar"}, {Name: "TS_USERSPACE", Value: "true"}, {Name: "bar"}}...)
wantSS.Spec.Template.Spec.Containers[0].ImagePullPolicy = "IfNotPresent"
wantSS.Spec.Template.Spec.Containers[0].Image = "ghcr.io/my-repo/tailscale:v0.01testsomething"
gotSS = applyProxyClassToStatefulSet(proxyClassAllOpts, userspaceProxySS.DeepCopy(), new(tailscaleSTSConfig), zl.Sugar())
if diff := cmp.Diff(gotSS, wantSS); diff != "" {
t.Fatalf("Unexpected result applying ProxyClass with custom labels and annotations to a StatefulSet for a userspace proxy (-got +want):\n%s", diff)
t.Fatalf("Unexpected result applying ProxyClass with all options to a StatefulSet for a userspace proxy (-got +want):\n%s", diff)
}
// 4. Test that a ProxyClass with custom labels and annotations gets correctly applied
@ -187,10 +218,19 @@ func Test_applyProxyClassToStatefulSet(t *testing.T) {
wantSS.ObjectMeta.Annotations = mergeMapKeys(wantSS.ObjectMeta.Annotations, proxyClassJustLabels.Spec.StatefulSet.Annotations)
wantSS.Spec.Template.Labels = proxyClassJustLabels.Spec.StatefulSet.Pod.Labels
wantSS.Spec.Template.Annotations = proxyClassJustLabels.Spec.StatefulSet.Pod.Annotations
gotSS = applyProxyClassToStatefulSet(proxyClassJustLabels, userspaceProxySS.DeepCopy())
gotSS = applyProxyClassToStatefulSet(proxyClassJustLabels, userspaceProxySS.DeepCopy(), new(tailscaleSTSConfig), zl.Sugar())
if diff := cmp.Diff(gotSS, wantSS); diff != "" {
t.Fatalf("Unexpected result applying ProxyClass with custom labels and annotations to a StatefulSet for a userspace proxy (-got +want):\n%s", diff)
}
// 5. Test that a ProxyClass with metrics enabled gets correctly applied to a StatefulSet.
wantSS = nonUserspaceProxySS.DeepCopy()
wantSS.Spec.Template.Spec.Containers[0].Env = append(wantSS.Spec.Template.Spec.Containers[0].Env, corev1.EnvVar{Name: "TS_TAILSCALED_EXTRA_ARGS", Value: "--debug=$(POD_IP):9001"})
wantSS.Spec.Template.Spec.Containers[0].Ports = []corev1.ContainerPort{{Name: "metrics", Protocol: "TCP", ContainerPort: 9001, HostPort: 9001}}
gotSS = applyProxyClassToStatefulSet(proxyClassMetrics, nonUserspaceProxySS.DeepCopy(), new(tailscaleSTSConfig), zl.Sugar())
if diff := cmp.Diff(gotSS, wantSS); diff != "" {
t.Fatalf("Unexpected result applying ProxyClass with metrics enabled to a StatefulSet (-got +want):\n%s", diff)
}
}
func mergeMapKeys(a, b map[string]string) map[string]string {

@ -7,6 +7,7 @@ package main
import (
"context"
"errors"
"fmt"
"net/netip"
"slices"
@ -15,17 +16,32 @@ import (
"go.uber.org/zap"
corev1 "k8s.io/api/core/v1"
apiequality "k8s.io/apimachinery/pkg/api/equality"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/tools/record"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
tsoperator "tailscale.com/k8s-operator"
tsapi "tailscale.com/k8s-operator/apis/v1alpha1"
"tailscale.com/net/dns/resolvconffile"
"tailscale.com/tstime"
"tailscale.com/util/clientmetric"
"tailscale.com/util/dnsname"
"tailscale.com/util/set"
)
const (
resolvConfPath = "/etc/resolv.conf"
defaultClusterDomain = "cluster.local"
reasonProxyCreated = "ProxyCreated"
reasonProxyInvalid = "ProxyInvalid"
reasonProxyFailed = "ProxyFailed"
reasonProxyPending = "ProxyPending"
)
type ServiceReconciler struct {
client.Client
ssr *tailscaleSTSReconciler
@ -42,6 +58,10 @@ type ServiceReconciler struct {
managedEgressProxies set.Slice[types.UID]
recorder record.EventRecorder
tsNamespace string
clock tstime.Clock
}
var (
@ -68,6 +88,12 @@ func childResourceLabels(name, ns, typ string) map[string]string {
}
}
func (a *ServiceReconciler) isTailscaleService(svc *corev1.Service) bool {
targetIP := tailnetTargetAnnotation(svc)
targetFQDN := svc.Annotations[AnnotationTailnetTargetFQDN]
return a.shouldExpose(svc) || targetIP != "" || targetFQDN != ""
}
func (a *ServiceReconciler) Reconcile(ctx context.Context, req reconcile.Request) (_ reconcile.Result, err error) {
logger := a.logger.With("service-ns", req.Namespace, "service-name", req.Name)
logger.Debugf("starting reconcile")
@ -82,9 +108,8 @@ func (a *ServiceReconciler) Reconcile(ctx context.Context, req reconcile.Request
} else if err != nil {
return reconcile.Result{}, fmt.Errorf("failed to get svc: %w", err)
}
targetIP := a.tailnetTargetAnnotation(svc)
targetFQDN := svc.Annotations[AnnotationTailnetTargetFQDN]
if !svc.DeletionTimestamp.IsZero() || !a.shouldExpose(svc) && targetIP == "" && targetFQDN == "" {
if !svc.DeletionTimestamp.IsZero() || !a.isTailscaleService(svc) {
logger.Debugf("service is being deleted or is (no longer) referring to Tailscale ingress/egress, ensuring any created resources are cleaned up")
return reconcile.Result{}, a.maybeCleanup(ctx, logger, svc)
}
@ -96,7 +121,14 @@ func (a *ServiceReconciler) Reconcile(ctx context.Context, req reconcile.Request
//
// This function is responsible for removing the finalizer from the service,
// once all associated resources are gone.
func (a *ServiceReconciler) maybeCleanup(ctx context.Context, logger *zap.SugaredLogger, svc *corev1.Service) error {
func (a *ServiceReconciler) maybeCleanup(ctx context.Context, logger *zap.SugaredLogger, svc *corev1.Service) (err error) {
oldSvcStatus := svc.Status.DeepCopy()
defer func() {
if !apiequality.Semantic.DeepEqual(oldSvcStatus, svc.Status) {
// An error encountered here should get returned by the Reconcile function.
err = errors.Join(err, a.Client.Status().Update(ctx, svc))
}
}()
ix := slices.Index(svc.Finalizers, FinalizerName)
if ix < 0 {
logger.Debugf("no finalizer, nothing to do")
@ -106,6 +138,10 @@ func (a *ServiceReconciler) maybeCleanup(ctx context.Context, logger *zap.Sugare
a.managedEgressProxies.Remove(svc.UID)
gaugeIngressProxies.Set(int64(a.managedIngressProxies.Len()))
gaugeEgressProxies.Set(int64(a.managedEgressProxies.Len()))
if !a.isTailscaleService(svc) {
tsoperator.RemoveServiceCondition(svc, tsapi.ProxyReady)
}
return nil
}
@ -125,7 +161,7 @@ func (a *ServiceReconciler) maybeCleanup(ctx context.Context, logger *zap.Sugare
// exactly once at the very end of cleanup, because the final step of
// cleanup removes the tailscale finalizer, which will make all future
// reconciles exit early.
logger.Infof("unexposed service from tailnet")
logger.Infof("unexposed Service from tailnet")
a.mu.Lock()
defer a.mu.Unlock()
@ -133,6 +169,10 @@ func (a *ServiceReconciler) maybeCleanup(ctx context.Context, logger *zap.Sugare
a.managedEgressProxies.Remove(svc.UID)
gaugeIngressProxies.Set(int64(a.managedIngressProxies.Len()))
gaugeEgressProxies.Set(int64(a.managedEgressProxies.Len()))
if !a.isTailscaleService(svc) {
tsoperator.RemoveServiceCondition(svc, tsapi.ProxyReady)
}
return nil
}
@ -141,7 +181,15 @@ func (a *ServiceReconciler) maybeCleanup(ctx context.Context, logger *zap.Sugare
//
// This function adds a finalizer to svc, ensuring that we can handle orderly
// deprovisioning later.
func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.SugaredLogger, svc *corev1.Service) error {
func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.SugaredLogger, svc *corev1.Service) (err error) {
oldSvcStatus := svc.Status.DeepCopy()
defer func() {
if !apiequality.Semantic.DeepEqual(oldSvcStatus, svc.Status) {
// An error encountered here should get returned by the Reconcile function.
err = errors.Join(err, a.Client.Status().Update(ctx, svc))
}
}()
// Run for proxy config related validations here as opposed to running
// them earlier. This is to prevent cleanup being blocked on a
// misconfigured proxy param.
@ -149,30 +197,31 @@ func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
msg := fmt.Sprintf("unable to provision proxy resources: invalid config: %v", err)
a.recorder.Event(svc, corev1.EventTypeWarning, "INVALIDCONFIG", msg)
a.logger.Error(msg)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyInvalid, msg, a.clock, logger)
return nil
}
if violations := validateService(svc); len(violations) > 0 {
msg := fmt.Sprintf("unable to provision proxy resources: invalid Service: %s", strings.Join(violations, ", "))
a.recorder.Event(svc, corev1.EventTypeWarning, "INVALIDSERVCICE", msg)
a.recorder.Event(svc, corev1.EventTypeWarning, "INVALIDSERVICE", msg)
a.logger.Error(msg)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyInvalid, msg, a.clock, logger)
return nil
}
proxyClass := proxyClassForObject(svc)
if proxyClass != "" {
if ready, err := proxyClassIsReady(ctx, proxyClass, a.Client); err != nil {
return fmt.Errorf("error verifying ProxyClass for Service: %w", err)
errMsg := fmt.Errorf("error verifying ProxyClass for Service: %w", err)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyFailed, errMsg.Error(), a.clock, logger)
return errMsg
} else if !ready {
logger.Infof("ProxyClass %s specified for the Service, but is not (yet) Ready, waiting..", proxyClass)
msg := fmt.Sprintf("ProxyClass %s specified for the Service, but is not (yet) Ready, waiting..", proxyClass)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyPending, msg, a.clock, logger)
logger.Info(msg)
return nil
}
}
hostname, err := nameForService(svc)
if err != nil {
return err
}
if !slices.Contains(svc.Finalizers, FinalizerName) {
// This log line is printed exactly once during initial provisioning,
// because once the finalizer is in place this block gets skipped. So,
@ -181,7 +230,9 @@ func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
logger.Infof("exposing service over tailscale")
svc.Finalizers = append(svc.Finalizers, FinalizerName)
if err := a.Update(ctx, svc); err != nil {
return fmt.Errorf("failed to add finalizer: %w", err)
errMsg := fmt.Errorf("failed to add finalizer: %w", err)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyFailed, errMsg.Error(), a.clock, logger)
return errMsg
}
}
crl := childResourceLabels(svc.Name, svc.Namespace, "svc")
@ -193,18 +244,22 @@ func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
sts := &tailscaleSTSConfig{
ParentResourceName: svc.Name,
ParentResourceUID: string(svc.UID),
Hostname: hostname,
Hostname: nameForService(svc),
Tags: tags,
ChildResourceLabels: crl,
ProxyClass: proxyClass,
ProxyClassName: proxyClass,
}
a.mu.Lock()
if a.shouldExpose(svc) {
if a.shouldExposeClusterIP(svc) {
sts.ClusterTargetIP = svc.Spec.ClusterIP
a.managedIngressProxies.Add(svc.UID)
gaugeIngressProxies.Set(int64(a.managedIngressProxies.Len()))
} else if ip := a.tailnetTargetAnnotation(svc); ip != "" {
} else if a.shouldExposeDNSName(svc) {
sts.ClusterTargetDNSName = svc.Spec.ExternalName
a.managedIngressProxies.Add(svc.UID)
gaugeIngressProxies.Set(int64(a.managedIngressProxies.Len()))
} else if ip := tailnetTargetAnnotation(svc); ip != "" {
sts.TailnetTargetIP = ip
a.managedEgressProxies.Add(svc.UID)
gaugeEgressProxies.Set(int64(a.managedEgressProxies.Len()))
@ -221,27 +276,31 @@ func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
var hsvc *corev1.Service
if hsvc, err = a.ssr.Provision(ctx, logger, sts); err != nil {
return fmt.Errorf("failed to provision: %w", err)
errMsg := fmt.Errorf("failed to provision: %w", err)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyFailed, errMsg.Error(), a.clock, logger)
return errMsg
}
if sts.TailnetTargetIP != "" || sts.TailnetTargetFQDN != "" {
// TODO (irbekrm): cluster.local is the default DNS name, but
// can be changed by users. Make this configurable or figure out
// how to discover the DNS name from within operator
headlessSvcName := hsvc.Name + "." + hsvc.Namespace + ".svc.cluster.local"
if sts.TailnetTargetIP != "" || sts.TailnetTargetFQDN != "" { // if an egress proxy
clusterDomain := retrieveClusterDomain(a.tsNamespace, logger)
headlessSvcName := hsvc.Name + "." + hsvc.Namespace + ".svc." + clusterDomain
if svc.Spec.ExternalName != headlessSvcName || svc.Spec.Type != corev1.ServiceTypeExternalName {
svc.Spec.ExternalName = headlessSvcName
svc.Spec.Selector = nil
svc.Spec.Type = corev1.ServiceTypeExternalName
if err := a.Update(ctx, svc); err != nil {
return fmt.Errorf("failed to update service: %w", err)
errMsg := fmt.Errorf("failed to update service: %w", err)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyFailed, errMsg.Error(), a.clock, logger)
return errMsg
}
}
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionTrue, reasonProxyCreated, reasonProxyCreated, a.clock, logger)
return nil
}
if !a.hasLoadBalancerClass(svc) {
if !isTailscaleLoadBalancerService(svc, a.isDefaultLoadBalancer) {
logger.Debugf("service is not a LoadBalancer, so not updating ingress")
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionTrue, reasonProxyCreated, reasonProxyCreated, a.clock, logger)
return nil
}
@ -250,22 +309,23 @@ func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
return fmt.Errorf("failed to get device ID: %w", err)
}
if tsHost == "" {
logger.Debugf("no Tailscale hostname known yet, waiting for proxy pod to finish auth")
msg := "no Tailscale hostname known yet, waiting for proxy pod to finish auth"
logger.Debug(msg)
// No hostname yet. Wait for the proxy pod to auth.
svc.Status.LoadBalancer.Ingress = nil
if err := a.Status().Update(ctx, svc); err != nil {
return fmt.Errorf("failed to update service status: %w", err)
}
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyPending, msg, a.clock, logger)
return nil
}
logger.Debugf("setting ingress to %q, %s", tsHost, strings.Join(tsIPs, ", "))
logger.Debugf("setting Service LoadBalancer status to %q, %s", tsHost, strings.Join(tsIPs, ", "))
ingress := []corev1.LoadBalancerIngress{
{Hostname: tsHost},
}
clusterIPAddr, err := netip.ParseAddr(svc.Spec.ClusterIP)
if err != nil {
return fmt.Errorf("failed to parse cluster IP: %w", err)
msg := fmt.Sprintf("failed to parse cluster IP: %v", err)
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionFalse, reasonProxyFailed, msg, a.clock, logger)
return fmt.Errorf(msg)
}
for _, ip := range tsIPs {
addr, err := netip.ParseAddr(ip)
@ -277,53 +337,64 @@ func (a *ServiceReconciler) maybeProvision(ctx context.Context, logger *zap.Suga
}
}
svc.Status.LoadBalancer.Ingress = ingress
if err := a.Status().Update(ctx, svc); err != nil {
return fmt.Errorf("failed to update service status: %w", err)
}
tsoperator.SetServiceCondition(svc, tsapi.ProxyReady, metav1.ConditionTrue, reasonProxyCreated, reasonProxyCreated, a.clock, logger)
return nil
}
func validateService(svc *corev1.Service) []string {
violations := make([]string, 0)
if svc.Annotations[AnnotationTailnetTargetFQDN] != "" && svc.Annotations[AnnotationTailnetTargetIP] != "" {
violations = append(violations, "only one of annotations %s and %s can be set", AnnotationTailnetTargetIP, AnnotationTailnetTargetFQDN)
violations = append(violations, fmt.Sprintf("only one of annotations %s and %s can be set", AnnotationTailnetTargetIP, AnnotationTailnetTargetFQDN))
}
if fqdn := svc.Annotations[AnnotationTailnetTargetFQDN]; fqdn != "" {
if !isMagicDNSName(fqdn) {
violations = append(violations, fmt.Sprintf("invalid value of annotation %s: %q does not appear to be a valid MagicDNS name", AnnotationTailnetTargetFQDN, fqdn))
}
}
svcName := nameForService(svc)
if err := dnsname.ValidLabel(svcName); err != nil {
if _, ok := svc.Annotations[AnnotationHostname]; ok {
violations = append(violations, fmt.Sprintf("invalid Tailscale hostname specified %q: %s", svcName, err))
} else {
violations = append(violations, fmt.Sprintf("invalid Tailscale hostname %q, use %q annotation to override: %s", svcName, AnnotationHostname, err))
}
}
return violations
}
func (a *ServiceReconciler) shouldExpose(svc *corev1.Service) bool {
// Headless services can't be exposed, since there is no ClusterIP to
// forward to.
return a.shouldExposeClusterIP(svc) || a.shouldExposeDNSName(svc)
}
func (a *ServiceReconciler) shouldExposeDNSName(svc *corev1.Service) bool {
return hasExposeAnnotation(svc) && svc.Spec.Type == corev1.ServiceTypeExternalName && svc.Spec.ExternalName != ""
}
func (a *ServiceReconciler) shouldExposeClusterIP(svc *corev1.Service) bool {
if svc.Spec.ClusterIP == "" || svc.Spec.ClusterIP == "None" {
return false
}
return a.hasLoadBalancerClass(svc) || a.hasExposeAnnotation(svc)
return isTailscaleLoadBalancerService(svc, a.isDefaultLoadBalancer) || hasExposeAnnotation(svc)
}
func (a *ServiceReconciler) hasLoadBalancerClass(svc *corev1.Service) bool {
func isTailscaleLoadBalancerService(svc *corev1.Service, isDefaultLoadBalancer bool) bool {
return svc != nil &&
svc.Spec.Type == corev1.ServiceTypeLoadBalancer &&
(svc.Spec.LoadBalancerClass != nil && *svc.Spec.LoadBalancerClass == "tailscale" ||
svc.Spec.LoadBalancerClass == nil && a.isDefaultLoadBalancer)
svc.Spec.LoadBalancerClass == nil && isDefaultLoadBalancer)
}
// hasExposeAnnotation reports whether Service has the tailscale.com/expose
// annotation set
func (a *ServiceReconciler) hasExposeAnnotation(svc *corev1.Service) bool {
func hasExposeAnnotation(svc *corev1.Service) bool {
return svc != nil && svc.Annotations[AnnotationExpose] == "true"
}
// hasTailnetTargetAnnotation returns the value of tailscale.com/tailnet-ip
// tailnetTargetAnnotation returns the value of tailscale.com/tailnet-ip
// annotation or of the deprecated tailscale.com/ts-tailnet-target-ip
// annotation. If neither is set, it returns an empty string. If both are set,
// it returns the value of the new annotation.
func (a *ServiceReconciler) tailnetTargetAnnotation(svc *corev1.Service) string {
func tailnetTargetAnnotation(svc *corev1.Service) string {
if svc == nil {
return ""
}
@ -344,3 +415,51 @@ func proxyClassIsReady(ctx context.Context, name string, cl client.Client) (bool
}
return tsoperator.ProxyClassIsReady(proxyClass), nil
}
// retrieveClusterDomain determines and retrieves cluster domain i.e
// (cluster.local) in which this Pod is running by parsing search domains in
// /etc/resolv.conf. If an error is encountered at any point during the process,
// defaults cluster domain to 'cluster.local'.
func retrieveClusterDomain(namespace string, logger *zap.SugaredLogger) string {
logger.Infof("attempting to retrieve cluster domain..")
conf, err := resolvconffile.ParseFile(resolvConfPath)
if err != nil {
// Vast majority of clusters use the cluster.local domain, so it
// is probably better to fall back to that than error out.
logger.Infof("[unexpected] error parsing /etc/resolv.conf to determine cluster domain, defaulting to 'cluster.local'.")
return defaultClusterDomain
}
return clusterDomainFromResolverConf(conf, namespace, logger)
}
// clusterDomainFromResolverConf attempts to retrieve cluster domain from the provided resolver config.
// It expects the first three search domains in the resolver config to be be ['<namespace>.svc.<cluster-domain>, svc.<cluster-domain>, <cluster-domain>, ...]
// If the first three domains match the expected structure, it returns the third.
// If the domains don't match the expected structure or an error is encountered, it defaults to 'cluster.local' domain.
func clusterDomainFromResolverConf(conf *resolvconffile.Config, namespace string, logger *zap.SugaredLogger) string {
if len(conf.SearchDomains) < 3 {
logger.Infof("[unexpected] resolver config contains only %d search domains, at least three expected.\nDefaulting cluster domain to 'cluster.local'.")
return defaultClusterDomain
}
first := conf.SearchDomains[0]
if !strings.HasPrefix(string(first), namespace+".svc") {
logger.Infof("[unexpected] first search domain in resolver config is %s; expected %s.\nDefaulting cluster domain to 'cluster.local'.", first, namespace+".svc.<cluster-domain>")
return defaultClusterDomain
}
second := conf.SearchDomains[1]
if !strings.HasPrefix(string(second), "svc") {
logger.Infof("[unexpected] second search domain in resolver config is %s; expected 'svc.<cluster-domain>'.\nDefaulting cluster domain to 'cluster.local'.", second)
return defaultClusterDomain
}
// Trim the trailing dot for backwards compatibility purposes as the
// cluster domain was previously hardcoded to 'cluster.local' without a
// trailing dot.
probablyClusterDomain := strings.TrimPrefix(second.WithoutTrailingDot(), "svc.")
third := conf.SearchDomains[2]
if !strings.EqualFold(third.WithoutTrailingDot(), probablyClusterDomain) {
logger.Infof("[unexpected] expected resolver config to contain serch domains <namespace>.svc.<cluster-domain>, svc.<cluster-domain>, <cluster-domain>; got %s %s %s\n. Defaulting cluster domain to 'cluster.local'.", first, second, third)
return defaultClusterDomain
}
logger.Infof("Cluster domain %q extracted from resolver config", probablyClusterDomain)
return probablyClusterDomain
}

@ -15,11 +15,13 @@ import (
"time"
"github.com/google/go-cmp/cmp"
"go.uber.org/zap"
appsv1 "k8s.io/api/apps/v1"
corev1 "k8s.io/api/core/v1"
apierrors "k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/types"
"k8s.io/client-go/tools/record"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/reconcile"
"tailscale.com/client/tailscale"
@ -42,6 +44,7 @@ type configOpts struct {
tailnetTargetIP string
tailnetTargetFQDN string
clusterTargetIP string
clusterTargetDNS string
subnetRoutes string
isExitNode bool
confFileHash string
@ -52,6 +55,10 @@ type configOpts struct {
func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.StatefulSet {
t.Helper()
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
tsContainer := corev1.Container{
Name: "tailscale",
Image: "tailscale/tailscale",
@ -60,6 +67,7 @@ func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.Statef
{Name: "POD_IP", ValueFrom: &corev1.EnvVarSource{FieldRef: &corev1.ObjectFieldSelector{APIVersion: "", FieldPath: "status.podIP"}, ResourceFieldRef: nil, ConfigMapKeyRef: nil, SecretKeyRef: nil}},
{Name: "TS_KUBE_SECRET", Value: opts.secretName},
{Name: "EXPERIMENTAL_TS_CONFIGFILE_PATH", Value: "/etc/tsconfig/tailscaled"},
{Name: "TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR", Value: "/etc/tsconfig"},
},
SecurityContext: &corev1.SecurityContext{
Capabilities: &corev1.Capabilities{
@ -82,12 +90,6 @@ func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.Statef
VolumeSource: corev1.VolumeSource{
Secret: &corev1.SecretVolumeSource{
SecretName: opts.secretName,
Items: []corev1.KeyToPath{
{
Key: "tailscaled",
Path: "tailscaled",
},
},
},
},
},
@ -125,15 +127,19 @@ func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.Statef
Value: opts.clusterTargetIP,
})
annots["tailscale.com/operator-last-set-cluster-ip"] = opts.clusterTargetIP
} else if opts.clusterTargetDNS != "" {
tsContainer.Env = append(tsContainer.Env, corev1.EnvVar{
Name: "TS_EXPERIMENTAL_DEST_DNS_NAME",
Value: opts.clusterTargetDNS,
})
annots["tailscale.com/operator-last-set-cluster-dns-name"] = opts.clusterTargetDNS
}
if opts.serveConfig != nil {
tsContainer.Env = append(tsContainer.Env, corev1.EnvVar{
Name: "TS_SERVE_CONFIG",
Value: "/etc/tailscaled/serve-config",
})
volumes = append(volumes, corev1.Volume{
Name: "serve-config", VolumeSource: corev1.VolumeSource{Secret: &corev1.SecretVolumeSource{SecretName: opts.secretName, Items: []corev1.KeyToPath{{Path: "serve-config", Key: "serve-config"}}}},
})
volumes = append(volumes, corev1.Volume{Name: "serve-config", VolumeSource: corev1.VolumeSource{Secret: &corev1.SecretVolumeSource{SecretName: opts.secretName, Items: []corev1.KeyToPath{{Key: "serve-config", Path: "serve-config"}}}}})
tsContainer.VolumeMounts = append(tsContainer.VolumeMounts, corev1.VolumeMount{Name: "serve-config", ReadOnly: true, MountPath: "/etc/tailscaled"})
}
ss := &appsv1.StatefulSet{
@ -176,8 +182,8 @@ func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.Statef
{
Name: "sysctler",
Image: "tailscale/tailscale",
Command: []string{"/bin/sh"},
Args: []string{"-c", "sysctl -w net.ipv4.ip_forward=1 net.ipv6.conf.all.forwarding=1"},
Command: []string{"/bin/sh", "-c"},
Args: []string{"sysctl -w net.ipv4.ip_forward=1 && if sysctl net.ipv6.conf.all.forwarding; then sysctl -w net.ipv6.conf.all.forwarding=1; fi"},
SecurityContext: &corev1.SecurityContext{
Privileged: ptr.To(true),
},
@ -197,20 +203,26 @@ func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.Statef
if err := cl.Get(context.Background(), types.NamespacedName{Name: opts.proxyClass}, proxyClass); err != nil {
t.Fatalf("error getting ProxyClass: %v", err)
}
return applyProxyClassToStatefulSet(proxyClass, ss)
return applyProxyClassToStatefulSet(proxyClass, ss, new(tailscaleSTSConfig), zl.Sugar())
}
return ss
}
func expectedSTSUserspace(t *testing.T, cl client.Client, opts configOpts) *appsv1.StatefulSet {
t.Helper()
zl, err := zap.NewDevelopment()
if err != nil {
t.Fatal(err)
}
tsContainer := corev1.Container{
Name: "tailscale",
Image: "tailscale/tailscale",
Env: []corev1.EnvVar{
{Name: "TS_USERSPACE", Value: "true"},
{Name: "POD_IP", ValueFrom: &corev1.EnvVarSource{FieldRef: &corev1.ObjectFieldSelector{APIVersion: "", FieldPath: "status.podIP"}, ResourceFieldRef: nil, ConfigMapKeyRef: nil, SecretKeyRef: nil}},
{Name: "TS_KUBE_SECRET", Value: opts.secretName},
{Name: "EXPERIMENTAL_TS_CONFIGFILE_PATH", Value: "/etc/tsconfig/tailscaled"},
{Name: "TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR", Value: "/etc/tsconfig"},
{Name: "TS_SERVE_CONFIG", Value: "/etc/tailscaled/serve-config"},
},
ImagePullPolicy: "Always",
@ -225,20 +237,12 @@ func expectedSTSUserspace(t *testing.T, cl client.Client, opts configOpts) *apps
VolumeSource: corev1.VolumeSource{
Secret: &corev1.SecretVolumeSource{
SecretName: opts.secretName,
Items: []corev1.KeyToPath{
{
Key: "tailscaled",
Path: "tailscaled",
},
},
},
},
},
{Name: "serve-config",
VolumeSource: corev1.VolumeSource{
Secret: &corev1.SecretVolumeSource{SecretName: opts.secretName,
Items: []corev1.KeyToPath{{Key: "serve-config", Path: "serve-config"}}}},
},
Secret: &corev1.SecretVolumeSource{SecretName: opts.secretName, Items: []corev1.KeyToPath{{Key: "serve-config", Path: "serve-config"}}}}},
}
ss := &appsv1.StatefulSet{
TypeMeta: metav1.TypeMeta{
@ -293,17 +297,13 @@ func expectedSTSUserspace(t *testing.T, cl client.Client, opts configOpts) *apps
if err := cl.Get(context.Background(), types.NamespacedName{Name: opts.proxyClass}, proxyClass); err != nil {
t.Fatalf("error getting ProxyClass: %v", err)
}
return applyProxyClassToStatefulSet(proxyClass, ss)
return applyProxyClassToStatefulSet(proxyClass, ss, new(tailscaleSTSConfig), zl.Sugar())
}
return ss
}
func expectedHeadlessService(name string, parentType string) *corev1.Service {
return &corev1.Service{
TypeMeta: metav1.TypeMeta{
Kind: "Service",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: name,
GenerateName: "ts-test-",
@ -324,13 +324,9 @@ func expectedHeadlessService(name string, parentType string) *corev1.Service {
}
}
func expectedSecret(t *testing.T, opts configOpts) *corev1.Secret {
func expectedSecret(t *testing.T, cl client.Client, opts configOpts) *corev1.Secret {
t.Helper()
s := &corev1.Secret{
TypeMeta: metav1.TypeMeta{
Kind: "Secret",
APIVersion: "v1",
},
ObjectMeta: metav1.ObjectMeta{
Name: opts.secretName,
Namespace: "operator-ns",
@ -351,6 +347,16 @@ func expectedSecret(t *testing.T, opts configOpts) *corev1.Secret {
AuthKey: ptr.To("secret-authkey"),
AcceptRoutes: "false",
}
if opts.proxyClass != "" {
t.Logf("applying configuration from ProxyClass %s", opts.proxyClass)
proxyClass := new(tsapi.ProxyClass)
if err := cl.Get(context.Background(), types.NamespacedName{Name: opts.proxyClass}, proxyClass); err != nil {
t.Fatalf("error getting ProxyClass: %v", err)
}
if proxyClass.Spec.TailscaleConfig != nil && proxyClass.Spec.TailscaleConfig.AcceptRoutes {
conf.AcceptRoutes = "true"
}
}
var routes []netip.Prefix
if opts.subnetRoutes != "" || opts.isExitNode {
r := opts.subnetRoutes
@ -370,7 +376,17 @@ func expectedSecret(t *testing.T, opts configOpts) *corev1.Secret {
if err != nil {
t.Fatalf("error marshalling tailscaled config")
}
if opts.tailnetTargetFQDN != "" || opts.tailnetTargetIP != "" {
conf.NoStatefulFiltering = "true"
} else {
conf.NoStatefulFiltering = "false"
}
bn, err := json.Marshal(conf)
if err != nil {
t.Fatalf("error marshalling tailscaled config")
}
mak.Set(&s.StringData, "tailscaled", string(b))
mak.Set(&s.StringData, "cap-95.hujson", string(bn))
labels := map[string]string{
"tailscale.com/managed": "true",
"tailscale.com/parent-resource": "test",
@ -441,11 +457,11 @@ func mustUpdateStatus[T any, O ptrObject[T]](t *testing.T, client client.Client,
// expectEqual accepts a Kubernetes object and a Kubernetes client. It tests
// whether an object with equivalent contents can be retrieved by the passed
// client. If you want to NOT test some object fields for equality, ensure that
// they are not present in the passed object and use the modify func to remove
// them from the cluster object. If no such modifications are needed, you can
// pass nil in place of the modify function.
func expectEqual[T any, O ptrObject[T]](t *testing.T, client client.Client, want O, modify func(O)) {
// client. If you want to NOT test some object fields for equality, use the
// modify func to ensure that they are removed from the cluster object and the
// object passed as 'want'. If no such modifications are needed, you can pass
// nil in place of the modify function.
func expectEqual[T any, O ptrObject[T]](t *testing.T, client client.Client, want O, modifier func(O)) {
t.Helper()
got := O(new(T))
if err := client.Get(context.Background(), types.NamespacedName{
@ -459,8 +475,9 @@ func expectEqual[T any, O ptrObject[T]](t *testing.T, client client.Client, want
// so just remove it from both got and want.
got.SetResourceVersion("")
want.SetResourceVersion("")
if modify != nil {
modify(got)
if modifier != nil {
modifier(want)
modifier(got)
}
if diff := cmp.Diff(got, want); diff != "" {
t.Fatalf("unexpected object (-got +want):\n%s", diff)
@ -515,6 +532,34 @@ func expectRequeue(t *testing.T, sr reconcile.Reconciler, ns, name string) {
}
}
// expectEvents accepts a test recorder and a list of events, tests that expected
// events are sent down the recorder's channel. Waits for 5s for each event.
func expectEvents(t *testing.T, rec *record.FakeRecorder, wantsEvents []string) {
t.Helper()
// Events are not expected to arrive in order.
seenEvents := make([]string, 0)
for range len(wantsEvents) {
timer := time.NewTimer(time.Second * 5)
defer timer.Stop()
select {
case gotEvent := <-rec.Events:
found := false
for _, wantEvent := range wantsEvents {
if wantEvent == gotEvent {
found = true
seenEvents = append(seenEvents, gotEvent)
break
}
}
if !found {
t.Errorf("got unexpected event %q, expected events: %+#v", gotEvent, wantsEvents)
}
case <-timer.C:
t.Errorf("timeout waiting for an event, wants events %#+v, got events %+#v", wantsEvents, seenEvents)
}
}
}
type fakeTSClient struct {
sync.Mutex
keyRequests []tailscale.KeyCapabilities
@ -566,3 +611,33 @@ func (c *fakeTSClient) Deleted() []string {
func removeHashAnnotation(sts *appsv1.StatefulSet) {
delete(sts.Spec.Template.Annotations, podAnnotationLastSetConfigFileHash)
}
func removeAuthKeyIfExistsModifier(t *testing.T) func(s *corev1.Secret) {
return func(secret *corev1.Secret) {
t.Helper()
if len(secret.StringData["tailscaled"]) != 0 {
conf := &ipn.ConfigVAlpha{}
if err := json.Unmarshal([]byte(secret.StringData["tailscaled"]), conf); err != nil {
t.Fatalf("error unmarshalling 'tailscaled' contents: %v", err)
}
conf.AuthKey = nil
b, err := json.Marshal(conf)
if err != nil {
t.Fatalf("error marshalling updated 'tailscaled' config: %v", err)
}
mak.Set(&secret.StringData, "tailscaled", string(b))
}
if len(secret.StringData["cap-95.hujson"]) != 0 {
conf := &ipn.ConfigVAlpha{}
if err := json.Unmarshal([]byte(secret.StringData["cap-95.hujson"]), conf); err != nil {
t.Fatalf("error umarshalling 'cap-95.hujson' contents: %v", err)
}
conf.AuthKey = nil
b, err := json.Marshal(conf)
if err != nil {
t.Fatalf("error marshalling 'cap-95.huson' contents: %v", err)
}
mak.Set(&secret.StringData, "cap-95.hujson", string(b))
}
}
}

@ -0,0 +1,567 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
// The natc command is a work-in-progress implementation of a NAT based
// connector for Tailscale. It is intended to be used to route traffic to a
// specific domain through a specific node.
package main
import (
"context"
"encoding/binary"
"errors"
"flag"
"fmt"
"log"
"math/rand/v2"
"net"
"net/http"
"net/netip"
"os"
"strings"
"sync"
"time"
"github.com/gaissmai/bart"
"github.com/inetaf/tcpproxy"
"github.com/peterbourgon/ff/v3"
"golang.org/x/net/dns/dnsmessage"
"tailscale.com/client/tailscale"
"tailscale.com/envknob"
"tailscale.com/hostinfo"
"tailscale.com/ipn"
"tailscale.com/net/netutil"
"tailscale.com/syncs"
"tailscale.com/tailcfg"
"tailscale.com/tsnet"
"tailscale.com/tsweb"
"tailscale.com/util/dnsname"
"tailscale.com/util/mak"
)
func main() {
hostinfo.SetApp("natc")
if !envknob.UseWIPCode() {
log.Fatal("cmd/natc is a work in progress and has not been security reviewed;\nits use requires TAILSCALE_USE_WIP_CODE=1 be set in the environment for now.")
}
// Parse flags
fs := flag.NewFlagSet("natc", flag.ExitOnError)
var (
debugPort = fs.Int("debug-port", 8893, "Listening port for debug/metrics endpoint")
hostname = fs.String("hostname", "", "Hostname to register the service under")
siteID = fs.Uint("site-id", 1, "an integer site ID to use for the ULA prefix which allows for multiple proxies to act in a HA configuration")
v4PfxStr = fs.String("v4-pfx", "100.64.1.0/24", "comma-separated list of IPv4 prefixes to advertise")
verboseTSNet = fs.Bool("verbose-tsnet", false, "enable verbose logging in tsnet")
printULA = fs.Bool("print-ula", false, "print the ULA prefix and exit")
ignoreDstPfxStr = fs.String("ignore-destinations", "", "comma-separated list of prefixes to ignore")
wgPort = fs.Uint("wg-port", 0, "udp port for wireguard and peer to peer traffic")
)
ff.Parse(fs, os.Args[1:], ff.WithEnvVarPrefix("TS_NATC"))
if *printULA {
fmt.Println(ula(uint16(*siteID)))
return
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if *siteID == 0 {
log.Fatalf("site-id must be set")
} else if *siteID > 0xffff {
log.Fatalf("site-id must be in the range [0, 65535]")
}
var ignoreDstTable *bart.Table[bool]
for _, s := range strings.Split(*ignoreDstPfxStr, ",") {
s := strings.TrimSpace(s)
if s == "" {
continue
}
if ignoreDstTable == nil {
ignoreDstTable = &bart.Table[bool]{}
}
pfx, err := netip.ParsePrefix(s)
if err != nil {
log.Fatalf("unable to parse prefix: %v", err)
}
if pfx.Masked() != pfx {
log.Fatalf("prefix %v is not normalized (bits are set outside the mask)", pfx)
}
ignoreDstTable.Insert(pfx, true)
}
var v4Prefixes []netip.Prefix
for _, s := range strings.Split(*v4PfxStr, ",") {
p := netip.MustParsePrefix(strings.TrimSpace(s))
if p.Masked() != p {
log.Fatalf("v4 prefix %v is not a masked prefix", p)
}
v4Prefixes = append(v4Prefixes, p)
}
if len(v4Prefixes) == 0 {
log.Fatalf("no v4 prefixes specified")
}
dnsAddr := v4Prefixes[0].Addr()
ts := &tsnet.Server{
Hostname: *hostname,
}
if *wgPort != 0 {
if *wgPort >= 1<<16 {
log.Fatalf("wg-port must be in the range [0, 65535]")
}
ts.Port = uint16(*wgPort)
}
defer ts.Close()
if *verboseTSNet {
ts.Logf = log.Printf
}
// Start special-purpose listeners: dns, http promotion, debug server
if *debugPort != 0 {
mux := http.NewServeMux()
tsweb.Debugger(mux)
dln, err := ts.Listen("tcp", fmt.Sprintf(":%d", *debugPort))
if err != nil {
log.Fatalf("failed listening on debug port: %v", err)
}
defer dln.Close()
go func() {
log.Fatalf("debug serve: %v", http.Serve(dln, mux))
}()
}
lc, err := ts.LocalClient()
if err != nil {
log.Fatalf("LocalClient() failed: %v", err)
}
if _, err := ts.Up(ctx); err != nil {
log.Fatalf("ts.Up: %v", err)
}
c := &connector{
ts: ts,
lc: lc,
dnsAddr: dnsAddr,
v4Ranges: v4Prefixes,
v6ULA: ula(uint16(*siteID)),
ignoreDsts: ignoreDstTable,
}
c.run(ctx)
}
type connector struct {
// ts is the tsnet.Server used to host the connector.
ts *tsnet.Server
// lc is the LocalClient used to interact with the tsnet.Server hosting this
// connector.
lc *tailscale.LocalClient
// dnsAddr is the IPv4 address to listen on for DNS requests. It is used to
// prevent the app connector from assigning it to a domain.
dnsAddr netip.Addr
// v4Ranges is the list of IPv4 ranges to advertise and assign addresses from.
// These are masked prefixes.
v4Ranges []netip.Prefix
// v6ULA is the ULA prefix used by the app connector to assign IPv6 addresses.
v6ULA netip.Prefix
perPeerMap syncs.Map[tailcfg.NodeID, *perPeerState]
// ignoreDsts is initialized at start up with the contents of --ignore-destinations (if none it is nil)
// It is never mutated, only used for lookups.
// Users who want to natc a DNS wildcard but not every address record in that domain can supply the
// exceptions in --ignore-destinations. When we receive a dns request we will look up the fqdn
// and if any of the ip addresses in response to the lookup match any 'ignore destinations' prefix we will
// return a dns response that contains the ip addresses we discovered with the lookup (ie not the
// natc behavior, which would return a dummy ip address pointing at natc).
ignoreDsts *bart.Table[bool]
}
// v6ULA is the ULA prefix used by the app connector to assign IPv6 addresses.
// The 8th and 9th bytes are used to encode the site ID which allows for
// multiple proxies to act in a HA configuration.
// mnemonic: a99c = appc
var v6ULA = netip.MustParsePrefix("fd7a:115c:a1e0:a99c::/64")
func ula(siteID uint16) netip.Prefix {
as16 := v6ULA.Addr().As16()
as16[8] = byte(siteID >> 8)
as16[9] = byte(siteID)
return netip.PrefixFrom(netip.AddrFrom16(as16), 64+16)
}
// run runs the connector.
//
// The passed in context is only used for the initial setup. The connector runs
// forever.
func (c *connector) run(ctx context.Context) {
if _, err := c.lc.EditPrefs(ctx, &ipn.MaskedPrefs{
AdvertiseRoutesSet: true,
Prefs: ipn.Prefs{
AdvertiseRoutes: append(c.v4Ranges, c.v6ULA),
},
}); err != nil {
log.Fatalf("failed to advertise routes: %v", err)
}
c.ts.RegisterFallbackTCPHandler(c.handleTCPFlow)
c.serveDNS()
}
func (c *connector) serveDNS() {
pc, err := c.ts.ListenPacket("udp", net.JoinHostPort(c.dnsAddr.String(), "53"))
if err != nil {
log.Fatalf("failed listening on port 53: %v", err)
}
defer pc.Close()
log.Printf("Listening for DNS on %s", pc.LocalAddr().String())
for {
buf := make([]byte, 1500)
n, addr, err := pc.ReadFrom(buf)
if err != nil {
if errors.Is(err, net.ErrClosed) {
return
}
log.Printf("serveDNS.ReadFrom failed: %v", err)
continue
}
go c.handleDNS(pc, buf[:n], addr.(*net.UDPAddr))
}
}
func lookupDestinationIP(domain string) ([]netip.Addr, error) {
netIPs, err := net.LookupIP(domain)
if err != nil {
var dnsError *net.DNSError
if errors.As(err, &dnsError) && dnsError.IsNotFound {
return nil, nil
} else {
return nil, err
}
}
var addrs []netip.Addr
for _, ip := range netIPs {
a, ok := netip.AddrFromSlice(ip)
if ok {
addrs = append(addrs, a)
}
}
return addrs, nil
}
// handleDNS handles a DNS request to the app connector.
// It generates a response based on the request and the node that sent it.
//
// Each node is assigned a unique pair of IP addresses for each domain it
// queries. This assignment is done lazily and is not persisted across restarts.
// A per-peer assignment allows the connector to reuse a limited number of IP
// addresses across multiple nodes and domains. It also allows for clear
// failover behavior when an app connector is restarted.
//
// This assignment later allows the connector to determine where to forward
// traffic based on the destination IP address.
func (c *connector) handleDNS(pc net.PacketConn, buf []byte, remoteAddr *net.UDPAddr) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
who, err := c.lc.WhoIs(ctx, remoteAddr.String())
if err != nil {
log.Printf("HandleDNS: WhoIs failed: %v\n", err)
return
}
var msg dnsmessage.Message
err = msg.Unpack(buf)
if err != nil {
log.Printf("HandleDNS: dnsmessage unpack failed: %v\n ", err)
return
}
// If there are destination ips that we don't want to route, we
// have to do a dns lookup here to find the destination ip.
if c.ignoreDsts != nil {
if len(msg.Questions) > 0 {
q := msg.Questions[0]
switch q.Type {
case dnsmessage.TypeAAAA, dnsmessage.TypeA:
dstAddrs, err := lookupDestinationIP(q.Name.String())
if err != nil {
log.Printf("HandleDNS: lookup destination failed: %v\n ", err)
return
}
if c.ignoreDestination(dstAddrs) {
bs, err := dnsResponse(&msg, dstAddrs)
// TODO (fran): treat as SERVFAIL
if err != nil {
log.Printf("HandleDNS: generate ignore response failed: %v\n", err)
return
}
_, err = pc.WriteTo(bs, remoteAddr)
if err != nil {
log.Printf("HandleDNS: write failed: %v\n", err)
}
return
}
}
}
}
// None of the destination IP addresses match an ignore destination prefix, do
// the natc thing.
resp, err := c.generateDNSResponse(&msg, who.Node.ID)
// TODO (fran): treat as SERVFAIL
if err != nil {
log.Printf("HandleDNS: connector handling failed: %v\n", err)
return
}
// TODO (fran): treat as NXDOMAIN
if len(resp) == 0 {
return
}
// This connector handled the DNS request
_, err = pc.WriteTo(resp, remoteAddr)
if err != nil {
log.Printf("HandleDNS: write failed: %v\n", err)
}
}
// tsMBox is the mailbox used in SOA records.
// The convention is to replace the @ symbol with a dot.
// So in this case, the mailbox is support.tailscale.com. with the trailing dot
// to indicate that it is a fully qualified domain name.
var tsMBox = dnsmessage.MustNewName("support.tailscale.com.")
// generateDNSResponse generates a DNS response for the given request. The from
// argument is the NodeID of the node that sent the request.
func (c *connector) generateDNSResponse(req *dnsmessage.Message, from tailcfg.NodeID) ([]byte, error) {
pm, _ := c.perPeerMap.LoadOrStore(from, &perPeerState{c: c})
var addrs []netip.Addr
if len(req.Questions) > 0 {
switch req.Questions[0].Type {
case dnsmessage.TypeAAAA, dnsmessage.TypeA:
var err error
addrs, err = pm.ipForDomain(req.Questions[0].Name.String())
if err != nil {
return nil, err
}
}
}
return dnsResponse(req, addrs)
}
// dnsResponse makes a DNS response for the natc. If the dnsmessage is requesting TypeAAAA
// or TypeA the provided addrs of the requested type will be used.
func dnsResponse(req *dnsmessage.Message, addrs []netip.Addr) ([]byte, error) {
b := dnsmessage.NewBuilder(nil,
dnsmessage.Header{
ID: req.Header.ID,
Response: true,
Authoritative: true,
})
b.EnableCompression()
if len(req.Questions) == 0 {
return b.Finish()
}
q := req.Questions[0]
if err := b.StartQuestions(); err != nil {
return nil, err
}
if err := b.Question(q); err != nil {
return nil, err
}
if err := b.StartAnswers(); err != nil {
return nil, err
}
switch q.Type {
case dnsmessage.TypeAAAA, dnsmessage.TypeA:
want6 := q.Type == dnsmessage.TypeAAAA
for _, ip := range addrs {
if want6 != ip.Is6() {
continue
}
if want6 {
if err := b.AAAAResource(
dnsmessage.ResourceHeader{Name: q.Name, Class: q.Class, TTL: 5},
dnsmessage.AAAAResource{AAAA: ip.As16()},
); err != nil {
return nil, err
}
} else {
if err := b.AResource(
dnsmessage.ResourceHeader{Name: q.Name, Class: q.Class, TTL: 5},
dnsmessage.AResource{A: ip.As4()},
); err != nil {
return nil, err
}
}
}
case dnsmessage.TypeSOA:
if err := b.SOAResource(
dnsmessage.ResourceHeader{Name: q.Name, Class: q.Class, TTL: 120},
dnsmessage.SOAResource{NS: q.Name, MBox: tsMBox, Serial: 2023030600,
Refresh: 120, Retry: 120, Expire: 120, MinTTL: 60},
); err != nil {
return nil, err
}
case dnsmessage.TypeNS:
if err := b.NSResource(
dnsmessage.ResourceHeader{Name: q.Name, Class: q.Class, TTL: 120},
dnsmessage.NSResource{NS: tsMBox},
); err != nil {
return nil, err
}
}
return b.Finish()
}
// handleTCPFlow handles a TCP flow from the given source to the given
// destination. It uses the source address to determine the node that sent the
// request and the destination address to determine the domain that the request
// is for based on the IP address assigned to the destination in the DNS
// response.
func (c *connector) handleTCPFlow(src, dst netip.AddrPort) (handler func(net.Conn), intercept bool) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
who, err := c.lc.WhoIs(ctx, src.Addr().String())
cancel()
if err != nil {
log.Printf("HandleTCPFlow: WhoIs failed: %v\n", err)
return nil, false
}
from := who.Node.ID
ps, ok := c.perPeerMap.Load(from)
if !ok {
log.Printf("handleTCPFlow: no perPeerState for %v", from)
return nil, false
}
domain, ok := ps.domainForIP(dst.Addr())
if !ok {
log.Printf("handleTCPFlow: no domain for IP %v\n", dst.Addr())
return nil, false
}
return func(conn net.Conn) {
proxyTCPConn(conn, domain)
}, true
}
// ignoreDestination reports whether any of the provided dstAddrs match the prefixes configured
// in --ignore-destinations
func (c *connector) ignoreDestination(dstAddrs []netip.Addr) bool {
for _, a := range dstAddrs {
if _, ok := c.ignoreDsts.Get(a); ok {
return true
}
}
return false
}
func proxyTCPConn(c net.Conn, dest string) {
addrPortStr := c.LocalAddr().String()
_, port, err := net.SplitHostPort(addrPortStr)
if err != nil {
log.Printf("tcpRoundRobinHandler.Handle: bogus addrPort %q", addrPortStr)
c.Close()
return
}
p := &tcpproxy.Proxy{
ListenFunc: func(net, laddr string) (net.Listener, error) {
return netutil.NewOneConnListener(c, nil), nil
},
}
p.AddRoute(addrPortStr, &tcpproxy.DialProxy{
Addr: fmt.Sprintf("%s:%s", dest, port),
})
p.Start()
}
// perPeerState holds the state for a single peer.
type perPeerState struct {
c *connector
mu sync.Mutex
domainToAddr map[string][]netip.Addr
addrToDomain *bart.Table[string]
}
// domainForIP returns the domain name assigned to the given IP address and
// whether it was found.
func (ps *perPeerState) domainForIP(ip netip.Addr) (_ string, ok bool) {
ps.mu.Lock()
defer ps.mu.Unlock()
return ps.addrToDomain.Get(ip)
}
// ipForDomain assigns a pair of unique IP addresses for the given domain and
// returns them. The first address is an IPv4 address and the second is an IPv6
// address. If the domain already has assigned addresses, it returns them.
func (ps *perPeerState) ipForDomain(domain string) ([]netip.Addr, error) {
fqdn, err := dnsname.ToFQDN(domain)
if err != nil {
return nil, err
}
domain = fqdn.WithoutTrailingDot()
ps.mu.Lock()
defer ps.mu.Unlock()
if addrs, ok := ps.domainToAddr[domain]; ok {
return addrs, nil
}
addrs := ps.assignAddrsLocked(domain)
return addrs, nil
}
// isIPUsedLocked reports whether the given IP address is already assigned to a
// domain.
// ps.mu must be held.
func (ps *perPeerState) isIPUsedLocked(ip netip.Addr) bool {
_, ok := ps.addrToDomain.Get(ip)
return ok
}
// unusedIPv4Locked returns an unused IPv4 address from the available ranges.
func (ps *perPeerState) unusedIPv4Locked() netip.Addr {
// TODO: skip ranges that have been exhausted
for _, r := range ps.c.v4Ranges {
ip := randV4(r)
for r.Contains(ip) {
if !ps.isIPUsedLocked(ip) && ip != ps.c.dnsAddr {
return ip
}
ip = ip.Next()
}
}
return netip.Addr{}
}
// randV4 returns a random IPv4 address within the given prefix.
func randV4(maskedPfx netip.Prefix) netip.Addr {
bits := 32 - maskedPfx.Bits()
randBits := rand.Uint32N(1 << uint(bits))
ip4 := maskedPfx.Addr().As4()
pn := binary.BigEndian.Uint32(ip4[:])
binary.BigEndian.PutUint32(ip4[:], randBits|pn)
return netip.AddrFrom4(ip4)
}
// assignAddrsLocked assigns a pair of unique IP addresses for the given domain
// and returns them. The first address is an IPv4 address and the second is an
// IPv6 address. It does not check if the domain already has assigned addresses.
// ps.mu must be held.
func (ps *perPeerState) assignAddrsLocked(domain string) []netip.Addr {
if ps.addrToDomain == nil {
ps.addrToDomain = &bart.Table[string]{}
}
v4 := ps.unusedIPv4Locked()
as16 := ps.c.v6ULA.Addr().As16()
as4 := v4.As4()
copy(as16[12:], as4[:])
v6 := netip.AddrFrom16(as16)
addrs := []netip.Addr{v4, v6}
mak.Set(&ps.domainToAddr, domain, addrs)
for _, a := range addrs {
ps.addrToDomain.Insert(netip.PrefixFrom(a, a.BitLen()), domain)
}
return addrs
}

@ -314,7 +314,7 @@ func mustMakeNamesByAddr() map[netip.Addr]string {
seen := make(map[string]bool)
namesByAddr := make(map[netip.Addr]string)
retry:
for i := 0; i < 10; i++ {
for i := range 10 {
clear(seen)
clear(namesByAddr)
for _, d := range m.Devices {
@ -354,7 +354,7 @@ func fieldPrefix(s string, n int) string {
}
func appendRepeatByte(b []byte, c byte, n int) []byte {
for i := 0; i < n; i++ {
for range n {
b = append(b, c)
}
return b

@ -28,7 +28,6 @@ import (
"tailscale.com/metrics"
"tailscale.com/tsnet"
"tailscale.com/tsweb"
"tailscale.com/types/logger"
)
var (
@ -58,8 +57,6 @@ func main() {
ts := &tsnet.Server{
Dir: *tailscaleDir,
Hostname: *hostname,
// Make the stdout logs a clean audit log of connections.
Logf: logger.Discard,
}
if os.Getenv("TS_AUTHKEY") == "" {

@ -46,6 +46,7 @@ var (
backendAddr = flag.String("backend-addr", "", "Address of the Grafana server served over HTTP, in host:port format. Typically localhost:nnnn.")
tailscaleDir = flag.String("state-dir", "./", "Alternate directory to use for Tailscale state storage. If empty, a default is used.")
useHTTPS = flag.Bool("use-https", false, "Serve over HTTPS via your *.ts.net subdomain if enabled in Tailscale admin.")
loginServer = flag.String("login-server", "", "URL to alternative control server. If empty, the default Tailscale control is used.")
)
func main() {
@ -59,6 +60,7 @@ func main() {
ts := &tsnet.Server{
Dir: *tailscaleDir,
Hostname: *hostname,
ControlURL: *loginServer,
}
// TODO(bradfitz,maisem): move this to a method on tsnet.Server probably.
@ -88,7 +90,7 @@ func main() {
go func() {
// wait for tailscale to start before trying to fetch cert names
for i := 0; i < 60; i++ {
for range 60 {
st, err := localClient.Status(context.Background())
if err != nil {
log.Printf("error retrieving tailscale status; retrying: %v", err)

@ -7,7 +7,7 @@ import (
"context"
"fmt"
"log"
"math/rand"
"math/rand/v2"
"net"
"net/netip"
"slices"
@ -47,7 +47,7 @@ func (h *tcpRoundRobinHandler) Handle(c net.Conn) {
return netutil.NewOneConnListener(c, nil), nil
}
dest := h.To[rand.Intn(len(h.To))]
dest := h.To[rand.IntN(len(h.To))]
dial := &tcpproxy.DialProxy{
Addr: fmt.Sprintf("%s:%s", dest, port),
DialContext: h.DialContext,

@ -8,6 +8,7 @@ import (
"encoding/json"
"flag"
"fmt"
"log"
"net"
"net/http/httptest"
"net/netip"
@ -24,6 +25,7 @@ import (
"tailscale.com/tsnet"
"tailscale.com/tstest/integration"
"tailscale.com/tstest/integration/testcontrol"
"tailscale.com/tstest/nettest"
"tailscale.com/types/appctype"
"tailscale.com/types/ipproto"
"tailscale.com/types/key"
@ -98,8 +100,8 @@ func startNode(t *testing.T, ctx context.Context, controlURL, hostname string) (
Store: new(mem.Store),
Ephemeral: true,
}
if !*verboseNodes {
s.Logf = logger.Discard
if *verboseNodes {
s.Logf = log.Printf
}
t.Cleanup(func() { s.Close() })
@ -111,6 +113,7 @@ func startNode(t *testing.T, ctx context.Context, controlURL, hostname string) (
}
func TestSNIProxyWithNetmapConfig(t *testing.T) {
nettest.SkipIfNoNetwork(t)
c, controlURL := startControl(t)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
@ -158,7 +161,7 @@ func TestSNIProxyWithNetmapConfig(t *testing.T) {
t.Fatal(err)
}
gotConfigured := false
for i := 0; i < 100; i++ {
for range 100 {
s, err := l.StatusWithoutPeers(ctx)
if err != nil {
t.Fatal(err)
@ -189,6 +192,7 @@ func TestSNIProxyWithNetmapConfig(t *testing.T) {
}
func TestSNIProxyWithFlagConfig(t *testing.T) {
nettest.SkipIfNoNetwork(t)
_, controlURL := startControl(t)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()

@ -8,6 +8,7 @@ import (
"log"
"net"
"os"
"strconv"
"tailscale.com/net/stun"
)
@ -15,12 +16,20 @@ import (
func main() {
log.SetFlags(0)
if len(os.Args) != 2 {
log.Fatalf("usage: %s <hostname>", os.Args[0])
if len(os.Args) < 2 || len(os.Args) > 3 {
log.Fatalf("usage: %s <hostname> [port]", os.Args[0])
}
host := os.Args[1]
port := "3478"
if len(os.Args) == 3 {
port = os.Args[2]
}
_, err := strconv.ParseUint(port, 10, 16)
if err != nil {
log.Fatalf("invalid port: %v", err)
}
uaddr, err := net.ResolveUDPAddr("udp", host+":3478")
uaddr, err := net.ResolveUDPAddr("udp", net.JoinHostPort(host, port))
if err != nil {
log.Fatal(err)
}

@ -20,6 +20,7 @@ tailscale.com/cmd/stund dependencies: (generated by github.com/tailscale/depawar
google.golang.org/protobuf/internal/descfmt from google.golang.org/protobuf/internal/filedesc
google.golang.org/protobuf/internal/descopts from google.golang.org/protobuf/internal/filedesc+
google.golang.org/protobuf/internal/detrand from google.golang.org/protobuf/internal/descfmt+
google.golang.org/protobuf/internal/editiondefaults from google.golang.org/protobuf/internal/filedesc
google.golang.org/protobuf/internal/encoding/defval from google.golang.org/protobuf/internal/encoding/tag+
google.golang.org/protobuf/internal/encoding/messageset from google.golang.org/protobuf/encoding/prototext+
google.golang.org/protobuf/internal/encoding/tag from google.golang.org/protobuf/internal/impl
@ -152,7 +153,7 @@ tailscale.com/cmd/stund dependencies: (generated by github.com/tailscale/depawar
math/big from crypto/dsa+
math/bits from compress/flate+
math/rand from math/big+
math/rand/v2 from tailscale.com/util/fastuuid
math/rand/v2 from tailscale.com/util/fastuuid+
mime from github.com/prometheus/common/expfmt+
mime/multipart from net/http
mime/quotedprintable from mime/multipart

@ -0,0 +1,142 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
package main
import (
"compress/gzip"
"encoding/json"
"errors"
"net/http"
"net/url"
"strconv"
"strings"
"time"
sq "github.com/Masterminds/squirrel"
)
type api struct {
db *db
mux *http.ServeMux
}
func newAPI(db *db) *api {
a := &api{
db: db,
}
mux := http.NewServeMux()
mux.HandleFunc("/query", a.query)
a.mux = mux
return a
}
type apiResult struct {
At int `json:"at"` // time.Time.Unix()
RegionID int `json:"regionID"`
Hostname string `json:"hostname"`
Af int `json:"af"` // 4 or 6
Addr string `json:"addr"`
Source int `json:"source"` // timestampSourceUserspace (0) or timestampSourceKernel (1)
StableConn bool `json:"stableConn"`
DstPort int `json:"dstPort"`
RttNS *int `json:"rttNS"`
}
func getTimeBounds(vals url.Values) (from time.Time, to time.Time, err error) {
lastForm, ok := vals["last"]
if ok && len(lastForm) > 0 {
dur, err := time.ParseDuration(lastForm[0])
if err != nil {
return time.Time{}, time.Time{}, err
}
now := time.Now()
return now.Add(-dur), now, nil
}
fromForm, ok := vals["from"]
if ok && len(fromForm) > 0 {
fromUnixSec, err := strconv.Atoi(fromForm[0])
if err != nil {
return time.Time{}, time.Time{}, err
}
from = time.Unix(int64(fromUnixSec), 0)
toForm, ok := vals["to"]
if ok && len(toForm) > 0 {
toUnixSec, err := strconv.Atoi(toForm[0])
if err != nil {
return time.Time{}, time.Time{}, err
}
to = time.Unix(int64(toUnixSec), 0)
} else {
return time.Time{}, time.Time{}, errors.New("from specified without to")
}
return from, to, nil
}
// no time bounds specified, default to last 1h
now := time.Now()
return now.Add(-time.Hour), now, nil
}
func (a *api) ServeHTTP(w http.ResponseWriter, r *http.Request) {
a.mux.ServeHTTP(w, r)
}
func (a *api) query(w http.ResponseWriter, r *http.Request) {
err := r.ParseForm()
if err != nil {
http.Error(w, err.Error(), 500)
return
}
from, to, err := getTimeBounds(r.Form)
if err != nil {
http.Error(w, err.Error(), 500)
return
}
sb := sq.Select("at_unix", "region_id", "hostname", "af", "address", "timestamp_source", "stable_conn", "dst_port", "rtt_ns").From("rtt")
sb = sb.Where(sq.And{
sq.GtOrEq{"at_unix": from.Unix()},
sq.LtOrEq{"at_unix": to.Unix()},
})
query, args, err := sb.ToSql()
if err != nil {
return
}
rows, err := a.db.Query(query, args...)
if err != nil {
http.Error(w, err.Error(), 500)
return
}
results := make([]apiResult, 0)
for rows.Next() {
rtt := 0
result := apiResult{
RttNS: &rtt,
}
err = rows.Scan(&result.At, &result.RegionID, &result.Hostname, &result.Af, &result.Addr, &result.Source, &result.StableConn, &result.DstPort, &result.RttNS)
if err != nil {
http.Error(w, err.Error(), 500)
return
}
results = append(results, result)
}
if rows.Err() != nil {
http.Error(w, rows.Err().Error(), 500)
return
}
if strings.Contains(r.Header.Get("Accept-Encoding"), "gzip") {
gz := gzip.NewWriter(w)
defer gz.Close()
w.Header().Set("Content-Encoding", "gzip")
err = json.NewEncoder(gz).Encode(&results)
} else {
err = json.NewEncoder(w).Encode(&results)
}
if err != nil {
http.Error(w, err.Error(), 500)
return
}
}

@ -0,0 +1,886 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
// The stunstamp binary measures STUN round-trip latency with DERPs.
package main
import (
"bytes"
"cmp"
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"log"
"math"
"math/rand/v2"
"net"
"net/http"
"net/netip"
"net/url"
"os"
"os/signal"
"slices"
"strconv"
"strings"
"sync"
"syscall"
"time"
"github.com/golang/snappy"
"github.com/prometheus/prometheus/prompb"
"tailscale.com/logtail/backoff"
"tailscale.com/net/stun"
"tailscale.com/tailcfg"
)
var (
flagDERPMap = flag.String("derp-map", "https://login.tailscale.com/derpmap/default", "URL to DERP map")
flagOut = flag.String("out", "", "output sqlite filename")
flagInterval = flag.Duration("interval", time.Minute, "interval to probe at in time.ParseDuration() format")
flagAPI = flag.String("api", "", "listen addr for HTTP API")
flagIPv6 = flag.Bool("ipv6", false, "probe IPv6 addresses")
flagRetention = flag.Duration("retention", time.Hour*24*7, "sqlite retention period in time.ParseDuration() format")
flagRemoteWriteURL = flag.String("rw-url", "", "prometheus remote write URL")
flagInstance = flag.String("instance", "", "instance label value; defaults to hostname if unspecified")
flagDstPorts = flag.String("dst-ports", "", "comma-separated list of destination ports to monitor")
)
const (
minInterval = time.Second
maxBufferDuration = time.Hour
)
func getDERPMap(ctx context.Context, url string) (*tailcfg.DERPMap, error) {
req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
if err != nil {
return nil, err
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
dm := tailcfg.DERPMap{}
err = json.NewDecoder(resp.Body).Decode(&dm)
if err != nil {
return nil, nil
}
return &dm, nil
}
type timestampSource int
const (
timestampSourceUserspace timestampSource = iota
timestampSourceKernel
)
func (t timestampSource) String() string {
switch t {
case timestampSourceUserspace:
return "userspace"
case timestampSourceKernel:
return "kernel"
default:
return "unknown"
}
}
// resultKey contains the stable dimensions and their values for a given
// timeseries, i.e. not time and not rtt/timeout.
type resultKey struct {
meta nodeMeta
timestampSource timestampSource
connStability connStability
dstPort int
}
type result struct {
key resultKey
at time.Time
rtt *time.Duration // nil signifies failure, e.g. timeout
}
func measureRTT(conn io.ReadWriteCloser, dst *net.UDPAddr) (rtt time.Duration, err error) {
uconn, ok := conn.(*net.UDPConn)
if !ok {
return 0, fmt.Errorf("unexpected conn type: %T", conn)
}
err = uconn.SetReadDeadline(time.Now().Add(time.Second * 2))
if err != nil {
return 0, fmt.Errorf("error setting read deadline: %w", err)
}
txID := stun.NewTxID()
req := stun.Request(txID)
txAt := time.Now()
_, err = uconn.WriteToUDP(req, dst)
if err != nil {
return 0, fmt.Errorf("error writing to udp socket: %w", err)
}
b := make([]byte, 1460)
for {
n, err := uconn.Read(b)
rxAt := time.Now()
if err != nil {
return 0, fmt.Errorf("error reading from udp socket: %w", err)
}
gotTxID, _, err := stun.ParseResponse(b[:n])
if err != nil || gotTxID != txID {
continue
}
return rxAt.Sub(txAt), nil
}
}
func isTemporaryOrTimeoutErr(err error) bool {
if errors.Is(err, os.ErrDeadlineExceeded) || errors.Is(err, context.DeadlineExceeded) {
return true
}
if err, ok := err.(interface{ Temporary() bool }); ok {
return err.Temporary()
}
return false
}
type nodeMeta struct {
regionID int
regionCode string
hostname string
addr netip.Addr
}
type measureFn func(conn io.ReadWriteCloser, dst *net.UDPAddr) (rtt time.Duration, err error)
// probe measures STUN round trip time for the node described by meta over
// conn against dstPort. It may return a nil duration and nil error if the
// STUN request timed out. A non-nil error indicates an unrecoverable or
// non-temporary error.
func probe(meta nodeMeta, conn io.ReadWriteCloser, fn measureFn, dstPort int) (*time.Duration, error) {
ua := &net.UDPAddr{
IP: net.IP(meta.addr.AsSlice()),
Port: dstPort,
}
time.Sleep(rand.N(200 * time.Millisecond)) // jitter across tx
rtt, err := fn(conn, ua)
if err != nil {
if isTemporaryOrTimeoutErr(err) {
log.Printf("temp error measuring RTT to %s(%s): %v", meta.hostname, ua.String(), err)
return nil, nil
}
return nil, err
}
return &rtt, nil
}
// nodeMetaFromDERPMap parses the provided DERP map in order to update nodeMeta
// in the provided nodeMetaByAddr. It returns a slice of nodeMeta containing
// the nodes that are no longer seen in the DERP map, but were previously held
// in nodeMetaByAddr.
func nodeMetaFromDERPMap(dm *tailcfg.DERPMap, nodeMetaByAddr map[netip.Addr]nodeMeta, ipv6 bool) (stale []nodeMeta, err error) {
// Parse the new derp map before making any state changes in nodeMetaByAddr.
// If parse fails we just stick with the old state.
updated := make(map[netip.Addr]nodeMeta)
for regionID, region := range dm.Regions {
for _, node := range region.Nodes {
v4, err := netip.ParseAddr(node.IPv4)
if err != nil || !v4.Is4() {
return nil, fmt.Errorf("invalid ipv4 addr for node in derp map: %v", node.Name)
}
metas := make([]nodeMeta, 0, 2)
metas = append(metas, nodeMeta{
regionID: regionID,
regionCode: region.RegionCode,
hostname: node.HostName,
addr: v4,
})
if ipv6 {
v6, err := netip.ParseAddr(node.IPv6)
if err != nil || !v6.Is6() {
return nil, fmt.Errorf("invalid ipv6 addr for node in derp map: %v", node.Name)
}
metas = append(metas, metas[0])
metas[1].addr = v6
}
for _, meta := range metas {
updated[meta.addr] = meta
}
}
}
// Find nodeMeta that have changed
for addr, updatedMeta := range updated {
previousMeta, ok := nodeMetaByAddr[addr]
if ok {
if previousMeta == updatedMeta {
continue
}
stale = append(stale, previousMeta)
nodeMetaByAddr[addr] = updatedMeta
} else {
nodeMetaByAddr[addr] = updatedMeta
}
}
// Find nodeMeta that no longer exist
for addr, potentialStale := range nodeMetaByAddr {
_, ok := updated[addr]
if !ok {
stale = append(stale, potentialStale)
}
}
return stale, nil
}
func getStableConns(stableConns map[netip.Addr]map[int][2]io.ReadWriteCloser, addr netip.Addr, dstPort int) ([2]io.ReadWriteCloser, error) {
conns := [2]io.ReadWriteCloser{}
byDstPort, ok := stableConns[addr]
if ok {
conns, ok = byDstPort[dstPort]
if ok {
return conns, nil
}
}
if supportsKernelTS() {
kconn, err := getConnKernelTimestamp()
if err != nil {
return conns, err
}
conns[timestampSourceKernel] = kconn
}
uconn, err := net.ListenUDP("udp", &net.UDPAddr{})
if err != nil {
if supportsKernelTS() {
conns[timestampSourceKernel].Close()
}
return conns, err
}
conns[timestampSourceUserspace] = uconn
if byDstPort == nil {
byDstPort = make(map[int][2]io.ReadWriteCloser)
}
byDstPort[dstPort] = conns
stableConns[addr] = byDstPort
return conns, nil
}
// probeNodes measures the round-trip time for STUN binding requests against the
// DERP nodes described by nodeMetaByAddr while using/updating stableConns for
// UDP sockets that should be recycled across runs. It returns the results or
// an error if one occurs.
func probeNodes(nodeMetaByAddr map[netip.Addr]nodeMeta, stableConns map[netip.Addr]map[int][2]io.ReadWriteCloser, dstPorts []int) ([]result, error) {
wg := sync.WaitGroup{}
results := make([]result, 0)
resultsCh := make(chan result)
errCh := make(chan error)
doneCh := make(chan struct{})
numProbes := 0
at := time.Now()
addrsToProbe := make(map[netip.Addr]bool)
doProbe := func(conn io.ReadWriteCloser, meta nodeMeta, source timestampSource, dstPort int) {
defer wg.Done()
r := result{
key: resultKey{
meta: meta,
timestampSource: source,
dstPort: dstPort,
},
at: at,
}
if conn == nil {
var err error
if source == timestampSourceKernel {
conn, err = getConnKernelTimestamp()
} else {
conn, err = net.ListenUDP("udp", &net.UDPAddr{})
}
if err != nil {
select {
case <-doneCh:
return
case errCh <- err:
return
}
}
defer conn.Close()
} else {
r.key.connStability = stableConn
}
fn := measureRTT
if source == timestampSourceKernel {
fn = measureRTTKernel
}
rtt, err := probe(meta, conn, fn, dstPort)
if err != nil {
select {
case <-doneCh:
return
case errCh <- err:
return
}
}
r.rtt = rtt
select {
case <-doneCh:
case resultsCh <- r:
}
}
for _, meta := range nodeMetaByAddr {
addrsToProbe[meta.addr] = true
for _, port := range dstPorts {
stable, err := getStableConns(stableConns, meta.addr, port)
if err != nil {
close(doneCh)
wg.Wait()
return nil, err
}
wg.Add(2)
numProbes += 2
go doProbe(stable[timestampSourceUserspace], meta, timestampSourceUserspace, port)
go doProbe(nil, meta, timestampSourceUserspace, port)
if supportsKernelTS() {
wg.Add(2)
numProbes += 2
go doProbe(stable[timestampSourceKernel], meta, timestampSourceKernel, port)
go doProbe(nil, meta, timestampSourceKernel, port)
}
}
}
// cleanup conns we no longer need
for k, byDstPort := range stableConns {
if !addrsToProbe[k] {
for _, conns := range byDstPort {
if conns[timestampSourceKernel] != nil {
conns[timestampSourceKernel].Close()
}
conns[timestampSourceUserspace].Close()
delete(stableConns, k)
}
}
}
for {
select {
case err := <-errCh:
close(doneCh)
wg.Wait()
return nil, err
case result := <-resultsCh:
results = append(results, result)
if len(results) == numProbes {
return results, nil
}
}
}
}
type connStability bool
const (
unstableConn connStability = false
stableConn connStability = true
)
const (
rttMetricName = "stunstamp_derp_stun_rtt_ns"
timeoutsMetricName = "stunstamp_derp_stun_timeouts_total"
)
func timeSeriesLabels(metricName string, meta nodeMeta, instance string, source timestampSource, stability connStability, dstPort int) []prompb.Label {
addressFamily := "ipv4"
if meta.addr.Is6() {
addressFamily = "ipv6"
}
labels := make([]prompb.Label, 0)
labels = append(labels, prompb.Label{
Name: "job",
Value: "stunstamp-rw",
})
labels = append(labels, prompb.Label{
Name: "instance",
Value: instance,
})
labels = append(labels, prompb.Label{
Name: "region_id",
Value: fmt.Sprintf("%d", meta.regionID),
})
labels = append(labels, prompb.Label{
Name: "region_code",
Value: meta.regionCode,
})
labels = append(labels, prompb.Label{
Name: "address_family",
Value: addressFamily,
})
labels = append(labels, prompb.Label{
Name: "hostname",
Value: meta.hostname,
})
labels = append(labels, prompb.Label{
Name: "dst_port",
Value: strconv.Itoa(dstPort),
})
labels = append(labels, prompb.Label{
Name: "__name__",
Value: metricName,
})
labels = append(labels, prompb.Label{
Name: "timestamp_source",
Value: source.String(),
})
labels = append(labels, prompb.Label{
Name: "stable_conn",
Value: fmt.Sprintf("%v", stability),
})
slices.SortFunc(labels, func(a, b prompb.Label) int {
// prometheus remote-write spec requires lexicographically sorted label names
return cmp.Compare(a.Name, b.Name)
})
return labels
}
const (
// https://prometheus.io/docs/concepts/remote_write_spec/#stale-markers
staleNaN uint64 = 0x7ff0000000000002
)
func staleMarkersFromNodeMeta(stale []nodeMeta, instance string, dstPorts []int) []prompb.TimeSeries {
staleMarkers := make([]prompb.TimeSeries, 0)
now := time.Now()
for _, s := range stale {
for _, dstPort := range dstPorts {
samples := []prompb.Sample{
{
Timestamp: now.UnixMilli(),
Value: math.Float64frombits(staleNaN),
},
}
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(rttMetricName, s, instance, timestampSourceUserspace, unstableConn, dstPort),
Samples: samples,
})
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(rttMetricName, s, instance, timestampSourceUserspace, stableConn, dstPort),
Samples: samples,
})
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(timeoutsMetricName, s, instance, timestampSourceUserspace, unstableConn, dstPort),
Samples: samples,
})
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(timeoutsMetricName, s, instance, timestampSourceUserspace, stableConn, dstPort),
Samples: samples,
})
if supportsKernelTS() {
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(rttMetricName, s, instance, timestampSourceKernel, unstableConn, dstPort),
Samples: samples,
})
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(rttMetricName, s, instance, timestampSourceKernel, stableConn, dstPort),
Samples: samples,
})
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(timeoutsMetricName, s, instance, timestampSourceKernel, unstableConn, dstPort),
Samples: samples,
})
staleMarkers = append(staleMarkers, prompb.TimeSeries{
Labels: timeSeriesLabels(timeoutsMetricName, s, instance, timestampSourceKernel, stableConn, dstPort),
Samples: samples,
})
}
}
}
return staleMarkers
}
// resultsToPromTimeSeries returns a slice of prometheus TimeSeries for the
// provided results and instance. timeouts is updated based on results, i.e.
// all result.key's are added to timeouts if they do not exist, and removed
// from timeouts if they are not present in results.
func resultsToPromTimeSeries(results []result, instance string, timeouts map[resultKey]uint64) []prompb.TimeSeries {
all := make([]prompb.TimeSeries, 0, len(results)*2)
seenKeys := make(map[resultKey]bool)
for _, r := range results {
timeoutsCount := timeouts[r.key] // a non-existent key will return a zero val
seenKeys[r.key] = true
rttLabels := timeSeriesLabels(rttMetricName, r.key.meta, instance, r.key.timestampSource, r.key.connStability, r.key.dstPort)
rttSamples := make([]prompb.Sample, 1)
rttSamples[0].Timestamp = r.at.UnixMilli()
if r.rtt != nil {
rttSamples[0].Value = float64(*r.rtt)
} else {
rttSamples[0].Value = math.NaN()
timeoutsCount++
}
rttTS := prompb.TimeSeries{
Labels: rttLabels,
Samples: rttSamples,
}
all = append(all, rttTS)
timeouts[r.key] = timeoutsCount
timeoutsLabels := timeSeriesLabels(timeoutsMetricName, r.key.meta, instance, r.key.timestampSource, r.key.connStability, r.key.dstPort)
timeoutsSamples := make([]prompb.Sample, 1)
timeoutsSamples[0].Timestamp = r.at.UnixMilli()
timeoutsSamples[0].Value = float64(timeoutsCount)
timeoutsTS := prompb.TimeSeries{
Labels: timeoutsLabels,
Samples: timeoutsSamples,
}
all = append(all, timeoutsTS)
}
for k := range timeouts {
if !seenKeys[k] {
delete(timeouts, k)
}
}
return all
}
type remoteWriteClient struct {
c *http.Client
url string
}
type recoverableErr struct {
error
}
func newRemoteWriteClient(url string) *remoteWriteClient {
return &remoteWriteClient{
c: &http.Client{
Timeout: time.Second * 30,
},
url: url,
}
}
func (r *remoteWriteClient) write(ctx context.Context, ts []prompb.TimeSeries) error {
wr := &prompb.WriteRequest{
Timeseries: ts,
}
b, err := wr.Marshal()
if err != nil {
return fmt.Errorf("unable to marshal write request: %w", err)
}
compressed := snappy.Encode(nil, b)
req, err := http.NewRequestWithContext(ctx, "POST", r.url, bytes.NewReader(compressed))
if err != nil {
return fmt.Errorf("unable to create write request: %w", err)
}
req.Header.Add("Content-Encoding", "snappy")
req.Header.Set("Content-Type", "application/x-protobuf")
req.Header.Set("User-Agent", "stunstamp")
req.Header.Set("X-Prometheus-Remote-Write-Version", "0.1.0")
resp, err := r.c.Do(req)
if err != nil {
return recoverableErr{fmt.Errorf("error performing write request: %w", err)}
}
if resp.StatusCode/100 != 2 {
err = fmt.Errorf("remote server %s returned HTTP status %d", r.url, resp.StatusCode)
}
if resp.StatusCode/100 == 5 || resp.StatusCode == http.StatusTooManyRequests {
return recoverableErr{err}
}
return err
}
func remoteWriteTimeSeries(client *remoteWriteClient, tsCh chan []prompb.TimeSeries) {
bo := backoff.NewBackoff("remote-write", log.Printf, time.Second*30)
// writeErr may contribute to bo's backoff schedule across tsCh read ops,
// i.e. if an unrecoverable error occurs for client.write(ctx, A), that
// should be accounted against bo prior to attempting to
// client.write(ctx, B).
var writeErr error
for ts := range tsCh {
for {
bo.BackOff(context.Background(), writeErr)
reqCtx, cancel := context.WithTimeout(context.Background(), time.Second*30)
writeErr = client.write(reqCtx, ts)
cancel()
var re recoverableErr
recoverable := errors.As(writeErr, &re)
if writeErr != nil {
log.Printf("remote write error(recoverable=%v): %v", recoverable, writeErr)
}
if !recoverable {
// a nil err is not recoverable
break
}
}
}
}
func main() {
flag.Parse()
if len(*flagDstPorts) == 0 {
log.Fatal("dst-ports flag is unset")
}
dstPortsSplit := strings.Split(*flagDstPorts, ",")
slices.Sort(dstPortsSplit)
dstPortsSplit = slices.Compact(dstPortsSplit)
dstPorts := make([]int, 0, len(dstPortsSplit))
for _, d := range dstPortsSplit {
i, err := strconv.ParseUint(d, 10, 16)
if err != nil {
log.Fatal("invalid dst-ports")
}
dstPorts = append(dstPorts, int(i))
}
if len(*flagDERPMap) < 1 {
log.Fatal("derp-map flag is unset")
}
if len(*flagOut) < 1 {
log.Fatal("out flag is unset")
}
if *flagInterval < minInterval || *flagInterval > maxBufferDuration {
log.Fatalf("interval must be >= %s and <= %s", minInterval, maxBufferDuration)
}
if *flagRetention < *flagInterval {
log.Fatal("retention must be >= interval")
}
if len(*flagRemoteWriteURL) < 1 {
log.Fatal("rw-url flag is unset")
}
_, err := url.Parse(*flagRemoteWriteURL)
if err != nil {
log.Fatalf("invalid rw-url flag value: %v", err)
}
if len(*flagInstance) < 1 {
hostname, err := os.Hostname()
if err != nil {
log.Fatalf("failed to get hostname: %v", err)
}
*flagInstance = hostname
}
sigCh := make(chan os.Signal, 1)
signal.Notify(sigCh, syscall.SIGINT, syscall.SIGTERM)
dmCh := make(chan *tailcfg.DERPMap)
go func() {
bo := backoff.NewBackoff("derp-map", log.Printf, time.Second*30)
for {
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
dm, err := getDERPMap(ctx, *flagDERPMap)
cancel()
bo.BackOff(context.Background(), err)
if err != nil {
continue
}
dmCh <- dm
return
}
}()
nodeMetaByAddr := make(map[netip.Addr]nodeMeta)
select {
case <-sigCh:
return
case dm := <-dmCh:
_, err := nodeMetaFromDERPMap(dm, nodeMetaByAddr, *flagIPv6)
if err != nil {
log.Fatalf("error parsing derp map on startup: %v", err)
}
}
db, err := newDB(*flagOut)
if err != nil {
log.Fatalf("error opening output file for writing: %v", err)
}
defer db.Close()
_, err = db.Exec("PRAGMA journal_mode=WAL")
if err != nil {
log.Fatalf("error enabling WAL mode: %v", err)
}
// No indices or primary key. Keep it simple for now. Reads will be full
// scans. We can AUTOINCREMENT rowid in the future and hold an in-memory
// index to at_unix if needed as reads are almost always going to be
// time-bound (e.g. WHERE at_unix >= ?). At the time of authorship we have
// ~300 data points per-interval w/o ipv6 w/kernel timestamping resulting
// in ~2.6m rows in 24h w/a 10s probe interval.
_, err = db.Exec(`
CREATE TABLE IF NOT EXISTS rtt(at_unix INT, region_id INT, hostname TEXT, af INT, address TEXT, timestamp_source INT, stable_conn INT, dst_port INT, rtt_ns INT)
`)
if err != nil {
log.Fatalf("error initializing db: %v", err)
}
wg := sync.WaitGroup{}
httpErrCh := make(chan error, 1)
var httpServer *http.Server
if len(*flagAPI) > 0 {
api := newAPI(db)
httpServer = &http.Server{
Addr: *flagAPI,
Handler: api,
ReadTimeout: time.Second * 60,
WriteTimeout: time.Second * 60,
}
wg.Add(1)
go func() {
err := httpServer.ListenAndServe()
httpErrCh <- err
wg.Done()
}()
}
tsCh := make(chan []prompb.TimeSeries, maxBufferDuration / *flagInterval)
remoteWriteDoneCh := make(chan struct{})
rwc := newRemoteWriteClient(*flagRemoteWriteURL)
go func() {
remoteWriteTimeSeries(rwc, tsCh)
close(remoteWriteDoneCh)
}()
shutdown := func() {
if httpServer != nil {
httpServer.Close()
}
close(tsCh)
select {
case <-time.After(time.Second * 10): // give goroutine some time to flush
case <-remoteWriteDoneCh:
}
// send stale markers on shutdown
staleMeta := make([]nodeMeta, 0, len(nodeMetaByAddr))
for _, v := range nodeMetaByAddr {
staleMeta = append(staleMeta, v)
}
staleMarkers := staleMarkersFromNodeMeta(staleMeta, *flagInstance, dstPorts)
if len(staleMarkers) > 0 {
ctx, cancel := context.WithTimeout(context.Background(), time.Second*5)
rwc.write(ctx, staleMarkers)
cancel()
}
wg.Wait()
return
}
log.Println("stunstamp started")
// Re-using sockets means we get the same 5-tuple across runs. This results
// in a higher probability of the packets traversing the same underlay path.
// Comparison of stable and unstable 5-tuple results can shed light on
// differences between paths where hashing (multipathing/load balancing)
// comes into play.
stableConns := make(map[netip.Addr]map[int][2]io.ReadWriteCloser)
// timeouts holds counts of timeout events. Values are persisted for the
// lifetime of the related node in the DERP map.
timeouts := make(map[resultKey]uint64)
derpMapTicker := time.NewTicker(time.Minute * 5)
defer derpMapTicker.Stop()
probeTicker := time.NewTicker(*flagInterval)
defer probeTicker.Stop()
cleanupTicker := time.NewTicker(time.Hour)
defer cleanupTicker.Stop()
for {
select {
case <-cleanupTicker.C:
older := time.Now().Add(-*flagRetention)
log.Printf("cleaning up measurements older than %v", older)
_, err := db.Exec("DELETE FROM rtt WHERE at_unix < ?", older.Unix())
if err != nil {
log.Printf("error cleaning up old data: %v", err)
shutdown()
return
}
case <-probeTicker.C:
results, err := probeNodes(nodeMetaByAddr, stableConns, dstPorts)
if err != nil {
log.Printf("unrecoverable error while probing: %v", err)
shutdown()
return
}
ts := resultsToPromTimeSeries(results, *flagInstance, timeouts)
select {
case tsCh <- ts:
default:
select {
case <-tsCh:
log.Println("prometheus remote-write buffer full, dropped measurements")
default:
tsCh <- ts
}
}
tx, err := db.Begin()
if err != nil {
log.Printf("error beginning sqlite tx: %v", err)
shutdown()
return
}
for _, result := range results {
af := 4
if result.key.meta.addr.Is6() {
af = 6
}
_, err = tx.Exec("INSERT INTO rtt(at_unix, region_id, hostname, af, address, timestamp_source, stable_conn, dst_port, rtt_ns) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?)",
result.at.Unix(), result.key.meta.regionID, result.key.meta.hostname, af, result.key.meta.addr.String(), result.key.timestampSource, result.key.connStability, result.key.dstPort, result.rtt)
if err != nil {
tx.Rollback()
log.Printf("error adding result to tx: %v", err)
shutdown()
return
}
}
err = tx.Commit()
if err != nil {
log.Printf("error committing tx: %v", err)
shutdown()
return
}
case dm := <-dmCh:
staleMeta, err := nodeMetaFromDERPMap(dm, nodeMetaByAddr, *flagIPv6)
if err != nil {
log.Printf("error parsing DERP map, continuing with stale map: %v", err)
continue
}
staleMarkers := staleMarkersFromNodeMeta(staleMeta, *flagInstance, dstPorts)
if len(staleMarkers) < 1 {
continue
}
select {
case tsCh <- staleMarkers:
default:
select {
case <-tsCh:
log.Println("prometheus remote-write buffer full, dropped measurements")
default:
tsCh <- staleMarkers
}
}
case <-derpMapTicker.C:
go func() {
ctx, cancel := context.WithTimeout(context.Background(), time.Second*10)
defer cancel()
updatedDM, err := getDERPMap(ctx, *flagDERPMap)
if err != nil {
dmCh <- updatedDM
}
}()
case err := <-httpErrCh:
log.Printf("http server error: %v", err)
shutdown()
return
case <-sigCh:
shutdown()
return
}
}
}

@ -0,0 +1,26 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !(windows && 386)
package main
import (
"database/sql"
_ "modernc.org/sqlite"
)
type db struct {
*sql.DB
}
func newDB(path string) (*db, error) {
d, err := sql.Open("sqlite", *flagOut)
if err != nil {
return nil, err
}
return &db{
DB: d,
}, nil
}

@ -0,0 +1,17 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
package main
import (
"database/sql"
"errors"
)
type db struct {
*sql.DB
}
func newDB(path string) (*db, error) {
return nil, errors.New("unsupported platform")
}

@ -0,0 +1,25 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build !linux
package main
import (
"errors"
"io"
"net"
"time"
)
func getConnKernelTimestamp() (io.ReadWriteCloser, error) {
return nil, errors.New("unimplemented")
}
func measureRTTKernel(conn io.ReadWriteCloser, dst *net.UDPAddr) (rtt time.Duration, err error) {
return 0, errors.New("unimplemented")
}
func supportsKernelTS() bool {
return false
}

@ -0,0 +1,143 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
package main
import (
"bytes"
"context"
"encoding/binary"
"errors"
"fmt"
"io"
"net"
"time"
"github.com/mdlayher/socket"
"golang.org/x/sys/unix"
"tailscale.com/net/stun"
)
const (
flags = unix.SOF_TIMESTAMPING_TX_SOFTWARE | // tx timestamp generation in device driver
unix.SOF_TIMESTAMPING_RX_SOFTWARE | // rx timestamp generation in the kernel
unix.SOF_TIMESTAMPING_SOFTWARE // report software timestamps
)
func getConnKernelTimestamp() (io.ReadWriteCloser, error) {
sconn, err := socket.Socket(unix.AF_INET6, unix.SOCK_DGRAM, unix.IPPROTO_UDP, "udp", nil)
if err != nil {
return nil, err
}
sa := unix.SockaddrInet6{}
err = sconn.Bind(&sa)
if err != nil {
return nil, err
}
err = sconn.SetsockoptInt(unix.SOL_SOCKET, unix.SO_TIMESTAMPING_NEW, flags)
if err != nil {
return nil, err
}
return sconn, nil
}
func parseTimestampFromCmsgs(oob []byte) (time.Time, error) {
msgs, err := unix.ParseSocketControlMessage(oob)
if err != nil {
return time.Time{}, fmt.Errorf("error parsing oob as cmsgs: %w", err)
}
for _, msg := range msgs {
if msg.Header.Level == unix.SOL_SOCKET && msg.Header.Type == unix.SO_TIMESTAMPING_NEW && len(msg.Data) >= 16 {
sec := int64(binary.NativeEndian.Uint64(msg.Data[:8]))
ns := int64(binary.NativeEndian.Uint64(msg.Data[8:16]))
return time.Unix(sec, ns), nil
}
}
return time.Time{}, errors.New("failed to parse timestamp from cmsgs")
}
func measureRTTKernel(conn io.ReadWriteCloser, dst *net.UDPAddr) (rtt time.Duration, err error) {
sconn, ok := conn.(*socket.Conn)
if !ok {
return 0, fmt.Errorf("conn of unexpected type: %T", conn)
}
var to unix.Sockaddr
to4 := dst.IP.To4()
if to4 != nil {
to = &unix.SockaddrInet4{
Port: dst.Port,
}
copy(to.(*unix.SockaddrInet4).Addr[:], to4)
} else {
to = &unix.SockaddrInet6{
Port: dst.Port,
}
copy(to.(*unix.SockaddrInet6).Addr[:], dst.IP)
}
txID := stun.NewTxID()
req := stun.Request(txID)
err = sconn.Sendto(context.Background(), req, 0, to)
if err != nil {
return 0, fmt.Errorf("sendto error: %v", err) // don't wrap
}
txCtx, txCancel := context.WithTimeout(context.Background(), time.Second*2)
defer txCancel()
buf := make([]byte, 1024)
oob := make([]byte, 1024)
var txAt time.Time
for {
n, oobn, _, _, err := sconn.Recvmsg(txCtx, buf, oob, unix.MSG_ERRQUEUE)
if err != nil {
return 0, fmt.Errorf("recvmsg (MSG_ERRQUEUE) error: %v", err) // don't wrap
}
buf = buf[:n]
if n < len(req) || !bytes.Equal(req, buf[len(buf)-len(req):]) {
// Spin until we find the message we sent. We get the full packet
// looped including eth header so match against the tail.
continue
}
txAt, err = parseTimestampFromCmsgs(oob[:oobn])
if err != nil {
return 0, fmt.Errorf("failed to get tx timestamp: %v", err) // don't wrap
}
break
}
rxCtx, rxCancel := context.WithTimeout(context.Background(), time.Second*2)
defer rxCancel()
for {
n, oobn, _, _, err := sconn.Recvmsg(rxCtx, buf, oob, 0)
if err != nil {
return 0, fmt.Errorf("recvmsg error: %w", err) // wrap for timeout-related error unwrapping
}
gotTxID, _, err := stun.ParseResponse(buf[:n])
if err != nil || gotTxID != txID {
// Spin until we find the txID we sent. We may end up reading
// extremely late arriving responses from previous intervals. As
// such, we can't be certain if we're parsing the "current"
// response, so spin for parse errors too.
continue
}
rxAt, err := parseTimestampFromCmsgs(oob[:oobn])
if err != nil {
return 0, fmt.Errorf("failed to get rx timestamp: %v", err) // don't wrap
}
return rxAt.Sub(txAt), nil
}
}
func supportsKernelTS() bool {
return true
}

@ -22,6 +22,7 @@ import (
"github.com/mattn/go-isatty"
"github.com/peterbourgon/ff/v3/ffcli"
"tailscale.com/client/tailscale"
"tailscale.com/cmd/tailscale/cli/ffcomplete"
"tailscale.com/envknob"
"tailscale.com/paths"
"tailscale.com/version/distro"
@ -77,7 +78,9 @@ func CleanUpArgs(args []string) []string {
return out
}
var localClient tailscale.LocalClient
var localClient = tailscale.LocalClient{
Socket: paths.DefaultTailscaledSocket(),
}
// Run runs the CLI. The args do not include the binary name.
func Run(args []string) (err error) {
@ -99,14 +102,32 @@ func Run(args []string) (err error) {
if errors.Is(err, flag.ErrHelp) {
return nil
}
if noexec := (ffcli.NoExecError{}); errors.As(err, &noexec) {
// When the user enters an unknown subcommand, ffcli tries to run
// the closest valid parent subcommand with everything else as args,
// returning NoExecError if it doesn't have an Exec function.
cmd := noexec.Command
args := cmd.FlagSet.Args()
if len(cmd.Subcommands) > 0 {
if len(args) > 0 {
return fmt.Errorf("%s: unknown subcommand: %s", fullCmd(rootCmd, cmd), args[0])
}
subs := make([]string, 0, len(cmd.Subcommands))
for _, sub := range cmd.Subcommands {
subs = append(subs, sub.Name)
}
return fmt.Errorf("%s: missing subcommand: %s", fullCmd(rootCmd, cmd), strings.Join(subs, ", "))
}
}
return err
}
if envknob.Bool("TS_DUMP_HELP") {
walkCommands(rootCmd, func(c *ffcli.Command) {
walkCommands(rootCmd, func(w cmdWalk) bool {
fmt.Println("===")
// UsageFuncs are typically called during Command.Run which ensures
// FlagSet is not nil.
c := w.Command
if c.FlagSet == nil {
c.FlagSet = flag.NewFlagSet(c.Name, flag.ContinueOnError)
}
@ -115,17 +136,11 @@ func Run(args []string) (err error) {
} else {
fmt.Println(ffcli.DefaultUsageFunc(c))
}
return true
})
return
}
localClient.Socket = rootArgs.socket
rootCmd.FlagSet.Visit(func(f *flag.Flag) {
if f.Name == "socket" {
localClient.UseSocketOnly = true
}
})
err = rootCmd.Run(context.Background())
if tailscale.IsAccessDeniedError(err) && os.Getuid() != 0 && runtime.GOOS != "windows" {
return fmt.Errorf("%v\n\nUse 'sudo tailscale %s' or 'tailscale up --operator=$USER' to not require root.", err, strings.Join(args, " "))
@ -138,7 +153,12 @@ func Run(args []string) (err error) {
func newRootCmd() *ffcli.Command {
rootfs := newFlagSet("tailscale")
rootfs.StringVar(&rootArgs.socket, "socket", paths.DefaultTailscaledSocket(), "path to tailscaled socket")
rootfs.Func("socket", "path to tailscaled socket", func(s string) error {
localClient.Socket = s
localClient.UseSocketOnly = true
return nil
})
rootfs.Lookup("socket").DefValue = localClient.Socket
rootCmd := &ffcli.Command{
Name: "tailscale",
@ -178,25 +198,29 @@ change in the future.
whoisCmd,
debugCmd,
driveCmd,
idTokenCmd,
},
FlagSet: rootfs,
Exec: func(context.Context, []string) error { return flag.ErrHelp },
Exec: func(ctx context.Context, args []string) error {
if len(args) > 0 {
return fmt.Errorf("tailscale: unknown subcommand: %s", args[0])
}
if envknob.UseWIPCode() {
rootCmd.Subcommands = append(rootCmd.Subcommands,
idTokenCmd,
)
return flag.ErrHelp
},
}
if runtime.GOOS == "linux" && distro.Get() == distro.Synology {
rootCmd.Subcommands = append(rootCmd.Subcommands, configureHostCmd)
}
walkCommands(rootCmd, func(c *ffcli.Command) {
if c.UsageFunc == nil {
c.UsageFunc = usageFunc
walkCommands(rootCmd, func(w cmdWalk) bool {
if w.UsageFunc == nil {
w.UsageFunc = usageFunc
}
return true
})
ffcomplete.Inject(rootCmd, func(c *ffcli.Command) { c.LongHelp = hidden + c.LongHelp }, usageFunc)
return rootCmd
}
@ -212,15 +236,57 @@ func fatalf(format string, a ...any) {
// Fatalf, if non-nil, is used instead of log.Fatalf.
var Fatalf func(format string, a ...any)
var rootArgs struct {
socket string
type cmdWalk struct {
*ffcli.Command
parents []*ffcli.Command
}
func (w cmdWalk) Path() string {
if len(w.parents) == 0 {
return w.Name
}
var sb strings.Builder
for _, p := range w.parents {
sb.WriteString(p.Name)
sb.WriteString(" ")
}
sb.WriteString(w.Name)
return sb.String()
}
func walkCommands(cmd *ffcli.Command, f func(*ffcli.Command)) {
f(cmd)
// walkCommands calls f for root and all of its nested subcommands until f
// returns false or all have been visited.
func walkCommands(root *ffcli.Command, f func(w cmdWalk) (more bool)) {
var walk func(cmd *ffcli.Command, parents []*ffcli.Command, f func(cmdWalk) bool) bool
walk = func(cmd *ffcli.Command, parents []*ffcli.Command, f func(cmdWalk) bool) bool {
if !f(cmdWalk{cmd, parents}) {
return false
}
parents = append(parents, cmd)
for _, sub := range cmd.Subcommands {
walkCommands(sub, f)
if !walk(sub, parents, f) {
return false
}
}
return true
}
walk(root, nil, f)
}
// fullCmd returns the full "tailscale ... cmd" invocation for a subcommand.
func fullCmd(root, cmd *ffcli.Command) (full string) {
walkCommands(root, func(w cmdWalk) bool {
if w.Command == cmd {
full = w.Path()
return false
}
return true
})
if full == "" {
return cmd.Name
}
return full
}
// usageFuncNoDefaultValues is like usageFunc but doesn't print default values.
@ -232,9 +298,12 @@ func usageFunc(c *ffcli.Command) string {
return usageFuncOpt(c, true)
}
// hidden is the prefix that hides subcommands and flags from --help output when
// found at the start of the subcommand's LongHelp or flag's Usage.
const hidden = "HIDDEN: "
func usageFuncOpt(c *ffcli.Command, withDefaults bool) string {
var b strings.Builder
const hiddenPrefix = "HIDDEN: "
if c.ShortHelp != "" {
fmt.Fprintf(&b, "%s\n\n", c.ShortHelp)
@ -248,8 +317,7 @@ func usageFuncOpt(c *ffcli.Command, withDefaults bool) string {
}
fmt.Fprintf(&b, "\n")
if c.LongHelp != "" {
help, _ := strings.CutPrefix(c.LongHelp, hiddenPrefix)
if help := strings.TrimPrefix(c.LongHelp, hidden); help != "" {
fmt.Fprintf(&b, "%s\n\n", help)
}
@ -257,7 +325,7 @@ func usageFuncOpt(c *ffcli.Command, withDefaults bool) string {
fmt.Fprintf(&b, "SUBCOMMANDS\n")
tw := tabwriter.NewWriter(&b, 0, 2, 2, ' ', 0)
for _, subcommand := range c.Subcommands {
if strings.HasPrefix(subcommand.LongHelp, hiddenPrefix) {
if strings.HasPrefix(subcommand.LongHelp, hidden) {
continue
}
fmt.Fprintf(tw, " %s\t%s\n", subcommand.Name, subcommand.ShortHelp)
@ -272,7 +340,7 @@ func usageFuncOpt(c *ffcli.Command, withDefaults bool) string {
c.FlagSet.VisitAll(func(f *flag.Flag) {
var s string
name, usage := flag.UnquoteUsage(f)
if strings.HasPrefix(usage, hiddenPrefix) {
if strings.HasPrefix(usage, hidden) {
return
}
if isBoolFlag(f) {

@ -16,7 +16,6 @@ import (
qt "github.com/frankban/quicktest"
"github.com/google/go-cmp/cmp"
"github.com/peterbourgon/ff/v3/ffcli"
"tailscale.com/envknob"
"tailscale.com/health/healthmsg"
"tailscale.com/ipn"
@ -25,6 +24,7 @@ import (
"tailscale.com/tka"
"tailscale.com/tstest"
"tailscale.com/types/logger"
"tailscale.com/types/opt"
"tailscale.com/types/persist"
"tailscale.com/types/preftype"
"tailscale.com/version/distro"
@ -34,28 +34,106 @@ func TestPanicIfAnyEnvCheckedInInit(t *testing.T) {
envknob.PanicIfAnyEnvCheckedInInit()
}
func TestShortUsage_FullCmd(t *testing.T) {
func TestShortUsage(t *testing.T) {
t.Setenv("TAILSCALE_USE_WIP_CODE", "1")
if !envknob.UseWIPCode() {
t.Fatal("expected envknob.UseWIPCode() to be true")
}
// Some commands have more than one path from the root, so investigate all
// paths before we report errors.
ok := make(map[*ffcli.Command]bool)
root := newRootCmd()
walkCommands(root, func(c *ffcli.Command) {
if !ok[c] {
ok[c] = strings.HasPrefix(c.ShortUsage, "tailscale ") && (c.Name == "tailscale" || strings.Contains(c.ShortUsage, " "+c.Name+" ") || strings.HasSuffix(c.ShortUsage, " "+c.Name))
walkCommands(newRootCmd(), func(w cmdWalk) bool {
c, parents := w.Command, w.parents
// Words that we expect to be in the usage.
words := make([]string, len(parents)+1)
for i, parent := range parents {
words[i] = parent.Name
}
})
walkCommands(root, func(c *ffcli.Command) {
if !ok[c] {
t.Errorf("subcommand %s should show full usage ('tailscale ... %s ...') in ShortUsage (%q)", c.Name, c.Name, c.ShortUsage)
words[len(parents)] = c.Name
// Check the ShortHelp starts with a capital letter.
if prefix, help := trimPrefixes(c.ShortHelp, "HIDDEN: ", "[ALPHA] ", "[BETA] "); help != "" {
if 'a' <= help[0] && help[0] <= 'z' {
if len(help) > 20 {
help = help[:20] + "…"
}
caphelp := string(help[0]-'a'+'A') + help[1:]
t.Errorf("command: %s: ShortHelp %q should start with a capital letter %q", strings.Join(words, " "), prefix+help, prefix+caphelp)
}
}
// Check all words appear in the usage.
usage := c.ShortUsage
for _, word := range words {
var ok bool
usage, ok = cutWord(usage, word)
if !ok {
full := strings.Join(words, " ")
t.Errorf("command: %s: usage %q should contain the full path %q", full, c.ShortUsage, full)
return true
}
}
return true
})
}
func trimPrefixes(full string, prefixes ...string) (trimmed, remaining string) {
s := full
start:
for _, p := range prefixes {
var ok bool
s, ok = strings.CutPrefix(s, p)
if ok {
goto start
}
}
return full[:len(full)-len(s)], s
}
// cutWord("tailscale debug scale 123", "scale") returns (" 123", true).
func cutWord(s, w string) (after string, ok bool) {
var p string
for {
p, s, ok = strings.Cut(s, w)
if !ok {
return "", false
}
if p != "" && isWordChar(p[len(p)-1]) {
continue
}
if s != "" && isWordChar(s[0]) {
continue
}
return s, true
}
}
func isWordChar(r byte) bool {
return r == '_' ||
('0' <= r && r <= '9') ||
('A' <= r && r <= 'Z') ||
('a' <= r && r <= 'z')
}
func TestCutWord(t *testing.T) {
tests := []struct {
in string
word string
out string
ok bool
}{
{"tailscale debug", "debug", "", true},
{"tailscale debug", "bug", "", false},
{"tailscale debug", "tail", "", false},
{"tailscale debug scaley scale 123", "scale", " 123", true},
}
for _, test := range tests {
out, ok := cutWord(test.in, test.word)
if out != test.out || ok != test.ok {
t.Errorf("cutWord(%q, %q) = (%q, %t), wanted (%q, %t)", test.in, test.word, out, ok, test.out, test.ok)
}
}
}
// geese is a collection of gooses. It need not be complete.
// But it should include anything handled specially (e.g. linux, windows)
// and at least one thing that's not (darwin, freebsd).
@ -102,6 +180,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
ControlURL: ipn.DefaultControlURL,
WantRunning: false,
Hostname: "foo",
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -114,7 +193,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
Hostname: "foo",
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AllowSingleHosts: true,
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --accept-dns --hostname=foo",
},
@ -125,8 +204,8 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
ControlURL: ipn.DefaultControlURL,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AllowSingleHosts: true,
Hostname: "foo",
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -137,8 +216,8 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
ControlURL: ipn.DefaultControlURL,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AllowSingleHosts: true,
Hostname: "foo",
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -156,9 +235,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
OperatorUser: "alice",
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
curUser: "eve",
want: accidentalUpPrefix + " --hostname=foo --operator=alice",
@ -168,10 +247,10 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--hostname=foo"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
OperatorUser: "alice",
NoStatefulFiltering: opt.NewBool(true),
},
curUser: "alice",
want: "",
@ -181,7 +260,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-routes=10.0.42.0/24"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
@ -189,6 +267,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
netip.MustParsePrefix("0.0.0.0/0"),
netip.MustParsePrefix("::/0"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --advertise-routes=10.0.42.0/24 --advertise-exit-node",
},
@ -197,7 +276,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-routes=10.0.42.0/24", "--advertise-exit-node=false"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
@ -205,6 +283,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
netip.MustParsePrefix("0.0.0.0/0"),
netip.MustParsePrefix("::/0"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -213,7 +292,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-routes=11.1.43.0/24,0.0.0.0/0,::/0"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
@ -221,6 +299,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
netip.MustParsePrefix("0.0.0.0/0"),
netip.MustParsePrefix("::/0"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -229,9 +308,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-exit-node"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -240,13 +319,13 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-exit-node"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
netip.MustParsePrefix("1.2.0.0/16"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --advertise-exit-node --advertise-routes=1.2.0.0/16",
},
@ -255,7 +334,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-exit-node"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
@ -263,6 +341,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
netip.MustParsePrefix("::/0"),
netip.MustParsePrefix("1.2.0.0/16"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --advertise-exit-node --advertise-routes=1.2.0.0/16",
},
@ -271,11 +350,11 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--exit-node="},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
ExitNodeID: "fooID",
NoStatefulFiltering: opt.NewBool(true),
},
want: "",
},
@ -286,7 +365,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
WantRunning: true,
ControlURL: ipn.DefaultControlURL,
RouteAll: true,
AllowSingleHosts: false,
ExitNodeIP: netip.MustParseAddr("100.64.5.6"),
CorpDNS: false,
ShieldsUp: true,
@ -300,9 +378,10 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
},
NetfilterMode: preftype.NetfilterNoDivert,
OperatorUser: "alice",
NoStatefulFiltering: opt.NewBool(true),
},
curUser: "eve",
want: accidentalUpPrefix + " --force-reauth --accept-dns=false --accept-routes --advertise-exit-node --advertise-routes=10.0.0.0/16 --advertise-tags=tag:foo,tag:bar --exit-node=100.64.5.6 --host-routes=false --hostname=myhostname --netfilter-mode=nodivert --operator=alice --shields-up",
want: accidentalUpPrefix + " --force-reauth --accept-dns=false --accept-routes --advertise-exit-node --advertise-routes=10.0.0.0/16 --advertise-tags=tag:foo,tag:bar --exit-node=100.64.5.6 --hostname=myhostname --netfilter-mode=nodivert --operator=alice --shields-up",
},
{
name: "remove_all_implicit_except_hostname",
@ -311,7 +390,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
WantRunning: true,
ControlURL: ipn.DefaultControlURL,
RouteAll: true,
AllowSingleHosts: false,
ExitNodeIP: netip.MustParseAddr("100.64.5.6"),
CorpDNS: false,
ShieldsUp: true,
@ -323,9 +401,10 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
},
NetfilterMode: preftype.NetfilterNoDivert,
OperatorUser: "alice",
NoStatefulFiltering: opt.NewBool(true),
},
curUser: "eve",
want: accidentalUpPrefix + " --hostname=newhostname --accept-dns=false --accept-routes --advertise-routes=10.0.0.0/16 --advertise-tags=tag:foo,tag:bar --exit-node=100.64.5.6 --host-routes=false --netfilter-mode=nodivert --operator=alice --shields-up",
want: accidentalUpPrefix + " --hostname=newhostname --accept-dns=false --accept-routes --advertise-routes=10.0.0.0/16 --advertise-tags=tag:foo,tag:bar --exit-node=100.64.5.6 --netfilter-mode=nodivert --operator=alice --shields-up",
},
{
name: "loggedout_is_implicit",
@ -333,9 +412,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
LoggedOut: true,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
want: "", // not an error. LoggedOut is implicit.
},
@ -346,7 +425,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--advertise-exit-node"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
RouteAll: true,
@ -361,7 +439,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--accept-dns"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
NetfilterMode: preftype.NetfilterNoDivert, // we never had this bug, but pretend it got set non-zero on Windows somehow
},
@ -373,7 +450,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--operator=expbits"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
@ -381,6 +457,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
netip.MustParsePrefix("::/0"),
netip.MustParsePrefix("1.2.0.0/16"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --operator=expbits --advertise-exit-node --advertise-routes=1.2.0.0/16",
},
@ -389,7 +466,6 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--operator=expbits", "--advertise-routes=1.2.0.0/16"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AdvertiseRoutes: []netip.Prefix{
@ -397,6 +473,7 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
netip.MustParsePrefix("::/0"),
netip.MustParsePrefix("1.2.0.0/16"),
},
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --advertise-routes=1.2.0.0/16 --operator=expbits --advertise-exit-node",
},
@ -408,9 +485,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
WantRunning: false,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AllowSingleHosts: true,
Hostname: "foo",
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --auth-key=secretrand --force-reauth=false --reset --hostname=foo",
},
@ -419,11 +496,11 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--hostname=foo"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
ExitNodeIP: netip.MustParseAddr("100.64.5.4"),
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --hostname=foo --exit-node=100.64.5.4",
},
@ -433,11 +510,11 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curExitNodeIP: netip.MustParseAddr("100.64.5.7"),
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
ExitNodeID: "some_stable_id",
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --hostname=foo --exit-node=100.64.5.7",
},
@ -447,12 +524,12 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curExitNodeIP: netip.MustParseAddr("100.2.3.4"),
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
ExitNodeAllowLANAccess: true,
ExitNodeID: "some_stable_id",
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --hostname=foo --exit-node-allow-lan-access --exit-node=100.2.3.4",
},
@ -461,9 +538,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--login-server=https://controlplane.tailscale.com"},
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
want: "", // not an error
},
@ -472,9 +549,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
flags: []string{"--netfilter-mode=off"},
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
AllowSingleHosts: true,
CorpDNS: false,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
want: accidentalUpPrefix + " --netfilter-mode=off --accept-dns=false",
},
@ -486,9 +563,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
CorpDNS: true,
AllowSingleHosts: true,
RouteAll: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
goos: "linux",
distro: distro.Synology,
@ -502,9 +579,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
CorpDNS: true,
AllowSingleHosts: true,
RouteAll: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
goos: "linux",
distro: "", // not Synology
@ -516,9 +593,9 @@ func TestCheckForAccidentalSettingReverts(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
CorpDNS: true,
AllowSingleHosts: true,
NetfilterMode: preftype.NetfilterOn,
ProfileName: "foo",
NoStatefulFiltering: opt.NewBool(true),
},
goos: "linux",
want: "",
@ -581,9 +658,9 @@ func TestPrefsFromUpArgs(t *testing.T) {
ControlURL: ipn.DefaultControlURL,
WantRunning: true,
NoSNAT: false,
NoStatefulFiltering: "true",
NetfilterMode: preftype.NetfilterOn,
CorpDNS: true,
AllowSingleHosts: true,
AutoUpdate: ipn.AutoUpdatePrefs{
Check: true,
},
@ -597,8 +674,9 @@ func TestPrefsFromUpArgs(t *testing.T) {
ControlURL: ipn.DefaultControlURL,
WantRunning: true,
CorpDNS: true,
AllowSingleHosts: true,
RouteAll: true,
NoSNAT: false,
NoStatefulFiltering: "true",
NetfilterMode: preftype.NetfilterOn,
AutoUpdate: ipn.AutoUpdatePrefs{
Check: true,
@ -611,12 +689,12 @@ func TestPrefsFromUpArgs(t *testing.T) {
want: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
WantRunning: true,
AllowSingleHosts: true,
CorpDNS: true,
AdvertiseRoutes: []netip.Prefix{
netip.MustParsePrefix("0.0.0.0/0"),
netip.MustParsePrefix("::/0"),
},
NoStatefulFiltering: "true",
NetfilterMode: preftype.NetfilterOn,
AutoUpdate: ipn.AutoUpdatePrefs{
Check: true,
@ -707,6 +785,7 @@ func TestPrefsFromUpArgs(t *testing.T) {
WantRunning: true,
NetfilterMode: preftype.NetfilterNoDivert,
NoSNAT: true,
NoStatefulFiltering: "true",
AutoUpdate: ipn.AutoUpdatePrefs{
Check: true,
},
@ -723,6 +802,7 @@ func TestPrefsFromUpArgs(t *testing.T) {
WantRunning: true,
NetfilterMode: preftype.NetfilterOff,
NoSNAT: true,
NoStatefulFiltering: "true",
AutoUpdate: ipn.AutoUpdatePrefs{
Check: true,
},
@ -738,6 +818,7 @@ func TestPrefsFromUpArgs(t *testing.T) {
want: &ipn.Prefs{
WantRunning: true,
NoSNAT: true,
NoStatefulFiltering: "true",
AdvertiseRoutes: []netip.Prefix{
netip.MustParsePrefix("fd7a:115c:a1e0:b1a::bb:10.0.0.0/112"),
},
@ -756,6 +837,7 @@ func TestPrefsFromUpArgs(t *testing.T) {
want: &ipn.Prefs{
WantRunning: true,
NoSNAT: true,
NoStatefulFiltering: "true",
AdvertiseRoutes: []netip.Prefix{
netip.MustParsePrefix("fd7a:115c:a1e0:b1a::aabb:10.0.0.0/112"),
},
@ -831,12 +913,15 @@ func TestPrefFlagMapping(t *testing.T) {
}
prefType := reflect.TypeFor[ipn.Prefs]()
for i := 0; i < prefType.NumField(); i++ {
for i := range prefType.NumField() {
prefName := prefType.Field(i).Name
if prefHasFlag[prefName] {
continue
}
switch prefName {
case "AllowSingleHosts":
// Fake pref for downgrade compat. See #12058.
continue
case "WantRunning", "Persist", "LoggedOut":
// All explicitly handled (ignored) by checkForAccidentalSettingReverts.
continue
@ -944,7 +1029,6 @@ func TestUpdatePrefs(t *testing.T) {
wantJustEditMP: &ipn.MaskedPrefs{
AdvertiseRoutesSet: true,
AdvertiseTagsSet: true,
AllowSingleHostsSet: true,
AppConnectorSet: true,
ControlURLSet: true,
CorpDNSSet: true,
@ -954,6 +1038,7 @@ func TestUpdatePrefs(t *testing.T) {
HostnameSet: true,
NetfilterModeSet: true,
NoSNATSet: true,
NoStatefulFilteringSet: true,
OperatorUserSet: true,
RouteAllSet: true,
RunSSHSet: true,
@ -978,9 +1063,9 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
env: upCheckEnv{backendState: "Running"},
wantSimpleUp: true,
@ -993,9 +1078,9 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
env: upCheckEnv{backendState: "Running"},
},
@ -1006,9 +1091,9 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
CorpDNS: true,
AllowSingleHosts: true,
NetfilterMode: preftype.NetfilterOn,
OperatorUser: "somebody",
NoStatefulFiltering: opt.NewBool(true),
},
env: upCheckEnv{user: "somebody", backendState: "Running"},
wantJustEditMP: &ipn.MaskedPrefs{
@ -1027,9 +1112,9 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
RunSSHSet: true,
@ -1048,10 +1133,10 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
RunSSH: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
RunSSHSet: true,
@ -1073,10 +1158,10 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
RunSSH: true,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
RunSSHSet: true,
@ -1097,9 +1182,9 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
RunSSHSet: true,
@ -1120,9 +1205,9 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
RunSSHSet: true,
@ -1142,10 +1227,10 @@ func TestUpdatePrefs(t *testing.T) {
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
Persist: &persist.Persist{UserProfile: tailcfg.UserProfile{LoginName: "crawshaw.github"}},
AllowSingleHosts: true,
CorpDNS: true,
RunSSH: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
RunSSHSet: true,
@ -1164,9 +1249,9 @@ func TestUpdatePrefs(t *testing.T) {
sshOverTailscale: true,
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
env: upCheckEnv{backendState: "Running"},
wantErrSubtr: "aborted, no changes made",
@ -1177,9 +1262,9 @@ func TestUpdatePrefs(t *testing.T) {
sshOverTailscale: true,
curPrefs: &ipn.Prefs{
ControlURL: "https://login.tailscale.com",
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: nil,
env: upCheckEnv{backendState: "Running"},
@ -1189,9 +1274,9 @@ func TestUpdatePrefs(t *testing.T) {
flags: []string{"--advertise-connector"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
AppConnectorSet: true,
@ -1209,12 +1294,12 @@ func TestUpdatePrefs(t *testing.T) {
flags: []string{"--advertise-connector=false"},
curPrefs: &ipn.Prefs{
ControlURL: ipn.DefaultControlURL,
AllowSingleHosts: true,
CorpDNS: true,
NetfilterMode: preftype.NetfilterOn,
AppConnector: ipn.AppConnectorPrefs{
Advertise: true,
},
NoStatefulFiltering: opt.NewBool(true),
},
wantJustEditMP: &ipn.MaskedPrefs{
AppConnectorSet: true,

@ -24,9 +24,9 @@ import (
var configureHostCmd = &ffcli.Command{
Name: "configure-host",
Exec: runConfigureSynology,
ShortUsage: "tailscale configure-host",
ShortUsage: "tailscale configure-host\n" + synologyConfigureCmd.ShortUsage,
ShortHelp: synologyConfigureCmd.ShortHelp,
LongHelp: synologyConfigureCmd.LongHelp,
LongHelp: hidden + synologyConfigureCmd.LongHelp,
FlagSet: (func() *flag.FlagSet {
fs := newFlagSet("configure-host")
return fs

@ -4,7 +4,6 @@
package cli
import (
"context"
"flag"
"runtime"
"strings"
@ -26,9 +25,6 @@ services on the host to use Tailscale in more ways.
return fs
})(),
Subcommands: configureSubcommands(),
Exec: func(ctx context.Context, args []string) error {
return flag.ErrHelp
},
}
func configureSubcommands() (out []*ffcli.Command) {

@ -28,10 +28,12 @@ import (
"github.com/peterbourgon/ff/v3/ffcli"
"golang.org/x/net/http/httpproxy"
"golang.org/x/net/http2"
"tailscale.com/client/tailscale"
"tailscale.com/client/tailscale/apitype"
"tailscale.com/control/controlhttp"
"tailscale.com/hostinfo"
"tailscale.com/internal/noiseconn"
"tailscale.com/ipn"
"tailscale.com/net/tsaddr"
"tailscale.com/net/tshttpproxy"
@ -48,7 +50,8 @@ var debugCmd = &ffcli.Command{
Name: "debug",
Exec: runDebug,
ShortUsage: "tailscale debug <debug-flags | subcommand>",
LongHelp: `HIDDEN: "tailscale debug" contains misc debug facilities; it is not a stable interface.`,
ShortHelp: "Debug commands",
LongHelp: hidden + `"tailscale debug" contains misc debug facilities; it is not a stable interface.`,
FlagSet: (func() *flag.FlagSet {
fs := newFlagSet("debug")
fs.StringVar(&debugArgs.file, "file", "", "get, delete:NAME, or NAME")
@ -227,8 +230,8 @@ var debugCmd = &ffcli.Command{
},
{
Name: "via",
ShortUsage: "tailscale via <site-id> <v4-cidr>\n" +
"tailscale via <v6-route>",
ShortUsage: "tailscale debug via <site-id> <v4-cidr>\n" +
"tailscale debug via <v6-route>",
Exec: runVia,
ShortHelp: "Convert between site-specific IPv4 CIDRs and IPv6 'via' routes",
},
@ -346,7 +349,7 @@ func outName(dst string) string {
func runDebug(ctx context.Context, args []string) error {
if len(args) > 0 {
return errors.New("unknown arguments")
return fmt.Errorf("tailscale debug: unknown subcommand: %s", args[0])
}
var usedFlag bool
if out := debugArgs.cpuFile; out != "" {
@ -401,7 +404,7 @@ func runDebug(ctx context.Context, args []string) error {
// to subcommands.
return nil
}
return errors.New("see 'tailscale debug --help")
return errors.New("tailscale debug: subcommand or flag required")
}
func runLocalCreds(ctx context.Context, args []string) error {
@ -800,7 +803,10 @@ func runTS2021(ctx context.Context, args []string) error {
log.Printf("Dial(%q, %q) ...", network, address)
c, err := dialer.DialContext(ctx, network, address)
if err != nil {
// skip logging context cancellation errors
if !errors.Is(err, context.Canceled) {
log.Printf("Dial(%q, %q) = %v", network, address, err)
}
} else {
log.Printf("Dial(%q, %q) = %v / %v", network, address, c.LocalAddr(), c.RemoteAddr())
}
@ -833,6 +839,52 @@ func runTS2021(ctx context.Context, args []string) error {
}
log.Printf("final underlying conn: %v / %v", conn.LocalAddr(), conn.RemoteAddr())
h2Transport, err := http2.ConfigureTransports(&http.Transport{
IdleConnTimeout: time.Second,
})
if err != nil {
return fmt.Errorf("http2.ConfigureTransports: %w", err)
}
// Now, create a Noise conn over the existing conn.
nc, err := noiseconn.New(conn.Conn, h2Transport, 0, nil)
if err != nil {
return fmt.Errorf("noiseconn.New: %w", err)
}
defer nc.Close()
// Reserve a RoundTrip for the whoami request.
ok, _, err := nc.ReserveNewRequest(ctx)
if err != nil {
return fmt.Errorf("ReserveNewRequest: %w", err)
}
if !ok {
return errors.New("ReserveNewRequest failed")
}
// Make a /whoami request to the server to verify that we can actually
// communicate over the newly-established connection.
whoamiURL := "http://" + ts2021Args.host + "/machine/whoami"
req, err = http.NewRequestWithContext(ctx, "GET", whoamiURL, nil)
if err != nil {
return err
}
resp, err := nc.RoundTrip(req)
if err != nil {
return fmt.Errorf("RoundTrip whoami request: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != 200 {
log.Printf("whoami request returned status %v", resp.Status)
} else {
body, err := io.ReadAll(resp.Body)
if err != nil {
return fmt.Errorf("reading whoami response: %w", err)
}
log.Printf("whoami response: %q", body)
}
return nil
}
@ -842,7 +894,7 @@ var debugComponentLogsArgs struct {
func runDebugComponentLogs(ctx context.Context, args []string) error {
if len(args) != 1 {
return errors.New("usage: debug component-logs [" + strings.Join(ipn.DebuggableComponents, "|") + "]")
return errors.New("usage: tailscale debug component-logs [" + strings.Join(ipn.DebuggableComponents, "|") + "]")
}
component := args[0]
dur := debugComponentLogsArgs.forDur
@ -865,7 +917,7 @@ var devStoreSetArgs struct {
func runDevStoreSet(ctx context.Context, args []string) error {
if len(args) != 2 {
return errors.New("usage: dev-store-set --danger <key> <value>")
return errors.New("usage: tailscale debug dev-store-set --danger <key> <value>")
}
if !devStoreSetArgs.danger {
return errors.New("this command is dangerous; use --danger to proceed")
@ -883,7 +935,7 @@ func runDevStoreSet(ctx context.Context, args []string) error {
func runDebugDERP(ctx context.Context, args []string) error {
if len(args) != 1 {
return errors.New("usage: debug derp <region>")
return errors.New("usage: tailscale debug derp <region>")
}
st, err := localClient.DebugDERPRegion(ctx, args[0])
if err != nil {

@ -5,7 +5,6 @@ package cli
import (
"context"
"errors"
"fmt"
"strings"
@ -36,40 +35,33 @@ var driveCmd = &ffcli.Command{
Name: "share",
ShortUsage: driveShareUsage,
Exec: runDriveShare,
ShortHelp: "[ALPHA] create or modify a share",
UsageFunc: usageFunc,
ShortHelp: "[ALPHA] Create or modify a share",
},
{
Name: "rename",
ShortUsage: driveRenameUsage,
ShortHelp: "[ALPHA] rename a share",
ShortHelp: "[ALPHA] Rename a share",
Exec: runDriveRename,
UsageFunc: usageFunc,
},
{
Name: "unshare",
ShortUsage: driveUnshareUsage,
ShortHelp: "[ALPHA] remove a share",
ShortHelp: "[ALPHA] Remove a share",
Exec: runDriveUnshare,
UsageFunc: usageFunc,
},
{
Name: "list",
ShortUsage: driveListUsage,
ShortHelp: "[ALPHA] list current shares",
ShortHelp: "[ALPHA] List current shares",
Exec: runDriveList,
UsageFunc: usageFunc,
},
},
Exec: func(context.Context, []string) error {
return errors.New("drive subcommand required; run 'tailscale drive -h' for details")
},
}
// runDriveShare is the entry point for the "tailscale drive share" command.
func runDriveShare(ctx context.Context, args []string) error {
if len(args) != 2 {
return fmt.Errorf("usage: tailscale %v", driveShareUsage)
return fmt.Errorf("usage: %s", driveShareUsage)
}
name, path := args[0], args[1]
@ -87,7 +79,7 @@ func runDriveShare(ctx context.Context, args []string) error {
// runDriveUnshare is the entry point for the "tailscale drive unshare" command.
func runDriveUnshare(ctx context.Context, args []string) error {
if len(args) != 1 {
return fmt.Errorf("usage: tailscale %v", driveUnshareUsage)
return fmt.Errorf("usage: %s", driveUnshareUsage)
}
name := args[0]
@ -101,7 +93,7 @@ func runDriveUnshare(ctx context.Context, args []string) error {
// runDriveRename is the entry point for the "tailscale drive rename" command.
func runDriveRename(ctx context.Context, args []string) error {
if len(args) != 2 {
return fmt.Errorf("usage: tailscale %v", driveRenameUsage)
return fmt.Errorf("usage: %s", driveRenameUsage)
}
oldName := args[0]
newName := args[1]
@ -116,7 +108,7 @@ func runDriveRename(ctx context.Context, args []string) error {
// runDriveList is the entry point for the "tailscale drive list" command.
func runDriveList(ctx context.Context, args []string) error {
if len(args) != 0 {
return fmt.Errorf("usage: tailscale %v", driveListUsage)
return fmt.Errorf("usage: %s", driveListUsage)
}
shares, err := localClient.DriveShareList(ctx)
@ -189,19 +181,9 @@ In order to access this share, other machines on the tailnet can connect to the
http://100.100.100.100:8080/mydomain.com/mylaptop/docs
Permissions to access shares are controlled via ACLs. For example, to give yourself read/write access and give the group "home" read-only access to the above share, use the below ACL grants:
Permissions to access shares are controlled via ACLs. For example, to give the group "home" read-only access to the above share, use the below ACL grant:
"grants": [
{
"src": ["mylogin@domain.com"],
"dst": ["mylaptop's ip address"],
"app": {
"tailscale.com/cap/drive": [{
"shares": ["docs"],
"access": "rw"
}]
}
},
{
"src": ["group:home"],
"dst": ["mylaptop"],
@ -213,7 +195,9 @@ Permissions to access shares are controlled via ACLs. For example, to give yours
}
}]
To categorically give yourself access to all your shares, you can use the below ACL grant:
Whenever anyone in the group "home" connects to the share, they connect as if they are using your local machine user. They'll be able to read the same files as your user, and if they create files, those files will be owned by your user.%s
On small tailnets, it may be convenient to categorically give all users full access to their own shares. That can be accomplished with the below grant.
"grants": [
{
@ -227,8 +211,6 @@ To categorically give yourself access to all your shares, you can use the below
}
}]
Whenever either you or anyone in the group "home" connects to the share, they connect as if they are using your local machine user. They'll be able to read the same files as your user and if they create files, those files will be owned by your user.%s
You can rename shares, for example you could rename the above share by running:
$ tailscale drive rename docs newdocs

@ -25,10 +25,6 @@ func exitNodeCmd() *ffcli.Command {
Name: "exit-node",
ShortUsage: "tailscale exit-node [flags]",
ShortHelp: "Show machines on your tailnet configured as exit nodes",
LongHelp: "Show machines on your tailnet configured as exit nodes",
Exec: func(context.Context, []string) error {
return errors.New("exit-node subcommand required; run 'tailscale exit-node -h' for details")
},
Subcommands: append([]*ffcli.Command{
{
Name: "list",
@ -40,6 +36,12 @@ func exitNodeCmd() *ffcli.Command {
fs.StringVar(&exitNodeArgs.filter, "filter", "", "filter exit nodes by country")
return fs
})(),
},
{
Name: "suggest",
ShortUsage: "tailscale exit-node suggest",
ShortHelp: "Suggests the best available exit node",
Exec: runExitNodeSuggest,
}},
(func() []*ffcli.Command {
if !envknob.UseWIPCode() {
@ -49,13 +51,13 @@ func exitNodeCmd() *ffcli.Command {
{
Name: "connect",
ShortUsage: "tailscale exit-node connect",
ShortHelp: "connect to most recently used exit node",
ShortHelp: "Connect to most recently used exit node",
Exec: exitNodeSetUse(true),
},
{
Name: "disconnect",
ShortUsage: "tailscale exit-node disconnect",
ShortHelp: "disconnect from current exit node, if any",
ShortHelp: "Disconnect from current exit node, if any",
Exec: exitNodeSetUse(false),
},
}
@ -134,11 +136,37 @@ func runExitNodeList(ctx context.Context, args []string) error {
}
fmt.Fprintln(w)
fmt.Fprintln(w)
fmt.Fprintln(w, "# To use an exit node, use `tailscale set --exit-node=` followed by the hostname or IP")
fmt.Fprintln(w, "# To use an exit node, use `tailscale set --exit-node=` followed by the hostname or IP.")
if hasAnyExitNodeSuggestions(peers) {
fmt.Fprintln(w, "# To have Tailscale suggest an exit node, use `tailscale exit-node suggest`.")
}
return nil
}
// runExitNodeSuggest returns a suggested exit node ID to connect to and shows the chosen exit node tailcfg.StableNodeID.
// If there are no derp based exit nodes to choose from or there is a failure in finding a suggestion, the command will return an error indicating so.
func runExitNodeSuggest(ctx context.Context, args []string) error {
res, err := localClient.SuggestExitNode(ctx)
if err != nil {
return fmt.Errorf("suggest exit node: %w", err)
}
if res.ID == "" {
fmt.Println("No exit node suggestion is available.")
return nil
}
fmt.Printf("Suggested exit node: %v\nTo accept this suggestion, use `tailscale set --exit-node=%v`.\n", res.Name, res.ID)
return nil
}
func hasAnyExitNodeSuggestions(peers []*ipnstate.PeerStatus) bool {
for _, peer := range peers {
if peer.HasCap(tailcfg.NodeAttrSuggestExitNode) {
return true
}
}
return false
}
// peerStatus returns a string representing the current state of
// a peer. If there is no notable state, a - is returned.
func peerStatus(peer *ipnstate.PeerStatus) string {

@ -0,0 +1,160 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build go1.19 && !ts_omit_completion
// Package ffcomplete provides shell tab-completion of subcommands, flags and
// arguments for Go programs written with [ffcli].
//
// The shell integration scripts have been extracted from Cobra
// (https://cobra.dev/), whose authors deserve most of the credit for this work.
// These shell completion functions invoke `$0 completion __complete -- ...`
// which is wired up to [Complete].
package ffcomplete
import (
"context"
"flag"
"fmt"
"io"
"log"
"os"
"strconv"
"strings"
"github.com/peterbourgon/ff/v3/ffcli"
"tailscale.com/cmd/tailscale/cli/ffcomplete/internal"
"tailscale.com/tempfork/spf13/cobra"
)
type compOpts struct {
showFlags bool
showDescs bool
}
func newFS(name string, opts *compOpts) *flag.FlagSet {
fs := flag.NewFlagSet(name, flag.ContinueOnError)
fs.BoolVar(&opts.showFlags, "flags", true, "Suggest flag completions with subcommands")
fs.BoolVar(&opts.showDescs, "descs", true, "Include flag, subcommand, and other descriptions in completions")
return fs
}
// Inject adds the 'completion' subcommand to the root command which provide the
// user with shell scripts for calling `completion __command` to provide
// tab-completion suggestions.
//
// root.Name needs to match the command that the user is tab-completing for the
// shell script to work as expected by default.
//
// The hide function is called with the __complete Command instance to provide a
// hook to omit it from the help output, if desired.
func Inject(root *ffcli.Command, hide func(*ffcli.Command), usageFunc func(*ffcli.Command) string) {
var opts compOpts
compFS := newFS("completion", &opts)
completeCmd := &ffcli.Command{
Name: "__complete",
ShortUsage: root.Name + " completion __complete -- <args to complete...>",
ShortHelp: "Tab-completion suggestions for interactive shells",
UsageFunc: usageFunc,
FlagSet: compFS,
Exec: func(ctx context.Context, args []string) error {
// Set up debug logging for the rest of this function call.
if t := os.Getenv("BASH_COMP_DEBUG_FILE"); t != "" {
tf, err := os.OpenFile(t, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o600)
if err != nil {
return fmt.Errorf("opening debug file: %w", err)
}
defer func(origW io.Writer, origPrefix string, origFlags int) {
log.SetOutput(origW)
log.SetFlags(origFlags)
log.SetPrefix(origPrefix)
tf.Close()
}(log.Writer(), log.Prefix(), log.Flags())
log.SetOutput(tf)
log.SetFlags(log.Lshortfile)
log.SetPrefix("debug: ")
}
// Send back the results to the shell.
words, dir, err := internal.Complete(root, args, opts.showFlags, opts.showDescs)
if err != nil {
dir = ShellCompDirectiveError
}
for _, word := range words {
fmt.Println(word)
}
fmt.Println(":" + strconv.Itoa(int(dir)))
return err
},
}
if hide != nil {
hide(completeCmd)
}
root.Subcommands = append(
root.Subcommands,
&ffcli.Command{
Name: "completion",
ShortUsage: root.Name + " completion <shell> [--flags] [--descs]",
ShortHelp: "Shell tab-completion scripts",
LongHelp: fmt.Sprintf(cobra.UsageTemplate, root.Name),
// Print help if run without args.
Exec: func(ctx context.Context, args []string) error { return flag.ErrHelp },
// Omit the '__complete' subcommand from the 'completion' help.
UsageFunc: func(c *ffcli.Command) string {
// Filter the subcommands to omit '__complete'.
s := make([]*ffcli.Command, 0, len(c.Subcommands))
for _, sub := range c.Subcommands {
if !strings.HasPrefix(sub.Name, "__") {
s = append(s, sub)
}
}
// Swap in the filtered subcommands list for the rest of the call.
defer func(r []*ffcli.Command) { c.Subcommands = r }(c.Subcommands)
c.Subcommands = s
// Render the usage.
if usageFunc == nil {
return ffcli.DefaultUsageFunc(c)
}
return usageFunc(c)
},
Subcommands: append(
scriptCmds(root, usageFunc),
completeCmd,
),
},
)
}
// Flag registers a completion function for the flag in fs with given name.
// comp will always called with a 1-element slice.
//
// comp will be called to return suggestions when the user tries to tab-complete
// '--name=<TAB>' or '--name <TAB>' for the commands using fs.
func Flag(fs *flag.FlagSet, name string, comp CompleteFunc) {
f := fs.Lookup(name)
if f == nil {
panic(fmt.Errorf("ffcomplete.Flag: flag %s not found", name))
}
if internal.CompleteFlags == nil {
internal.CompleteFlags = make(map[*flag.Flag]CompleteFunc)
}
internal.CompleteFlags[f] = comp
}
// Args registers a completion function for the args of cmd.
//
// comp will be called to return suggestions when the user tries to tab-complete
// `prog <TAB>` or `prog subcmd arg1 <TAB>`, for example.
func Args(cmd *ffcli.Command, comp CompleteFunc) {
if internal.CompleteCmds == nil {
internal.CompleteCmds = make(map[*ffcli.Command]CompleteFunc)
}
internal.CompleteCmds[cmd] = comp
}

@ -0,0 +1,17 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
//go:build go1.19 && ts_omit_completion
package ffcomplete
import (
"flag"
"github.com/peterbourgon/ff/v3/ffcli"
)
func Inject(root *ffcli.Command, hide func(*ffcli.Command), usageFunc func(*ffcli.Command) string) {}
func Flag(fs *flag.FlagSet, name string, comp CompleteFunc) {}
func Args(cmd *ffcli.Command, comp CompleteFunc) *ffcli.Command { return cmd }

@ -0,0 +1,60 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
package ffcomplete
import (
"strings"
"tailscale.com/cmd/tailscale/cli/ffcomplete/internal"
"tailscale.com/tempfork/spf13/cobra"
)
type ShellCompDirective = cobra.ShellCompDirective
const (
ShellCompDirectiveError = cobra.ShellCompDirectiveError
ShellCompDirectiveNoSpace = cobra.ShellCompDirectiveNoSpace
ShellCompDirectiveNoFileComp = cobra.ShellCompDirectiveNoFileComp
ShellCompDirectiveFilterFileExt = cobra.ShellCompDirectiveFilterFileExt
ShellCompDirectiveFilterDirs = cobra.ShellCompDirectiveFilterDirs
ShellCompDirectiveKeepOrder = cobra.ShellCompDirectiveKeepOrder
ShellCompDirectiveDefault = cobra.ShellCompDirectiveDefault
)
// CompleteFunc is used to return tab-completion suggestions to the user as they
// are typing command-line instructions. It returns the list of things to
// suggest and an additional directive to the shell about what extra
// functionality to enable.
type CompleteFunc = internal.CompleteFunc
// LastArg returns the last element of args, or the empty string if args is
// empty.
func LastArg(args []string) string {
if len(args) == 0 {
return ""
}
return args[len(args)-1]
}
// Fixed returns a CompleteFunc which suggests the given words.
func Fixed(words ...string) CompleteFunc {
return func(args []string) ([]string, cobra.ShellCompDirective, error) {
match := LastArg(args)
matches := make([]string, 0, len(words))
for _, word := range words {
if strings.HasPrefix(word, match) {
matches = append(matches, word)
}
}
return matches, cobra.ShellCompDirectiveNoFileComp, nil
}
}
// FilesWithExtensions returns a CompleteFunc that tells the shell to limit file
// suggestions to those with the given extensions.
func FilesWithExtensions(exts ...string) CompleteFunc {
return func(args []string) ([]string, cobra.ShellCompDirective, error) {
return exts, cobra.ShellCompDirectiveFilterFileExt, nil
}
}

@ -0,0 +1,270 @@
// Copyright (c) Tailscale Inc & AUTHORS
// SPDX-License-Identifier: BSD-3-Clause
package internal
import (
"flag"
"fmt"
"strings"
"github.com/peterbourgon/ff/v3"
"github.com/peterbourgon/ff/v3/ffcli"
"tailscale.com/tempfork/spf13/cobra"
)
var (
CompleteCmds map[*ffcli.Command]CompleteFunc
CompleteFlags map[*flag.Flag]CompleteFunc
)
type CompleteFunc func([]string) ([]string, cobra.ShellCompDirective, error)
// Complete returns the autocomplete suggestions for the root program and args.
//
// The returned words do not necessarily need to be prefixed with the last arg
// which is being completed. For example, '--bool-flag=' will have completions
// 'true' and 'false'.
//
// "HIDDEN: " is trimmed from the start of Flag Usage's.
func Complete(root *ffcli.Command, args []string, startFlags, descs bool) (words []string, dir cobra.ShellCompDirective, err error) {
// Explicitly log panics.
defer func() {
if r := recover(); r != nil {
if rerr, ok := err.(error); ok {
err = fmt.Errorf("panic: %w", rerr)
} else {
err = fmt.Errorf("panic: %v", r)
}
}
}()
// Set up the arguments.
if len(args) == 0 {
args = []string{""}
}
// Completion criteria.
completeArg := args[len(args)-1]
args = args[:len(args)-1]
emitFlag := startFlags || strings.HasPrefix(completeArg, "-")
emitArgs := true
// Traverse the command-tree to find the cmd command whose
// subcommand, flags, or arguments are being completed.
cmd := root
walk:
for {
// Ensure there's a flagset with ContinueOnError set.
if cmd.FlagSet == nil {
cmd.FlagSet = flag.NewFlagSet(cmd.Name, flag.ContinueOnError)
}
cmd.FlagSet.Init(cmd.FlagSet.Name(), flag.ContinueOnError)
// Manually split the args so we know when we're completing flags/args.
flagArgs, argArgs, flagNeedingValue := splitFlagArgs(cmd.FlagSet, args)
if flagNeedingValue != "" {
completeArg = flagNeedingValue + "=" + completeArg
emitFlag = true
}
args = argArgs
// Parse the flags.
err := ff.Parse(cmd.FlagSet, flagArgs, cmd.Options...)
if err != nil {
return nil, 0, fmt.Errorf("%s flag parsing: %w", cmd.Name, err)
}
if cmd.FlagSet.NArg() > 0 {
// This shouldn't happen if splitFlagArgs is accurately finding the
// split between flags and args.
_ = false
}
if len(args) == 0 {
break
}
// Check if the first argument is actually a subcommand.
for _, sub := range cmd.Subcommands {
if strings.EqualFold(sub.Name, args[0]) {
args = args[1:]
cmd = sub
continue walk
}
}
break
}
if len(args) > 0 {
emitFlag = false
}
// Complete '-flag=...'. If the args ended with '-flag ...' we will have
// rewritten to '-flag=...' by now.
if emitFlag && strings.HasPrefix(completeArg, "-") && strings.Contains(completeArg, "=") {
// Don't complete '-flag' later on as the
// flag name is terminated by a '='.
emitFlag = false
emitArgs = false
dashFlag, completeVal, _ := strings.Cut(completeArg, "=")
_, f := cutDash(dashFlag)
flag := cmd.FlagSet.Lookup(f)
if flag != nil {
if comp := CompleteFlags[flag]; comp != nil {
// Complete custom flag values.
var err error
words, dir, err = comp([]string{completeVal})
if err != nil {
return nil, 0, fmt.Errorf("completing %s flag %s: %w", cmd.Name, flag.Name, err)
}
} else if isBoolFlag(flag) {
// Complete true/false.
for _, vals := range [][]string{
{"true", "TRUE", "True", "1"},
{"false", "FALSE", "False", "0"},
} {
for _, val := range vals {
if strings.HasPrefix(val, completeVal) {
words = append(words, val)
break
}
}
}
}
}
}
// Complete '-flag...'.
if emitFlag {
used := make(map[string]struct{})
cmd.FlagSet.Visit(func(f *flag.Flag) {
used[f.Name] = struct{}{}
})
cd, cf := cutDash(completeArg)
cmd.FlagSet.VisitAll(func(f *flag.Flag) {
if !strings.HasPrefix(f.Name, cf) {
return
}
// Skip flags already set by the user.
if _, seen := used[f.Name]; seen {
return
}
// Suggest single-dash '-v' for single-char flags and
// double-dash '--verbose' for longer.
d := cd
if (d == "" || d == "-") && cf == "" && len(f.Name) > 1 {
d = "--"
}
if descs {
_, usage := flag.UnquoteUsage(f)
usage = strings.TrimPrefix(usage, "HIDDEN: ")
if usage != "" {
words = append(words, d+f.Name+"\t"+usage)
return
}
}
words = append(words, d+f.Name)
})
}
if emitArgs {
// Complete 'sub...'.
for _, sub := range cmd.Subcommands {
if strings.HasPrefix(sub.Name, completeArg) {
if descs {
if sub.ShortHelp != "" {
words = append(words, sub.Name+"\t"+sub.ShortHelp)
continue
}
}
words = append(words, sub.Name)
}
}
// Complete custom args.
if comp := CompleteCmds[cmd]; comp != nil {
w, d, err := comp(append(args, completeArg))
if err != nil {
return nil, 0, fmt.Errorf("completing %s args: %w", cmd.Name, err)
}
dir = d
words = append(words, w...)
}
}
// Strip any descriptions if they were suppressed.
clean := words[:0]
for _, w := range words {
if !descs {
w, _, _ = strings.Cut(w, "\t")
}
w = cutAny(w, "\n\r")
if w == "" || w[0] == '\t' {
continue
}
clean = append(clean, w)
}
return clean, dir, nil
}
func cutAny(s, cutset string) string {
i := strings.IndexAny(s, cutset)
if i == -1 {
return s
}
return s[:i]
}
// splitFlagArgs separates a list of command-line arguments into arguments
// comprising flags and their values, preceding arguments to be passed to the
// command. This follows the stdlib 'flag' parsing conventions. If the final
// argument is a flag name which takes a value but has no value specified, it is
// omitted from flagArgs and argArgs and instead returned in needValue.
func splitFlagArgs(fs *flag.FlagSet, args []string) (flagArgs, argArgs []string, flagNeedingValue string) {
for i := 0; i < len(args); i++ {
a := args[i]
if a == "--" {
return args[:i], args[i+1:], ""
}
d, f := cutDash(a)
if d == "" {
return args[:i], args[i:], ""
}
if strings.Contains(f, "=") {
continue
}
flag := fs.Lookup(f)
if flag == nil {
return args[:i], args[i:], ""
}
if isBoolFlag(flag) {
continue
}
// Consume an extra argument for the flag value.
if i == len(args)-1 {
return args[:i], nil, args[i]
}
i++
}
return args, nil, ""
}
func cutDash(s string) (dashes, flag string) {
if strings.HasPrefix(s, "-") {
if strings.HasPrefix(s[1:], "-") {
return "--", s[2:]
}
return "-", s[1:]
}
return "", s
}
func isBoolFlag(f *flag.Flag) bool {
bf, ok := f.Value.(interface {
IsBoolFlag() bool
})
return ok && bf.IsBoolFlag()
}

Some files were not shown because too many files have changed in this diff Show More

Loading…
Cancel
Save