Commit Graph

177 Commits (a7a394e7d9d32991ddd3b6f99ed9efa95e88e01f)

Author SHA1 Message Date
Irbe Krumina a21bf100f3
cmd/k8s-operator,k8s-operator/sessionrecording,sessionrecording,ssh/tailssh: refactor session recording functionality (#12945)
cmd/k8s-operator,k8s-operator/sessionrecording,sessionrecording,ssh/tailssh: refactor session recording functionality

Refactor SSH session recording functionality (mostly the bits related to
Kubernetes API server proxy 'kubectl exec' session recording):

- move the session recording bits used by both Tailscale SSH
and the Kubernetes API server proxy into a shared sessionrecording package,
to avoid having the operator to import ssh/tailssh

- move the Kubernetes API server proxy session recording functionality
into a k8s-operator/sessionrecording package, add some abstractions
in preparation for adding support for a second streaming protocol (WebSockets)

Updates tailscale/corp#19821

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
4 months ago
Irbe Krumina ba517ab388
cmd/k8s-operator,ssh/tailssh,tsnet: optionally record 'kubectl exec' sessions via Kubernetes operator's API server proxy (#12274)
cmd/k8s-operator,ssh/tailssh,tsnet: optionally record kubectl exec sessions

The Kubernetes operator's API server proxy, when it receives a request
for 'kubectl exec' session now reads 'RecorderAddrs', 'EnforceRecorder'
fields from tailcfg.KubernetesCapRule.
If 'RecorderAddrs' is set to one or more addresses (of a tsrecorder instance(s)),
it attempts to connect to those and sends the session contents
to the recorder before forwarding the request to the kube API
server. If connection cannot be established or fails midway,
it is only allowed if 'EnforceRecorder' is not true (fail open).

Updates tailscale/corp#19821

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Co-authored-by: Maisem Ali <maisem@tailscale.com>
4 months ago
Irbe Krumina 07063bc5c7
ssh/tailssh: fix integration test (#12562)
Updates#cleanup

Signed-off-by: Irbe Krumina <irbe@tailscale.com>
5 months ago
Percy Wegmann 730f0368d0 ssh/tailssh: replace incubator process with su instead of running su as child
This allows the SSH_AUTH_SOCK environment variable to work inside of
su and agent forwarding to succeed.

Fixes #12467

Signed-off-by: Percy Wegmann <percy@tailscale.com>
5 months ago
Andrew Dunham 45d2f4301f proxymap, various: distinguish between different protocols
Previously, we were registering TCP and UDP connections in the same map,
which could result in erroneously removing a mapping if one of the two
connections completes while the other one is still active.

Add a "proto string" argument to these functions to avoid this.
Additionally, take the "proto" argument in LocalAPI, and plumb that
through from the CLI and add a new LocalClient method.

Updates tailscale/corp#20600

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I35d5efaefdfbf4721e315b8ca123f0c8af9125fb
5 months ago
Percy Wegmann d7fdc01f7f ssh/tailssh: check IsSELinuxEnforcing in tailscaled process
Checking in the incubator as this used to do fails because
the getenforce command is not on the PATH.

Updates #12442

Signed-off-by: Percy Wegmann <percy@tailscale.com>
5 months ago
Percy Wegmann 08a9551a73 ssh/tailssh: fall back to using su when no TTY available on Linux
This allows pam authentication to run for ssh sessions, triggering
automation like pam_mkhomedir.

Updates #11854

Signed-off-by: Percy Wegmann <percy@tailscale.com>
6 months ago
Brad Fitzpatrick 46f3feae96 ssh/tailssh: plumb health.Tracker in test
In prep for it being required in more places.

Updates #11874

Change-Id: Ib743205fc2a6c6ff3d2c4ed3a2b28cac79156539
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 months ago
Brad Fitzpatrick 4fa6cbec27 ssh/tailssh: use ptr.To in test
Updates #cleanup

Change-Id: Ic98ba1b63c8205084b30f59f0ca343788edea5b0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
6 months ago
Percy Wegmann 843afe7c53 ssh/tailssh: add integration test
Updates tailscale/corp#11854

Signed-off-by: Percy Wegmann <percy@tailscale.com>
7 months ago
Andrew Dunham e985c6e58f ssh/tailssh: try fetching group IDs for user with the 'id' command
Since the tailscaled binaries that we distribute are static and don't
link cgo, we previously wouldn't fetch group IDs that are returned via
NSS. Try shelling out to the 'id' command, similar to how we call
'getent', to detect such cases.

Updates #11682

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I9bdc938bd76c71bc130d44a97cc2233064d64799
7 months ago
Andrew Lytvynov b743b85dad
ipn/ipnlocal,ssh/tailssh: reject c2n /update if SSH conns are active (#11820)
Since we already track active SSH connections, it's not hard to
proactively reject updates until those finish. We attempt to do the same
on the control side, but the detection latency for new connections is in
the minutes, which is not fast enough for common short sessions.

Handle a `force=true` query parameter to override this behavior, so that
control can still trigger an update on a server where some long-running
abandoned SSH session is open.

Updates https://github.com/tailscale/corp/issues/18556

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
7 months ago
Brad Fitzpatrick 7c1d6e35a5 all: use Go 1.22 range-over-int
Updates #11058

Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
Joe Tsai 94a4f701c2
all: use reflect.TypeFor now available in Go 1.22 (#11078)
Updates #cleanup

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
9 months ago
Andrew Lytvynov 29e98e18f8
ssh/tailssh: use a local error instead of gossh.ErrDenied (#10743)
ErrDenied was added in [our fork of
x/crypto/ssh](acc6f8fe8d)
to short-circuit auth attempts once one fails.

In the case of our callbacks, this error is returned when SSH policy
check determines that a connection should not be allowed. Both
`NoClientAuthCallback` and `PublicKeyHandler` check the policy and will
fail anyway. The `fakePasswordHandler` returns true only if
`NoClientAuthCallback` succeeds the policy check, so it checks it
indirectly too.

The difference here is that a client might attempt all 2-3 auth methods
instead of just `none` but will fail to authenticate regardless.

Updates #8593

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
10 months ago
Andrew Lytvynov 1302bd1181
all: cleanup unused code, part 1 (#10661)
Run `staticcheck` with `U1000` to find unused code. This cleans up about
a half of it. I'll do the other half separately to keep PRs manageable.

Updates #cleanup

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
11 months ago
Ox Cart 719ee4415e ssh/tailssh: use control server time instead of local time
This takes advantage of existing functionality in ipn/ipnlocal to adjust
the local clock based on periodic time signals from the control server.
This way, when checking things like SSHRule expirations, calculations are
protected incorrectly set local clocks.

Fixes tailscale/corp#15796

Signed-off-by: Percy Wegmann <percy@tailscale.com>
1 year ago
Brad Fitzpatrick 53c4adc982 ssh/tailssh: add envknobs to force override forwarding, sftp, pty
Updates tailscale/corp#15735

Change-Id: Ib1303406be925c3231ce7e0950a173ad12626492
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick ffabe5fe21 ssh/tailssh: fix sftp metric increment location
We were incrementing the sftp metric on regular sessions
too, not just sftp.

Updates #cleanup

Change-Id: I63027a39cffb3e03397c6e4829b1620c10fa3130
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Andrew Lytvynov 1fc1077052
ssh/tailssh,util: extract new osuser package from ssh code (#10170)
This package is a wrapper for os/user that handles non-cgo builds,
gokrazy and user shells.

Updates tailscale/corp#15405

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
1 year ago
Brad Fitzpatrick b775a3799e util/httpm, all: add a test to make sure httpm is used consistently
Updates #cleanup

Change-Id: I7dbf8a02de22fc6b317ab5e29cc97792dd75352c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Joe Tsai 0a0adb68ad
ssh/tailssh: log when recording starts and finishes (#9294)
Updates tailscale/corp#14579

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
1 year ago
Marwan Sulaiman 9c07f4f512 all: replace deprecated ioutil references
This PR removes calls to ioutil library and replaces them
with their new locations in the io and os packages.

Fixes #9034
Updates #5210

Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
1 year ago
Brad Fitzpatrick 84b94b3146 types/netmap, all: make NetworkMap.SelfNode a tailcfg.NodeView
Updates #1909

Change-Id: I8c470cbc147129a652c1d58eac9b790691b87606
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick 58a4fd43d8 types/netmap, all: use read-only tailcfg.NodeView in NetworkMap
Updates #8948

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick e8551d6b40 all: use Go 1.21 slices, maps instead of x/exp/{slices,maps}
Updates #8419

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick bd02d00608 ssh/tailssh: fix gokrazy SSH crash
Stupid mistake in earlier refactor.

Updates gokrazy/gokrazy#209

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Joe Tsai bb4b35e923 ssh: ignore io.EOF from sftp.Server.Serve
If the connection provided to sftp.NewServer is closed,
Serve returns the io.EOF error verbatim from io.Reader.Read.
This is an odd error since this is an expected situation,
so we manually ignore io.EOF.
This is somewhat buggy since the sftp package itself
incorrectly reports io.EOF in cases where it should actually
be reporting io.ErrUnexpectedEOF.
See https://github.com/pkg/sftp/pull/554 which patches Serve to
return nil on clean closes and fixes buggy uses of io.ReadFull.

Fixes #8592

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
1 year ago
Joe Tsai 61886e031e
ssh/tailssh: fix double race condition with non-pty command (#8405)
There are two race conditions in output handling.

The first race condition is due to a misuse of exec.Cmd.StdoutPipe.
The documentation explicitly forbids concurrent use of StdoutPipe
with exec.Cmd.Wait (see golang/go#60908) because Wait will
close both sides of the pipe once the process ends without
any guarantees that all data has been read from the pipe.
To fix this, we allocate the os.Pipes ourselves and
manage cleanup ourselves when the process has ended.

The second race condition is because sshSession.run waits
upon exec.Cmd to finish and then immediately proceeds to call ss.Exit,
which will close all output streams going to the SSH client.
This may interrupt any asynchronous io.Copy still copying data.
To fix this, we close the write-side of the os.Pipes after
the process has finished (and before calling ss.Exit) and
synchronously wait for the io.Copy routines to finish.

Fixes #7601

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
Co-authored-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Brad Fitzpatrick 32b8f25ed1 Revert "ssh/tailssh: change to user directory when running login/command"
This reverts commit dc5bc32d8f.

It broke tests. (sadly, ones which we have disabled on CI, but go test
./ssh/tailssh broke)
1 year ago
Derek Burdick dc5bc32d8f ssh/tailssh: change to user directory when running login/command
On redhat 9 and similarly locked down systems, root user does not have
access to a users directory. This fix does not set a directory for the
incubator process and instead sets the directory when the actual process
requested by remote user is executed.

Fixes #8118

Signed-off-by: Derek Burdick <derek-burdick@users.noreply.github.com>
1 year ago
Maisem Ali 2ae670eb71 ssh/tailssh: work around lack of scontext in SELinux
Trying to SSH when SELinux is enforced results in errors like:

```
➜  ~ ssh ec2-user@<ip>
Last login: Thu Jun  1 22:51:44 from <ip2>
ec2-user: no shell: Permission denied
Connection to <ip> closed.
```

while the `/var/log/audit/audit.log` has
```
type=AVC msg=audit(1685661291.067:465): avc:  denied  { transition } for  pid=5296 comm="login" path="/usr/bin/bash" dev="nvme0n1p1" ino=2564 scontext=system_u:system_r:unconfined_service_t:s0 tcontext=unconfined_u:unconfined_r:unconfined_t:s0 tclass=process permissive=0
```

The right fix here would be to somehow install the appropriate context when
tailscale is installed on host, but until we figure out a way to do that
stop using the `login` cmd in these situations.

Updates #4908

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Maisem Ali 2e0aa151c9 ssh/tailssh: add support for remote/reverse port forwarding
This basically allows running services on the SSH client and reaching
them from the SSH server during the session.

Updates #6575

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Derek Burdick d3c8c3dd00 ssh/tailssh: Max Username Length 256 for linux
Max username length is increased to 256 on linux to match /usr/include/bits/local_lim.h

Fixes #8277

Signed-off-by: Derek Burdick <derek-burdick@users.noreply.github.com>
1 year ago
Charlotte Brandhorst-Satzkorn 4e86857313 ssh/tailssh: add ssh session recording failed event type
This change introduces a SSHSessionRecordingFailed event type
that is used when a session recording fails to start or fails during a
session, and the on failure indicates that it should fail open.

Updates tailscale/corp#9967

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Brad Fitzpatrick a4fd4fd845 ssh/tailssh: fix regression after LDAP support
58ab66ec51 added LDAP support
for #4945 by shelling out to getdent.

It was supposed to fall back to the old method when getdent wasn't
found, but some variable name confusion (uid vs username) meant the
old path wasn't calling the right lookup function (user.LookupId
instead of user.Lookup).

Which meant that changed probably also broke FreeBSD and macOS SSH
support in addition to the reported OpenWRT regression.

The gokrazy support didn't look right either.

Fixes #8180

Change-Id: I273bbe96fe98b2517fbf0335fd476b483c051554
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Maisem Ali 0ca8bf1e26 ssh/tailssh: close tty on session close
We were only closing on side of the pty/tty pair.
Close the other side too.

Thanks to @fritterhoff for reporting and debugging the issue!

Fixes #8119

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Brad Fitzpatrick a743b66f9d ssh/tailssh: move some user-related code into new user.go
The previous commit 58ab66e added ssh/tailssh/user.go as part of
working on #4945. So move some more user-related code over to it.

Updates #cleanup

Change-Id: I24de66df25ffb8f867e1a0a540d410f9ef16d7b0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 58ab66ec51 ssh/tailssh: support LDAP users for Tailscale SSH
Fixes #4945

Change-Id: Ie013cb47684cb87928a44f92c66352310bfe53f1
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Charlotte Brandhorst-Satzkorn 29ded8f9f9 ssh/tailssh,tailcfg: add connID to ssheventnotifyrequest and castheader
This change adds a ConnectionID field to both SSHEventNotifyRequest and
CastHeader that identifies the ID of a connection to the SSH server.

Updates tailscale/corp#9967

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Charlotte Brandhorst-Satzkorn 68307c1411 ssh/tailssh: send ssh event notifications on recording failures
This change sends an SSHEventNotificationRequest over noise when a
SSH session is set to fail closed and the session is unable to start
because a recorder is not available or a session is terminated because
connection to the recorder is ended. Each of these scenarios have their
own event type.

Updates tailscale/corp#9967

Signed-off-by: Charlotte Brandhorst-Satzkorn <charlotte@tailscale.com>
2 years ago
Brad Fitzpatrick 6e967446e4 tsd: add package with System type to unify subsystem init, discovery
This is part of an effort to clean up tailscaled initialization between
tailscaled, tailscaled Windows service, tsnet, and the mac GUI.

Updates #8036

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Maisem Ali be190e990f ssh/tailssh: restore support for recording locally
We removed it earlier in 916aa782af, but we still want to support it for some time longer.

Updates tailscale/corp#9967

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Tom DNetto c5bf868940 ssh/tailssh: improve debug logging around revoked sessions
Updates https://github.com/tailscale/corp/issues/10943
Signed-off-by: Tom DNetto <tom@tailscale.com>
2 years ago
Maisem Ali 1b8a0dfe5e ssh/tailssh: also handle recording upload failure during writes
Previously we would error out when the recording server disappeared after the in memory
buffer filled up for the io.Copy. This makes it so that we handle failing open correctly
in that path.

Updates tailscale/corp#9967

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Maisem Ali 7778d708a6 ssh/tailssh: handle dialing multiple recorders and failing open
This adds support to try dialing out to multiple recorders each
with a 5s timeout and an overall 30s timeout. It also starts respecting
the actions `OnRecordingFailure` field if set, if it is not set
it fails open.

Updates tailscale/corp#9967

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Andrew Dunham 280255acae
various: add golangci-lint, fix issues (#7905)
This adds an initial and intentionally minimal configuration for
golang-ci, fixes the issues reported, and adds a GitHub Action to check
new pull requests against this linter configuration.

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I8f38fbc315836a19a094d0d3e986758b9313f163
2 years ago
Maisem Ali d42d570066
ssh/tailssh: handle output matching better in tests (#7799) 2 years ago
Brad Fitzpatrick 2c0bda6e2e ssh/tailssh: make Tailscale SSH work on gokrazy
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Maisem Ali e04acabfde ssh/tailssh: fix race in errors returned when starting recorder
There were two code paths that could fail depending on how fast
the recorder responses. This fixes that by returning the correct
error from both paths.

Fixes #7707

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago