You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
Go to file
Joe Tsai d4bfe34ba7
util/zstdframe: add package for stateless zstd compression (#11481)
The Go zstd package is not friendly for stateless zstd compression.
Passing around multiple zstd.Encoder just for stateless compression
is a waste of memory since the memory is never freed and seldom
used if no compression operations are happening.

For performance, we pool the relevant Encoder/Decoder
with the specific options set.

Functionally, this package is a wrapper over the Go zstd package
with a more ergonomic API for stateless operations.

This package can be used to cleanup various pre-existing zstd.Encoder
pools or one-off handlers spread throughout our codebases.

Performance:

	BenchmarkEncode/Best               1690        610926 ns/op      25.78 MB/s           1 B/op          0 allocs/op
	    zstd_test.go:137: memory: 50.336 MiB
	    zstd_test.go:138: ratio:  3.269x
	BenchmarkEncode/Better            10000        100939 ns/op     156.04 MB/s           0 B/op          0 allocs/op
	    zstd_test.go:137: memory: 20.399 MiB
	    zstd_test.go:138: ratio:  3.131x
	BenchmarkEncode/Default            15775         74976 ns/op     210.08 MB/s         105 B/op          0 allocs/op
	    zstd_test.go:137: memory: 1.586 MiB
	    zstd_test.go:138: ratio:  3.064x
	BenchmarkEncode/Fastest            23222         53977 ns/op     291.81 MB/s          26 B/op          0 allocs/op
	    zstd_test.go:137: memory: 599.458 KiB
	    zstd_test.go:138: ratio:  2.898x
	BenchmarkEncode/FastestLowMemory                   23361         50789 ns/op     310.13 MB/s          15 B/op          0 allocs/op
	    zstd_test.go:137: memory: 334.458 KiB
	    zstd_test.go:138: ratio:  2.898x
	BenchmarkEncode/FastestNoChecksum                  23086         50253 ns/op     313.44 MB/s          26 B/op          0 allocs/op
	    zstd_test.go:137: memory: 599.458 KiB
	    zstd_test.go:138: ratio:  2.900x

	BenchmarkDecode/Checksum                           70794         17082 ns/op     300.96 MB/s           4 B/op          0 allocs/op
	    zstd_test.go:163: memory: 316.438 KiB
	BenchmarkDecode/NoChecksum                         74935         15990 ns/op     321.51 MB/s           4 B/op          0 allocs/op
	    zstd_test.go:163: memory: 316.438 KiB
	BenchmarkDecode/LowMemory                          71043         16739 ns/op     307.13 MB/s           0 B/op          0 allocs/op
	    zstd_test.go:163: memory: 79.347 KiB

We can see that the options are taking effect where compression ratio improves
with higher levels and compression speed diminishes.
We can also see that LowMemory takes effect where the pooled coder object
references less memory than other cases.
We can see that the pooling is taking effect as there are 0 amortized allocations.

Additional performance:

	BenchmarkEncodeParallel/zstd-24                     1857        619264 ns/op        1796 B/op         49 allocs/op
	BenchmarkEncodeParallel/zstdframe-24                1954        532023 ns/op        4293 B/op         49 allocs/op
	BenchmarkDecodeParallel/zstd-24                     5288        197281 ns/op        2516 B/op         49 allocs/op
	BenchmarkDecodeParallel/zstdframe-24                6441        196254 ns/op        2513 B/op         49 allocs/op

In concurrent usage, handling the pooling in this package
has a marginal benefit over the zstd package,
which relies on a Go channel as the pooling mechanism.
In particular, coders can be freed by the GC when not in use.
Coders can be shared throughout the program if they use this package
instead of multiple independent pools doing the same thing.
The allocations are unrelated to pooling as they're caused by the spawning of goroutines.

Updates #cleanup
Updates tailscale/corp#18514
Updates tailscale/corp#17653
Updates tailscale/corp#18005

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
8 months ago
.bencher bencher: add config to suppress failures on benchmark regressions. 3 years ago
.github .github/workflows: fix regular breakage of go toolchains 9 months ago
appc appc: optimize dns response observation for large route tables 10 months ago
atomicfile various: add golangci-lint, fix issues (#7905) 2 years ago
chirp all: update copyright and license headers 2 years ago
client ipn,cmd/tailscale,client/tailscale: add support for renaming TailFS shares 9 months ago
clientupdate clientupdate: do not allow msiexec to reboot the OS (#11409) 8 months ago
cmd cmd/k8s-operator,ipn/conf.go: fix --accept-routes for proxies (#11453) 8 months ago
control control/controlclient: do not alias peer CapMap 8 months ago
derp cmd/derper, derp, tailcfg: add admission controller URL option 9 months ago
disco disco: correct noun for nacl box type in disco docs 11 months ago
docs docs/k8s: don't run subnet router in userspace mode (#11363) 9 months ago
doctor doctor/ethtool, ipn/ipnlocal: add ethtool bugreport check 9 months ago
envknob envknob: ensure f is not nil before using it 8 months ago
health health: warn about reverse path filtering and exit nodes 9 months ago
hostinfo hostinfo: fix a couple of logic simplification lints 1 year ago
internal/tooldeps various: add golangci-lint, fix issues (#7905) 2 years ago
ipn ipn/ipnlocal: add c2n pprof option to force a GC 8 months ago
jsondb all: update copyright and license headers 2 years ago
k8s-operator cmd/k8s-operator,k8s-operator,go.{mod,sum},tstest/tools: add Tailscale Kubernetes operator API docs (#11246) 9 months ago
kube cmd/k8s-operator,ipn/store/kubestore: patch secrets instead of updating 1 year ago
licenses licenses: update android licenses 9 months ago
log tailcfg: move LogHeapPprof from Debug to c2n [capver 69] 1 year ago
logpolicy logpolicy: allow longer idle log upload connections 9 months ago
logtail tailscale/logtail: redact public ipv6 and ipv4 ip addresses within tailscaled. (#10531) 11 months ago
metrics metrics: add missing comma in histogram JSON export 1 year ago
net net/routetable: increase route limit used by the test 8 months ago
packages/deb go.mod: upgrade nfpm to v2 (#8786) 1 year ago
paths safesocket, paths: add Plan 9 support 1 year ago
portlist all: cleanup unused code, part 2 (#10670) 11 months ago
posture ci: run 'go vet' in golangci-lint; fix errors in tests 12 months ago
prober prober: add a DERP bandwidth probe 8 months ago
proxymap wgengine, proxymap: split out port mapping from Engine to new type 1 year ago
release version/mkversion: enforce synology versions within int32 range 9 months ago
safesocket version,cli,safesocket: detect non-sandboxed macOS GUI (#11369) 8 months ago
scripts scripts/installer.sh: auto-start tailscale on Alpine (#11214) 9 months ago
smallzstd all: update copyright and license headers 2 years ago
ssh/tailssh all: use reflect.TypeFor now available in Go 1.22 (#11078) 10 months ago
syncs syncs: add Swap method 8 months ago
tailcfg control/controlclient: send load balancing hint HTTP request header 8 months ago
taildrop all: use new AppendEncode methods available in Go 1.22 (#11079) 10 months ago
tailfs tailfs: include whitespace in test share and filenames 8 months ago
tempfork all: use Go 1.21 slices, maps instead of x/exp/{slices,maps} 1 year ago
tka all: use new AppendEncode methods available in Go 1.22 (#11079) 10 months ago
tool tool/gocross: add android autoflags (#11465) 8 months ago
tsconst all: update copyright and license headers 2 years ago
tsd tailfs: clean up naming and package structure 10 months ago
tsnet all: remove LenIter, use Go 1.22 range-over-int instead 9 months ago
tstest cmd/k8s-operator,k8s-operator,go.{mod,sum},tstest/tools: add Tailscale Kubernetes operator API docs (#11246) 9 months ago
tstime tstime/rate: implement Value.{Marshal,Unmarshal}JSON (#8481) 10 months ago
tsweb tsweb: add String method to tsweb.RequestID 9 months ago
types types/views: optimize slices contains under some conditions (#11449) 8 months ago
util util/zstdframe: add package for stateless zstd compression (#11481) 8 months ago
version version/prop: remove IsMacAppSandboxEnabled (#11461) 8 months ago
wf go.mod, all: move away from inet.af domain seized by Taliban 9 months ago
wgengine wgengine/magicsock: don't use endpoint debug ringbuffer on mobile. 8 months ago
words words: add a list of things you should yahoo! 10 months ago
.gitattributes .: add .gitattributes entry to use Go hunk-header driver 3 years ago
.gitignore client/web: always use new web client; remove old client 1 year ago
.golangci.yml ci: run 'go vet' in golangci-lint; fix errors in tests 12 months ago
ALPINE.txt ALPINE.txt,Dockerfile{.base},build_docker.sh: bump alpine (#10543) 12 months ago
AUTHORS Move Linux client & common packages into a public repo. 5 years ago
CODEOWNERS CODEOWNERS: add the start of an owners file 1 year ago
CODE_OF_CONDUCT.md Add a code of conduct. 5 years ago
Dockerfile Dockerfile: use Go 1.22 10 months ago
Dockerfile.base ALPINE.txt,Dockerfile{.base},build_docker.sh: bump alpine (#10543) 12 months ago
LICENSE all: update tools that manage copyright headers 2 years ago
Makefile Makefile,build_docker.sh: allow to configure target platform. (#10806) 11 months ago
PATENTS Move Linux client & common packages into a public repo. 5 years ago
README.md go.mod, README.md: use Go 1.22 10 months ago
SECURITY.md Add a SECURITY.md for vulnerability reports. 5 years ago
VERSION.txt VERSION.txt: this is v1.63.0 8 months ago
api.md api.md: add missing backtick to GET searchpaths doc (#11459) 8 months ago
build_dist.sh client/web: switch to using prebuilt web client assets 1 year ago
build_docker.sh Code Improvements (#11311) 9 months ago
flake.lock flake.nix: build tailscale with go 1.22 9 months ago
flake.nix go.mod.sri: update SRI hash for go.mod changes 9 months ago
go.mod go.mod: bump gvisor 9 months ago
go.mod.sri go.mod.sri: update SRI hash for go.mod changes 9 months ago
go.sum go.mod: bump gvisor 9 months ago
go.toolchain.branch go.toolchain.rev: bump to Go 1.22.0 (#11055) 10 months ago
go.toolchain.rev go.toolchain.rev: bump Go toolchain to 1.22.1 9 months ago
gomod_test.go go.mod: add test that replace directives aren't added in oss 1 year ago
header.txt cmd/k8s-operator: operator can create subnetrouter (#9505) 11 months ago
pull-toolchain.sh pull-toolchain.sh: don't run update-flake.sh 2 years ago
shell.nix go.mod.sri: update SRI hash for go.mod changes 9 months ago
staticcheck.conf all: cleanup unused code, part 2 (#10670) 11 months ago
update-flake.sh Code Improvements (#11311) 9 months ago
version-embed.go version: return correct Meta.MajorMinorPatch in non-dev builds 2 years ago
version_test.go go.mod,wgengine/netstack: bump gvisor 10 months ago

README.md

Tailscale

https://tailscale.com

Private WireGuard® networks made easy

Overview

This repository contains the majority of Tailscale's open source code. Notably, it includes the tailscaled daemon and the tailscale CLI tool. The tailscaled daemon runs on Linux, Windows, macOS, and to varying degrees on FreeBSD and OpenBSD. The Tailscale iOS and Android apps use this repo's code, but this repo doesn't contain the mobile GUI code.

Other Tailscale repos of note:

For background on which parts of Tailscale are open source and why, see https://tailscale.com/opensource/.

Using

We serve packages for a variety of distros and platforms at https://pkgs.tailscale.com.

Other clients

The macOS, iOS, and Windows clients use the code in this repository but additionally include small GUI wrappers. The GUI wrappers on non-open source platforms are themselves not open source.

Building

We always require the latest Go release, currently Go 1.22. (While we build releases with our Go fork, its use is not required.)

go install tailscale.com/cmd/tailscale{,d}

If you're packaging Tailscale for distribution, use build_dist.sh instead, to burn commit IDs and version info into the binaries:

./build_dist.sh tailscale.com/cmd/tailscale
./build_dist.sh tailscale.com/cmd/tailscaled

If your distro has conventions that preclude the use of build_dist.sh, please do the equivalent of what it does in your distro's way, so that bug reports contain useful version information.

Bugs

Please file any issues about this code or the hosted service on the issue tracker.

Contributing

PRs welcome! But please file bugs. Commit messages should reference bugs.

We require Developer Certificate of Origin Signed-off-by lines in commits.

See git log for our commit message style. It's basically the same as Go's style.

About Us

Tailscale is primarily developed by the people at https://github.com/orgs/tailscale/people. For other contributors, see:

WireGuard is a registered trademark of Jason A. Donenfeld.