You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
Go to file
Joe Tsai d4bfe34ba7
util/zstdframe: add package for stateless zstd compression (#11481)
The Go zstd package is not friendly for stateless zstd compression.
Passing around multiple zstd.Encoder just for stateless compression
is a waste of memory since the memory is never freed and seldom
used if no compression operations are happening.

For performance, we pool the relevant Encoder/Decoder
with the specific options set.

Functionally, this package is a wrapper over the Go zstd package
with a more ergonomic API for stateless operations.

This package can be used to cleanup various pre-existing zstd.Encoder
pools or one-off handlers spread throughout our codebases.

Performance:

	BenchmarkEncode/Best               1690        610926 ns/op      25.78 MB/s           1 B/op          0 allocs/op
	    zstd_test.go:137: memory: 50.336 MiB
	    zstd_test.go:138: ratio:  3.269x
	BenchmarkEncode/Better            10000        100939 ns/op     156.04 MB/s           0 B/op          0 allocs/op
	    zstd_test.go:137: memory: 20.399 MiB
	    zstd_test.go:138: ratio:  3.131x
	BenchmarkEncode/Default            15775         74976 ns/op     210.08 MB/s         105 B/op          0 allocs/op
	    zstd_test.go:137: memory: 1.586 MiB
	    zstd_test.go:138: ratio:  3.064x
	BenchmarkEncode/Fastest            23222         53977 ns/op     291.81 MB/s          26 B/op          0 allocs/op
	    zstd_test.go:137: memory: 599.458 KiB
	    zstd_test.go:138: ratio:  2.898x
	BenchmarkEncode/FastestLowMemory                   23361         50789 ns/op     310.13 MB/s          15 B/op          0 allocs/op
	    zstd_test.go:137: memory: 334.458 KiB
	    zstd_test.go:138: ratio:  2.898x
	BenchmarkEncode/FastestNoChecksum                  23086         50253 ns/op     313.44 MB/s          26 B/op          0 allocs/op
	    zstd_test.go:137: memory: 599.458 KiB
	    zstd_test.go:138: ratio:  2.900x

	BenchmarkDecode/Checksum                           70794         17082 ns/op     300.96 MB/s           4 B/op          0 allocs/op
	    zstd_test.go:163: memory: 316.438 KiB
	BenchmarkDecode/NoChecksum                         74935         15990 ns/op     321.51 MB/s           4 B/op          0 allocs/op
	    zstd_test.go:163: memory: 316.438 KiB
	BenchmarkDecode/LowMemory                          71043         16739 ns/op     307.13 MB/s           0 B/op          0 allocs/op
	    zstd_test.go:163: memory: 79.347 KiB

We can see that the options are taking effect where compression ratio improves
with higher levels and compression speed diminishes.
We can also see that LowMemory takes effect where the pooled coder object
references less memory than other cases.
We can see that the pooling is taking effect as there are 0 amortized allocations.

Additional performance:

	BenchmarkEncodeParallel/zstd-24                     1857        619264 ns/op        1796 B/op         49 allocs/op
	BenchmarkEncodeParallel/zstdframe-24                1954        532023 ns/op        4293 B/op         49 allocs/op
	BenchmarkDecodeParallel/zstd-24                     5288        197281 ns/op        2516 B/op         49 allocs/op
	BenchmarkDecodeParallel/zstdframe-24                6441        196254 ns/op        2513 B/op         49 allocs/op

In concurrent usage, handling the pooling in this package
has a marginal benefit over the zstd package,
which relies on a Go channel as the pooling mechanism.
In particular, coders can be freed by the GC when not in use.
Coders can be shared throughout the program if they use this package
instead of multiple independent pools doing the same thing.
The allocations are unrelated to pooling as they're caused by the spawning of goroutines.

Updates #cleanup
Updates tailscale/corp#18514
Updates tailscale/corp#17653
Updates tailscale/corp#18005

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
.bencher bencher: add config to suppress failures on benchmark regressions. 4 years ago
.github .github/workflows: fix regular breakage of go toolchains 2 years ago
appc appc: optimize dns response observation for large route tables 2 years ago
atomicfile various: add golangci-lint, fix issues (#7905) 3 years ago
chirp all: update copyright and license headers 3 years ago
client ipn,cmd/tailscale,client/tailscale: add support for renaming TailFS shares 2 years ago
clientupdate clientupdate: do not allow msiexec to reboot the OS (#11409) 2 years ago
cmd cmd/k8s-operator,ipn/conf.go: fix --accept-routes for proxies (#11453) 2 years ago
control control/controlclient: do not alias peer CapMap 2 years ago
derp cmd/derper, derp, tailcfg: add admission controller URL option 2 years ago
disco disco: correct noun for nacl box type in disco docs 2 years ago
docs docs/k8s: don't run subnet router in userspace mode (#11363) 2 years ago
doctor doctor/ethtool, ipn/ipnlocal: add ethtool bugreport check 2 years ago
envknob envknob: ensure f is not nil before using it 2 years ago
health health: warn about reverse path filtering and exit nodes 2 years ago
hostinfo hostinfo: fix a couple of logic simplification lints 2 years ago
internal/tooldeps various: add golangci-lint, fix issues (#7905) 3 years ago
ipn ipn/ipnlocal: add c2n pprof option to force a GC 2 years ago
jsondb all: update copyright and license headers 3 years ago
k8s-operator cmd/k8s-operator,k8s-operator,go.{mod,sum},tstest/tools: add Tailscale Kubernetes operator API docs (#11246) 2 years ago
kube cmd/k8s-operator,ipn/store/kubestore: patch secrets instead of updating 2 years ago
licenses licenses: update android licenses 2 years ago
log tailcfg: move LogHeapPprof from Debug to c2n [capver 69] 2 years ago
logpolicy logpolicy: allow longer idle log upload connections 2 years ago
logtail tailscale/logtail: redact public ipv6 and ipv4 ip addresses within tailscaled. (#10531) 2 years ago
metrics metrics: add missing comma in histogram JSON export 2 years ago
net net/routetable: increase route limit used by the test 2 years ago
packages/deb go.mod: upgrade nfpm to v2 (#8786) 2 years ago
paths safesocket, paths: add Plan 9 support 2 years ago
portlist all: cleanup unused code, part 2 (#10670) 2 years ago
posture ci: run 'go vet' in golangci-lint; fix errors in tests 2 years ago
prober prober: add a DERP bandwidth probe 2 years ago
proxymap wgengine, proxymap: split out port mapping from Engine to new type 2 years ago
release version/mkversion: enforce synology versions within int32 range 2 years ago
safesocket version,cli,safesocket: detect non-sandboxed macOS GUI (#11369) 2 years ago
scripts scripts/installer.sh: auto-start tailscale on Alpine (#11214) 2 years ago
smallzstd all: update copyright and license headers 3 years ago
ssh/tailssh all: use reflect.TypeFor now available in Go 1.22 (#11078) 2 years ago
syncs syncs: add Swap method 2 years ago
tailcfg control/controlclient: send load balancing hint HTTP request header 2 years ago
taildrop all: use new AppendEncode methods available in Go 1.22 (#11079) 2 years ago
tailfs tailfs: include whitespace in test share and filenames 2 years ago
tempfork all: use Go 1.21 slices, maps instead of x/exp/{slices,maps} 2 years ago
tka all: use new AppendEncode methods available in Go 1.22 (#11079) 2 years ago
tool tool/gocross: add android autoflags (#11465) 2 years ago
tsconst all: update copyright and license headers 3 years ago
tsd tailfs: clean up naming and package structure 2 years ago
tsnet all: remove LenIter, use Go 1.22 range-over-int instead 2 years ago
tstest cmd/k8s-operator,k8s-operator,go.{mod,sum},tstest/tools: add Tailscale Kubernetes operator API docs (#11246) 2 years ago
tstime tstime/rate: implement Value.{Marshal,Unmarshal}JSON (#8481) 2 years ago
tsweb tsweb: add String method to tsweb.RequestID 2 years ago
types types/views: optimize slices contains under some conditions (#11449) 2 years ago
util util/zstdframe: add package for stateless zstd compression (#11481) 2 years ago
version version/prop: remove IsMacAppSandboxEnabled (#11461) 2 years ago
wf go.mod, all: move away from inet.af domain seized by Taliban 2 years ago
wgengine wgengine/magicsock: don't use endpoint debug ringbuffer on mobile. 2 years ago
words words: add a list of things you should yahoo! 2 years ago
.gitattributes .: add .gitattributes entry to use Go hunk-header driver 4 years ago
.gitignore client/web: always use new web client; remove old client 2 years ago
.golangci.yml ci: run 'go vet' in golangci-lint; fix errors in tests 2 years ago
ALPINE.txt ALPINE.txt,Dockerfile{.base},build_docker.sh: bump alpine (#10543) 2 years ago
AUTHORS Move Linux client & common packages into a public repo. 6 years ago
CODEOWNERS CODEOWNERS: add the start of an owners file 2 years ago
CODE_OF_CONDUCT.md Add a code of conduct. 6 years ago
Dockerfile Dockerfile: use Go 1.22 2 years ago
Dockerfile.base ALPINE.txt,Dockerfile{.base},build_docker.sh: bump alpine (#10543) 2 years ago
LICENSE all: update tools that manage copyright headers 3 years ago
Makefile Makefile,build_docker.sh: allow to configure target platform. (#10806) 2 years ago
PATENTS Move Linux client & common packages into a public repo. 6 years ago
README.md go.mod, README.md: use Go 1.22 2 years ago
SECURITY.md Add a SECURITY.md for vulnerability reports. 6 years ago
VERSION.txt VERSION.txt: this is v1.63.0 2 years ago
api.md api.md: add missing backtick to GET searchpaths doc (#11459) 2 years ago
build_dist.sh client/web: switch to using prebuilt web client assets 2 years ago
build_docker.sh Code Improvements (#11311) 2 years ago
flake.lock flake.nix: build tailscale with go 1.22 2 years ago
flake.nix go.mod.sri: update SRI hash for go.mod changes 2 years ago
go.mod go.mod: bump gvisor 2 years ago
go.mod.sri go.mod.sri: update SRI hash for go.mod changes 2 years ago
go.sum go.mod: bump gvisor 2 years ago
go.toolchain.branch go.toolchain.rev: bump to Go 1.22.0 (#11055) 2 years ago
go.toolchain.rev go.toolchain.rev: bump Go toolchain to 1.22.1 2 years ago
gomod_test.go go.mod: add test that replace directives aren't added in oss 2 years ago
header.txt cmd/k8s-operator: operator can create subnetrouter (#9505) 2 years ago
pull-toolchain.sh pull-toolchain.sh: don't run update-flake.sh 3 years ago
shell.nix go.mod.sri: update SRI hash for go.mod changes 2 years ago
staticcheck.conf all: cleanup unused code, part 2 (#10670) 2 years ago
update-flake.sh Code Improvements (#11311) 2 years ago
version-embed.go version: return correct Meta.MajorMinorPatch in non-dev builds 3 years ago
version_test.go go.mod,wgengine/netstack: bump gvisor 2 years ago

README.md

Tailscale

https://tailscale.com

Private WireGuard® networks made easy

Overview

This repository contains the majority of Tailscale's open source code. Notably, it includes the tailscaled daemon and the tailscale CLI tool. The tailscaled daemon runs on Linux, Windows, macOS, and to varying degrees on FreeBSD and OpenBSD. The Tailscale iOS and Android apps use this repo's code, but this repo doesn't contain the mobile GUI code.

Other Tailscale repos of note:

For background on which parts of Tailscale are open source and why, see https://tailscale.com/opensource/.

Using

We serve packages for a variety of distros and platforms at https://pkgs.tailscale.com.

Other clients

The macOS, iOS, and Windows clients use the code in this repository but additionally include small GUI wrappers. The GUI wrappers on non-open source platforms are themselves not open source.

Building

We always require the latest Go release, currently Go 1.22. (While we build releases with our Go fork, its use is not required.)

go install tailscale.com/cmd/tailscale{,d}

If you're packaging Tailscale for distribution, use build_dist.sh instead, to burn commit IDs and version info into the binaries:

./build_dist.sh tailscale.com/cmd/tailscale
./build_dist.sh tailscale.com/cmd/tailscaled

If your distro has conventions that preclude the use of build_dist.sh, please do the equivalent of what it does in your distro's way, so that bug reports contain useful version information.

Bugs

Please file any issues about this code or the hosted service on the issue tracker.

Contributing

PRs welcome! But please file bugs. Commit messages should reference bugs.

We require Developer Certificate of Origin Signed-off-by lines in commits.

See git log for our commit message style. It's basically the same as Go's style.

About Us

Tailscale is primarily developed by the people at https://github.com/orgs/tailscale/people. For other contributors, see:

WireGuard is a registered trademark of Jason A. Donenfeld.