Commit Graph

23 Commits (9cd4e6519108f3f4f34bf6e9ee497d900e5b73aa)

Author SHA1 Message Date
David Anderson 9cd4e65191 smallzstd: new package that constructs zstd small encoders/decoders.
It's just a config wrapper that passes "use less memory at the
expense of compression" parameters by default, so that we don't
accidentally construct resource-hungry (de)compressors.

Also includes a benchmark that measures the memory cost of the
small variants vs. the stock variants. The savings are significant
on both compressors (~8x less memory) and decompressors (~1.4x less,
not including the savings from the significantly smaller
window on the compression side - with those savings included it's
more like ~140x smaller).

BenchmarkSmallEncoder-8            	   56174	     19354 ns/op	      31 B/op	       0 allocs/op
BenchmarkSmallEncoderWithBuild-8   	    2900	    382940 ns/op	 1746547 B/op	      36 allocs/op
BenchmarkStockEncoder-8            	   48921	     25761 ns/op	     286 B/op	       0 allocs/op
BenchmarkStockEncoderWithBuild-8   	     426	   2630241 ns/op	13843842 B/op	     124 allocs/op
BenchmarkSmallDecoder-8            	  123814	      9344 ns/op	       0 B/op	       0 allocs/op
BenchmarkSmallDecoderWithBuild-8   	   41547	     27455 ns/op	   27694 B/op	      31 allocs/op
BenchmarkStockDecoder-8            	  129832	      9417 ns/op	       1 B/op	       0 allocs/op
BenchmarkStockDecoderWithBuild-8   	   25561	     51751 ns/op	   39607 B/op	      92 allocs/op

Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson c71754eba2 ipn/ipnserver: revert decoder memory limit.
The zstd library treats that limit as a hard cap on decompressed
size, in the mode we're using it, rather than a window size.

Signed-off-by: David Anderson <danderson@tailscale.com>
5 years ago
Avery Pennarun db051fb013 ipnserver and logpolicy: configure zstd with low-memory settings.
The compressed blobs we send back and forth are small and infrequent,
which doesn't justify the 8MB * GOMAXPROCS memory that was being
allocated. This was the overwhelming majority of memory use in
tailscaled. On my system it goes from ~100M RSS to ~15M RSS (which is
still suspiciously high, but we can worry about that more later).

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
Wendi Yu bb55694c95
wgengine: log node IDs when peers are added/removed (#381)
Also stop logging data sent/received from nodes we're not connected to (ie all those `x`s being logged in the `peers: ` line)
Signed-off-by: Wendi <wendi.yu@yahoo.ca>
5 years ago
Avery Pennarun 08acb502e5 Add tstest.PanicOnLog(), and fix various problems detected by this.
If a test calls log.Printf, 'go test' horrifyingly rearranges the
output to no longer be in chronological order, which makes debugging
virtually impossible. Let's stop that from happening by making
log.Printf panic if called from any module, no matter how deep, during
tests.

This required us to change the default error handler in at least one
http.Server, as well as plumbing a bunch of logf functions around,
especially in magicsock and wgengine, but also in logtail and backoff.

To add insult to injury, 'go test' also rearranges the output when a
parent test has multiple sub-tests (all the sub-test's t.Logf is always
printed after all the parent tests t.Logf), so we need to screw around
with a special Logf that can point at the "current" t (current_t.Logf)
in some places. Probably our entire way of using subtests is wrong,
since 'go test' would probably like to run them all in parallel if you
called t.Parallel(), but it definitely can't because the're all
manipulating the shared state created by the parent test. They should
probably all be separate toplevel tests instead, with common
setup/teardown logic. But that's a job for another time.

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
Wendi Yu 0c69b4e00d
Implement rate limiting on log messages (#356)
Implement rate limiting on log messages

Addresses issue #317, where logs can get spammed with the same message
nonstop. Created a rate limiting closure on logging functions, which
limits the number of messages being logged per second based on format
string. To keep memory usage as constant as possible, the previous cache
purging at periodic time intervals has been replaced by an LRU that
discards the oldest string when the capacity of the cache is reached.


Signed-off-by: Wendi Yu <wendi.yu@yahoo.ca>
5 years ago
Brad Fitzpatrick 45f2b53aca all: remove unnecessary trailing newlines in format patterns for consistency
And document on logger.Logf that it's unnecessary.
5 years ago
Brad Fitzpatrick a4ef345737 cmd/tailscale: add status subcommand
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 322499473e cmd/tailscaled, wgengine, ipn: add /debug/ipn handler with world state
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Avery Pennarun b4897e7de8 controlclient/netmap: write our own b.ConciseDiffFrom(a) function.
This removes the need for go-cmp, which is extremely bloaty so we had
to leave it out of iOS. As a result, we had also left it out of macOS,
and so we didn't print netmap diffs at all on darwin-based platforms.
Oops.

As a bonus, the output format of the new function is way better.

Minor oddity: because I used the dumbest possible diff algorithm, the
sort order is a bit dumb. We print all "removed" lines and then print
all "added" lines, rather than doing the usual diff-like thing of
interspersing them. This probably doesn't matter (maybe it's an
improvement).
5 years ago
Brad Fitzpatrick 4cf5ac3060 ipn/ipnserver: remove unnecessary closure parameter 5 years ago
Brad Fitzpatrick 06092a3af3 ipn/ipnserver: document potential race, start on test for it 5 years ago
Brad Fitzpatrick 3988ddc85d types/logger: add WithPrefix, use it in two places
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Brad Fitzpatrick 367ffde21a ipn: misc cleanup
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Anderson 0c55777fed ipn: temporary support for loading legacy relaynode configs.
Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson cf1e386cbd ipn: move Options.ServerURL into Prefs.
We can't rely on a frontend to provide a control
server URL, so this naturally belongs in server-persisted
state.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson 4460bd638b safesocket: simplify API.
On unix, we want to provide a full path to the desired unix socket.

On windows, currently we want to provide a TCP port, but someday
we'll also provide a "path-ish" object for a named pipe.

For now, simplify the API down to exactly a path and a TCP port.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson 62fb652eef cmd/tailscaled: run off internal state autonomously.
With this change, tailscaled can be restarted and reconnect
without interaction from `tailscale`, and `tailscale` is merely
there to provide login assistance and adjust preferences.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
Brad Fitzpatrick 2896be60db Move "logger" package to under types, now that we have it.
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
David Anderson dec5d64593 ipn/ipnserver: remove Options.AllowQuit.
It was previously used by the MacOS client, but it now does
something different. ipnserver should never obey a client's
request to exit.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
David Anderson 5bc632271b Introduce a state store to LocalBackend.
The store is passed-in by callers of NewLocalBackend and
ipnserver.Run, but currently all callers are hardcoded to
an in-memory store. The store is unused.

Signed-Off-By: David Anderson <dave@natulte.net>
5 years ago
David Anderson 81eedcd5be ipnserver: ignore lint error for unused context.
The linter is strictly correct, but the code is structured
this way to avoid variable shadowing problems in the following
for loop. The context doesn't leak.

Staticcheck is correctly pointing out that this code is hard to
follow. However, this chunk of code is in service of enforcing
one frontend <> one backend, and we want to remove that limitation.
So, we'll just ignore the lint warning until this entire piece of
code goes away.

Signed-off-by: David Anderson <dave@natulte.net>
5 years ago
Earl Lee a8d8b8719a Move Linux client & common packages into a public repo. 5 years ago