Commit Graph

23 Commits (f352f8a0e6e55eb6a0397853d9acbbaa2cf7fe4e)

Author SHA1 Message Date
Joe Tsai 283a84724f
types/logid: simplify implementation (#7415)
Share the same underlying implementation for both PrivateID and PublicID.
For the shared methods, declare them in the same order.
Only keep documentation on methods without obvious meaning.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Joe Tsai 7e4788e383
logtail: delete ID types and functions (#7412)
These have been moved to the types/logid package.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Mihai Parparita f0f2b2e22b logtail: increase maximum log line size in low memory mode
The 255 byte limit was chosen more than 3 years ago (tailscale/corp@929635c9d9),
when iOS was operating under much more significant memory constraints.
With iOS 15 the network extension has an increased limit, so increasing
it to 4K should be fine.

The motivating factor was that the network interfaces being logged
by linkChange in wgengine/userspace.go were getting truncated, and it
would be useful to know why in some cases we're choosing the pdp_ip1
cell interface instead of the pdp_ip0 one.

Updates #7184
Updates #7188

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Will Norris 71029cea2d all: update copyright and license headers
This updates all source files to use a new standard header for copyright
and license declaration.  Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.

This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.

Updates #6865

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Eng Zer Jun f0347e841f refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Reference: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2 years ago
Mihai Parparita 3222bce02d logtail: add instance metadata to the entry logtail
Allows instances that are running with the same machine ID (due to
cloning) to be distinguished.

Also adds sequence numbers to detect duplicates.

For tailscale/corp#5244

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
3 years ago
Josh Bleecher Snyder 0868329936 all: use any instead of interface{}
My favorite part of generics.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick 84138450a4 types/logger, logtail: add mechanism to do structured JSON logs
e.g. the change to ipnlocal in this commit ultimately logs out:

{"logtail":{"client_time":"2022-02-17T20:40:30.511381153-08:00","server_time":"2022-02-18T04:40:31.057771504Z"},"type":"Hostinfo","val":{"GoArch":"amd64","Hostname":"tsdev","IPNVersion":"1.21.0-date.20220107","OS":"linux","OSVersion":"Debian 11.2 (bullseye); kernel=5.10.0-10-amd64"},"v":1}

Change-Id: I668646b19aeae4a2fed05170d7b279456829c844
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 5d9ab502f3 logtail: don't strip verbose level on upload
For analysis of log spam.

Bandwidth is ~unchanged from had we not stripped the "[vN] " from
text; it just gets restructed intot he new "v":N, field.  I guess it
adds one byte.

Updates #1548

Change-Id: Ie00a4e0d511066a33d10dc38d765d92b0b044697
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Josh Bleecher Snyder 9fe5ece833 logtail: cap the buffer size in encodeText
This started as an attempt to placate GitHub's code scanner,
but it's also probably generally a good idea.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 94fb42d4b2 all: use testingutil.MinAllocsPerRun
There are a few remaining uses of testing.AllocsPerRun:
Two in which we only log the number of allocations,
and one in which dynamically calculate the allocations
target based on a different AllocsPerRun run.

This also allows us to tighten the "no allocs"
test in wgengine/filter.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Josh Bleecher Snyder 278e7de9c9 logtail: always send a json array
The code goes to some effort to send a single JSON object
when there's only a single line and a JSON array when there
are multiple lines.

It makes the code more complex and more expensive;
when we add a second line, we have to use a second buffer
to duplicate the first one after adding a leading square brackets.

The savings come to two bytes. Instead, always send an array.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
Brad Fitzpatrick b34fbb24e8 logtail: reduce PublicID.UnmarshalText from 2 allocs to 0
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Denton Gentry 013da6660e logtail: add tests
+ add a test for parseAndRemoveLogLevel()
+ add a test for drainPendingMessages()
+ test JSON log encoding including several special cases

Other tests frequently send logs but a) don't check the result and
b) do so by happenstance, such that the code in encode() was not
consistently being exercised and leading to spurious changes in
code coverage. These tests attempt to more systematically test
the logging function.

This is the second attempt to add these tests, the first attempt
(in https://github.com/tailscale/tailscale/pull/1114) had two issues:
1. httptest.NewServer creates multiple goroutine handlers, and
   logtail uses goroutines to upload, but the first version had no
   locking in the server to guard this.
   Moved data handling into channels to get synchronization.
2. The channel to notify the test of the arrival of data had a depth
   of 1, in cases where the Logger sent multiple uploads it would
   block the server.

This resulted in the first iteration of these tests being flaky,
and we reverted it.

This new version of the tests has passed with
    go test -race -count=10000
and seems solid.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
4 years ago
Denton Gentry ce058c8280
Revert "Add logtail tests (#1114)" (#1116)
This reverts commit e4f53e9b6f.

At least two of these tests are flakey, reverting until they can be
made more robust.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
4 years ago
Denton Gentry e4f53e9b6f
Add logtail tests (#1114)
* logtail: test parseAndRemoveLogLevel()

Signed-off-by: Denton Gentry <dgentry@tailscale.com>

* logtail: test JSON log encoding.

Expand TestUploadMessages to also exercise the encoding functions
in logtail, like JSON logging and timestamps.

Other tests frequently send logs but a) don't check the result and
b) do so by happenstance, such that the lines in encode() were not
consistently being exercised and leading to spurious changes in
code coverage.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>

* logtail: add a test for drainPendingMessages

Make the client buffer some messages before the upload server
becomes available.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>

* logtail: use %q, raw strings, and io.WriteString

%q escapes binary characters for us.

raw strings avoid so much backslash escaping

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
4 years ago
Denton Gentry b771a1363b logtail: start a local server for TestFastShutdown
Right now TestFastShutdown tries to upload logs to localhost:1234,
which will most likely respond with an error. However if one has an
actual service running on port 1234, it would receive a connection
attempting to POST every time the unit test runs.

Start a local server and direct the upload there instead.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
4 years ago
Denton Gentry 2c328da094 logtail: add a test to upload logs to local server
Start an HTTP server to accept POST requests, and upload some logs to
it. Check that uploaded logs were received.

Code in logtail:drainPending was not being reliably exercised by other
tests. This shows up in code coverage reports, as lines of code in
drainPending are alternately added and subtracted from code coverage.
This test will reliably exercise and verify this code.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
4 years ago
Brad Fitzpatrick d97ee12179 logtail, logpolicy: remove an unidiomatic use of an interface 4 years ago
Elias Naur bca9fe35ba logtail: return correct write size from logger.Write
Signed-off-by: Elias Naur <mail@eliasnaur.com>
4 years ago
Avery Pennarun 08acb502e5 Add tstest.PanicOnLog(), and fix various problems detected by this.
If a test calls log.Printf, 'go test' horrifyingly rearranges the
output to no longer be in chronological order, which makes debugging
virtually impossible. Let's stop that from happening by making
log.Printf panic if called from any module, no matter how deep, during
tests.

This required us to change the default error handler in at least one
http.Server, as well as plumbing a bunch of logf functions around,
especially in magicsock and wgengine, but also in logtail and backoff.

To add insult to injury, 'go test' also rearranges the output when a
parent test has multiple sub-tests (all the sub-test's t.Logf is always
printed after all the parent tests t.Logf), so we need to screw around
with a special Logf that can point at the "current" t (current_t.Logf)
in some places. Probably our entire way of using subtests is wrong,
since 'go test' would probably like to run them all in parallel if you
called t.Parallel(), but it definitely can't because the're all
manipulating the shared state created by the parent test. They should
probably all be separate toplevel tests instead, with common
setup/teardown logic. But that's a job for another time.

Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>
5 years ago
Brad Fitzpatrick fcb6a34f4b logtail: reduce allocations encoding text
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
5 years ago
Earl Lee a8d8b8719a Move Linux client & common packages into a public repo. 5 years ago