tailscale

Commit Graph

Author	SHA1	Message	Date
Joe Tsai	6f55309f34	logtail/filch: fix panic in concurrent file access (#18555 ) In the event of multiple Filch intances being backed by the same file, it is possible that concurrent rotateLocked calls occur. One operation might clear the file, resulting in another skipping the call to resetReadBuffer, resulting in a later panic because the read index is invalid. To at least avoid the panic, always call resetReadBuffer. Note that the behavior of Filch is undefined when using the same file. While this avoids the panic, we may still experience data corruption or less. Fixes #18552 Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 days ago
Will Norris	3ec5be3f51	all: remove AUTHORS file and references to it This file was never truly necessary and has never actually been used in the history of Tailscale's open source releases. A Brief History of AUTHORS files --- The AUTHORS file was a pattern developed at Google, originally for Chromium, then adopted by Go and a bunch of other projects. The problem was that Chromium originally had a copyright line only recognizing Google as the copyright holder. Because Google (and most open source projects) do not require copyright assignemnt for contributions, each contributor maintains their copyright. Some large corporate contributors then tried to add their own name to the copyright line in the LICENSE file or in file headers. This quickly becomes unwieldy, and puts a tremendous burden on anyone building on top of Chromium, since the license requires that they keep all copyright lines intact. The compromise was to create an AUTHORS file that would list all of the copyright holders. The LICENSE file and source file headers would then include that list by reference, listing the copyright holder as "The Chromium Authors". This also become cumbersome to simply keep the file up to date with a high rate of new contributors. Plus it's not always obvious who the copyright holder is. Sometimes it is the individual making the contribution, but many times it may be their employer. There is no way for the proejct maintainer to know. Eventually, Google changed their policy to no longer recommend trying to keep the AUTHORS file up to date proactively, and instead to only add to it when requested: https://opensource.google/docs/releasing/authors. They are also clear that: > Adding contributors to the AUTHORS file is entirely within the > project's discretion and has no implications for copyright ownership. It was primarily added to appease a small number of large contributors that insisted that they be recognized as copyright holders (which was entirely their right to do). But it's not truly necessary, and not even the most accurate way of identifying contributors and/or copyright holders. In practice, we've never added anyone to our AUTHORS file. It only lists Tailscale, so it's not really serving any purpose. It also causes confusion because Tailscalars put the "Tailscale Inc & AUTHORS" header in other open source repos which don't actually have an AUTHORS file, so it's ambiguous what that means. Instead, we just acknowledge that the contributors to Tailscale (whoever they are) are copyright holders for their individual contributions. We also have the benefit of using the DCO (developercertificate.org) which provides some additional certification of their right to make the contribution. The source file changes were purely mechanical with: git ls-files \| xargs sed -i -e 's/$Tailscale Inc &$ AUTHORS/\1 contributors/g' Updates #cleanup Change-Id: Ia101a4a3005adb9118051b3416f5a64a4a45987d Signed-off-by: Will Norris <will@tailscale.com>	7 days ago
Joe Tsai	9613b4eecc	logtail: add metrics (#18184 ) Add metrics about logtail uploading and underlying buffer. Add metrics to the in-memory buffer implementation. Updates tailscale/corp#21363 Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 months ago
Joe Tsai	6428ba01ef	logtail/filch: rewrite the package (#18143 ) The filch implementation is fairly broken: * When Filch.cur exceeds MaxFileSize, it calls moveContents to copy the entirety of cur into alt (while holding the write lock). By nature, this is the movement of a lot of data in a hot path, meaning that all log calls will be globally blocked! It also means that log uploads will be blocked during the move. * The implementation of moveContents is buggy in that it copies data from cur into the start of alt, but fails to truncate alt to the number of bytes copied. Consequently, there are unrelated lines near the end, leading to out-of-order lines when being read back. * Data filched via stderr do not directly respect MaxFileSize, which is only checked every 100 Filch.Write calls. This means that it is possible that the file grows far beyond the specified max file size before moveContents is called. * If both log files have data when New is called, it also copies the entirety of cur into alt. This can block the startup of a process copying lots of data before the process can do any useful work. * TryReadLine is implemented using bufio.Scanner. Unfortunately, it will choke on any lines longer than bufio.MaxScanTokenSize, rather than gracefully skip over them. The re-implementation avoids a lot of these problems by fundamentally eliminating the need for moveContent. We enforce MaxFileSize by simply rotating the log files whenever the current file exceeds MaxFileSize/2. This is a constant-time operation regardless of file size. To more gracefully handle lines longer than bufio.MaxScanTokenSize, we skip over these lines (without growing the read buffer) and report an error. This allows subsequent lines to be read. In order to improve debugging, we add a lot of metrics. Note that the the mechanism of dup2 with stderr is inherently racy with a the two file approach. The order of operations during a rotation is carefully chosen to reduce the race window to be as short as possible. Thus, this is slightly less racy than before. Updates tailscale/corp#21363 Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 months ago
Brad Fitzpatrick	7c1d6e35a5	all: use Go 1.22 range-over-int Updates #11058 Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Andrea Barisani	d61494db68	adjust build tags for tamago Signed-off-by: Andrea Barisani <andrea@inversepath.com>	2 years ago
Andrea Barisani	0a74d46568	adjust build tags for tamago Signed-off-by: Andrea Barisani <andrea@inversepath.com>	2 years ago
Brad Fitzpatrick	98a5116434	all: adjust some build tags for plan9 I'm not saying it works, but it compiles. Updates #5794 Change-Id: I2f3c99732e67fe57a05edb25b758d083417f083e Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	67e912824a	all: adjust some build tags for wasi A start. Updates #8320 Change-Id: I64057f977be51ba63ce635c56d67de7ecec415d1 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Maisem Ali	1a30b2d73f	all: use tstest.Replace more Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Will Norris	71029cea2d	all: update copyright and license headers This updates all source files to use a new standard header for copyright and license declaration. Notably, copyright no longer includes a date, and we now use the standard SPDX-License-Identifier header. This commit was done almost entirely mechanically with perl, and then some minimal manual fixes. Updates #6865 Signed-off-by: Will Norris <will@tailscale.com>	3 years ago
Brad Fitzpatrick	da8def8e13	all: remove old +build tags The //go:build syntax was introduced in Go 1.17: https://go.dev/doc/go1.17#build-lines gofmt has kept the +build and go:build lines in sync since then, but enough time has passed. Time to remove them. Done with: perl -i -npe 's,^// \+build.*\n,,' $(git grep -l -F '+build') Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Eng Zer Jun	f0347e841f	refactor: move from io/ioutil to io and os packages The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Reference: https://golang.org/doc/go1.16#ioutil Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	3 years ago
Mihai Parparita	f371a1afd9	cmd/tsconnect: make logtail uploading work Initialize logtail and provide an uploader that works in the browser (we make a no-cors cross-origin request to avoid having to open up the logcatcher servers to CORS). Fixes #5147 Signed-off-by: Mihai Parparita <mihai@tailscale.com>	4 years ago
Maisem Ali	05e55f4a0b	logtail/filch: limit buffer file size to 50MB Signed-off-by: Maisem Ali <maisem@tailscale.com>	4 years ago
Josh Bleecher Snyder	93284209bc	logtail/filch: preallocate a scanner buffer Scanning log lines is a frequent source of allocations. Pre-allocate a re-usable buffer. This still doesn't help when there are giant log lines. Those will still be problematic from an iOS memory perspective. For more on that, see https://github.com/tailscale/corp/issues/2423. (For those who cannot follow that link, it is a discussion of particular problematic types of log lines for particular categories of customers. The "categories of customers" part is the reason that it is a private issue.) There is also a latent bug here. If we ever encounter a log line longer than bufio.MaxScanTokenSize, then bufio.Scan will return an error, and we'll truncate the file and discard the rest of the log. That's not good, but bufio.MaxScanTokenSize is really big, so it probably doesn't matter much in practice now. Unfortunately, it does prevent us from easily capping the potential memory usage here, on pain of losing log entries. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	5 years ago
Josh Bleecher Snyder	a5da4ed981	all: gofmt with Go 1.17 This adds "//go:build" lines and tidies up existing "// +build" lines. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	5 years ago
David Crawshaw	297b3d6fa4	staticcheck.conf: turn off noisy lint errors Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	5 years ago
Josh Bleecher Snyder	61e411344f	logtail/filch: add staticcheck annotation To work around a staticcheck bug when running with GOOS=windows. Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>	5 years ago
Brad Fitzpatrick	d6e9fb1df0	all: adjust Unix permissions for those without umasks Fixes tailscale/corp#1165 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	5 years ago
Brad Fitzpatrick	7a2a3955d3	logtail/filch: skip a broken test on Windows Add a TODO with some notes about why it's skipped for now. Updates #50	5 years ago
Josh Bleecher Snyder	585a0d8997	all: use testing.T.TempDir Bit of Friday cleanup. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	5 years ago
Avery Pennarun	5eb09c8f5e	filch_test: clarify the use of os.RemoveAll(). Signed-off-by: Avery Pennarun <apenwarr@tailscale.com>	6 years ago
David Crawshaw	51a12d1307	filch: a few minor comments Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	6 years ago
David Anderson	de0239375a	logtail/filch: use x/sys/unix instead of syscall. In particular, the Dup2 syscall is not defined in the syscall package on arm64, but is defined in x/sys/unix. Signed-off-by: David Anderson <dave@natulte.net>	6 years ago
David Anderson	80ecb8342f	logtail/filch: use io's seek constants. Signed-off-by: David Anderson <dave@natulte.net>	6 years ago
Earl Lee	a8d8b8719a	Move Linux client & common packages into a public repo.	6 years ago

27 Commits (main)