74 Commits (723858665272582e4240716ddc98b36d154704d2)

Author	SHA1	Message	Date
Brad Fitzpatrick	52e4f24c58	portlist: populate Pid field on Linux ... The Port.Pid was always more of an implementation detail on some platforms and isn't necessary on Linux so it was never populated. (Nothing outside the portlist package ever used it) But might as well populate it for consistency since we have it in memory and its absence confused people. Updates #cleanup Change-Id: I869768a75c9fedeff242a5452206e2b2947a17cb Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	1 year ago
Brad Fitzpatrick	e8551d6b40	all: use Go 1.21 slices, maps instead of x/exp/{slices,maps} ... Updates #8419 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	1 year ago
Marwan Sulaiman	f8f0b981ac	portlist: remove async functionality ... This PR removes all async functionality from the portlist package which may be a breaking change for non-tailscale importers. The only importer within this codebase (LocalBackend) is already using the synchronous API so no further action needed. Fixes #8171 Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>	2 years ago
Marwan Sulaiman	5dd0b02133	portlist: add a synchronous Poll method ... This is a follow up on PR #8172 that adds a synchronous Poll method which allows for the Poller to be used as a zero value without needing the constructor. The local backend is also changed to use the new API. A follow up PR will remove the async functionality from the portlist package. Updates #8171 Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>	2 years ago
Marwan Sulaiman	e32e5c0d0c	portlist: add Poller.IncludeLocalhost option ... This PR parameterizes receiving loopback updates from the portlist package. Callers can now include services bound to localhost if they want. Note that this option is off by default still. Fixes #8171 Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>	2 years ago
Andrew Dunham	280255acae	various: add golangci-lint, fix issues (#7905) ... This adds an initial and intentionally minimal configuration for golang-ci, fixes the issues reported, and adds a GitHub Action to check new pull requests against this linter configuration. Signed-off-by: Andrew Dunham <andrew@du.nham.ca> Change-Id: I8f38fbc315836a19a094d0d3e986758b9313f163	2 years ago
Will Norris	71029cea2d	all: update copyright and license headers ... This updates all source files to use a new standard header for copyright and license declaration. Notably, copyright no longer includes a date, and we now use the standard SPDX-License-Identifier header. This commit was done almost entirely mechanically with perl, and then some minimal manual fixes. Updates #6865 Signed-off-by: Will Norris <will@tailscale.com>	2 years ago
Aaron Klotz	296f53524c	netstat, portlist: update Windows implementation to disambiguate svchost processes ... We change our invocations of GetExtendedTcpTable to request additional information about the "module" responsible for the port. In addition to pid, this output also includes sufficient metadata to enable Windows to resolve process names and disambiguate svchost processes. We store the OS-specific output in an OSMetadata field in netstat.Entry, which portlist may then use as necessary to actually resolve the process/module name. Signed-off-by: Aaron Klotz <aaron@tailscale.com>	2 years ago
Maisem Ali	b5299d7d0e	portlist: wait for lsof cmd to exit ... We were leaking processes otherwise. Co-authored-by: Mihai Parparita <mihai@tailscale.com> Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Brad Fitzpatrick	f81351fdef	portlist: fix data race ... Maisem spotted the bug. The initial getList call in NewPoller wasn't making a clone (only the Run loop's getList calls). Fixes #6314 Change-Id: I8ab8799fcccea8e799140340d0ff88a825bb6ff0 Co-authored-by: Maisem Ali <maisem@tailscale.com> Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	db2cc393af	util/dirwalk, metrics, portlist: add new package for fast directory walking ... This is similar to the golang.org/x/tools/internal/fastwalk I'd previously written but not recursive and using mem.RO. The metrics package already had some Linux-specific directory reading code in it. Move that out to a new general package that can be reused by portlist too, which helps its scanning of all /proc files: name old time/op new time/op delta FindProcessNames-8 2.79ms ± 6% 2.45ms ± 7% -12.11% (p=0.000 n=10+10) name old alloc/op new alloc/op delta FindProcessNames-8 62.9kB ± 0% 33.5kB ± 0% -46.76% (p=0.000 n=9+10) name old allocs/op new allocs/op delta FindProcessNames-8 2.25k ± 0% 0.38k ± 0% -82.98% (p=0.000 n=9+10) Change-Id: I75db393032c328f12d95c39f71c9742c375f207a Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	21ef7e5c35	portlist: add macOS osImpl, finish migration to new style ... Previously: * 036f70b7b4 for linux * 35bee36549 for windows This does macOS. And removes all the compat code for the old style. (e.g. iOS, js are no longer mentioned; all platforms without implementations just default to not doing anything) One possible regression is that platforms without explicit implementations previously tried to do the "netstat -na" style to get open ports (but not process names). Maybe that worked on FreeBSD and OpenBSD previously, but nobody ever really tested it. And it was kinda useless without associated process names. So better off removing those for now until they get a good implementation. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	da8def8e13	all: remove old +build tags ... The //go:build syntax was introduced in Go 1.17: https://go.dev/doc/go1.17#build-lines gofmt has kept the +build and go:build lines in sync since then, but enough time has passed. Time to remove them. Done with: perl -i -npe 's,^// \+build.*\n,,' $(git grep -l -F '+build') Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	35bee36549	portlist: use win32 calls instead of running netstat process [windows] ... Turns out using win32 instead of shelling out to child processes is a bit faster: name old time/op new time/op delta GetListIncremental-4 278ms ± 2% 0ms ± 7% -99.93% (p=0.000 n=8+10) name old alloc/op new alloc/op delta GetListIncremental-4 238kB ± 0% 9kB ± 0% -96.12% (p=0.000 n=10+8) name old allocs/op new allocs/op delta GetListIncremental-4 1.19k ± 0% 0.02k ± 0% -98.49% (p=0.000 n=10+10) Fixes #3876 (sadly) Change-Id: I1195ac5de21a8a8b3cdace5871d263e81aa27e91 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	575c599410	portlist: add a test that verifies changes are picked up over time ... To avoid annoying firewall dialogs on macOS and Windows, only run it on Linux by default without the flag. Change-Id: If8486c31d4243ade54b0131f673237c6c9184c08 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	036f70b7b4	portlist: refactor, introduce OS-specific types ... Add an osImpl interface that can be stateful and thus more efficient between calls. It will later be implemented by all OSes but for now this change only adds a Linux implementation. Remove Port.inode. It was only used by Linux and moves into its osImpl. Don't reopen /proc/net/* files on each run. Turns out you can just keep then open and seek to the beginning and reread and the contents are fresh. name old time/op new time/op delta GetListIncremental-8 7.29ms ± 2% 6.53ms ± 1% -10.50% (p=0.000 n=9+9) name old alloc/op new alloc/op delta GetListIncremental-8 1.30kB ±13% 0.70kB ± 5% -46.38% (p=0.000 n=9+10) name old allocs/op new allocs/op delta GetListIncremental-8 33.2 ±11% 18.0 ± 0% -45.82% (p=0.000 n=9+10) Updates #5958 Change-Id: I4be83463cbd23c2e2fa5d0bdf38560004f53401b Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	70dde89c34	portlist: add package doc, file comments, move a method to the right file ... And respect envknob earlier. NewPoller has one caller and ignores errors; they just signal ipnlocal to log a warning and not use the portlist poller. Change-Id: I4a33af936fe780cca8c7197d4d74ac31a1dc01e3 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	774fa72d32	portlist: add BenchmarkGetListIncremental ... In contrast to BenchmarkGetList, this new BenchmarkGetListIncremental acts like what happens in practice, remembering the previous run and avoiding work that's already been done previously. Currently: BenchmarkGetList BenchmarkGetList-8 100 11011100 ns/op 68411 B/op 2211 allocs/op BenchmarkGetList-8 100 11443410 ns/op 69073 B/op 2223 allocs/op BenchmarkGetList-8 100 11217311 ns/op 68421 B/op 2197 allocs/op BenchmarkGetList-8 100 11035559 ns/op 68801 B/op 2220 allocs/op BenchmarkGetList-8 100 10921596 ns/op 69226 B/op 2225 allocs/op BenchmarkGetListIncremental BenchmarkGetListIncremental-8 168 7187217 ns/op 1192 B/op 28 allocs/op BenchmarkGetListIncremental-8 172 7004525 ns/op 1194 B/op 28 allocs/op BenchmarkGetListIncremental-8 162 7235889 ns/op 1221 B/op 29 allocs/op BenchmarkGetListIncremental-8 164 7035671 ns/op 1219 B/op 29 allocs/op BenchmarkGetListIncremental-8 174 7095448 ns/op 1114 B/op 27 allocs/op Updates #5958 Change-Id: I1bd5a4b206df4173e2cb8e8a780429d9daa6ef1d Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	3697609aaa	portlist: remove unix.Readlink allocs on Linux ... name old time/op new time/op delta GetList-8 11.2ms ± 5% 11.1ms ± 3% ~ (p=0.661 n=10+9) name old alloc/op new alloc/op delta GetList-8 83.3kB ± 1% 67.4kB ± 1% -19.05% (p=0.000 n=10+10) name old allocs/op new allocs/op delta GetList-8 2.89k ± 2% 2.19k ± 1% -24.24% (p=0.000 n=10+10) (real issue is we're calling this code as much as we are, but easy enough to make it efficient because it'll still need to be called sometimes in any case) Updates #5958 Change-Id: I90c20278d73e80315a840aed1397d24faa308d93 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	7149155b80	portlist: further reduce allocations on Linux ... Make Linux parsePorts also an append-style API and attach it to caller's provided append base memory. And add a little string intern pool in front of the []byte to string for inode names. name old time/op new time/op delta GetList-8 11.1ms ± 4% 9.8ms ± 6% -11.68% (p=0.000 n=9+10) name old alloc/op new alloc/op delta GetList-8 92.8kB ± 2% 79.7kB ± 0% -14.11% (p=0.000 n=10+9) name old allocs/op new allocs/op delta GetList-8 2.94k ± 1% 2.76k ± 0% -6.16% (p=0.000 n=10+10) More coming. (the bulk of the allocations are in addProcesses and filesystem operations, most of which we should usually be able to skip) Updates #5958 Change-Id: I3f0c03646d314a16fef7f8346aefa7d5c96701e7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	def089f9c9	portlist: unexport all Poller fields, removing unused one, rework channels ... Poller.C and Poller.c were duplicated for one caller. Add an accessor returning the receive-only version instead. It'll inline. Poller.Err was unused. Remove. Then Poller is opaque. The channel usage and shutdown was a bit sketchy. Clean it up. And document some things. Change-Id: I5669e54f51a6a13492cf5485c83133bda7ea3ce9 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	46ce80758d	portlist: update some internals to use append-style APIs ... In prep for reducing garbage, being able to reuse memory. So far this doesn't actually reuse much. This is just changing signatures around. But some improvement in any case: bradfitz@tsdev:~/src/tailscale.com$ ~/go/bin/benchstat before after name old time/op new time/op delta GetList-8 11.8ms ± 9% 9.9ms ± 3% -15.98% (p=0.000 n=10+10) name old alloc/op new alloc/op delta GetList-8 99.5kB ± 2% 91.9kB ± 0% -7.62% (p=0.000 n=9+9) name old allocs/op new allocs/op delta GetList-8 3.05k ± 1% 2.93k ± 0% -3.83% (p=0.000 n=8+9) More later, once parsers can reuse strings from previous parses. Updates #5958 Change-Id: I76cd5048246dd24d11c4e263d8bb8041747fb2b0 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	67597bfc9e	portlist: unexport GetList ... It's an internal implementation detail, and I plan to refactor it for performance (garbage) reasons anyway, so start by hiding it. Updates #5958 Change-Id: I2c0d1f743d3495c5f798d1d8afc364692cd9d290 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Eng Zer Jun	f0347e841f	refactor: move from io/ioutil to io and os packages ... The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit replaces the existing io/ioutil functions with their new definitions in io and os packages. Reference: https://golang.org/doc/go1.16#ioutil Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	2 years ago
Brad Fitzpatrick	74674b110d	envknob: support changing envknobs post-init ... Updates #5114 Change-Id: Ia423fc7486e1b3f3180a26308278be0086fae49b Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Kristoffer Dalby	81574a5c8d	portlist: normalise space delimited process names (#5634)	2 years ago
Brad Fitzpatrick	4950fe60bd	syncs, all: move to using Go's new atomic types instead of ours ... Fixes #5185 Change-Id: I850dd532559af78c3895e2924f8237ccc328449d Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Mihai Parparita	27a1ad6a70	wasm: exclude code that's not used on iOS for Wasm too ... It has similar size constraints. Saves ~1.9MB from the Wasm build. Updates #3157 Signed-off-by: Mihai Parparita <mihai@tailscale.com>	3 years ago
Brad Fitzpatrick	41fd4eab5c	envknob: add new package for all the strconv.ParseBool(os.Getenv(..)) ... A new package can also later record/report which knobs are checked and set. It also makes the code cleaner & easier to grep for env knobs. Change-Id: Id8a123ab7539f1fadbd27e0cbeac79c2e4f09751 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	486059589b	all: gofmt -w -s (simplify) tests ... And it updates the build tag style on a couple files. Change-Id: I84478d822c8de3f84b56fa1176c99d2ea5083237 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	9310713bfb	all: fix some js/wasm compilation issues ... Change-Id: I05a3a4835e225a1e413ec3540a7c7e4a2d477084 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	a05086ef86	portlist: add debug knob to disable portlist collection ... For big servers. Per discussion with @crawshaw. Updates tailscale/corp#2566 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	a3c5de641b	portlist: stop logging stray UDP ports ... These "weird" port lines show up in logs frequently. They're the result of uninteresting races, and they're not actionable. Remove the noise. Remove the isLoopbackAddr case to placate staticcheck. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	0eb6cc9321	portlist: cache field index position between runs, cut two more allocs (Linux) ... name old time/op new time/op delta ParsePorts-6 6.41ms ± 7% 3.15ms ± 2% -50.84% (p=0.000 n=9+9) name old alloc/op new alloc/op delta ParsePorts-6 408B ± 0% 216B ± 0% -47.06% (p=0.002 n=8+10) name old allocs/op new allocs/op delta ParsePorts-6 7.00 ± 0% 4.00 ± 0% -42.86% (p=0.000 n=10+10) Updates tailscale/corp#2566 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	61f201f33d	portlist: reuse bufio.Reader between files ... name old time/op new time/op delta ListPorts-6 1.18ms ± 5% 1.16ms ± 5% ~ (p=0.075 n=10+10) name old alloc/op new alloc/op delta ListPorts-6 27.2kB ± 0% 14.9kB ± 0% -45.14% (p=0.001 n=8+9) name old allocs/op new allocs/op delta ListPorts-6 90.0 ± 0% 84.0 ± 0% -6.67% (p=0.000 n=10+10) Updates tailscale/corp#2566 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	5a9d977c78	portlist: reduce CPU parsing portlist ... Avoid splitting fields in the common case. Field splitting was 84% of the overall CPU. name old time/op new time/op delta ParsePorts-6 33.3ms ± 2% 6.3ms ± 4% -80.97% (p=0.000 n=9+10) name old alloc/op new alloc/op delta ParsePorts-6 520B ±79% 408B ± 0% -21.49% (p=0.046 n=10+8) name old allocs/op new allocs/op delta ParsePorts-6 7.00 ± 0% 7.00 ± 0% ~ (all equal) Updates tailscale/corp#2566 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	64e9ce8df1	portlist: reduce allocs on Linux ... Notably, it no longer allocates proportional to the number of open sockets on the machine. Any alloc reduction numbers are a little contrived with such a reduction but e.g. on a machine with 50,000 connections open: name old time/op new time/op delta ParsePorts-6 57.7ms ± 6% 32.8ms ± 3% -43.04% (p=0.000 n=9+10) name old alloc/op new alloc/op delta ParsePorts-6 24.0MB ± 0% 0.0MB ± 0% -100.00% (p=0.000 n=10+9) name old allocs/op new allocs/op delta ParsePorts-6 100k ± 0% 0k ± 0% -99.99% (p=0.000 n=10+10) Updates tailscale/corp#2566 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Anderson	bf8556ab86	portlist: fix build tag to build only on macOS, not macOS+iOS.	3 years ago
Josh Bleecher Snyder	6da6d47a83	all: simplify build tags involving iOS ... Prior to Go 1.16, iOS used GOOS=darwin, so we had to distinguish macOS from iOS during GOARCH. We now require Go 1.16 in our go.mod, so we can simplify. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	a5da4ed981	all: gofmt with Go 1.17 ... This adds "//go:build" lines and tidies up existing "// +build" lines. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
David Crawshaw	297b3d6fa4	staticcheck.conf: turn off noisy lint errors ... Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	4 years ago
Josh Bleecher Snyder	bfd2b71926	portlist: suppress staticcheck error ... Signed-off-by: Josh Bleecher Snyder <josharian@gmail.com>	4 years ago
Brad Fitzpatrick	2df6372b67	portlist: de-dup services on same (proto, port) on both IPv4/IPv6 ... Fixes #1703 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	c0befee188	portlist: use windows OpenCurrentProcessToken, not GetCurrentProcessToken ... The latter only works on Windows 8+. Also add a TODO to get do this all more efficiently. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	e619296ece	portlist: filter out all of 127.0.0.0/8, not just 127.0.0.1/32 ... Per user private bug report. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	f325aa7e38	portlist: exclude services bound to IPv6 loopback address ... Fixes #1683 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	5c5acadb2a	portlist: unexport SameInodes method ... Signed-off-by: Brad Fitzpatrick <brad@danga.com>	4 years ago
Brad Fitzpatrick	b0af15ff5c	portlist: remove some old TODOs ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
David Anderson	63a9adeb6c	portlist: collect IPv6 listening sockets on linux. ... This is important because some of those v6 sockets are actually dual-stacked sockets, so this is our only chance of discovering some services. Fixes #1443. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	ad6edf5ecd	portlist: report a better process name for .Net on linux. ... Fixes #1440. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago