406 Commits (6ab39b7bcd259a7cf4adb9331586a64698c85dcc)

Author	SHA1	Message	Date
Cuong Manh Le	a7efc7bd17	util/singleflight: sync with upstream ... Sync with golang.org/x/sync/singleflight at commit 8fcdb60fdcc0539c5e357b2308249e4e752147f1 Fixes #5790 Signed-off-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>	2 years ago
Josh Soref	d4811f11a0	all: fix spelling mistakes ... Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>	2 years ago
Aaron Klotz	44f13d32d7	cmd/tailscaled, util/winutil: log Windows service diagnostics when the wintun device fails to install ... I added new functions to winutil to obtain the state of a service and all its depedencies, serialize them to JSON, and write them to a Logf. When tstun.New returns a wrapped ERROR_DEVICE_NOT_AVAILABLE, we know that wintun installation failed. We then log the service graph rooted at "NetSetupSvc". We are interested in that specific service because network devices will not install if that service is not running. Updates https://github.com/tailscale/tailscale/issues/5531 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	2 years ago
Andrew Dunham	4b996ad5e3	util/deephash: add AppendSum method (#5768) ... This method can be used to obtain the hex-formatted deephash.Sum instance without allocations. Signed-off-by: Andrew Dunham <andrew@du.nham.ca>	2 years ago
Brad Fitzpatrick	9bdf0cd8cd	ipn/ipnlocal: add c2n /debug/{goroutines,prefs,metrics} ... * and move goroutine scrubbing code to its own package for reuse * bump capver to 45 Change-Id: I9b4dfa5af44d2ecada6cc044cd1b5674ee427575 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Aaron Klotz	acc7baac6d	tailcfg, util/deephash: add DataPlaneAuditLogID to Node and DomainDataPlaneAuditLogID to MapResponse ... We're adding two log IDs to facilitate data-plane audit logging: a node-specific log ID, and a domain-specific log ID. Updated util/deephash/deephash_test.go with revised expectations for tailcfg.Node. Updates https://github.com/tailscale/corp/issues/6991 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	2 years ago
Brad Fitzpatrick	f3ce1e2536	util/mak: deprecate NonNil, add type-safe NonNilSliceForJSON, NonNilMapForJSON ... And put the rationale in the name too to save the callers the need for a comment. Change-Id: I090f51b749a5a0641897ee89a8fb2e2080c8b782 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Andrew Dunham	70ed22ccf9	util/uniq: add ModifySliceFunc (#5504) ... Follow-up to #5491. This was used in control... oops! Signed-off-by: Andrew Dunham <andrew@tailscale.com>	2 years ago
Andrew Dunham	e945d87d76	util/uniq: use generics instead of reflect (#5491) ... This takes 75% less time per operation per some benchmarks on my mac. Signed-off-by: Andrew Dunham <andrew@du.nham.ca>	2 years ago
Joe Tsai	7e40071571	util/deephash: handle slice edge-cases (#5471) ... It is unclear whether the lack of checking nil-ness of slices was an oversight or a deliberate feature. Lacking a comment, the assumption is that this was an oversight. Also, expand the logic to perform cycle detection for recursive slices. We do this on a per-element basis since a slice is semantically equivalent to a list of pointers. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Andrew Dunham	58cc049a9f	util/cstruct: add package for decoding padded C structures (#5429) ... I was working on my "dump iptables rules using only syscalls" branch and had a bunch of C structure decoding to do. Rather than manually calculating the padding or using unsafe trickery to actually cast variable-length structures to Go types, I'd rather use a helper package that deals with padding for me. Padding rules were taken from the following article: http://www.catb.org/esr/structure-packing/ Signed-off-by: Andrew Dunham <andrew@du.nham.ca>	2 years ago
Joe Tsai	9bf13fc3d1	util/deephash: remove getTypeInfo (#5469) ... Add a new lookupTypeHasher function that is just a cached front-end around the makeTypeHasher function. We do not need to worry about the recursive type cycle issue that made getTypeInfo more complicated since makeTypeHasher is not directly recursive. All calls to itself happen lazily through a sync.Once upon first use. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	ab7e6f3f11	util/deephash: require pointer in API (#5467) ... The entry logic of Hash has extra complexity to make sure we always have an addressable value on hand. If not, we heap allocate the input. For this reason we document that there are performance benefits to always providing a pointer. Rather than documenting this, just enforce it through generics. Also, delete the unused HasherForType function. It's an interesting use of generics, but not well tested. We can resurrect it from code history if there's a need for it. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	c5b1565337	util/deephash: move pointer and interface logic to separate function (#5465) ... This helps pprof better identify which Go kinds take the most time since the kind is always in the function name. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	d2e2d8438b	util/deephash: move map logic to separate function (#5464) ... This helps pprof better identify which Go kinds take the most time since the kind is always in the function name. There is a minor adjustment where we hash the length of the map to be more on the cautious side. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	23c3831ff9	util/deephash: coalesce struct logic (#5466) ... Rather than having two copies []fieldInfo, just maintain one and perform merging in the same pass. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	296b008b9f	util/deephash: move array and slice logic to separate function (#5463) ... This helps pprof better identify which Go kinds take the most time since the kind is always in the function name. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	31bf3874d6	util/deephash: use unsafe.Pointer instead of reflect.Value (#5459) ... Use of reflect.Value.SetXXX panics if the provided argument was obtained from an unexported struct field. Instead, pass an unsafe.Pointer around and convert to a reflect.Value when necessary (i.e., for maps and interfaces). Converting from unsafe.Pointer to reflect.Value guarantees that none of the read-only bits will be populated. When running in race mode, we attach type information to the pointer so that we can type check every pointer operation. This also type-checks that direct memory hashing is within the valid range of a struct value. We add test cases that previously caused deephash to panic, but now pass. Performance: name old time/op new time/op delta Hash 14.1µs ± 1% 14.1µs ± 1% ~ (p=0.590 n=10+9) HashPacketFilter 2.53µs ± 2% 2.44µs ± 1% -3.79% (p=0.000 n=9+10) TailcfgNode 1.45µs ± 1% 1.43µs ± 0% -1.36% (p=0.000 n=9+9) HashArray 318ns ± 2% 318ns ± 2% ~ (p=0.541 n=10+10) HashMapAcyclic 32.9µs ± 1% 31.6µs ± 1% -4.16% (p=0.000 n=10+9) There is a slight performance gain due to the use of unsafe.Pointer over reflect.Value methods. Also, passing an unsafe.Pointer (1 word) on the stack is cheaper than passing a reflect.Value (3 words). Performance gains are diminishing since SHA-256 hashing now dominates the runtime. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	e0c5ac1f02	util/deephash: add debug printer (#5460) ... When built with "deephash_debug", print the set of HashXXX methods. Example usage: $ go test -run=GetTypeHasher/string_slice -tags=deephash_debug U64(2)+U64(3)+S("foo")+U64(3)+S("bar")+FIN Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	70f9fc8c7a	util/deephash: rely on direct memory hashing for primitive kinds (#5457) ... Rather than separate functions to hash each kind, just rely on the fact that these are direct memory hashable, thus simplifying the code. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	531ccca648	util/deephash: delete slow path (#5423) ... Every implementation of typeHasherFunc always returns true, which implies that the slow path is no longer executed. Delete it. h.hashValueWithType(v, ti, ...) is deleted as it is equivalent to: ti.hasher()(h, v) h.hashValue(v, ...) is deleted as it is equivalent to: ti := getTypeInfo(v.Type()) ti.hasher()(h, v) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	3fc8683585	util/deephash: expand fast-path capabilities (#5404) ... Add support for maps and interfaces to the fast path. Add cycle-detection to the pointer handling logic. This logic is mostly copied from the slow path. A future commit will delete the slow path once the fast path never falls back to the slow path. Performance: name old time/op new time/op delta Hash-24 18.5µs ± 1% 14.9µs ± 2% -19.52% (p=0.000 n=10+10) HashPacketFilter-24 2.54µs ± 1% 2.60µs ± 1% +2.19% (p=0.000 n=10+10) HashMapAcyclic-24 31.6µs ± 1% 30.5µs ± 1% -3.42% (p=0.000 n=9+8) TailcfgNode-24 1.44µs ± 2% 1.43µs ± 1% ~ (p=0.171 n=10+10) HashArray-24 324ns ± 1% 324ns ± 2% ~ (p=0.425 n=9+9) The additional cycle detection logic doesn't incur much slow down since it only activates if a type is recursive, which does not apply for any of the types that we care about. There is a notable performance boost since we switch from the fath path to the slow path less often. Most notably, a struct with a field that could not be handled by the fast path would previously cause the entire struct to go through the slow path. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Tom DNetto	18edd79421	control/controlclient,tailcfg: [capver 40] create KeySignature field in tailcfg.Node ... We calve out a space to put the node-key signature (used on tailnets where network lock is enabled). Signed-off-by: Tom DNetto <tom@tailscale.com>	2 years ago
Joe Tsai	d32700c7b2	util/deephash: specialize for netip.Addr and drop AppendTo support (#5402) ... There are 5 types that we care about that implement AppendTo: key.DiscoPublic key.NodePublic netip.Prefix netipx.IPRange netip.Addr The key types are thin wrappers around [32]byte and are memory hashable. The netip.Prefix and netipx.IPRange types are thin wrappers over netip.Addr and are hashable by default if netip.Addr is hashable. The netip.Addr type is the only one with a complex structure where the default behavior of deephash does not hash it correctly due to the presence of the intern.Value type. Drop support for AppendTo and instead add specialized hashing for netip.Addr that would be semantically equivalent to == on the netip.Addr values. The AppendTo support was already broken prior to this change. It was fully removed (intentionally or not) in #4870. It was partially restored in #4858 for the fast path, but still broken in the slow path. Just drop support for it altogether. This does mean we lack any ability for types to self-hash themselves. In the future we can add support for types that implement: interface { DeepHash() Sum } Test and fuzz cases were added for the relevant types that used to rely on the AppendTo method. FuzzAddr has been executed on 1 billion samples without issues. Signed-off-by: Joe Tsai joetsai@digital-static.net	2 years ago
Joe Tsai	03f7e4e577	util/hashx: move from sha256x (#5388)	2 years ago
Joe Tsai	f061d20c9d	util/sha256x: rename Hash as Block512 (#5351) ... Rename Hash as Block512 to indicate that this is a general-purpose hash.Hash for any algorithm that operates on 512-bit block sizes. While we rename the package as hashx in this commit, a subsequent commit will move the sha256x package to hashx. This is done separately to avoid confusing git. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	44d62b65d0	util/deephash: move typeIsRecursive and canMemHash to types.go (#5386) ... Also, rename canMemHash to typeIsMemHashable to be consistent. There are zero changes to the semantics. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	d53eb6fa11	util/deephash: simplify typeIsRecursive (#5385) ... Any type that is memory hashable must not be recursive since there are definitely no pointers involved to make a cycle. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	23ec3c104a	util/deephash: remove unused stack slice in typeIsRecursive (#5363) ... No operation ever reads from this variable. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	c200229f9e	util/deephash: simplify canMemHash (#5384) ... Put the t.Size() == 0 check first since this is applicable in all cases. Drop the last struct field conditional since this is covered by the sumFieldSize check at the end. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	32a1a3d1c0	util/deephash: avoid variadic argument for Update (#5372) ... Hashing []any is slow since hashing of interfaces is slow. Hashing of interfaces is slow since we pessimistically assume that cycles can occur through them and start cycle tracking. Drop the variadic signature of Update and fix callers to pass in an anonymous struct so that we are hashing concrete types near the root of the value tree. Signed-off-by: Joe Tsai <joetsai@digital-static.net> Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Maisem Ali	545639ee44	util/winutil: consolidate interface specific registry keys ... Code movement to allow reuse in a follow up PR. Updates #1659 Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Joe Tsai	548fa63e49	util/deephash: use binary encoding of time.Time (#5352) ... Formatting a time.Time as RFC3339 is slow. See https://go.dev/issue/54093 Now that we have efficient hashing of fixed-width integers, just hash the time.Time as a binary value. Performance: Hash-24 19.0µs ± 1% 18.6µs ± 1% -2.03% (p=0.000 n=10+9) TailcfgNode-24 1.79µs ± 1% 1.40µs ± 1% -21.74% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	1f7479466e	util/deephash: use sha256x (#5339) ... Switch deephash to use sha256x.Hash. We add sha256x.HashString to efficiently hash a string. It uses unsafe under the hood to convert a string to a []byte. We also modify sha256x.Hash to export the underlying hash.Hash for testing purposes so that we can intercept all hash.Hash calls. Performance: name old time/op new time/op delta Hash-24 19.8µs ± 1% 19.2µs ± 1% -3.01% (p=0.000 n=10+10) HashPacketFilter-24 2.61µs ± 0% 2.53µs ± 1% -3.01% (p=0.000 n=8+10) HashMapAcyclic-24 31.3µs ± 1% 29.8µs ± 0% -4.80% (p=0.000 n=10+9) TailcfgNode-24 1.83µs ± 1% 1.82µs ± 2% ~ (p=0.305 n=10+10) HashArray-24 344ns ± 2% 323ns ± 1% -6.02% (p=0.000 n=9+10) The performance gains is not as dramatic as sha256x over sha256 due to: 1. most of the hashing already occurring through the direct memory hashing logic, and 2. what does not go through direct memory hashing is slowed down by reflect. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	77a92f326d	util/deephash: avoid using sync.Pool for reflect.MapIter (#5333) ... In Go 1.19, the reflect.Value.MapRange method uses "function outlining" so that the allocation of reflect.MapIter is inlinable by the caller. If the iterator doesn't escape the caller, it can be stack allocated. See https://go.dev/cl/400675 Performance: name old time/op new time/op delta HashMapAcyclic-24 31.9µs ± 2% 32.1µs ± 1% ~ (p=0.075 n=10+10) name old alloc/op new alloc/op delta HashMapAcyclic-24 0.00B 0.00B ~ (all equal) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	1c3c6b5382	util/sha256x: make Hash.Sum non-escaping (#5338) ... Since Hash is a concrete type, we can make it such that Sum never escapes the input. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	76b0e578c5	util/sha256x: new package (#5337) ... The hash.Hash provided by sha256.New is much more efficient if we always provide it with data a multiple of the block size. This avoids double-copying of data into the internal block of sha256.digest.x. Effectively, we are managing a block ourselves to ensure we only ever call hash.Hash.Write with full blocks. Performance: name old time/op new time/op delta Hash 33.5µs ± 1% 20.6µs ± 1% -38.40% (p=0.000 n=10+9) The logic has gone through CPU-hours of fuzzing. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Joe Tsai	539c5e44c5	util/deephash: always keep values addressable (#5328) ... The logic of deephash is both simpler and easier to reason about if values are always addressable. In Go, the composite kinds are slices, arrays, maps, structs, interfaces, pointers, channels, and functions, where we define "composite" as a Go value that encapsulates some other Go value (e.g., a map is a collection of key-value entries). In the cases of pointers and slices, the sub-values are always addressable. In the cases of arrays and structs, the sub-values are always addressable if and only if the parent value is addressable. In the case of maps and interfaces, the sub-values are never addressable. To make them addressable, we need to copy them onto the heap. For the purposes of deephash, we do not care about channels and functions. For all non-composite kinds (e.g., strings and ints), they are only addressable if obtained from one of the composite kinds that produce addressable values (i.e., pointers, slices, addressable arrays, and addressable structs). A non-addressible, non-composite kind can be made addressable by allocating it on the heap, obtaining a pointer to it, and dereferencing it. Thus, if we can ensure that values are addressable at the entry points, and shallow copy sub-values whenever we encounter an interface or map, then we can ensure that all values are always addressable and assume such property throughout all the logic. Performance: name old time/op new time/op delta Hash-24 21.5µs ± 1% 19.7µs ± 1% -8.29% (p=0.000 n=9+9) HashPacketFilter-24 2.61µs ± 1% 2.62µs ± 0% +0.29% (p=0.037 n=10+9) HashMapAcyclic-24 30.8µs ± 1% 30.9µs ± 1% ~ (p=0.400 n=9+10) TailcfgNode-24 1.84µs ± 1% 1.84µs ± 2% ~ (p=0.928 n=10+10) HashArray-24 324ns ± 2% 332ns ± 2% +2.45% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Maisem Ali	40ec8617ac	util/must: rename Do->Get, add Do ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Brad Fitzpatrick	01e8ef7293	util/strs: add new package for string utility funcs ... Updates #5309 Change-Id: I677cc6e01050b6e10d8d6907d961b11c7a787a05 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Joe Tsai	622d80c007	util/must: new package (#5307) ... The Do function assists in calling functions that must succeed. It only interacts well with functions that return (T, err). Signatures with more return arguments are not supported. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	2 years ago
Maisem Ali	a9f6cd41fd	all: use syncs.AtomicValue ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	2 years ago
Mihai Parparita	4aa88bc2c0	cmd/tsconnect,util/precompress: move precompression to its own package ... We have very similar code in corp, moving it to util/precompress allows it to be reused. Updates #5133 Signed-off-by: Mihai Parparita <mihai@tailscale.com>	2 years ago
Brad Fitzpatrick	8725b14056	all: migrate more code code to net/netip directly ... Instead of going through the tailscale.com/net/netaddr transitional wrappers. Updates #5162 Change-Id: I3dafd1c2effa1a6caa9b7151ecf6edd1a3fda3dd Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	116f55ff66	all: gofmt for Go 1.19 ... Updates #5210 Change-Id: Ib02cd5e43d0a8db60c1f09755a8ac7b140b670be Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	04cf46a762	util/deephash: fix unexported time.Time hashing ... Updates tailscale/corp#6311 Change-Id: I33cd7e4040966261c2f2eb3d32f29936aeb7f632 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	a12aad6b47	all: convert more code to use net/netip directly ... perl -i -npe 's,netaddr.IPPrefixFrom,netip.PrefixFrom,' $(git grep -l -F netaddr.) perl -i -npe 's,netaddr.IPPortFrom,netip.AddrPortFrom,' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPrefix,netip.Prefix,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPPort,netip.AddrPort,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IP\b,netip.Addr,g' $(git grep -l -F netaddr. ) perl -i -npe 's,netaddr.IPv6Raw\b,netip.AddrFrom16,g' $(git grep -l -F netaddr. ) goimports -w . Then delete some stuff from the net/netaddr shim package which is no longer neeed. Updates #5162 Change-Id: Ia7a86893fe21c7e3ee1ec823e8aba288d4566cd8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	6a396731eb	all: use various net/netip parse funcs directly ... Mechanical change with perl+goimports. Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then goimports -d . Finally, removed the net/netaddr wrappers, to prevent future use. Updates #5162 Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	7eaf5e509f	net/netaddr: start migrating to net/netip via new netaddr adapter package ... Updates #5162 Change-Id: Id7bdec303b25471f69d542f8ce43805328d56c12 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	2a22ea3e83	util/deephash: generate type-specific hasher funcs ... name old time/op new time/op delta Hash-8 71.1µs ± 2% 71.5µs ± 1% ~ (p=0.114 n=9+8) HashPacketFilter-8 8.39µs ± 1% 4.83µs ± 2% -42.38% (p=0.000 n=8+9) HashMapAcyclic-8 56.2µs ± 1% 56.9µs ± 2% +1.17% (p=0.035 n=10+9) TailcfgNode-8 6.49µs ± 2% 3.54µs ± 1% -45.37% (p=0.000 n=9+9) HashArray-8 729ns ± 2% 566ns ± 3% -22.30% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashPacketFilter-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashPacketFilter-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I34c4e786e748fe60280646d40cc63a2adb2ea6fe Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Mihai Parparita	e37167b3ef	ipn/localapi: add API for uploading client metrics ... Clients may have platform-specific metrics they would like uploaded (e.g. extracted from MetricKit on iOS). Add a new local API endpoint that allows metrics to be updated by a simple name/value JSON-encoded struct. Signed-off-by: Mihai Parparita <mihai@tailscale.com>	2 years ago
Brad Fitzpatrick	6b71568eb7	util/cloudenv: add Azure support & DNS IPs ... And rewrite cloud detection to try to do only zero or one metadata discovery request for all clouds, only doing a first (or second) as confidence increases. Work remains for Windows, but a start. And add Cloud to tailcfg.Hostinfo, which helped with testing using "tailcfg debug hostinfo". Updates #4983 (Linux only) Updates #4984 Change-Id: Ib03337089122ce0cb38c34f724ba4b4812bc614e Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	aa37aece9c	ipn/ipnlocal, net/dns*, util/cloudenv: add AWS DNS support ... And remove the GCP special-casing from ipn/ipnlocal; do it only in the forwarder for *.internal. Fixes #4980 Fixes #4981 Change-Id: I5c481e96d91f3d51d274a80fbd37c38f16dfa5cb Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	88c2afd1e3	ipn/ipnlocal, net/dns*, util/cloudenv: specialize DNS config on Google Cloud ... This does three things: * If you're on GCP, it adds a *.internal DNS split route to the metadata server, so we never break GCP DNS names. This lets people have some Tailscale nodes on GCP and some not (e.g. laptops at home) without having to add a Tailnet-wide *.internal DNS route. If you already have such a route, though, it won't overwrite it. * If the 100.100.100.100 DNS forwarder has nowhere to forward to, it forwards it to the GCP metadata IP, which forwards to 8.8.8.8. This means there are never errNoUpstreams ("upstream nameservers not set") errors on GCP due to e.g. mangled /etc/resolv.conf (GCP default VMs don't have systemd-resolved, so it's likely a DNS supremacy fight) * makes the DNS fallback mechanism use the GCP metadata IP as a fallback before our hosted HTTP-based fallbacks I created a default GCP VM from their web wizard. It has no systemd-resolved. I then made its /etc/resolv.conf be empty and deleted its GCP hostnames in /etc/hosts. I then logged in to a tailnet with no global DNS settings. With this, tailscaled writes /etc/resolv.conf (direct mode, as no systemd-resolved) and sets it to 100.100.100.100, which then has regular DNS via the metadata IP and *.internal DNS via the metadata IP as well. If the tailnet configures explicit DNS servers, those are used instead, except for *.internal. This also adds a new util/cloudenv package based on version/distro where the cloud type is only detected once. We'll likely expand it in the future for other clouds, doing variants of this change for other popular cloud environments. Fixes #4911 RELNOTES=Google Cloud DNS improvements Change-Id: I19f3c2075983669b2b2c0f29a548da8de373c7cf Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	35782f891d	util/deephash: add canMemHash func + typeInfo property ... Currently unused. (breaking up a bigger change) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	7b9a901489	util/deephash: add packet filter benchmark ... (breaking up parts of another change) This adds a PacketFilter hashing benchmark with an input that both contains every possible field, but also is somewhat representative in the shape of what real packet filters contain. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	8c5c87be26	util/deephash: fix collisions between different types ... Updates #4883 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	59ed846277	util/singleflight: add fork of singleflight with generics ... Forked from golang.org/x/sync/singleflight at the x/sync repo's commit 67f06af15bc961c363a7260195bcd53487529a21 Updates golang/go#53427 Change-Id: Iec2b47b7777940017bb9b3db9bd7d93ba4a2e394 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	757ecf7e80	util/deephash: fix map hashing when key & element have the same type ... Regression from 09afb8e35b, in which the same reflect.Value scratch value was being used as the map iterator copy destination. Also: make nil and empty maps hash differently, add test. Fixes #4871 Co-authored-by: Josh Bleecher Snyder <josharian@gmail.com> Change-Id: I67f42524bc81f694c1b7259d6682200125ea4a66 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	f31588786f	util/deephash: don't track cycles on non-recursive types ... name old time/op new time/op delta Hash-8 67.3µs ±20% 76.5µs ±16% ~ (p=0.143 n=10+10) HashMapAcyclic-8 63.0µs ± 2% 56.3µs ± 1% -10.65% (p=0.000 n=10+8) TailcfgNode-8 9.18µs ± 2% 6.52µs ± 3% -28.96% (p=0.000 n=9+10) HashArray-8 732ns ± 3% 709ns ± 1% -3.21% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Hash-8 24.0B ± 0% 24.0B ± 0% ~ (all equal) HashMapAcyclic-8 0.00B 0.00B ~ (all equal) TailcfgNode-8 0.00B 0.00B ~ (all equal) HashArray-8 0.00B 0.00B ~ (all equal) name old allocs/op new allocs/op delta Hash-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) HashMapAcyclic-8 0.00 0.00 ~ (all equal) TailcfgNode-8 0.00 0.00 ~ (all equal) HashArray-8 0.00 0.00 ~ (all equal) Change-Id: I28642050d837dff66b2db54b2b0e6d272a930be8 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Brad Fitzpatrick	36ea837736	util/deephash: fix map hashing to actually hash elements ... Fixes #4868 Change-Id: I574fd139cb7f7033dd93527344e6aa0e625477c7 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	2 years ago
Aaron Klotz	b005b79236	net/dns, paths, util/winutil: change net/dns/windowsManager NRPT management to support more than 50 domains. ... AFAICT this isn't documented on MSDN, but based on the issue referenced below, NRPT rules are not working when a rule specifies > 50 domains. This patch modifies our NRPT rule generator to split the list of domains into chunks as necessary, and write a separate rule for each chunk. For compatibility reasons, we continue to use the hard-coded rule ID, but as additional rules are required, we generate new GUIDs. Those GUIDs are stored under the Tailscale registry path so that we know which rules are ours. I made some changes to winutils to add additional helper functions in support of both the code and its test: I added additional registry accessors, and also moved some token accessors from paths to util/winutil. Fixes https://github.com/tailscale/coral/issues/63 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Aaron Klotz	c163b2a3f1	util/winutil, util/winutil/vss: remove winrestore and vss as they are unnecessary. ... I wrote this code way back at the beginning of my tenure at Tailscale when we had concerns about needing to restore deleted machine keys from backups. We never ended up using this functionality, and the code is now getting in the way, so we might as well remove it. Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Mihai Parparita	dd5548771e	util/clientmetric: add gauge_ name prefix when uploading names ... The prefix is a signal to tsweb to treat this as a gauge metric when generating the Prometheus version. Signed-off-by: Mihai Parparita <mihai@tailscale.com>	3 years ago
Maisem Ali	fd99c54e10	tailcfg,all: change structs to []*dnstype.Resolver ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Maisem Ali	e409e59a54	cmd/cloner,util/codegen: refactor cloner internals to allow reuse ... Also run go generate again for Copyright updates. Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Mihai Parparita	025867fd07	util/clientmetric: switch to TestHooks struct for test-only functions (#4632) ... Followup to 7966aed1e0 to pick up review feedback that was accidentally left out. Signed-off-by: Mihai Parparita <mihai@tailscale.com>	3 years ago
Mihai Parparita	7966aed1e0	util/clientmetric: add test hooks and ResetLastDelta function ... Necessary to force flushing of client metrics more aggressively in dev/test mode. Signed-off-by: Mihai Parparita <mihai@tailscale.com>	3 years ago
Brad Fitzpatrick	373176ea54	util/codegen: format generated code with goimports, not gofmt ... goimports is a superset of gofmt that also groups imports. (the goimports tool also adds/removes imports as needed, but that part is disabled here) Change-Id: Iacf0408dfd9497f4ed3da4fa50e165359ce38498 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	910ae68e0b	util/mak: move tailssh's mapSet into a new package for reuse elsewhere ... Change-Id: Idfe95db82275fd2be6ca88f245830731a0d5aecf Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	53588f632d	Revert "wgengine/router,util/kmod: load & log xt_mark" ... This reverts commit 8d6793fd70. Reason: breaks Android build (cgo/pthreads addition) We can try again next cycle. Change-Id: I5e7e1730a8bf399a8acfce546a6d22e11fb835d5 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
James Tucker	8d6793fd70	wgengine/router,util/kmod: load & log xt_mark ... Attempt to load the xt_mark kernel module when it is not present. If the load fails, log error information. It may be tempting to promote this failure to an error once it has been in use for some time, so as to avoid reaching an error with the iptables invocation, however, there are conditions under which the two stages may disagree - this change adds more useful breadcrumbs. Example new output from tailscaled running under my WSL2: ``` router: ensure module xt_mark: "/usr/sbin/modprobe xt_mark" failed: exit status 1; modprobe: FATAL: Module xt_mark not found in directory /lib/modules/5.10.43.3-microsoft-standard-WSL2 ``` Background: There are two places to lookup modules, one is `/proc/modules` "old", the other is `/sys/module/` "new". There was query_modules(2) in linux <2.6, alas, it is gone. In a docker container in the default configuration, you would get /proc/modules and /sys/module/ both populated. lsmod may work file, modprobe will fail with EPERM at `finit_module()` for an unpriviliged container. In a priviliged container the load may *succeed*, if some conditions are met. This condition should be avoided, but the code landing in this change does not attempt to avoid this scenario as it is both difficult to detect, and has a very uncertain impact. In an nspawn container `/proc/modules` is populated, but `/sys/module` does not exist. Modern `lsmod` versions will fail to gather most module information, without sysfs being populated with module information. In WSL2 modules are likely missing, as the in-use kernel typically is not provided by the distribution filesystem, and WSL does not mount in a module filesystem of its own. Notably the WSL2 kernel supports iptables marks without listing the xt_mark module in /sys/module, and /proc/modules is empty. On a recent kernel, we can ask the capabilities system about SYS_MODULE, that will help to disambiguate between the non-privileged container case and just being root. On older kernels these calls may fail. Update #4329 Signed-off-by: James Tucker <james@tailscale.com>	3 years ago
Maisem Ali	909f40da84	util/groupmember: remove redundant code (#4298) ... Now that we have 30faf968b1 this is no longer needed. Fixes #3001 Signed-off-by: Maisem Ali <maisem@tailscale.com>	3 years ago
Josh Bleecher Snyder	32fd42430b	all: use cibuild.On ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	0868329936	all: use any instead of interface{} ... My favorite part of generics. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Brad Fitzpatrick	61ee72940c	all: use Go 1.18's strings.Cut ... More remain. Change-Id: I6ec562cc1f687600758deae1c9d7dbd0d04004cb Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Josh Bleecher Snyder	97a01b7b17	util/deephash: remove Tailscale toolchain compatibility shim ... The future is now. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	463728a885	util/netconv: add package to convert between netip and netaddr types ... Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Aaron Klotz	82cd98609f	util/winutil: migrate corp's winutil into OSS. ... It makes the most sense to have all our utility functions reside in one place. There was nothing in corp that could not reasonably live in OSS. I also updated `StartProcessAsChild` to no longer depend on `futureexec`, thus reducing the amount of code that needed migration. I tested this change with `tswin` and it is working correctly. I have a follow-up PR to remove the corresponding code from corp. The migrated code was mostly written by @alexbrainman. Sourced from corp revision 03e90cfcc4dd7b8bc9b25eb13a26ec3a24ae0ef9 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Aaron Klotz	3a74f2d2d7	cmd/tailscaled, util/winutil: add accessor functions for Windows system policies. ... This patch adds new functions to be used when accessing system policies, and revises callers to use the new functions. They first attempt the new registry path for policies, and if that fails, attempt to fall back to the legacy path. We keep non-policy variants of these functions because we should be able to retain the ability to read settings from locations that are not exposed to sysadmins for group policy edits. The remaining changes will be done in corp. Updates https://github.com/tailscale/tailscale/issues/3584 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Brad Fitzpatrick	0626cf4183	util/winutil: fix build ... It was broken on Windows: Error: util\winutil\winutil_windows.go:15:7: regBase redeclared in this block Error: D:\a\tailscale\tailscale\util\winutil\winutil_notwindows.go:7:17: previous declaration Error: util\winutil\winutil_windows.go:29:6: getRegString redeclared in this block Error: D:\a\tailscale\tailscale\util\winutil\winutil_notwindows.go:9:40: previous declaration Error: util\winutil\winutil_windows.go:47:6: getRegInteger redeclared in this block Error: D:\a\tailscale\tailscale\util\winutil\winutil_notwindows.go:11:48: previous declaration Error: util\winutil\winutil_windows.go:77:6: isSIDValidPrincipal redeclared in this block Error: D:\a\tailscale\tailscale\util\winutil\winutil_notwindows.go:13:38: previous declaration Change-Id: Ib1ce4b647f5711547840c736b933a6c42bf09583 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Aaron Klotz	d7962e3bcf	ipn/ipnserver, util/winutil: update workaround for os/user.LookupId failures on Windows to reject SIDs from deleted/invalid security principals. ... Our current workaround made the user check too lax, thus allowing deleted users. This patch adds a helper function to winutil that checks that the uid's SID represents a valid Windows security principal. Now if `lookupUserFromID` determines that the SID is invalid, we simply propagate the error. Updates https://github.com/tailscale/tailscale/issues/869 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Brad Fitzpatrick	486059589b	all: gofmt -w -s (simplify) tests ... And it updates the build tag style on a couple files. Change-Id: I84478d822c8de3f84b56fa1176c99d2ea5083237 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	0de1b74fbb	util/clientmetric: add tests omitted from earlier commit ... These were supposed to be part of 3b541c833e but I guess I forgot to "git add" them. Whoops. Updates #3307 Change-Id: I8c768a61ec7102a01799e81dc502a22399b9e9f0 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	5b5f032c9a	util/clientmetric: optimize memory layout for finding updates ... Updates #3307 Change-Id: I2840b190583467cc3f00688b96ce3d170df46a46 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	3b541c833e	util/clientmetric, logtail: log metric changes ... Updates #3307 Change-Id: I1399ebd786f6ff7defe6e11c0eb651144c071574 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	57b039c51d	util/clientmetrics: add new package to add metrics to the client ... And annotate magicsock as a start. And add localapi and debug handlers with the Prometheus-format exporter. Updates #3307 Change-Id: I47c5d535fe54424741df143d052760387248f8d3 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
David Anderson	0532eb30db	all: replace tailcfg.DiscoKey with key.DiscoPublic. ... Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Josh Bleecher Snyder	3fd5f4380f	util/multierr: new package ... github.com/go-multierror/multierror served us well. But we need a few feature from it (implement Is), and it's not worth maintaining a fork of such a small module. Instead, I did a clean room implementation inspired by its API. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
David Anderson	a9c78910bd	wgengine/wgcfg: convert to use new node key type. ... Updates #3206 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Aaron Klotz	c6ea282b3f	utils/winutil utils/winutil/vss: add utility function for extracting data from Windows System Restore Point backups. ... utils/winutil/vss contains just enough COM wrapping to query the Volume Shadow Copy service for snapshots. WalkSnapshotsForLegacyStateDir is the friendlier interface that adds awareness of our actual use case, mapping the snapshots and locating our legacy state directory. Updates #3011 Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Brad Fitzpatrick	0410f1a35a	util/groupmember: adjust build tags for osusergo ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	2a0d3f72c5	util/groupmember: fix typo of package's name in its package doc ... And canonicalize a func comment while I'm here. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Aaron Klotz	3386a86fe5	util/winutil: add GetRegInteger ... This helper allows us to retrieve `DWORD` and `QWORD` values from the Tailscale key in the Windows registry. Signed-off-by: Aaron Klotz <aaron@tailscale.com>	3 years ago
Josh Bleecher Snyder	b14db5d943	util/codegen: reorder AssertStructUnchanged args ... The fully qualified name of the type is thisPkg.tname, so write the args like that too. Suggested-by: Joe Tsai <joetsai@digital-static.net> Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	3cd85c0ca6	util/codegen: add ContainsPointers ... And use it in cmd/cloner. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	d8a8f70000	util/codegen: add NamedTypes ... And use it in cmd/cloner. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	618376dbc0	util/codegen: add AssertStructUnchanged ... Refactored out from cmd/cloner. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Josh Bleecher Snyder	fb66ff7c78	util/codegen: add package ... This is a package for shared utilities used in doing codegen programs. The inaugural API is for writing gofmt'd code to a file. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
David Anderson	bb10443edf	wgengine/wgcfg: use just the hexlified node key as the WireGuard endpoint. ... The node key is all magicsock needs to find the endpoint that WireGuard needs. Updates #2752 Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
David Anderson	54bc3b7d97	util/deephash: remove soon to be deleted field from wgcfg. ... Signed-off-by: David Anderson <danderson@tailscale.com>	3 years ago
Joe Tsai	3f1317e3e5	util/deephash: fix TestArrayAllocs ... Unfortunately this test fails on certain architectures. The problem comes down to inconsistencies in the Go escape analysis where specific variables are marked as escaping on certain architectures. The variables escaping to the heap are unfortunately in crypto/sha256, which makes it impossible to fixthis locally in deephash. For now, fix the test by compensating for the allocations that occur from calling sha256.digest.Sum. See golang/go#48055 Fixes #2727 Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
slowy07	ac0353e982	fix: typo spelling grammar ... Signed-off-by: slowy07 <slowy.arfy@gmail.com>	3 years ago
David Crawshaw	360223fccb	types/dnstype: introduce new package for Resolver ... So the type can be used in net/dns without introducing a tailcfg dependency. For #2596 Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Josh Bleecher Snyder	a5da4ed981	all: gofmt with Go 1.17 ... This adds "//go:build" lines and tidies up existing "// +build" lines. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	3 years ago
Joe Tsai	9d0c86b6ec	util/deephash: remove unnecessary formatting for structs and slices (#2571) ... The index for every struct field or slice element and the number of fields for the struct is unncessary. The hashing of Go values is unambiguous because every type (except maps) encodes in a parsable manner. So long as we know the type information, we could theoretically decode every value (except for maps). At a high level: * numbers are encoded as fixed-width records according to precision. * strings (and AppendTo output) are encoded with a fixed-width length, followed by the contents of the buffer. * slices are prefixed by a fixed-width length, followed by the encoding of each value. So long as we know the type of each element, we could theoretically decode each element. * arrays are encoded just like slices, but elide the length since it is determined from the Go type. * maps are encoded first with a byte indicating whether it is a cycle. If a cycle, it is followed by a fixed-width index for the pointer, otherwise followed by the SHA-256 hash of its contents. The encoding of maps is not decodeable, but a SHA-256 hash is sufficient to avoid ambiguities. * interfaces are encoded first with a byte indicating whether it is nil. If not nil, it is followed by a fixed-width index for the type, and then the encoding for the underlying value. Having the type be encoded first ensures that the value could theoretically be decoded next. * pointers are encoded first with a byte indicating whether it is 1) nil, 2) a cycle, or 3) newly seen. If a cycle, it is followed by a fixed-width index for the pointer. If newly seen, it is followed by the encoding for the pointed-at value. Removing unnecessary details speeds up hashing: name old time/op new time/op delta Hash-8 76.0µs ± 1% 55.8µs ± 2% -26.62% (p=0.000 n=10+10) HashMapAcyclic-8 61.9µs ± 0% 62.0µs ± 0% ~ (p=0.666 n=9+9) TailcfgNode-8 10.2µs ± 1% 7.5µs ± 1% -26.90% (p=0.000 n=10+9) HashArray-8 1.07µs ± 1% 0.70µs ± 1% -34.67% (p=0.000 n=10+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Joe Tsai	d8fbce7eef	util/deephash: hash uint{8,16,32,64} explicitly (#2502) ... Instead of hashing the humanly formatted forms of a number, hash the native machine bits of the integers themselves. There is a small performance gain for this: name old time/op new time/op delta Hash-8 75.7µs ± 1% 76.0µs ± 2% ~ (p=0.315 n=10+9) HashMapAcyclic-8 63.1µs ± 3% 61.3µs ± 1% -2.77% (p=0.000 n=10+10) TailcfgNode-8 10.3µs ± 1% 10.2µs ± 1% -1.48% (p=0.000 n=10+10) HashArray-8 1.07µs ± 1% 1.05µs ± 1% -1.79% (p=0.000 n=10+10) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Joe Tsai	01d4dd331d	util/deephash: simplify hasher.hashMap (#2503) ... The swapping of bufio.Writer between hasher and mapHasher is subtle. Just embed a hasher in mapHasher to avoid complexity here. No notable change in performance: name old time/op new time/op delta Hash-8 76.7µs ± 1% 77.0µs ± 1% ~ (p=0.182 n=9+10) HashMapAcyclic-8 62.4µs ± 1% 62.5µs ± 1% ~ (p=0.315 n=10+9) TailcfgNode-8 10.3µs ± 1% 10.3µs ± 1% -0.62% (p=0.004 n=10+9) HashArray-8 1.07µs ± 1% 1.06µs ± 1% -0.98% (p=0.001 n=8+9) Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Joe Tsai	d145c594ad	util/deephash: improve cycle detection (#2470) ... The previous algorithm used a map of all visited pointers. The strength of this approach is that it quickly prunes any nodes that we have ever visited before. The detriment of the approach is that pruning is heavily dependent on the order that pointers were visited. This is especially relevant for hashing a map where map entries are visited in a non-deterministic manner, which would cause the map hash to be non-deterministic (which defeats the point of a hash). This new algorithm uses a stack of all visited pointers, similar to how github.com/google/go-cmp performs cycle detection. When we visit a pointer, we push it onto the stack, and when we leave a pointer, we pop it from the stack. Before visiting a pointer, we first check whether the pointer exists anywhere in the stack. If yes, then we prune the node. The detriment of this approach is that we may hash a node more often than before since we do not prune as aggressively. The set of visited pointers up until any node is only the path of nodes up to that node and not any other pointers that may have been visited elsewhere. This provides us deterministic hashing regardless of visit order. We can now delete hashMapFallback and associated complexity, which only exists because the previous approach was non-deterministic in the presence of cycles. This fixes a failure of the old algorithm where obviously different values are treated as equal because the pruning was too aggresive. See https://github.com/tailscale/tailscale/issues/2443#issuecomment-883653534 The new algorithm is slightly slower since it prunes less aggresively: name old time/op new time/op delta Hash-8 66.1µs ± 1% 68.8µs ± 1% +4.09% (p=0.000 n=19+19) HashMapAcyclic-8 63.0µs ± 1% 62.5µs ± 1% -0.76% (p=0.000 n=18+19) TailcfgNode-8 9.79µs ± 2% 9.88µs ± 1% +0.95% (p=0.000 n=19+17) HashArray-8 643ns ± 1% 653ns ± 1% +1.64% (p=0.000 n=19+19) However, a slower but more correct algorithm seems more favorable than a faster but incorrect algorithm. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Joe Tsai	d666bd8533	util/deephash: disambiguate hashing of AppendTo (#2483) ... Prepend size to AppendTo output. Fixes #2443 Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Joe Tsai	23ad028414	util/deephash: include type as part of hash for interfaces (#2476) ... A Go interface may hold any number of different concrete types. Just because two underlying values hash to the same thing does not mean the two values are identical if they have different concrete types. As such, include the type in the hash.	3 years ago
Joe Tsai	a5fb8e0731	util/deephash: introduce deliberate instability (#2477) ... Seed the hash upon first use with the current time. This ensures that the stability of the hash is bounded within the lifetime of one program execution. Hopefully, this prevents future bugs where someone assumes that this hash is stable. Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Joe Tsai	9a0c8bdd20	util/deephash: make hash type opaque ... The fact that Hash returns a [sha256.Size]byte leaks details about the underlying hash implementation. This could very well be any other hashing algorithm with a possible different block size. Abstract this implementation detail away by declaring an opaque type that is comparable. While we are changing the signature of UpdateHash, rename it to just Update to reduce stutter (e.g., deephash.Update). Signed-off-by: Joe Tsai <joetsai@digital-static.net>	3 years ago
Brad Fitzpatrick	ddb8726c98	util/deephash: don't reflect.Copy if element type is a defined uint8 ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	df176c82f5	util/deephash: skip alloc test under race detector ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	6dc38ff25c	util/deephash: optimize hashing of byte arrays, reduce allocs in Hash ... name old time/op new time/op delta Hash-6 173µs ± 4% 101µs ± 3% -41.69% (p=0.000 n=10+9) HashMapAcyclic-6 101µs ± 5% 105µs ± 3% +3.52% (p=0.001 n=9+10) TailcfgNode-6 29.4µs ± 2% 16.4µs ± 3% -44.25% (p=0.000 n=8+10) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 1.13kB ± 0% -68.70% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.137 n=10+8) TailcfgNode-6 528B ± 0% 0B -100.00% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 40.0 ± 0% -52.38% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 0.0 -100.00% (p=0.000 n=10+10) Updates tailscale/corp#2130 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	3962744450	util/deephash: prevent infinite loop on map cycle ... Fixes #2340 Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	aceaa70b16	util/deephash: move funcs to methods ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	14f901da6d	util/deephash: fix sync.Pool usage ... Whoops. From yesterday's 9ae3bd0939 (not yet used by anything, fortunately) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	e0258ffd92	util/deephash: use keyed struct literal, fix vet ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	bf9f279768	util/deephash: optimize CPU a bit by by avoiding fmt in more places ... name old time/op new time/op delta Hash-6 179µs ± 5% 173µs ± 4% -3.12% (p=0.004 n=10+10) HashMapAcyclic-6 115µs ± 3% 101µs ± 5% -11.51% (p=0.000 n=9+9) TailcfgNode-6 30.8µs ± 4% 29.4µs ± 2% -4.51% (p=0.000 n=10+8) name old alloc/op new alloc/op delta Hash-6 3.60kB ± 0% 3.60kB ± 0% ~ (p=0.445 n=9+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (p=0.065 n=9+10) TailcfgNode-6 528B ± 0% 528B ± 0% ~ (all equal) name old allocs/op new allocs/op delta Hash-6 84.0 ± 0% 84.0 ± 0% ~ (all equal) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) TailcfgNode-6 11.0 ± 0% 11.0 ± 0% ~ (all equal) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	58f2ef6085	util/deephash: add a benchmark and some benchmark data ... No code changes. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	9ae3bd0939	util/deephash: export a Hash func for use by the control plane ... name old time/op new time/op delta Hash-6 69.4µs ± 6% 68.4µs ± 4% ~ (p=0.286 n=9+9) HashMapAcyclic-6 115µs ± 5% 115µs ± 4% ~ (p=1.000 n=10+10) name old alloc/op new alloc/op delta Hash-6 2.29kB ± 0% 1.88kB ± 0% -18.13% (p=0.000 n=10+10) HashMapAcyclic-6 2.53kB ± 0% 2.53kB ± 0% ~ (all equal) name old allocs/op new allocs/op delta Hash-6 58.0 ± 0% 54.0 ± 0% -6.90% (p=0.000 n=10+10) HashMapAcyclic-6 202 ± 0% 202 ± 0% ~ (all equal) Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	3 years ago
Brad Fitzpatrick	700badd8f8	util/deephash: move internal/deephash to util/deephash ... No code changes. Just a minor package doc addition about lack of API stability.	3 years ago
David Crawshaw	6b9f8208f4	net/dns: do not run wsl.exe as LocalSystem ... It doesn't work. It needs to run as the user. https://github.com/microsoft/WSL/issues/4803 The mechanism for doing this was extracted from: https://web.archive.org/web/20101009012531/http://blogs.msdn.com/b/winsdk/archive/2009/07/14/launching-an-interactive-process-from-windows-service-in-windows-vista-and-later.aspx While here, we also reclaculate WSL distro set on SetDNS. This accounts for: 1. potential inability to access wsl.exe on startup 2. WSL being installed while Tailscale is running 3. A new WSL distrobution being installed Signed-off-by: David Crawshaw <crawshaw@tailscale.com>	3 years ago
Maisem Ali	f944614c5c	cmd/tailscale/web: add support for QNAP ... Signed-off-by: Maisem Ali <maisem@tailscale.com>	4 years ago
David Anderson	c54cc24e87	util/dnsname: make ToFQDN take exactly 0 or 1 allocs for everything. ... name old time/op new time/op delta ToFQDN/www.tailscale.com.-32 9.55ns ± 2% 12.13ns ± 3% +27.03% (p=0.000 n=10+10) ToFQDN/www.tailscale.com-32 86.3ns ± 1% 40.7ns ± 1% -52.86% (p=0.000 n=10+9) ToFQDN/.www.tailscale.com-32 86.5ns ± 1% 40.4ns ± 1% -53.29% (p=0.000 n=10+9) ToFQDN/_ssh._tcp.www.tailscale.com.-32 12.8ns ± 2% 14.7ns ± 2% +14.24% (p=0.000 n=9+10) ToFQDN/_ssh._tcp.www.tailscale.com-32 104ns ± 1% 45ns ± 0% -57.16% (p=0.000 n=10+9) name old alloc/op new alloc/op delta ToFQDN/www.tailscale.com.-32 0.00B 0.00B ~ (all equal) ToFQDN/www.tailscale.com-32 72.0B ± 0% 24.0B ± 0% -66.67% (p=0.000 n=10+10) ToFQDN/.www.tailscale.com-32 72.0B ± 0% 24.0B ± 0% -66.67% (p=0.000 n=10+10) ToFQDN/_ssh._tcp.www.tailscale.com.-32 0.00B 0.00B ~ (all equal) ToFQDN/_ssh._tcp.www.tailscale.com-32 112B ± 0% 32B ± 0% -71.43% (p=0.000 n=10+10) name old allocs/op new allocs/op delta ToFQDN/www.tailscale.com.-32 0.00 0.00 ~ (all equal) ToFQDN/www.tailscale.com-32 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10) ToFQDN/.www.tailscale.com-32 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10) ToFQDN/_ssh._tcp.www.tailscale.com.-32 0.00 0.00 ~ (all equal) ToFQDN/_ssh._tcp.www.tailscale.com-32 2.00 ± 0% 1.00 ± 0% -50.00% (p=0.000 n=10+10) Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	d7f6ef3a79	util/dnsname: add a benchmark for ToFQDN. ... Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	caaefa00a0	util/dnsname: don't validate the contents of DNS labels. ... DNS names consist of labels, but outside of length limits, DNS itself permits any content within the labels. Some records require labels to conform to hostname limitations (which is what we implemented before), but not all. Fixes #2024. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	dc32b4695c	util/dnsname: normalize leading dots in ToFQDN. ... Fixes #1888. Signed-off-by: David Anderson <dave@natulte.net>	4 years ago
David Anderson	31f81b782e	util/cmpver: move into OSS from corp repo. ... Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
Aleksandar Pesic	7c985e4944	ipn/ipnlocal: add file sharing to windows shell ... Updates: tailscale/winmin#33 Signed-off-by: Aleksandar Pesic <peske.nis@gmail.com>	4 years ago
David Anderson	7055f870f8	control/controlclient: only use a single DNS label as the hostname. ... Fixes #971 Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	1a371b93be	util/dnsname: add FQDN type, use throughout codebase. ... Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
David Anderson	9f105d3968	net/dns/resolver: teach the forwarder to do per-domain routing. ... Given a DNS route map, the forwarder selects the right set of upstreams for a given name. Signed-off-by: David Anderson <danderson@tailscale.com>	4 years ago
Christine Dodrill	a480b1baa5	logpolicy: set log target on windows based on a registry key (#1542) ... Signed-off-by: Christine Dodrill <xe@tailscale.com>	4 years ago
Matt Layher	719de8f0e1	util/systemd: explicitly check for os.ErrNotExist from sdnotify ... Signed-off-by: Matt Layher <mdlayher@gmail.com>	4 years ago
Brad Fitzpatrick	2d5db90161	util/winutil: make it actually compile ... Helps to use the right GOOS after refactoring, sigh. Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	e98cdbb8b6	util/winutil: add little Windows utility package ... Code from Alex Brainman, split out of another change. I changed it to a comma-ok return and tweaked the docs a bit.	4 years ago
Sonia Appasamy	76fb27bea7	dnsname,tailcfg: add hostname sanitation logic to node display names (#1304) ... Signed-off-by: Sonia Appasamy <sonia@tailscale.com>	4 years ago
Josh Bleecher Snyder	635e4c7435	wgengine/magicsock: increase legacy ping timeout again ... I based my estimation of the required timeout based on locally observed behavior. But CI machines are worse than my local machine. 16s was enough to reduce flakiness but not eliminate it. Bump it up again. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	4 years ago
Sonia Appasamy	4dab0c1702	tailcfg: update node display name fields and methods (#1207) ... Signed-off-by: Sonia Appasamy <sonia@tailscale.com> Consolidates the node display name logic from each of the clients into tailcfg.Node. UI clients can use these names directly, rather than computing them independently.	4 years ago
Sonia Appasamy	567c5a6d9e	tailcfg, controlclient: add DisplayName field to tailcfg.Node and populate it from controlclient (#1191) ... Signed-off-by: Sonia Appasamy <sonia@tailscale.com>	4 years ago
Brad Fitzpatrick	5efb0a8bca	cmd/tailscale: change formatting of "tailscale status" ... * show DNS name over hostname, removing domain's common MagicDNS suffix. only show hostname if there's no DNS name. but still show shared devices' MagicDNS FQDN. * remove nerdy low-level details by default: endpoints, DERP relay, public key. They're available in JSON mode still for those who need them. * only show endpoint or DERP relay when it's active with the goal of making debugging easier. (so it's easier for users to understand what's happening) The asterisks are gone. * remove Tx/Rx numbers by default for idle peers; only show them when there's traffic. * include peers' owner login names * add CLI option to not show peers (matching --self=true, --peers= also defaults to true) * sort by DNS/host name, not public key * reorder columns	4 years ago
Brad Fitzpatrick	5f07da4854	util/systemd: don't log warnings when not running under systemd ... It caused our integration tests to fail, which prohibit logging to os.Stderr for test cleanliness reasons.	4 years ago
Christine Dodrill	7ea809897d	ipn/ipnserver: enable systemd-notify support ... Addresses #964 Still to be done: - Figure out the correct logging lines in util/systemd - Figure out if we need to slip the systemd.Status function anywhere else - Log util/systemd errors? (most of the errors are of the "you cannot do anything about this, but it might be a bad idea to crash the program if it errors" kind) Assistance in getting this over the finish line would help a lot. Signed-off-by: Christine Dodrill <me@christine.website> util/systemd: rename the nonlinux file to appease the magic Signed-off-by: Christine Dodrill <me@christine.website> util/systemd: fix package name Signed-off-by: Christine Dodrill <me@christine.website> util/systemd: fix review feedback from @mdlayher Signed-off-by: Christine Dodrill <me@christine.website> cmd/tailscale{,d}: update depaware manifests Signed-off-by: Christine Dodrill <me@christine.website> util/systemd: use sync.Once instead of func init Signed-off-by: Christine Dodrill <me@christine.website> control/controlclient: minor review feedback fixes Signed-off-by: Christine Dodrill <me@christine.website> {control,ipn,systemd}: fix review feedback Signed-off-by: Christine Dodrill <me@christine.website> review feedback fixes Signed-off-by: Christine Dodrill <me@christine.website> ipn: fix sprintf call Signed-off-by: Christine Dodrill <me@christine.website> ipn: make staticcheck less sad Signed-off-by: Christine Dodrill <me@christine.website> ipn: print IP address in connected status Signed-off-by: Christine Dodrill <me@christine.website> ipn: review feedback Signed-off-by: Christine Dodrill <me@christine.website> final fixups Signed-off-by: Christine Dodrill <me@christine.website>	4 years ago
Josh Bleecher Snyder	a5dd0bcb09	util/jsonutil: new package ... The cornerstone API is a more memory-efficient Unmarshal. The savings come from re-using a json.Decoder. BenchmarkUnmarshal-8 4016418 288 ns/op 8 B/op 1 allocs/op BenchmarkStdUnmarshal-8 4189261 283 ns/op 184 B/op 2 allocs/op It also includes a Bytes type to reduce allocations when unmarshalling a non-hex-encoded JSON string into a []byte. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	4 years ago
Josh Bleecher Snyder	b65eee0745	util/lineread: add docs to Reader ... In particular, point out how to stop reading and detect it on the other side. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	4 years ago
Brad Fitzpatrick	c8939ab7c7	util/endian: add Native variable to get the platform's native binary.ByteOrder ... Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>	4 years ago
Brad Fitzpatrick	ff50ddf1ee	util/racebuild: add package to export a race-is-enabled const	4 years ago
Brad Fitzpatrick	000347d4cf	util/endian: add package with const for whether platform is big endian	4 years ago
Josh Bleecher Snyder	9784cae23b	util/uniq: add new package ... This makes it easy to compact slices that contain duplicate elements by sorting and then uniqing. This is an alternative to constructing an intermediate map and then extracting elements from it. It also provides more control over equality than using a map key does. Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>	4 years ago
Brad Fitzpatrick	bc1751a376	util/pidowner: add missing copyright header	4 years ago
Brad Fitzpatrick	b14288f96c	util/pidowner: add two missing copyright headers	4 years ago
Brad Fitzpatrick	23f01174ea	util/pidowner: new package to map from process ID to its user ID	4 years ago
Brad Fitzpatrick	c3c607e78a	util/lineread: add little package to read lines from files/Readers	4 years ago