Commit Graph

131 Commits (e84751217a79e9e65be74f17b7595d04dcde0fef)

Author SHA1 Message Date
Marwan Sulaiman 7e357e1636 tsweb: rename AccessLogRecord's When to Time
This change makes our access log record more consistent with the
new log/tslog package formatting of "time". Note that we can
change slog itself to call "time" "when" but we're chosing
to make this breaking change to be consistent with the std lib's
defaults.

Updates tailscale/corp#17071

Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
6 months ago
Will Norris 80decd83c1 tsweb: remove redundant bumpStartIfNeeded func
Updates #12001

Signed-off-by: Will Norris <will@tailscale.com>
7 months ago
Will Norris 46980c9664 tsweb: ensure in-flight requests are always marked as finished
The inflight request tracker only starts recording a new bucket after
the first non-error request. Unfortunately, it's written in such a way
that ONLY successful requests are ever marked as being finished. Once a
bucket has had at least one successful request and begun to be tracked,
all subsequent error cases are never marked finished and always appear
as in-flight.

This change ensures that if a request is recorded has having been
started, we also mark it as finished at the end.

Updates tailscale/corp#19767

Signed-off-by: Will Norris <will@tailscale.com>
7 months ago
Brad Fitzpatrick 7c1d6e35a5 all: use Go 1.22 range-over-int
Updates #11058

Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
7 months ago
James Tucker 8fa3026614 tsweb: switch to fastuuid for request ID generation
Request ID generation appears prominently in some services cumulative
allocation rate, and while this does not eradicate this issue (the API
still makes UUID objects), it does improve the overhead of this API and
reduce the amount of garbage that it produces.

Updates tailscale/corp#18266
Updates tailscale/corp#19054

Signed-off-by: James Tucker <james@tailscale.com>
8 months ago
Maisem Ali 21b32b467e tsweb: handle panics in retHandler
We would have incomplete stats and missing logs in cases
of panics.

Updates tailscale/corp#18687

Signed-off-by: Maisem Ali <maisem@tailscale.com>
8 months ago
James Tucker 9eaa56df93 tsweb: update doc on BucketedStatsOptions.Finish to match behavior
I originally came to update this to match the documented behavior, but
the code is deliberately avoiding this behavior currently, making it
hard to decide how to update this. For now just align the documentation
to the behavior.

Updates #cleanup

Signed-off-by: James Tucker <james@tailscale.com>
8 months ago
Brad Fitzpatrick b0941b79d6 tsweb: make BucketedStats not track 400s, 404s, etc
Updates tailscale/corp#18687

Change-Id: I142ccb1301ec4201c70350799ff03222bce96668
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Brad Fitzpatrick 354cac74a9 tsweb/varz: add charset=utf-8 to varz handler
Some of our labels contain UTF-8 and get mojibaked in the browser
right now.

Updates tailscale/corp#18687

Change-Id: I6069cffd6cc8813df415f06bb308bc2fc3ab65c4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Brad Fitzpatrick 55baf9474f metrics, tsweb/varz: add multi-label map metrics
Updates tailscale/corp#18640

Change-Id: Ia9ae25956038e9d3266ea165537ac6f02485b74c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Brad Fitzpatrick 4992aca6ec tsweb/varz: flesh out munging of expvar keys into valid Prometheus metrics
From a problem we hit with how badger registers expvars; it broke
trunkd's exported metrics.

Updates tailscale/corp#1297

Change-Id: I42e1552e25f734c6f521b6e993d57a82849464b2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Andrew Dunham 8f27d519bb tsweb: add String method to tsweb.RequestID
In case we want to change the format to something opaque later.

Updates tailscale/corp#2549

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ie2eac8b885b694be607e9d5101d24b650026d89c
9 months ago
Chris Palmer 13853e7f29
tsweb: add more test cases for TestCleanRedirectURL (#11331)
Updates #cleanup

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
9 months ago
Andrew Lytvynov c7474431f1
tsweb: allow empty redirect URL in CleanRedirectURL (#11295)
Updates #cleanup

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
9 months ago
Andrew Dunham 23e9447871 tsweb: expose function to generate request IDs
For use in corp.

Updates tailscale/corp#2549

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I71debae1ce9ae48cf69cc44c2ab5c443fc3b2005
9 months ago
Patrick O'Doherty 256ecd0e8f
Revert "tsweb: update ServeMux matching to 1.22.0 syntax (#11090)" (#11125)
This reverts commit 30c9189ed3.

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
9 months ago
Patrick O'Doherty 30c9189ed3
tsweb: update ServeMux matching to 1.22.0 syntax (#11090)
* tsweb: update ServeMux matching to 1.22.0 syntax

Updates #cleanup

Go 1.22.0 introduced the ability to use more expressive routing patterns
that include HTTP method when constructing ServeMux entries.
Applications that attempted to use these patterns in combination with
the old `tsweb.Debugger` would experience a panic as Go would not permit
the use of matching rules with mixed level of specificity. We now
specify the method for each `/debug` handler to prevent
incompatibilities.

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
9 months ago
Patrick O'Doherty 7c52b27daf
Revert "tsweb: update ServeMux matching to 1.22.0 syntax (#11087)" (#11089)
This reverts commit 291f91d164.

Updates #cleanup

This PR needs additional changes to the registration of child handlers under /debug

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
10 months ago
Patrick O'Doherty 291f91d164
tsweb: update ServeMux matching to 1.22.0 syntax (#11087)
Updates #cleanup

Go 1.22.0 introduced the ability to use more expressive routing patterns
that include HTTP method when constructing ServeMux entries.
Applications that attempted to use these patterns in combination with
the old `tsweb.Debugger` would experience a panic as Go would not permit
the use of matching rules with mixed level of specificity.

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
10 months ago
Brad Fitzpatrick 2bd3c1474b util/cmpx: delete now that we're using Go 1.22
Updates #11058

Change-Id: I09dea8e86f03ec148b715efca339eab8b1f0f644
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
10 months ago
Tom DNetto 653721541c tsweb: normalize passkey identities in bucketed stats
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
Tom DNetto 8d6d9d28ba tsweb: normalize common StableID's in bucketed stats, export as LabelMap
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
Tom DNetto af931dcccd tsweb: replace domains/emails in paths when bucketing stats
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
Tom DNetto 36efc50817 tsweb: implementing bucketed statistics for started/finished counts
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
David Anderson ae79b2e784 tsweb: add a helper to validate redirect URLs
We issue redirects in a few different places, it's time to have
a common helper to do target validation.

Updates tailscale/corp#16875

Signed-off-by: David Anderson <danderson@tailscale.com>
10 months ago
Joe Tsai c25968e1c5
all: make use of ctxkey everywhere (#10846)
Also perform minor cleanups on the ctxkey package itself.
Provide guidance on when to use ctxkey.Key[T] over ctxkey.New.
Also, allow for interface kinds because the value wrapping trick
also happens to fix edge cases with interfaces in Go.

Updates #cleanup

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
10 months ago
Chris Palmer 1f1ab74250
tsweb: use object-src instead of plugin-types (#10719)
plugin-types is deprecated, and setting object-src: 'none' is best
practice. This should result in no functional change.

Fixes #10718

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
11 months ago
Marwan Sulaiman b819f66eb1 tsweb: propagate RequestID via context and entire request
The recent addition of RequestID was only populated if the
HTTP Request had returned an error. This meant that the underlying
handler has no access to this request id and any logs it may have
emitted were impossible to correlate to that request id. Therefore,
this PR adds a middleware to generate request ids and pass them
through the request context. The tsweb.StdHandler automatically
populates this request id if the middleware is being used. Finally,
inner handlers can use the context to retrieve that same request id
and use it so that all logs and events can be correlated.

Updates #2549

Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
12 months ago
Maisem Ali 5ee349e075 tsweb/varz: fix exporting histograms
Updates tailscale/corp#8641

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Dave Anderson 7b18ed293b
tsweb: check for key-based debug access before XFF check (#9093)
Fly apps all set X-Forwarded-For, which breaks debug access even
with a preshared key otherwise.

Updates tailscale/corp#3601

Signed-off-by: David Anderson <danderson@tailscale.com>
1 year ago
David Anderson 5cfa85e604 tsweb: clean up pprof handler registration, document why it's there
Updates #cleanup

Signed-off-by: David Anderson <danderson@tailscale.com>
1 year ago
Adrian Dewhurst f75a36f9bc tsweb: add request ID for errors
If an optional request ID generating func is supplied to StdHandler,
then requests that return an error will be logged with a request ID that
is also shown as part of the response.

Updates tailscale/corp#2549

Change-Id: Ic7499706df42f95b6878d44d4aab253e2fc6a69b
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
1 year ago
Andrew Lytvynov 7c04846eac
tsweb: relax CSP for debug handlers (#8649)
Allow inline CSS for debug handlers to make prototyping easier. These
are generally not accessible to the public and the small risk of CSS
injection via user content seems acceptable.

Also allow form submissions on the same domain, instead of banning all
forms. An example of such form is
http://webhooks.corp.ts.net:6359/debug/private-nodes/

Updates #3576

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
1 year ago
Maisem Ali 3304819739 metrics: add histogram support
Add initial histogram support.

Updates tailscale/corp#8641

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Andrew Lytvynov 96d7af3469
cmd/derper,tsweb: consistently add HTTP security headers (#8579)
Add a few helper functions in tsweb to add common security headers to handlers. Use those functions for all non-tailscaled-facing endpoints in derper.

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
1 year ago
Adrian Dewhurst cd4c71c122 tstest: prepare for Clock API changes
This change introduces tstime.NewClock and tstime.ClockOpts as a new way
to construct tstime.Clock. This is a subset of #8464 as a stepping stone
so that we can update our internal code to use the new API before making
the second round of changes.

Updates #8463

Change-Id: Ib26edb60e5355802aeca83ed60e4fdf806c90e27
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
1 year ago
Brad Fitzpatrick 79ee6d6e1e tsweb/varz: use default metrics.LabelMap.Label on serialization
To not break Prometheus if the label is unset.

Updates tailscale/corp#12830

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick eefee6f149 all: use cmpx.Or where it made sense
I left a few out where writing it explicitly was better
for various reasons.

Updates #8296

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
James Tucker 5c38f0979e tsweb/promvarz: fix repeated expvar definition in test
expvar can only be defined once, so running tests with a repeat counter
will fail if the variables are defined inside of the test function.

Observed failure:

```
--- FAIL: TestHandler (0.00s)
panic: Reuse of exported var name: gauge_promvarz_test_expvar
 [recovered]
        panic: Reuse of exported var name: gauge_promvarz_test_expvar

goroutine 9 [running]:
testing.tRunner.func1.2({0x100f267e0, 0x1400026e770})
        /usr/local/go/src/testing/testing.go:1526 +0x1c8
testing.tRunner.func1()
        /usr/local/go/src/testing/testing.go:1529 +0x364
panic({0x100f267e0, 0x1400026e770})
        /usr/local/go/src/runtime/panic.go:884 +0x1f4
log.Panicln({0x140000b8e20?, 0x1a?, 0x1400026e750?})
        /usr/local/go/src/log/log.go:398 +0x60
expvar.Publish({0x100e2b21d, 0x1a}, {0x100fd7a08?, 0x140000232c0})
        /usr/local/go/src/expvar/expvar.go:284 +0xc0
expvar.NewInt(...)
        /usr/local/go/src/expvar/expvar.go:304
tailscale.com/tsweb/promvarz.TestHandler(0x14000082b60)
        /Users/charlotte/ts-src/tailscale/tsweb/promvarz/promvarz_test.go:18 +0x5c
testing.tRunner(0x14000082b60, 0x100fd5858)
        /usr/local/go/src/testing/testing.go:1576 +0x104
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1629 +0x370
FAIL    tailscale.com/tsweb/promvarz    0.149s
```

Fixes #8065
Signed-off-by: James Tucker <james@tailscale.com>
2 years ago
Anton Tolchanov 8546ff98fb tsweb: move varz handler(s) into separate modules
This splits Prometheus metric handlers exposed by tsweb into two
modules:
- `varz.Handler` exposes Prometheus metrics generated by our expvar
  converter;
- `promvarz.Handler` combines our expvar-converted metrics and native
  Prometheus metrics.

By default, tsweb will use the promvarz handler, however users can keep
using only the expvar converter. Specifically, `tailscaled` now uses
`varz.Handler` explicitly, which avoids a dependency on the
(heavyweight) Prometheus client.

Updates https://github.com/tailscale/corp/issues/10205

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Anton Tolchanov c153e6ae2f prober: migrate to Prometheus metric library
This provides an example of using native Prometheus metrics with tsweb.

Prober library seems to be the only user of PrometheusVar, so I am
removing support for it in tsweb.

Updates https://github.com/tailscale/corp/issues/10205

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Anton Tolchanov 11e6247d2a tsweb: expose native Prometheus metrics in /debug/varz
The handler will expose built-in process and Go metrics by default,
which currently duplicate some of the expvar-proxied metrics
(`goroutines` vs `go_goroutines`, `memstats` vs `go_memstats`), but as
long as their names are different, Prometheus server will just scrape
both.

This will change /debug/varz behaviour for most tsweb binaries, but
notably not for control, which configures a `tsweb.VarzHandler`
[explicitly](a5b5d5167f/cmd/tailcontrol/tailcontrol.go (L779))

Updates https://github.com/tailscale/corp/issues/10205

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
2 years ago
Maisem Ali 1a30b2d73f all: use tstest.Replace more
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
David Anderson 8b2ae47c31 version: unexport all vars, turn Short/Long into funcs
The other formerly exported values aren't used outside the package,
so just unexport them.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
Brad Fitzpatrick b1248442c3 all: update to Go 1.20, use strings.CutPrefix/Suffix instead of our fork
Updates #7123
Updates #5309

Change-Id: I90bcd87a2fb85a91834a0dd4be6e03db08438672
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Will Norris 10085063fb util/vizerror: add As function to get wrapped Error
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Will Norris 51e1ab5560 fixup! util/vizerror: add new package for visible errors
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Will Norris 648aa00a28 fixup! util/vizerror: add new package for visible errors
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Will Norris 598ec463bc fixup! util/vizerror: add new package for visible errors
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Will Norris 71029cea2d all: update copyright and license headers
This updates all source files to use a new standard header for copyright
and license declaration.  Notably, copyright no longer includes a date,
and we now use the standard SPDX-License-Identifier header.

This commit was done almost entirely mechanically with perl, and then
some minimal manual fixes.

Updates #6865

Signed-off-by: Will Norris <will@tailscale.com>
2 years ago