Commit Graph

142 Commits (004dded0a87d713b57d8421495a5f0df17137816)

Author SHA1 Message Date
Paul Scott 1bf7ed0348
tsweb: add QuietLogging option (#12838)
Allows the use of tsweb.LogHandler exclusively for callbacks describing the
handler HTTP requests.

Fixes #12837

Signed-off-by: Paul Scott <paul@tailscale.com>
4 months ago
Paul Scott 855da47777
tsweb: Add MiddlewareStack func to apply lists of Middleware (#12907)
Fixes #12909

Signed-off-by: Paul Scott <paul@tailscale.com>
4 months ago
Paul Scott ba7f2d129e
tsweb: log all cancellations as 499s (#12894)
Updates #12141

Signed-off-by: Paul Scott <paul@tailscale.com>
4 months ago
Paul Scott b2c522ce95 tsweb: log cancelled requests as 499
Fixes #12860

Signed-off-by: Paul Scott <paul@tailscale.com>
4 months ago
Paul Scott d97cddd876 tsweb: swallow panics
With this change, the error handling and request logging are all done in defers
after calling inner.ServeHTTP. This ensures that any recovered values which we
want to re-panic with retain a useful stacktrace.  However, we now only
re-panic from errorHandler when there's no outside logHandler. Which if you're
using StdHandler there always is. We prefer this to ensure that we are able to
write a 500 Internal Server Error to the client. If a panic hits http.Server
then the response is not sent back.

Updates #12784

Signed-off-by: Paul Scott <paul@tailscale.com>
4 months ago
Paul Scott 646990a7d0 tsweb: log once per request
StdHandler/retHandler would previously emit one log line for each request.
If there were multiple StdHandler in the chain, there would be one log line
per instance of retHandler.

With this change, only the outermost StdHandler/logHandler actually logs the
request or invokes OnStart or OnCompletion callbacks. The error-rendering part
of retHandler lives on in errorHandler, and errorHandler passes those errors up
the stack to logHandler through a callback that logHandler places in the
request.Context().

Updates tailscale/corp#19999

Signed-off-by: Paul Scott <paul@tailscale.com>
5 months ago
Paul Scott 014bf25c0a tsweb: fix TestStdHandler_panic flake
Fixes #12816

Signed-off-by: Paul Scott <paul@tailscale.com>
5 months ago
Paul Scott fec41e4904 tsweb: add stack trace to panic error msg
Updates #12784

Signed-off-by: Paul Scott <paul@tailscale.com>
5 months ago
Anton Tolchanov 787ead835f tsweb: accept a function to call before request handling
To complement the existing `onCompletion` callback, which is called
after request handler.

Updates tailscale/corp#17075

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
5 months ago
Jordan Whited 65888d95c9
derp/xdp,cmd/xdpderper: initial skeleton (#12390)
This commit introduces a userspace program for managing an experimental
eBPF XDP STUN server program. derp/xdp contains the eBPF pseudo-C along
with a Go pkg for loading it and exporting its metrics.
cmd/xdpderper is a package main user of derp/xdp.

Updates tailscale/corp#20689

Signed-off-by: Jordan Whited <jordan@tailscale.com>
6 months ago
Kristoffer Dalby 95f266f1ce tsweb: add optional on completion callback func
Updates corp#17075

Co-Authored-By: Anton Tolchanov <anton@tailscale.com>
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
6 months ago
Marwan Sulaiman 7e357e1636 tsweb: rename AccessLogRecord's When to Time
This change makes our access log record more consistent with the
new log/tslog package formatting of "time". Note that we can
change slog itself to call "time" "when" but we're chosing
to make this breaking change to be consistent with the std lib's
defaults.

Updates tailscale/corp#17071

Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
6 months ago
Will Norris 80decd83c1 tsweb: remove redundant bumpStartIfNeeded func
Updates #12001

Signed-off-by: Will Norris <will@tailscale.com>
7 months ago
Will Norris 46980c9664 tsweb: ensure in-flight requests are always marked as finished
The inflight request tracker only starts recording a new bucket after
the first non-error request. Unfortunately, it's written in such a way
that ONLY successful requests are ever marked as being finished. Once a
bucket has had at least one successful request and begun to be tracked,
all subsequent error cases are never marked finished and always appear
as in-flight.

This change ensures that if a request is recorded has having been
started, we also mark it as finished at the end.

Updates tailscale/corp#19767

Signed-off-by: Will Norris <will@tailscale.com>
7 months ago
Brad Fitzpatrick 7c1d6e35a5 all: use Go 1.22 range-over-int
Updates #11058

Change-Id: I35e7ef9b90e83cac04ca93fd964ad00ed5b48430
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
James Tucker 8fa3026614 tsweb: switch to fastuuid for request ID generation
Request ID generation appears prominently in some services cumulative
allocation rate, and while this does not eradicate this issue (the API
still makes UUID objects), it does improve the overhead of this API and
reduce the amount of garbage that it produces.

Updates tailscale/corp#18266
Updates tailscale/corp#19054

Signed-off-by: James Tucker <james@tailscale.com>
8 months ago
Maisem Ali 21b32b467e tsweb: handle panics in retHandler
We would have incomplete stats and missing logs in cases
of panics.

Updates tailscale/corp#18687

Signed-off-by: Maisem Ali <maisem@tailscale.com>
8 months ago
James Tucker 9eaa56df93 tsweb: update doc on BucketedStatsOptions.Finish to match behavior
I originally came to update this to match the documented behavior, but
the code is deliberately avoiding this behavior currently, making it
hard to decide how to update this. For now just align the documentation
to the behavior.

Updates #cleanup

Signed-off-by: James Tucker <james@tailscale.com>
8 months ago
Brad Fitzpatrick b0941b79d6 tsweb: make BucketedStats not track 400s, 404s, etc
Updates tailscale/corp#18687

Change-Id: I142ccb1301ec4201c70350799ff03222bce96668
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Brad Fitzpatrick 354cac74a9 tsweb/varz: add charset=utf-8 to varz handler
Some of our labels contain UTF-8 and get mojibaked in the browser
right now.

Updates tailscale/corp#18687

Change-Id: I6069cffd6cc8813df415f06bb308bc2fc3ab65c4
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Brad Fitzpatrick 55baf9474f metrics, tsweb/varz: add multi-label map metrics
Updates tailscale/corp#18640

Change-Id: Ia9ae25956038e9d3266ea165537ac6f02485b74c
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Brad Fitzpatrick 4992aca6ec tsweb/varz: flesh out munging of expvar keys into valid Prometheus metrics
From a problem we hit with how badger registers expvars; it broke
trunkd's exported metrics.

Updates tailscale/corp#1297

Change-Id: I42e1552e25f734c6f521b6e993d57a82849464b2
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
8 months ago
Andrew Dunham 8f27d519bb tsweb: add String method to tsweb.RequestID
In case we want to change the format to something opaque later.

Updates tailscale/corp#2549

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: Ie2eac8b885b694be607e9d5101d24b650026d89c
9 months ago
Chris Palmer 13853e7f29
tsweb: add more test cases for TestCleanRedirectURL (#11331)
Updates #cleanup

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
9 months ago
Andrew Lytvynov c7474431f1
tsweb: allow empty redirect URL in CleanRedirectURL (#11295)
Updates #cleanup

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
9 months ago
Andrew Dunham 23e9447871 tsweb: expose function to generate request IDs
For use in corp.

Updates tailscale/corp#2549

Signed-off-by: Andrew Dunham <andrew@du.nham.ca>
Change-Id: I71debae1ce9ae48cf69cc44c2ab5c443fc3b2005
9 months ago
Patrick O'Doherty 256ecd0e8f
Revert "tsweb: update ServeMux matching to 1.22.0 syntax (#11090)" (#11125)
This reverts commit 30c9189ed3.

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
10 months ago
Patrick O'Doherty 30c9189ed3
tsweb: update ServeMux matching to 1.22.0 syntax (#11090)
* tsweb: update ServeMux matching to 1.22.0 syntax

Updates #cleanup

Go 1.22.0 introduced the ability to use more expressive routing patterns
that include HTTP method when constructing ServeMux entries.
Applications that attempted to use these patterns in combination with
the old `tsweb.Debugger` would experience a panic as Go would not permit
the use of matching rules with mixed level of specificity. We now
specify the method for each `/debug` handler to prevent
incompatibilities.

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
10 months ago
Patrick O'Doherty 7c52b27daf
Revert "tsweb: update ServeMux matching to 1.22.0 syntax (#11087)" (#11089)
This reverts commit 291f91d164.

Updates #cleanup

This PR needs additional changes to the registration of child handlers under /debug

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
10 months ago
Patrick O'Doherty 291f91d164
tsweb: update ServeMux matching to 1.22.0 syntax (#11087)
Updates #cleanup

Go 1.22.0 introduced the ability to use more expressive routing patterns
that include HTTP method when constructing ServeMux entries.
Applications that attempted to use these patterns in combination with
the old `tsweb.Debugger` would experience a panic as Go would not permit
the use of matching rules with mixed level of specificity.

Signed-off-by: Patrick O'Doherty <patrick@tailscale.com>
10 months ago
Brad Fitzpatrick 2bd3c1474b util/cmpx: delete now that we're using Go 1.22
Updates #11058

Change-Id: I09dea8e86f03ec148b715efca339eab8b1f0f644
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
10 months ago
Tom DNetto 653721541c tsweb: normalize passkey identities in bucketed stats
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
Tom DNetto 8d6d9d28ba tsweb: normalize common StableID's in bucketed stats, export as LabelMap
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
Tom DNetto af931dcccd tsweb: replace domains/emails in paths when bucketing stats
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
Tom DNetto 36efc50817 tsweb: implementing bucketed statistics for started/finished counts
Signed-off-by: Tom DNetto <tom@tailscale.com>
Updates: corp#17075
10 months ago
David Anderson ae79b2e784 tsweb: add a helper to validate redirect URLs
We issue redirects in a few different places, it's time to have
a common helper to do target validation.

Updates tailscale/corp#16875

Signed-off-by: David Anderson <danderson@tailscale.com>
10 months ago
Joe Tsai c25968e1c5
all: make use of ctxkey everywhere (#10846)
Also perform minor cleanups on the ctxkey package itself.
Provide guidance on when to use ctxkey.Key[T] over ctxkey.New.
Also, allow for interface kinds because the value wrapping trick
also happens to fix edge cases with interfaces in Go.

Updates #cleanup

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
11 months ago
Chris Palmer 1f1ab74250
tsweb: use object-src instead of plugin-types (#10719)
plugin-types is deprecated, and setting object-src: 'none' is best
practice. This should result in no functional change.

Fixes #10718

Signed-off-by: Chris Palmer <cpalmer@tailscale.com>
11 months ago
Marwan Sulaiman b819f66eb1 tsweb: propagate RequestID via context and entire request
The recent addition of RequestID was only populated if the
HTTP Request had returned an error. This meant that the underlying
handler has no access to this request id and any logs it may have
emitted were impossible to correlate to that request id. Therefore,
this PR adds a middleware to generate request ids and pass them
through the request context. The tsweb.StdHandler automatically
populates this request id if the middleware is being used. Finally,
inner handlers can use the context to retrieve that same request id
and use it so that all logs and events can be correlated.

Updates #2549

Signed-off-by: Marwan Sulaiman <marwan@tailscale.com>
1 year ago
Maisem Ali 5ee349e075 tsweb/varz: fix exporting histograms
Updates tailscale/corp#8641

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Dave Anderson 7b18ed293b
tsweb: check for key-based debug access before XFF check (#9093)
Fly apps all set X-Forwarded-For, which breaks debug access even
with a preshared key otherwise.

Updates tailscale/corp#3601

Signed-off-by: David Anderson <danderson@tailscale.com>
1 year ago
David Anderson 5cfa85e604 tsweb: clean up pprof handler registration, document why it's there
Updates #cleanup

Signed-off-by: David Anderson <danderson@tailscale.com>
1 year ago
Adrian Dewhurst f75a36f9bc tsweb: add request ID for errors
If an optional request ID generating func is supplied to StdHandler,
then requests that return an error will be logged with a request ID that
is also shown as part of the response.

Updates tailscale/corp#2549

Change-Id: Ic7499706df42f95b6878d44d4aab253e2fc6a69b
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
1 year ago
Andrew Lytvynov 7c04846eac
tsweb: relax CSP for debug handlers (#8649)
Allow inline CSS for debug handlers to make prototyping easier. These
are generally not accessible to the public and the small risk of CSS
injection via user content seems acceptable.

Also allow form submissions on the same domain, instead of banning all
forms. An example of such form is
http://webhooks.corp.ts.net:6359/debug/private-nodes/

Updates #3576

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
1 year ago
Maisem Ali 3304819739 metrics: add histogram support
Add initial histogram support.

Updates tailscale/corp#8641

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Andrew Lytvynov 96d7af3469
cmd/derper,tsweb: consistently add HTTP security headers (#8579)
Add a few helper functions in tsweb to add common security headers to handlers. Use those functions for all non-tailscaled-facing endpoints in derper.

Signed-off-by: Andrew Lytvynov <awly@tailscale.com>
1 year ago
Adrian Dewhurst cd4c71c122 tstest: prepare for Clock API changes
This change introduces tstime.NewClock and tstime.ClockOpts as a new way
to construct tstime.Clock. This is a subset of #8464 as a stepping stone
so that we can update our internal code to use the new API before making
the second round of changes.

Updates #8463

Change-Id: Ib26edb60e5355802aeca83ed60e4fdf806c90e27
Signed-off-by: Adrian Dewhurst <adrian@tailscale.com>
1 year ago
Brad Fitzpatrick 79ee6d6e1e tsweb/varz: use default metrics.LabelMap.Label on serialization
To not break Prometheus if the label is unset.

Updates tailscale/corp#12830

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick eefee6f149 all: use cmpx.Or where it made sense
I left a few out where writing it explicitly was better
for various reasons.

Updates #8296

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
James Tucker 5c38f0979e tsweb/promvarz: fix repeated expvar definition in test
expvar can only be defined once, so running tests with a repeat counter
will fail if the variables are defined inside of the test function.

Observed failure:

```
--- FAIL: TestHandler (0.00s)
panic: Reuse of exported var name: gauge_promvarz_test_expvar
 [recovered]
        panic: Reuse of exported var name: gauge_promvarz_test_expvar

goroutine 9 [running]:
testing.tRunner.func1.2({0x100f267e0, 0x1400026e770})
        /usr/local/go/src/testing/testing.go:1526 +0x1c8
testing.tRunner.func1()
        /usr/local/go/src/testing/testing.go:1529 +0x364
panic({0x100f267e0, 0x1400026e770})
        /usr/local/go/src/runtime/panic.go:884 +0x1f4
log.Panicln({0x140000b8e20?, 0x1a?, 0x1400026e750?})
        /usr/local/go/src/log/log.go:398 +0x60
expvar.Publish({0x100e2b21d, 0x1a}, {0x100fd7a08?, 0x140000232c0})
        /usr/local/go/src/expvar/expvar.go:284 +0xc0
expvar.NewInt(...)
        /usr/local/go/src/expvar/expvar.go:304
tailscale.com/tsweb/promvarz.TestHandler(0x14000082b60)
        /Users/charlotte/ts-src/tailscale/tsweb/promvarz/promvarz_test.go:18 +0x5c
testing.tRunner(0x14000082b60, 0x100fd5858)
        /usr/local/go/src/testing/testing.go:1576 +0x104
created by testing.(*T).Run
        /usr/local/go/src/testing/testing.go:1629 +0x370
FAIL    tailscale.com/tsweb/promvarz    0.149s
```

Fixes #8065
Signed-off-by: James Tucker <james@tailscale.com>
2 years ago