Commit Graph

128 Commits (main)

Author SHA1 Message Date
Anton Tolchanov 3c27632ffe tsweb: avoid dashes in Prometheus metric names
Ideally we should strip other invalid characters too, but that would
call for a regexp replacement which increases the number of allocations
and makes `TestVarzHandlerSorting` fail.

Signed-off-by: Anton Tolchanov <anton@tailscale.com>
1 year ago
Brad Fitzpatrick 614a24763b tsweb: sort top-level expvars after removing type prefixes
Fixes #5778

Change-Id: I56c367338fa5686da288cc6545209ef4d6b88549
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 718914b697 tsweb: remove allocs introduced by earlier change
This removes the ~9 allocs added by #5869, while still keeping struct
fields sorted (the previous commit's tests still pass). And add a test
to lock it in that this shouldn't allocate.

Updates #5778

Change-Id: I4c12b9e2a1334adc1ea5aba1777681cb9fc18fbf
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Hasnain Lakhani 8fe04b035c tsweb: sort varz by name after stripping prefix (#5778)
This makes it easier to view prometheus metrics.

Added a test case which demonstrates the new behavior - the test
initially failed as the output was ordered in the same order
as the fields were declared in the struct (i.e. foo_a, bar_a, foo_b,
bar_b). For that reason, I also had to change an existing test case
to sort the fields in the new expected order.

Signed-off-by: Hasnain Lakhani <m.hasnain.lakhani@gmail.com>
2 years ago
Will Norris 62bc1052a2 tsweb: allow HTTPError to unwrap errors
Signed-off-by: Will Norris <will@tailscale.com>
2 years ago
Josh Soref d4811f11a0 all: fix spelling mistakes
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2 years ago
Mihai Parparita c66e15772f tsweb: consider 304s as successful for quiet logging
Static resource handlers will generate lots of 304s, which are
effectively successful responses.

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
Brad Fitzpatrick aa5e494aba tsweb: export go_version in standard expvar vars
For monitoring.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Eng Zer Jun f0347e841f refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16 [1]. This commit
replaces the existing io/ioutil functions with their new definitions in
io and os packages.

Reference: https://golang.org/doc/go1.16#ioutil
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2 years ago
Brad Fitzpatrick 116f55ff66 all: gofmt for Go 1.19
Updates #5210

Change-Id: Ib02cd5e43d0a8db60c1f09755a8ac7b140b670be
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Joe Tsai 57275a4912
tsweb: add HTTPError.Header (#5251)
The Header field allows the server to specify specific headers to set.
Example use case: server returns 429 with the "Retry-After" header set.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Joe Tsai a794963e2f
tsweb: mark AccessLogRecord fields as omitempty (#5250)
If the field is the zero value, then avoid serializing the field.
This reduces verbosity in server logs.

Signed-off-by: Joe Tsai <joetsai@digital-static.net>
2 years ago
Brad Fitzpatrick 6a396731eb all: use various net/netip parse funcs directly
Mechanical change with perl+goimports.

Changed {Must,}Parse{IP,IPPrefix,IPPort} to their netip variants, then
goimports -d .

Finally, removed the net/netaddr wrappers, to prevent future use.

Updates #5162

Change-Id: I59c0e38b5fbca5a935d701645789cddf3d7863ad
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 7eaf5e509f net/netaddr: start migrating to net/netip via new netaddr adapter package
Updates #5162

Change-Id: Id7bdec303b25471f69d542f8ce43805328d56c12
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Denton Gentry 755396d6fe tsweb: add Float expvar support in varz
We make assertions about stringification of 0.5. IEEE floating point and
all reasonable proprietary floating point can exactly represent 0.5.
We don't make assertions about other floating point values, too brittle
in tests.

Signed-off-by: Denton Gentry <dgentry@tailscale.com>
2 years ago
Tom DNetto 32c6823cf5 tsweb: implement interceptor for error page presentation
Updates https://github.com/tailscale/corp/issues/5605

Signed-off-by: Tom DNetto <tom@tailscale.com>
2 years ago
Brad Fitzpatrick 0df3b76c25 tsweb: fix Port80Handler redirect to https with FQDN unset
Fixes the current http://pkgs.tailscale.com/ redirect to https:///
as that server doesn't configure the Port80Handler.FQDN field.

Change-Id: Iff56e6127a46c306ca97738d91b217bcab32a582
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
David Anderson e1c1d47991 tsweb: memoize the string forms of HTTP response codes.
Saves 1-2 allocs per HTTP request after warmup, thanks to
avoiding strconv and fmt.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson c48513b2be tsweb: support recording unabridged HTTP status codes as well.
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
Brad Fitzpatrick 2d1849a7b9 tsweb: remove JSONHandlerFunc
It's unused and we've decided it's not what we want.

Change-Id: I425a0104e8869630b498a0adfd0f455876d6f92b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
David Anderson 7b4960316b tsweb: add PrometheusVar, for vars that want to output varz themselves.
This enables the infrequent use of more complex Prometheus types, such as
timeseries with high/irregular label cardinality, without needing to
discover and implement generic abstracted type like LabelMap for each one.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
Maisem Ali 309c0a13a5 tsweb: add FQDN to Port80Handler to allow HTTPS redirects
When the request comes in say over http://mon, the current
implementation would rewrite it https://mon which causes the cert
validation to fail. This PR keeps the existing behavior intact but also
allows passing in a FQDN to the handler to reroute to the correct
hostname.

Related to https://github.com/tailscale/tailscale/pull/4208#pullrequestreview-913832340

Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
Josh Bleecher Snyder 0868329936 all: use any instead of interface{}
My favorite part of generics.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
2 years ago
Brad Fitzpatrick 61ee72940c all: use Go 1.18's strings.Cut
More remain.

Change-Id: I6ec562cc1f687600758deae1c9d7dbd0d04004cb
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 24c9dbd129 tsweb: fix JSONHandlerFunc regression where HTTP status was lost on gzip
Change-Id: Ia7add6cf7e8b46bb6dd45bd3c0371ea79402fb45
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick df8f02db3f tsweb: add gzip support to JSONHandlerFunc
Change-Id: I337e05f92f744bfc7e9d6fb8e67c87c191ba4da8
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
Brad Fitzpatrick 41fd4eab5c envknob: add new package for all the strconv.ParseBool(os.Getenv(..))
A new package can also later record/report which knobs are checked and
set. It also makes the code cleaner & easier to grep for env knobs.

Change-Id: Id8a123ab7539f1fadbd27e0cbeac79c2e4f09751
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
2 years ago
nicksherron f01ff18b6f all: fix spelling mistakes
Signed-off-by: nicksherron <nsherron90@gmail.com>
3 years ago
Brad Fitzpatrick 722942dd46 tsweb: restore CPU profiling handler
It was accidentally deleted in the earlier 0022c3d2e (#2143) refactor.
Lock it in with a test.

Fixes tailscale/corp#2503

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
slowy07 ac0353e982 fix: typo spelling grammar
Signed-off-by: slowy07 <slowy.arfy@gmail.com>
3 years ago
Brad Fitzpatrick 51976ab3a2 tsweb: add vars for unix process start time and version
To be scraped in the Go expvar JSON format, as a string is involved.

For a future tool to record when processes restarted exactly, and at
what version.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick c6740da624 tsweb: make VarzHandler support untyped expvar.Maps for compatibility
Updates #2635

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 7c7eb8094b tsweb: make VarzHandler support expvar.Funcs returning ints/floats
Updates #2635
3 years ago
Brad Fitzpatrick 5aba620fb9 tsweb: make VarzHandler capable of walking structs with reflect
To be used by control, per linked bug's plan.

Updates #2635

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 3700cf9ea4 tsweb: also support LabelMaps from expvar.Map, without metrics
We want to use tsweb to format Prometheus-style metrics from
our temporary golang.org/x/net/http2 fork, but we don't want http2
to depend on the tailscale.com module to use the concrete type
tailscale.com/metrics.LabelMap. Instead, let a expvar.Map be used
instead of it's annotated sufficiently in its name.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
Brad Fitzpatrick 5f45d8f8e6 tsweb: add VarzHandler tests
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
3 years ago
julianknodt 3377089583 tsweb: add float64 to logged metrics
A previously added metric which was float64 was being ignored in tsweb, because it previously
only accepted int64 and ints. It can be handled in the same way as ints.

Signed-off-by: julianknodt <julianknodt@gmail.com>
3 years ago
David Anderson 0022c3d2e2 tsweb: replace NewMux with a more flexible DebugHandler.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 48c25fa36f tsweb: fold StdHandlerOpts and StdHandler200s with StdHandler.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
David Anderson 72343fbbec tsweb: register expvars once at startup.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Josh Bleecher Snyder 516e8a4838 tsweb: add num_goroutines expvar
Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
3 years ago
David Anderson 692a011b54 net/interfaces: remove IsTailscaleIP, make callers use tsaddr.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
Brad Fitzpatrick 017dcd520f tsweb: export VarzHandler 3 years ago
David Anderson 57cd7738c2 tsweb: add an endpoint to manually trigger a GC.
Signed-off-by: David Anderson <danderson@tailscale.com>
3 years ago
chungdaniel e7ac9a4b90
tsweb: refactor JSONHandler to take status code from error if it is present (#905)
This change is to make JSONHandler error handling intuitive, as before there would be two sources of HTTP status code when HTTPErrors were generated: one as the first return value of the handler function, and one nested inside the HTTPError. Previously, it took the first return value as the status code, and ignored the code inside the HTTPError. Now, it should expect the first return value to be 0 if there is an error, and it takes the status code of the HTTPError to set as the response code.

Signed-off-by: Daniel Chung <daniel@tailscale.com>
4 years ago
Josh Bleecher Snyder 3b46655dbb tsweb: add StatusCodeCounters to HandlerOptions
This lets servers using tsweb register expvars
that will track the number of requests ending
in 200s/300s/400s/500s.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
Josh Bleecher Snyder e98f2c57d6 tsweb: add StdHandlerOpts that accepts an options struct
I'm about to add yet another StdHandler option.
Time to refactor.

Signed-off-by: Josh Bleecher Snyder <josh@tailscale.com>
4 years ago
David Crawshaw 815bf017fc tsweb: when unwrapping HTTPError, record the user-facing message also in the log
There's often some useful piece of information in there not already
repeated in the internal error.

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
4 years ago
David Crawshaw dea3ef0597 tsweb: make JSONHandlerFunc implement ReturnHandler, not http.Handler
This way something is capable of logging errors on the server.

Fixes #766

Signed-off-by: David Crawshaw <crawshaw@tailscale.com>
4 years ago
halulu 1835bb6f85
tsweb: rewrite JSONHandler without using reflect (#684)
Closes #656 #657

Signed-off-by: Zijie Lu <zijie@tailscale.com>
4 years ago
Zijie Lu c9e40abfb8 tsweb: jsonhandler: fix content type
Signed-off-by: Zijie Lu <zijie@tailscale.com>
4 years ago
Zijie Lu d1a30be275 tsweb: JSONHandler: supports HTTPError
Signed-off-by: Zijie Lu <zijie@tailscale.com>
4 years ago
Zijie Lu 5e1ee4be53 tsweb: fix JSONHandler nil response
Signed-off-by: Zijie Lu <zijie@tailscale.com>
4 years ago
halulu 1d2e497d47
tsweb: JSONHandler using reflect (#437)
Updates #395 #437

Signed-off-by: Zijie Lu <zijie@tailscale.com>
4 years ago
David Anderson 0038223632 tstest: rename from testy.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
David Anderson eccae0cd0c tsweb: add ReturnHandlerFunc.
Signed-off-by: David Anderson <danderson@tailscale.com>
4 years ago
Brad Fitzpatrick 7bf436ba30 tsweb: add some Benchmarks
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 64334143a1 tsweb: rename Handler to ReturnHandler
The name's been bugging me for a long time.

I liked neither the overlap between tsweb.Handler and http.Handler,
nor the name "ServeHTTPErr" which sounds like it's an error being
returned, like it's an error handler and not sometimes a happy path.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
David Anderson b446de103a tsweb: fix tests.
Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
David Anderson af3a9dfad6 tsweb: add a StdHandler that doesn't log 200 responses.
Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
Brad Fitzpatrick 2863e49db9 tsweb: don't flush, treat no-op Handler as 200, like Go
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
David Anderson 5d995d9d6b tsweb: add a test case for nil child errors in tsweb.Error.
Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
David Anderson 2c2dff9559 tsweb: don't panic if we get a tsweb.Error with no embedded error.
It's technically weird to return a tsweb.Error with no child err,
but it's a sensible thing to want to do, and we shouldn't panic
if it happens.

Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
David Anderson 12a6626a94 tsweb: use a ResponseWriter wrapper to log more accurately.
Also adds tests, because the logging handler is acquiring a fair
number of branches.

Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
Brad Fitzpatrick 6de37f4cc0 tsweb: move some comments, add a TODO 4 years ago
David Anderson df4636567f tsweb: adjust names and docs of the "handler with errors" functions.
Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
David Anderson 98eceae55e tsweb: add a Handler type.
Handler is like http.Handler, but returns errors. ErrHandler
converts back to an http.Handler, with added error handling
and logging.

Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
Brad Fitzpatrick c706731dc7 tsweb: add copyright header
And fix an unlikely but potential crash.
4 years ago
David Anderson a567c56971 tsweb: pull in tlog data structures from corp repo.
This is a prelude to having logging helpers in tsweb.

Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
Brad Fitzpatrick 5c1e443d34 wgengine/monitor: don't call LinkChange when interfaces look unchanged
Basically, don't trust the OS-level link monitor to only tell you
interesting things. Sanity check it.

Also, move the interfaces package into the net directory now that we
have it.
4 years ago
Brad Fitzpatrick a9cd8f31de tsweb: don't double import expvar; appease staticcheck 4 years ago
Brad Fitzpatrick 890f5dff40 tsweb: export runtime.MemStats var in Prometheus format 4 years ago
Brad Fitzpatrick 61f3fda405 tsweb: let expvar.Ints be gauges too 4 years ago
Brad Fitzpatrick 57de94c7aa tsweb: add /debug/ access via &debugkey + TS_DEBUG_KEY_PATH 4 years ago
David Anderson d580157921 tsweb: quote label values to comply with Prometheus format. 4 years ago
David Anderson f192c05413 metrics: add a LabelMap type for variables with 1 label dimension.
This lets us publish sets of vars that are breakdowns along one
dimension in a format that Prometheus and Grafana natively know
how to do useful things with.

Signed-off-by: David Anderson <dave@natulte.net>
4 years ago
Brad Fitzpatrick e371520cc5 tsweb, derp: add expvar http.Handler for Prometheus's format
And add some opinions.

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago
Brad Fitzpatrick 824f825552 tsweb, cmd/derper: move common web/debug stuff from derper to new tsweb
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
4 years ago