You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
tailscale/cmd/k8s-operator
Tom Proctor 74d4652144
cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user metrics (#14035)
containerboot:

Adds 3 new environment variables for containerboot, `TS_LOCAL_ADDR_PORT` (default
`"${POD_IP}:9002"`), `TS_METRICS_ENABLED` (default `false`), and `TS_DEBUG_ADDR_PORT`
(default `""`), to configure metrics and debug endpoints. In a follow-up PR, the
health check endpoint will be updated to use the `TS_LOCAL_ADDR_PORT` if
`TS_HEALTHCHECK_ADDR_PORT` hasn't been set.

Users previously only had access to internal debug metrics (which are unstable
and not recommended) via passing the `--debug` flag to tailscaled, but can now
set `TS_METRICS_ENABLED=true` to expose the stable metrics documented at
https://tailscale.com/kb/1482/client-metrics at `/metrics` on the addr/port
specified by `TS_LOCAL_ADDR_PORT`.

Users can also now configure a debug endpoint more directly via the
`TS_DEBUG_ADDR_PORT` environment variable. This is not recommended for production
use, but exposes an internal set of debug metrics and pprof endpoints.

operator:

The `ProxyClass` CRD's `.spec.metrics.enable` field now enables serving the
stable user metrics documented at https://tailscale.com/kb/1482/client-metrics
at `/metrics` on the same "metrics" container port that debug metrics were
previously served on. To smooth the transition for anyone relying on the way the
operator previously consumed this field, we also _temporarily_ serve tailscaled's
internal debug metrics on the same `/debug/metrics` path as before, until 1.82.0
when debug metrics will be turned off by default even if `.spec.metrics.enable`
is set. At that point, anyone who wishes to continue using the internal debug
metrics (not recommended) will need to set the new `ProxyClass` field
`.spec.statefulSet.pod.tailscaleContainer.debug.enable`.

Users who wish to opt out of the transitional behaviour, where enabling
`.spec.metrics.enable` also enables debug metrics, can set
`.spec.statefulSet.pod.tailscaleContainer.debug.enable` to false (recommended).

Separately but related, the operator will no longer specify a host port for the
"metrics" container port definition. This caused scheduling conflicts when k8s
needs to schedule more than one proxy per node, and was not necessary for allowing
the pod's port to be exposed to prometheus scrapers.

Updates #11292

---------

Co-authored-by: Kristoffer Dalby <kristoffer@tailscale.com>
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>
6 days ago
..
deploy cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user metrics (#14035) 6 days ago
generate cmd/k8s-operator,k8s-operator: add ProxyGroup CRD (#13591) 2 months ago
connector.go cmd/k8s-operator,k8s-operator,kube/kubetypes: add an option to configure app connector via Connector spec (#13950) 2 weeks ago
connector_test.go cmd/k8s-operator,k8s-operator,kube/kubetypes: add an option to configure app connector via Connector spec (#13950) 2 weeks ago
depaware.txt derp/derphttp: don't link websockets other than on GOOS=js 3 weeks ago
dnsrecords.go cmd/k8s-operator: fix DNS reconciler for dual-stack clusters (#13057) 4 months ago
dnsrecords_test.go cmd/k8s-operator: fix DNS reconciler for dual-stack clusters (#13057) 4 months ago
egress-eps.go cmd/k8s-operator: fix Pod IP selection (#13743) 2 months ago
egress-eps_test.go cmd/k8s-operator: fix Pod IP selection (#13743) 2 months ago
egress-services-readiness.go cmd/k8s-operator,k8s-operator/apis: set a readiness condition on egress Services for ProxyGroup (#13746) 2 months ago
egress-services-readiness_test.go cmd/k8s-operator,k8s-operator/apis: set a readiness condition on egress Services for ProxyGroup (#13746) 2 months ago
egress-services.go cmd/k8s-operator,k8s-operator/apis: set a readiness condition on egress Services for ProxyGroup (#13746) 2 months ago
egress-services_test.go cmd/k8s-operator,k8s-operator: create ConfigMap for egress services + small fixes for egress services (#13715) 2 months ago
ingress.go cmd/k8s-operator,k8s-operator: use default ProxyClass if set for ProxyGroup (#13720) 2 months ago
ingress_test.go cmd/k8s-operator,k8s-operator: use default ProxyClass if set for ProxyGroup (#13720) 2 months ago
nameserver.go kube,cmd/{k8s-operator,containerboot},envknob,ipn/store/kubestore,*/depaware.txt: rename packages (#13418) 3 months ago
nameserver_test.go cmd/k8s-operator,k8s-operator,kube: Add TSRecorder CRD + controller (#13299) 3 months ago
operator.go cmd/k8s-operator: allow to optionally configure tailscaled port (#14005) 3 weeks ago
operator_test.go cmd/{k8s-operator,containerboot},k8s-operator: remove support for proxies below capver 95. (#13986) 2 weeks ago
proxy.go kube,cmd/{k8s-operator,containerboot},envknob,ipn/store/kubestore,*/depaware.txt: rename packages (#13418) 3 months ago
proxy_test.go cmd/k8s-operator,ssh/tailssh,tsnet: optionally record 'kubectl exec' sessions via Kubernetes operator's API server proxy (#12274) 5 months ago
proxyclass.go cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user metrics (#14035) 6 days ago
proxyclass_test.go cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user metrics (#14035) 6 days ago
proxygroup.go cmd/k8s-operator: restart ProxyGroup pods less (#14045) 2 weeks ago
proxygroup_specs.go kube/{kubeapi,kubeclient},ipn/store/kubestore,cmd/{containerboot,k8s-operator}: emit kube store Events (#14112) 1 week ago
proxygroup_test.go cmd/k8s-operator: restart ProxyGroup pods less (#14045) 2 weeks ago
sts.go cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user metrics (#14035) 6 days ago
sts_test.go cmd/{containerboot,k8s-operator},k8s-operator: new options to expose user metrics (#14035) 6 days ago
svc.go cmd/k8s-operator: validate that tailscale.com/tailnet-ip annotation value is a valid IP 1 month ago
testutils_test.go kube/{kubeapi,kubeclient},ipn/store/kubestore,cmd/{containerboot,k8s-operator}: emit kube store Events (#14112) 1 week ago
tsrecorder.go cmd/{containerboot,k8s-operator},k8s-operator,kube: add ProxyGroup controller (#13684) 2 months ago
tsrecorder_specs.go cmd/k8s-operator,kube/kubeclient,docs/k8s: update rbac to emit events + small fixes (#14164) 1 week ago
tsrecorder_specs_test.go cmd/k8s-operator,k8s-operator,kube: Add TSRecorder CRD + controller (#13299) 3 months ago
tsrecorder_test.go cmd/{containerboot,k8s-operator},k8s-operator,kube: add ProxyGroup controller (#13684) 2 months ago