Commit Graph

36 Commits (67f3b2a525f58e44351f2b34ea1600b3d82c850c)

Author SHA1 Message Date
Percy Wegmann 17501ea31a ci: report test coverage to coveralls.io
This records test coverage for the amd64 no race tests and uploads the
results to coveralls.io.

Updates #cleanup

Signed-off-by: Ox Cart <ox.to.a.cart@gmail.com>
1 year ago
Will Norris 42abf13843 .github: run tests on all PRs, regardless of branch name
The branch name selector "*" doesn't match branches with a "/" in their
name. The vast majority of our PRs are against the main (or previously,
master) branch anyway, so this will have minimal impact. But in the rare
cases that we want to open a PR against a branch with a "/" in the name,
tests should still run.

```
gh pr list --limit 9999 --state all --json baseRefName | \
  jq -cs '.[] | group_by(.baseRefName) |
    map({ base: .[0].baseRefName, count: map(.baseRefName) | length}) |
    sort_by(-.count) | .[]'

{"base":"main","count":4593}
{"base":"master","count":226}
{"base":"release-branch/1.48","count":4}
{"base":"josh-and-adrian-io_uring","count":3}
{"base":"release-branch/1.30","count":3}
{"base":"release-branch/1.32","count":3}
{"base":"release-branch/1.20","count":2}
{"base":"release-branch/1.26","count":2}
{"base":"release-branch/1.34","count":2}
{"base":"release-branch/1.38","count":2}
{"base":"Aadi/speedtest-tailscaled","count":1}
{"base":"josh/io_uring","count":1}
{"base":"maisem/hi","count":1}
{"base":"rel-144","count":1}
{"base":"release-branch/1.18","count":1}
{"base":"release-branch/1.2","count":1}
{"base":"release-branch/1.22","count":1}
{"base":"release-branch/1.24","count":1}
{"base":"release-branch/1.4","count":1}
{"base":"release-branch/1.46","count":1}
{"base":"release-branch/1.8","count":1}
{"base":"web-client-main","count":1}
```

Updates #cleanup

Signed-off-by: Will Norris <will@tailscale.com>
1 year ago
Brad Fitzpatrick 1fc3573446 ipn/{conffile,ipnlocal}: start booting tailscaled from a config file w/ auth key
Updates #1412

Change-Id: Icd880035a31df59797b8379f4af19da5c4c453e2
Co-authored-by: Maisem Ali <maisem@tailscale.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick 4dec0c6eb9 tstest, tstest/integration, github/workflows: shard integration tests
Over four jobs for now.

Updates #cleanup

Change-Id: Ic2b1a739a454916893945a3f9efc480d6fcbd70b
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Maisem Ali e6ab7d3c14 cmd/testwrapper: parse args better
Previously we were just smushing together args and not trying
to parse the values at all. This resulted in the args to testwrapper
being limited and confusing.

This makes it so that testwrapper parses flags in the exact format as `go test`
command and passes them down in the provided order. It uses tesing.Init to
register flags that `go test` understands, however those are not the only
flags understood by `go test` (such as `-exec`) so we register these separately.

Updates tailscale/corp#14975

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Brad Fitzpatrick 56ebcd1ed4 .github/workflows: break up race builder a bit more
Move the compilation of everything to its own job too, separate
from test execution.

Updates #7894

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick 18e2936d25 github/workflows: move race tests to their own job
They're slow. Make them their own job that can run in parallel.

Also, only run them in race mode. No need to run them on 386
or non-race amd64.

Updates #7894

Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Brad Fitzpatrick c363b9055d tstest/integration: add tests for tun mode (requiring root)
Updates #7894

Change-Id: Iff0b07b21ae28c712dd665b12918fa28d6f601d0
Co-authored-by: Maisem Ali <maisem@tailscale.com>
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Maisem Ali 3280c81c95 .github,cmd/gitops-pusher: update to checkout@v4
checkout@v3 is broken:
actions/checkout#1448

Updates #cleanup

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Brad Fitzpatrick 346445acdd .github/workflows: only run bench all on packages with benchmarks
Drops time by several minutes.

Also, on top of that: skip building variant CLIs on the race builder
(29s), and getting qemu (15s).

Updates #9182

Change-Id: I979e02ab8c0daeebf5200459c9e4458a1f62f728
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Maisem Ali 1f12b3aedc .github: do not use testwrapper for benchmarks
Updates #9182

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Brad Fitzpatrick 98a5116434 all: adjust some build tags for plan9
I'm not saying it works, but it compiles.

Updates #5794

Change-Id: I2f3c99732e67fe57a05edb25b758d083417f083e
Signed-off-by: Brad Fitzpatrick <bradfitz@tailscale.com>
1 year ago
Maisem Ali 21e32b23f7 .github: use testwrapper on windows
Updates #cleanup

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Maisem Ali 8e840489ed cmd/testwrapper: only retry flaky failed tests
Redo the testwrapper to track and only retry flaky tests instead
of retrying the entire pkg. It also fails early if a non-flaky test fails.

This also makes it so that the go test caches are used.

Fixes #7975

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
Maisem Ali c11af12a49 .github: actually run tests in CI
Updates #cleanup

Signed-off-by: Maisem Ali <maisem@tailscale.com>
1 year ago
James Tucker 8032b966a1 .github/workflows: add recency bias to action cache keys
The action cache restore process either matches the restore key pattern
exactly, or uses a matching prefix with the most recent date.

If the restore key is an exact match, then no updates are uploaded, but
if we've just computed tests executions for more recent code then we
will likely want to use those results in future runs.

Appending run_id to the cache key will give us an always new key, and
then we will be restore a recently uploaded cache that is more likely
has a higher overlap with the code being tested.

Updates #7975

Signed-off-by: James Tucker <james@tailscale.com>
2 years ago
James Tucker 43819309e1 .github/workflows: split tests and benchmarks for caching
Benchmark flags prevent test caching, so benchmarks are now executed
independently of tests.

Fixes #7975

Signed-off-by: James Tucker <james@tailscale.com>
2 years ago
dependabot[bot] 5acc7c4b1e .github: Bump ruby/action-slack from 3.0.0 to 3.2.1
Bumps [ruby/action-slack](https://github.com/ruby/action-slack) from 3.0.0 to 3.2.1.
- [Release notes](https://github.com/ruby/action-slack/releases)
- [Commits](https://github.com/ruby/action-slack/compare/v3.0.0...v3.2.1)

---
updated-dependencies:
- dependency-name: ruby/action-slack
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
dependabot[bot] 28cb1221ba .github: Bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2 years ago
James Tucker b7f51a1468 .github/workflows: add artifact caching and remove double build on race
Go artifact caching will help provided that the cache remains small
enough - we can reuse the strategy from the Windows build where we only
cache and pull the zips, but let go(1) do the many-file unpacking as it
does so faster.

The race matrix was building once without race, then running all the
tests with race, so change the matrix to incldue a `buildflags`
parameter and use that both in the build and test steps.

Updates #cleanup

Signed-off-by: James Tucker <james@tailscale.com>
2 years ago
James Tucker 8dec1a8724 .github/workflows: reenable Windows CI, disable broken tests
We accidentally switched to ./tool/go in
4022796484 which resulted in no longer
running Windows builds, as this is attempting to run a bash script.

I was unable to quickly fix the various tests that have regressed, so
instead I've added skips referencing #7876, which we need to back and
fix.

Updates #7262
Updates #7876

Signed-off-by: James Tucker <james@tailscale.com>
2 years ago
Mihai Parparita 782ccb5655 .github/workflows: run one set of tests with the tailscale_go build tag
We use it to gate code that depends on custom Go toolchain, but it's
currently only passed in the corp runners. Add a set on OSS so that we
can catch regressions earlier.

To specifically test sockstats this required adding a build tag to
explicitly enable them -- they're normally on for iOS, macOS and Android
only, and we don't run tests on those platforms normally.

Signed-off-by: Mihai Parparita <mihai@tailscale.com>
2 years ago
David Anderson b0eba129e6 .github/workflows: add a pass/fail verdict job to the test workflow
Github requires explicitly listing every single job within a workflow
that is required for status checks, instead of letting you list entire
workflows. This is ludicrous, and apparently this nonsense is the
workaround.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 0ab6a7e7f5 .github/workflows: try to make the merge queue actually run CI
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 5a2fa3aa95 .github/workflows: add armv5 and armv7 cross tests
armv5 because that's what we ship to most downstreams right now,
armv7 becuase that's what we want to ship more of.

Fixes https://github.com/tailscale/tailscale/issues/7269

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 093139fafd .github/workflows: fix non-collapsing CI status in PRs
CI status doesn't collapse into "everything OK" if a job gets
skipped. Instead, always run the job, but skip its only step in PRs.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 306c8a713c .github/workflows: run CI and CodeQL in the merge queue
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 149de5e6d6 build_dist.sh: use cmd/mkversion to get version data
Replaces the former shell goop, which was a shell reimplementation
of a subset of version/mkversion.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
Maisem Ali 4a99481a11 .github/workflows: set TS_FUZZ_CURRENTLY_BROKEN to false
Signed-off-by: Maisem Ali <maisem@tailscale.com>
2 years ago
David Anderson 2eb25686d7 .github/workflows: simplify build-only go test invocation
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 253333b8a3 .github/workflows: support disabling fuzz testing safely
OSS-Fuzz doesn't update their version of Go as quickly as we do, so
we sometimes end up with OSS-Fuzz being unable to build our code for
a few weeks. We don't want CI to be red for that entire time, but
we also don't want to forget to reenable fuzzing when OSS-Fuzz does
start working again.

This change makes two configurations worthy of a CI pass:
 - Fuzzing works, and we expected it to work. This is a normal
   happy state.
 - Fuzzing didn't compile, and we expected it to not compile. This
   is the "OSS-Fuzz temporarily broken" state.

If fuzzing is unexpectedly broken, or unexpectedly not broken, that's
a CI failure because we need to either address a fuzz finding, or
update TS_FUZZ_CURRENTLY_BROKEN to reflect the state of OSS-Fuzz.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 5e186f9fbf .github/workflows: pin version of Windows we run on
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 471053a054 .github/workflows: pin version of Ubuntu we run on
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 2a43fa4421 .github/workflows: use variant=race instead of race=true for race test
Github's matrix runner formats the race variant as '(amd64, true)' if we
use race=true. So, change the way the variable is defined so that it says
'(amd64, race)' even if that makes the if statements a bit more complex.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 9fc3d00c17 .github/workflows: add back forgotten android CI job
Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago
David Anderson 4022796484 .github/workflows: unify and matrixify all our CI steps
Instead of having a dozen files that contribute CI steps with
inconsistent configs, this one file lists out everything that,
for us, constitutes "a CI run". It also enables the slack
notification webhook to notify us exactly once on a mass breakage,
rather than once for every sub-job that fails.

Signed-off-by: David Anderson <danderson@tailscale.com>
2 years ago