golangbuild, our LUCI Executable ("luciexe") command intentionally limited scope during the LUCI prototype stage: it left out some known test coverage that we wish to have and is provided by the current build system (plus some more we'd like to add, such as #34352), to be done after the prototype.

This is the tracking issue to address the remaining gaps in test coverage and get it to a point where it's viable to switch over to relying on it as the canonical source of Go project testing. This applies for the main Go repo and the golang.org/x repos, and most of the remaining work is in the latter category (e.g., running tests for nested modules, etc.).

CC @golang/release, @mknyszek.

Comment From: gopherbot

Change https://go.dev/cl/508276 mentions this issue: main.star: turn on no-network check in x/build

Comment From: dmitshur

By now I think all of the shortcuts we've taken during the prototyping stage that have reduced test coverage have been addressed, such that we can rely on test coverage provided by golangbuild not to drop compared to what we have today with x/build's cmd/coordinator.

There are some remaining nice-to-haves (such as test result streaming, keep-going in main repo), finishing checks (e.g., no-network check still needs to be turned up for main repo), some adjacent issues (such as #37827 that are more relevant to builder configuration policy), and some lower-priority exotic edge cases (empty repo, top-level module not having go.mod file, etc.). They can all be better tracked in other individual issues, so closing this as done.

Done:

  • [x] test all Go packages in nested modules, not just top-level one
  • [x] keep going for nested modules in golang.org/x repos (but not yet main Go repo)
  • [x] run subrepo tests from outside their repositories (equivalent to x/build/cmd/coordinator's go.dev/issue/34352)
  • [x] switch to "./..." import path pattern for testing packages in a module (equivalent to x/build/cmd/coordinator's go.dev/issue/51455)
  • [x] don't use predictable base name for module checkout dir (e.g., see go.dev/cl/506297)
  • [x] scale x-repo tests by GO_TEST_TIMEOUT_SCALE (equivalent to x/build/cmd/coordinator's go.dev/issue/56968) (intentionally decided not to do it yet; will reconsider in the future when running into a concrete need)
  • Update on 2024-02-21: out of band discussion and issue 65845 demonstrate that LUCI builders need this too