Go version

1.24.2

Output of go env in your module/workspace:

AR='ar'
CC='/usr/bin/gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='/usr/bin/g++'
GCCGO='gccgo'
GO111MODULE='auto'
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/lqw/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/lqw/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2711622146=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/home/lqw/mygit/go1.24.2/src/go.mod'
GOMODCACHE='/home/lqw/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/lqw/go'
GOPRIVATE=''
GOPROXY='https://goproxy.cn,direct'
GOROOT='/home/lqw/mygit/go1.24.2'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/lqw/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/lqw/mygit/go1.24.2/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.24.2'
GOWORK='/home/lqw/mygit/go1.24.2/src/go.work'
PKG_CONFIG='pkg-config'

What did you do?

I import gonum package in cmd/compile/internal/escape package to use the graph interface and export dot easily.

The minimal example is to add "gonum.org/v1/gonum/graph" in import section of cmd/compile/internal/escape/escape.go and add the following code in arbitrary function.

var g graph.Graph
_ = g

Then I execute the following instructions:

cd $(go env GOROOT)/src/cmd
go get gonum.org/v1/gonum/graph
go mod tidy
GOWORK=off go mod vendor # instructed in [README.vendor](https://github.com/golang/go/blob/master/src/README.vendor)

Indeed, gonum package is added in cmd/vendor directory, as the following tree shows:

(base) ➜  vendor git:(closure-go) ✗ pwd
/home/lqw/mygit/go1.24.2/src/cmd/vendor
(base) ➜  vendor git:(closure-go) ✗ tree -L 2
.
├── github.com
│   ├── google
│   └── ianlancetaylor
├── golang.org
│   └── x
├── gonum.org
│   └── v1
├── modules.txt
└── rsc.io
    └── markdown

9 directories, 1 file

What did you see happen?

But when I execute make.bash in src directory, I get the following error. The bootstrap version of go1.24.1 is the prebuilt binary downloaded from https://go.dev.

$ GOROOT_BOOTSTRAP=/home/lqw/golangs/go1.24.1 ./make.bash
Building Go cmd/dist using /home/lqw/golangs/go1.24.1. (go1.24.1 linux/amd64)
Building Go toolchain1 using /home/lqw/golangs/go1.24.1.
../../../../src/cmd/compile/internal/xxx: no required module provides package gonum.org/v1/gonum/graph; to add it:
        go get gonum.org/v1/gonum/graph
...(gonum subpackages)

What did you expect to see?

make.bash successfully rebuilds the toolchain although I import new vendor package like gonum.org/v1/gonum/graph in cmd/compile/interal/escape package.

Comment From: Lslightly

Scenario and Question

My scenario is that I'm learning the compiler but won't publish changes I make in the compiler, at least at the developing stage. So using toolstash restore and go install cmd/compile as mentioned in README.md#juggling compiler version is just enough. This can avoid the use of make.bash to rebuild the toolchain and get the above error.

However, the problem of executing make.bash after importing new package in cmd/compile/internal/escape and then raising no required module error is not solved. It is circumvented using go install cmd/compile. What if I want to import some new packages in cmd/compile?

Debugging Process

The following description is what I see when debugging the above error.

I inspect the vendor package that cmd/compiler/internal uses, only cmd/compiler/internal/ssa/_gen/rulegen.go uses golang.org/x/tools/go/ast/astutil, while other imported packages are all standard libraries. So is compile special when using vendor package?

I think it's limited to import vendor packages to avoid supply chain attack. Right? Or if using vendor package in compile, then it's hard to maintain the dependency among different versions since the API and behavior of vendor package might change?

I inspect the process of cmd/dist/dist bootstrap -a, it will create bootstrap project in pkg/bootstrap/src directory.

To reproduce the process before removing the bootstrap project: 1. set break point in run(base, ShowOutput|CheckExit, cmd...) or exit at that point. 2. run ./make.bash --dist-tool in src directory to build cmd/dist and generate pkg/dist/dist. 3. run or debug pkg/dist/dist.

The boostrap project just copies code from src directory. For go1.24.2, it copies cmd/asm, cmd/cgo, cmd/compile, cmd/internal, cmd/link.

I find that the go.mod of bootstrap project does not add any package in src/go.mod and src/cmd/go.mod. It is:

module bootstrap
go 1.22

So if cmd/asm, cmd/cgo, cmd/compile, cmd/internal, cmd/link import vendor packages, and these vendor packages does not reside in GOROOT_BOOSTRAP/bin/go's src/vendor and src/cmd/vendor directory, the above error occurs naturally.

Possible Solution

Solution: Is it possible to run go mod tidy after creating go.mod?

The code for creating go.mod in bootstrap project:

https://github.com/golang/go/blob/7a38975a48ac735e62b389957bfc898437d628dc/src/cmd/dist/buildtool.go#L168-L170

I think it's natural as most projects do. Besides, a NOTICE can be attached in README.vendor for using vendor packages in cmd directory(Or to be more specific, cmd/asm, cmd/cgo, ..., i.e. the packages used when bootstrapping). The notice is "please use vendor packages that are compatible to the minBootstrapVers go".

But there are also cases that the vendor packages for different boostrap version have some breaking changes. So bootstrap may fail if the behavior of vendor package of bootstrap is different from the one used when developing cmd/compile.

If adding go mod tidy is feasible, I'm happy to create a PR and add notices in README.vendor. If not, please kindly tell me the reason, which is invaluable experience I could learn. Thanks!

Comment From: randall77

Not sure about the rest of your question, but

only cmd/compiler/internal/ssa/_gen/rulegen.go uses golang.org/x/tools/go/ast/astutil, while other imported packages are all standard libraries. So is compile special when using vendor package?

That code is run on a go:generate trigger by a developer. It is not code that is built into the compiler. (Directories like testdata and _gen are not built by default.)