I propose that starting with the Go 1.18 dev cycle, we require Go 1.16 as the Go bootstrap version (instead of Go 1.4).
When we switched to writing the Go compiler in itself, I wrote two proposals related to the bootstrap process.
The original proposal, https://golang.org/s/go13compiler (December 2013), was about the overall process of converting the compiler to Go. In that proposal, I wrote:
With a Go compiler written in Go, there must be a plan for bootstrapping from scratch. The rule we plan to adopt is that the Go 1.3 compiler must compile using Go 1.2, Go 1.4 must compile using Go 1.3, and so on. Then there is a clear path to generating current binaries: build the Go 1.2 toolchain (written in C), use it to build the Go 1.3 toolchain, and so on. There will be a shell script to do this; it will take CPU time but not human time. The bootstrapping only needs to be done once per machine; the Go 1.x binaries can be kept in a known location and reused each time all.bash is run during the development of Go 1.(x+1).
Obviously, this bootstrapping path scales poorly over time. Before too many releases have gone by, it may make sense to write a back end for the compiler that generates C code. The code need not be efficient or readable, just correct. That C version would be checked in, just as today we check in the y.tab.c file generated by yacc. The bootstrap sequence would invoke gcc on that C code to build a bootstrap compiler, and the bootstrap compiler would be used to build the real compiler. Like in the other scheme, the bootstrap compiler binary can be kept in a known location and reused (not rebuilt) each time all.bash is run.
This was all fairly hypothetical, and I certainly no longer believe it makes any sense to write a C back end for the Go compiler. (In fact, for the windows/arm64 port I did, I now have a working Go toolchain but still don't know what C compiler I'm supposed to use!)
The final proposal, https://golang.org/s/go15bootstrap (January 2015), simplified the process from an iterative one to hard-coding Go 1.4 as the bootstrap toolchain:
To build Go 1.x, for x ≥ 5, it will be necessary to have Go 1.4 (or newer) installed already, in $GOROOT_BOOTSTRAP. The default value of $GOROOT_BOOTSTRAP is $HOME/go1.4. In general we'll keep using Go 1.4 as the bootstrap base version for as long as possible. The toolchain proper (compiler, assemblers, linkers) will need to be buildable with Go 1.4, whether by restricting their feature use to what is in Go 1.4 or by using build tags.
This was an important simplification, especially for people packaging Go for other systems. That decision has served us well.
But it has now been over six years since Go 1.4. Much has happened in the world of Go, and many bugs have been fixed. Many of the systems Go runs on today aren't supported by Go 1.4 (including darwin/arm64 for M1 Macs). Those are using newer toolchains to bootstrap, and the other systems could too. At a higher level, Go is far more mature and widely available now than it was in the Go 1.4 era. There are tons of available binary distributions to use for bootstrapping.
I propose that Go 1.17 be the last version of Go requiring Go 1.4 for bootstrapping, and that Go 1.18 require Go 1.16 for bootstrapping. Go 1.16 would remain the bootstrap version for the next few years at least.
Why not Go 1.15?
- Go 1.16 added
//go:buildsupport. We don't want the compiler to have to keep using+buildeven once the Go world has moved on. (Edit: this is only half true: see https://github.com/golang/go/issues/44505#issuecomment-783611413.)
Why not Go 1.17?
- There's no compelling feature being added in Go 1.17.
- Using Go 1.16 as the new bootstrap toolchain gives packagers the chance to test the changes they'd need to make using Go 1.17, which would still work with Go 1.4.
- Go 1.17 will be the first release to use the new register-based ABI, which will likely have some lingering bugs that take a while to shake out. (See next section.)
Why not Go 1.18?
- Go 1.18 will contain the initial implementation of generics, which is going to require many many new lines of code, and certainly some of them will contain subtle bugs. If we adopt Go 1.18 as the bootstrap toolchain, we will be stuck working around those bugs for many years to come. Better to lock in the last release before all that churn as the version that will need to hold up for the long term.
Why not a quickly rolling version?
As noted above, I think it has served us well to have a fixed version required to build Go, as opposed to an automatically sliding version as originally envisioned. Packagers benefit from not having to update their package-building scripts to provide a different environment to each new Go release.
At the same time, bumping the version forward every five years or so lets us take advantage of newer Go capabilities and ports and to let us retire old compatibility shims (standard library packages like sort are carrying various +build'ed files to keep them building with Go 1.4). Modern C compilers are not written in pre-ANSI C.
What about a slow-rolling version?
That's essentially what this proposal would establish as our practice, although without a specific timeline.
The next obvious entry in the sequence after Go 1.4 and Go 1.16 is Go 1.256, followed by Go 1.65536. (Or perhaps that is not quite the right pattern to establish.)
Using dates instead, assuming we switch to Go 1.16 in Go 1.18 (Feb 2022), it seems reasonable to me to revisit the bootstrap version four years later, which at our current release cycle would mean using Go 1.24 (Feb 2025) for Go 1.26 (Feb 2026).
But this proposal is not about establishing the sequence, which would depend on many other factors.
It is only about picking Go 1.16 as the bootstrap version starting in Go 1.18.
Comment From: rsc
Actually, I confused myself while writing this.
Go 1.16 does not know how to read //go:build lines,
but it does know to reject files that contain //go:build without // +build.
And all future gofmts will know to sync (≠ insert) // +build lines alongside //go:build
So that's enough for the bootstrap toolchain.
Comment From: mdempsky
Being able to assume a recent compiler for bootstrapping would be really nice for not having to remember all the quirks of Go 1.4, or needing to worry about working around long-fixed issues.
FWIW, OpenJDK appears to require the immediately previous release for bootstrapping (see https://openjdk.java.net/groups/build/doc/building.html, "Build JDK Requirements"), and also has a 6-month release cycle (https://en.wikipedia.org/wiki/OpenJDK#OpenJDK_versions), but building OpenJDK takes way longer than running make.bash does. I don't think we need to be nearly as aggressive at uprev'ing Go bootstrap versions, but I think it at least suggests end users could tolerate another uprev before 1.256. :)
Comment From: beoran
This is a good idea, however, perhaps it would make sense to additionaly declare that the bootstrap version of Go is also going to be the long term support or "LTS" version, which keeps receiving backports of bugfixes for 4 years (or whatever the support period for the bootstrap version is going to be). In some organizations and for some developments, having a stable LTS version that keeps receiving bug fixes is extremely valuable.
Comment From: tianon
FWIW, the level of aggressive bootstrap version bumps OpenJDK does are a big hurdle towards having recent OpenJDK packaged for backported distribution releases (and require a lot of extra coordination / "fancy footwork" from maintainers to coordinate or "hack" around).
As a concrete example for Go, this could make adding new architectures more difficult for a distribution until/unless gccgo gets updated to 1.16 (because gccgo is often used as a bootstrap-builder in order to onboard new architectures without having to do something drastic like uploading an old version of Go with the newer version number and "stepping" upwards until it's supported, which is really disruptive for the other architecture too -- the other alternative is cross-building, which has challenges of its own).
IMO this would be easier to understand/stomach if the proposed version were one that's already supported by at least the most recent gccgo release (which I believe in GCC 10 is 1.14?)
Comment From: ianlancetaylor
Just a note that GCC 11 will support Go 1.16.
Comment From: rsc
This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group
Comment From: rsc
Based on the discussion above, this proposal seems like a likely accept. — rsc for the proposal review group
Comment From: rsc
No change in consensus, so accepted. 🎉 This issue now tracks the work of implementing the proposal. — rsc for the proposal review group
Comment From: zephyr
This should be mentioned in the release notes for Go 1.18.
And maybe even in the release notes for Go 1.17, if still possible? (since the removal of support for old macOS versions is usually noted one release note in advance)
Comment From: gopherbot
Change https://golang.org/cl/344330 mentions this issue: cmd/dist: make Go 1.16 the minimal Go bootstrap version
Comment From: gopherbot
Change https://golang.org/cl/344351 mentions this issue: sort: always use internal/reflectlite Swapper and ValueOf functions
Comment From: rsc
We missed implementing this for Go 1.18. The builders are not ready and the release team has too much other work to do.
We will shoot for bumping the minimum bootstrap toolchain in Go 1.19 instead. Given that the bump is pushed forward a cycle, I suggest that we also push the minimum release forward a cycle: Go 1.19 would require Go 1.17, not Go 1.16.
(The main concern with Go 1.17 was the stability of the register ABI, but that has been rock solid.)
Thoughts? Concerns? Thanks.
Comment From: gopherbot
Change https://golang.org/cl/355690 mentions this issue: cmd/link/internal/ld: don't use linkname before 1.12
Comment From: gopherbot
Change https://golang.org/cl/369914 mentions this issue: build: for default bootstrap, use Go 1.17 if present, falling back to Go 1.4
Comment From: fzipp
Building from source no longer works for me after this change:
src % ./all.bash
Building Go cmd/dist using /usr/local/go. (go1.17.4 darwin/arm64)
Building Go toolchain1 using /Users/frederik/go1.4.
go tool dist: FAILED: /Users/frederik/go1.4/bin/go install -gcflags=-l -tags=math_big_pure_go compiler_bootstrap bootstrap/cmd/...: fork/exec /Users/frederik/go1.4/bin/go: no such file or directory
I have Go 1.17.4 installed under /usr/local/go, but it still looks for go1.4 in my home directory.
Comment From: zikaeroh
I'm experiencing the same thing on Arch (where I bootstrap with what's in $PATH, the system install).
Skimming the CL, make.bash no longer exports GOROOT_BOOTRSTRAP, so it's not set when dist is run (which then uses a different logic that only looks for go1.17 and go1.4).
Adding export GOROOT_BOOTSTRAP after the new code that checks $HOME/sdk/go1.17 and $HOME/go1.17 fixes the issue.
Comment From: gopherbot
Change https://golang.org/cl/370138 mentions this issue: Revert "build: for default bootstrap, use Go 1.17 if present, falling back to Go 1.4"
Comment From: prattmic
I am also seeing failures in a Google Cloud Build environment which effectively does apt-get update && apt-get install -y golang git && ./make.bash in an Ubuntu container.
Comment From: fsouza
I'm experiencing the same thing on Arch (where I bootstrap with what's in
$PATH, the system install).Skimming the CL,
make.bashno longer exportsGOROOT_BOOTRSTRAP, so it's not set whendistis run (which then uses a different logic that only looks forgo1.17andgo1.4).Adding
export GOROOT_BOOTSTRAPafter the new code that checks$HOME/sdk/go1.17and$HOME/go1.17fixes the issue.
Yeah I did that on my side too. Perhaps GOROOT_BOOTSTRAP should default to go env GOROOT? (assuming go is on the PATH)
Comment From: zikaeroh
I'm pretty sure it was just an oversight (as the new code "reassigns" the variable to set it up); the behavior you're describing is the intended behavior (see #48155), and what it'd do with the missing export added.
Comment From: zikaeroh
Critically, note that make.bat and make.rc implicitly export GOROOT_BOOTSTRAP to child processes; it's only bash where you have to explicitly do it (and so it was done before).
The odd thing to me is why dist has logic to conjure up a default for GOROOT_BOOTSTRAP at all, given it's just a part of the make process; I'm sure I've overlooked something there, but it means implementing that same fallback logic twice, which is awkward.
Comment From: ianlancetaylor
I sent https://golang.org/cl/370156 to restore the lost export of GOROOT_BOOTSTRAP.
Comment From: rsc
The odd thing to me is why dist has logic to conjure up a default for GOROOT_BOOTSTRAP at all, given it's just a part of the make process; I'm sure I've overlooked something there, but it means implementing that same fallback logic twice, which is awkward.
Dist can be run directly by hand too, in rare circumstances. But yes, the bug was the missing export. Thanks for finding it.
Comment From: gopherbot
Change https://golang.org/cl/370274 mentions this issue: build: for default bootstrap, use Go 1.17 if present, falling back to Go 1.4
Comment From: zikaeroh
I'm not sure whether or not this should be its own issue, but with the new code in make.bash, go in $PATH will be ignored even if it's new enough to satisfy 1.17, which may be surprising for some distro packagers as their builds may now use something that happens to be in their home. Maybe it doesn't matter thanks to the bootstrap process iterating until stability, but I'd probably be confused if I went to run make.bash and it chose that over my preferred install.
Additionally, the checks are for explicitly the directories $HOME/sdk/go1.17 and $HOME/go1.17, but I think the former is unlikely to exist given there are patch releases, where golang.org/dl downloaders put things into places like $HOME/sdk/go1.17.4, as go1.17 explicitly means the first release of 1.17 (or, 1.17.0 in semver).
Comment From: rsc
Retitled as suggested in https://github.com/golang/go/issues/44505#issuecomment-942593422.
Comment From: gopherbot
Change https://go.dev/cl/386774 mentions this issue: doc/go1.18: document Go 1.17 bootstrap and //go:build fix
Comment From: gopherbot
Change https://go.dev/cl/399514 mentions this issue: cmd/asm: use constraint.IsGoBuild and remove TODO
Comment From: mvdan
Are we on track for 1.19? I see https://go-review.googlesource.com/c/go/+/386774/ was merged, but the builders still seem to bootstrap with Go 1.4.x from what I can see.
Comment From: gopherbot
Change https://go.dev/cl/419115 mentions this issue: [release-branch.go1.17] bootstrap.bash: use git describe to get nicer file name
Comment From: rsc
We missed it for Go 1.19. I'm taking a look at whether it is likely for Go 1.20.
Comment From: gopherbot
Change https://go.dev/cl/419082 mentions this issue: cmd/genbootstrap: update for regular use
Comment From: gopherbot
Change https://go.dev/cl/419452 mentions this issue: make.bat, make.rc: show bootstrap toolchain version
Comment From: gopherbot
Change https://go.dev/cl/420215 mentions this issue: cmd/upload: remove special Go1.4 bootstrap support
Comment From: gopherbot
Change https://go.dev/cl/419993 mentions this issue: dashboard: update builders to Go 1.17.12 for bootstrap
Comment From: rsc
I've made good progress on this and am confident it can be done for Go 1.20. I've got about 80% of them converted and the others are failing in odd ways. (For example the bootstrap toolchains I built for OpenBSD don't actually run on OpenBSD!)
Comment From: gopherbot
Change https://go.dev/cl/420800 mentions this issue: cmd/makemac: update for new host name conventions
Comment From: gopherbot
Change https://go.dev/cl/420804 mentions this issue: dashboard: rename host-android-amd64-emu to host-linux-amd64-androidemu
Comment From: gopherbot
Change https://go.dev/cl/421095 mentions this issue: cmd/buildlet: update for new darwin-amd64-{11,12}_0 hosttype names
Comment From: gopherbot
Change https://go.dev/cl/421079 mentions this issue: all: clean up TODO after fixing issue 44505
Comment From: gopherbot
Change https://go.dev/cl/420903 mentions this issue: all: remove pre-Go 1.17 workarounds
Comment From: gopherbot
Change https://go.dev/cl/420902 mentions this issue: cmd/dist: require Go 1.17 for building Go
Comment From: gopherbot
Change https://go.dev/cl/421456 mentions this issue: dashboard: set HostArch for host-solaris-oracle-amd64-oraclerel
Comment From: gopherbot
Change https://go.dev/cl/421634 mentions this issue: all: use io.Seek* instead of deprecated os.SEEK_*
Comment From: 4a6f656c
I've made good progress on this and am confident it can be done for Go 1.20. I've got about 80% of them converted and the others are failing in odd ways. (For example the bootstrap toolchains I built for OpenBSD don't actually run on OpenBSD!)
@rsc for various reasons, some older Go versions will not run on current Go builders (the exact version depends on the architecture in question). This is in particular going to lead to some challenges when I upgrade builders - is there a way that we can override this to use a functional version?
Also, the bootstrap for openbsd/arm is currently broken - I'm not sure how this was built, however it looks like it was with GOARM=5 (rather than GOARM=7), which is tripping some code that needs to be fixed (I'll raise an issue/change for this).
Comment From: gopherbot
Change https://go.dev/cl/421438 mentions this issue: cmd/dist: do not run on ppc64le
Comment From: gopherbot
Change https://go.dev/cl/421774 mentions this issue: build: move openbsd-arm-joelsing to Go 1.19 for bootstrap
Comment From: gopherbot
Change https://go.dev/cl/423354 mentions this issue: build: update comments about Go 1.4 in make.bash, make.bat, make.rc
Comment From: gopherbot
Change https://go.dev/cl/421356 mentions this issue: make.bat: handle spaces in path when determining bootstrap version
Comment From: gopherbot
Change https://go.dev/cl/427114 mentions this issue: cmd/compile/internal/base: use runtime.KeepAlive in MapFile
Comment From: gopherbot
Change https://go.dev/cl/427958 mentions this issue: cmd/dist: simplify exec.Cmd helpers for Go 1.19
Comment From: gopherbot
Change https://go.dev/cl/427957 mentions this issue: cmd/dist: simplify exec.Cmd helpers
Comment From: gopherbot
Change https://go.dev/cl/428218 mentions this issue: cmd/dist: remove go1.4 code path
Comment From: gopherbot
Change https://go.dev/cl/430335 mentions this issue: cmd/link/internal/ld: drop Go 1.12 compatibility on darwin
Comment From: gopherbot
Change https://go.dev/cl/436915 mentions this issue: cmd/cgo, cmd/compile, cmd/link: remove old style build tags
Comment From: gopherbot
Change https://go.dev/cl/435473 mentions this issue: cmd/cgo, cmd/compile, cmd/link: remove old style build tags
Comment From: gopherbot
Change https://go.dev/cl/522197 mentions this issue: cmd/compile/internal/syntax: use strings.LastIndexByte in trailingDigits
Comment From: gopherbot
Change https://go.dev/cl/524335 mentions this issue: debug/elf: use io.Seek*
Comment From: gopherbot
Change https://go.dev/cl/524336 mentions this issue: debug/pe: use io.SeekStart
Comment From: gopherbot
Change https://go.dev/cl/562619 mentions this issue: _content/doc/install: document minumum bootstrap versions
Comment From: gopherbot
Change https://go.dev/cl/609955 mentions this issue: internal/pkgbits: cleanup pre-Go 1.17 workaround