This problem was originally reported in #74462, but I'm opening a new issue for visibility. In short, an optimization to the token.FileSet struct in 1.25 breaks older versions of the x/tools/internal/tokeninternal package that used unsafe to work around certain performance problems detailed in #73205. Specifically, this line breaks:

// If the size of token.FileSet changes, this will fail to compile.
const delta = int64(unsafe.Sizeof(tokenFileSet{})) - int64(unsafe.Sizeof(token.FileSet{}))
var _ [-delta * delta]int

Below is my initial analysis from #74462, though unfortunately it turns out that I was mistaken: the GetLines function is used indirectly by x/tools/go/gcexportdata.Write, and also contains this unsafe trick.

This is our mistake, sorry. A bit of background:

  • x/tools used unsafe to work around some major performance problems with the token.FileSet datastructure. This fix has since been upstreamed in #73205, so that we no longer need to use unsafe.
  • The tokeninternal package, which contained this unsafe workaround intended only for gopls, had another use in x/tools at v0.19.0. An unrelated function in the package (~not the one that uses unsafe~) was used by the gcimporter package, which is reachable through the public API of x/tools. This unintended edge was also subsequently removed, but not before some projects depended on affected versions.

So this was a latent bug in x/tools, which has since been fixed, but is now being encountered in projects depending on the affected versions. Apologies for the breakage.

Originally posted by @findleyr in #74462

This shouldn't have happened: x/tools should not depend on the internal layout of the token.FileSet struct, and while we need to analyze the impact of this bug, it seems quite bad: as @andydotxyz analyzed, this problem is present in versions of x/tools as recent as 2mo old. Fortunately, among packages in x/tools, go/gcexportdata is infrequently used.

We should investigate potential mitigations. It's gnarly, but here are some potential paths forward:

  1. Revert https://go.dev/cl/675736 for go 1.25.1.
  2. Patch older versions of x/tools (suggested by @jamietanna in #74462). This may be combinatorially complex (there may be many affected versions), and could lead to painful interactions with MVS, but might still be a safe path forward if we retract all the bad versions.
  3. Simply pad the go/token.FileSet struct to be the same size in 1.25 as it was in 1.24, and turn a compile-time error into a (likely unreachable) runtime error. This would have been more viable if not for the reachability of tokeninternal.GetLine through the x/tools/go/gcexportdata API.

Comment From: findleyr

Closing in favor of reopening the original: https://github.com/golang/go/issues/74462#issuecomment-3194549738.