What version of Go are you using (go version)?
$ go version go version go1.21rc2 windows/amd64
Does this issue reproduce with the latest release?
Reproducible with 1.21rc2 but not reproducible with 1.20.5. I'm not sure how to download 1.21rc1 to test that.
What operating system and processor architecture are you using (go env)?
Windows 11 Pro Version 10.0.22621 Build 22621
go env Output
$ go env set GO111MODULE= set GOARCH=amd64 set GOBIN= set GOCACHE=C:\Users\james\AppData\Local\go-build set GOENV=C:\Users\james\AppData\Roaming\go\env set GOEXE=.exe set GOEXPERIMENT= set GOFLAGS= set GOHOSTARCH=amd64 set GOHOSTOS=windows set GOINSECURE= set GOMODCACHE=C:\Users\james\go\pkg\mod set GONOPROXY=gerrit.levenlabs.com,gitlab.com/levenlabs set GONOSUMDB=gerrit.levenlabs.com,gitlab.com/levenlabs set GOOS=windows set GOPATH=C:\Users\james\go; set GOPRIVATE=gerrit.levenlabs.com,gitlab.com/levenlabs set GOPROXY=https://proxy.golang.org,direct set GOROOT=C:\Program Files\Go set GOSUMDB=sum.golang.org set GOTMPDIR= set GOTOOLCHAIN=auto set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64 set GOVCS= set GOVERSION=go1.21rc2 set GCCGO=gccgo set GOAMD64=v1 set AR=ar set CC=gcc set CXX=g++ set CGO_ENABLED=1 set GOMOD=C:\Users\james\Dropbox\aftermath\backend\go-llib\go.mod set GOWORK= set CGO_CFLAGS=-O2 -g set CGO_CPPFLAGS= set CGO_CXXFLAGS=-O2 -g set CGO_FFLAGS=-O2 -g set CGO_LDFLAGS=-O2 -g set PKG_CONFIG=pkg-config set GOGCCFLAGS=-m64 -mthreads -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=C:\Users\james\AppData\Local\Temp\go-build2611928154=/tmp/go-build -gno-record-gcc-switches
I had previously installed tdm-gcc 9.2.0 which causes the error:
gcc -v Output
> gcc -v Using built-in specs. COLLECT_GCC=C:\TDM-GCC-64\bin\gcc.exe COLLECT_LTO_WRAPPER=C:/TDM-GCC-64/bin/../libexec/gcc/x86_64-w64-mingw32/9.2.0/lto-wrapper.exe Target: x86_64-w64-mingw32 Configured with: ../../../src/gcc-git-9.2.0/configure --build=x86_64-w64-mingw32 --enable-targets=all --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-libgomp --enable-lto --enable-graphite --enable-cxx-flags=-DWINPTHREAD_STATIC --disable-build-with-cxx --disable-build-poststage1-with-cxx --enable-libstdcxx-debug --enable-threads=posix --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libstdcxx-threads --enable-libstdcxx-time --with-gnu-ld --disable-werror --disable-nls --disable-win32-registry --enable-large-address-aware --disable-rpath --disable-symvers --prefix=/mingw64tdm --with-local-prefix=/mingw64tdm --with-pkgversion=tdm64-1 --with-bugurl=http://tdm-gcc.tdragon.net/bugs Thread model: posix gcc version 9.2.0 (tdm64-1)
But upon upgrading to tdm-gcc 10.3.0 the error no longer occurs:
gcc -v Output
> gcc -v Using built-in specs. COLLECT_GCC=C:\TDM-GCC-64\bin\gcc.exe COLLECT_LTO_WRAPPER=C:/TDM-GCC-64/bin/../libexec/gcc/x86_64-w64-mingw32/10.3.0/lto-wrapper.exe Target: x86_64-w64-mingw32 Configured with: ../../../src/gcc-git-10.3.0/configure --build=x86_64-w64-mingw32 --enable-targets=all --enable-languages=ada,c,c++,fortran,jit,lto,objc,obj-c++ --enable-libgomp --enable-lto --enable-graphite --enable-cxx-flags=-DWINPTHREAD_STATIC --disable-build-with-cxx --disable-build-poststage1-with-cxx --enable-libstdcxx-debug --enable-threads=posix --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --disable-libstdcxx-pch --enable-libstdcxx-threads --enable-libstdcxx-time=yes --enable-mingw-wildcard --with-gnu-ld --disable-werror --enable-nls --disable-win32-registry --enable-large-address-aware --disable-rpath --disable-symvers --prefix=/mingw64tdm --with-local-prefix=/mingw64tdm --with-pkgversion=tdm64-1 --with-bugurl=https://github.com/jmeubank/tdm-gcc/issues Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 10.3.0 (tdm64-1)
What did you do?
Trying to run any tests with -race and gcc 9.2.0 ends up failing with exit status 0xc0000139.
https://go.dev/play/p/RFjBrSPywGg
> go.exe test ./... -v -race
exit status 0xc0000139
> go.exe test ./... -v
=== RUN TestAdd
--- PASS: TestAdd (0.00s)
PASS
But upon upgrading to gcc 10.3.0:
> go.exe test ./... -v -race
=== RUN TestAdd
--- PASS: TestAdd (0.00s)
PASS
What did you expect to see?
I expected for the race detector not to break between versions without something in the release notes pointing out an expectation of breakage. All I noticed in the release notes for 1.21 that might be related is:
On Windows AMD64, the linker (with help from the compiler) now emits SEH unwinding data by default, which improves the integration of Go applications with Windows debuggers and other tools.
But it's not obvious that this would require an upgraded gcc. It wasn't until after several hours of Googling and debugging that I ended up attempting to update gcc.
What did you see instead?
Instead I saw a bizarre exit status that led me down a deep rabbit hole on Google to try and figure out why my go tests suddenly don't work.
Comment From: jameshartig
After reading through #35006 it seems like the recommendation is to use llvm. Downloading the latest ucrt from https://github.com/mstorsjo/llvm-mingw/releases and putting that in my path seems to also be working.
Comment From: bcmills
Given that the binary works with a newer GCC, this sounds like a GCC bug rather than a Go bug.
That said, if we know of specific compilers that do or do not work with the race detector, or with Go in general, then probably we should add that detail to https://go.dev/wiki/MinimumRequirements and/or https://go.dev/doc/articles/race_detector#Requirements.
(CC @golang/windows, @golang/runtime)
Comment From: qmuntal
GCC 9.2 is still widely used, we should understand what happened before the final go1.22 release, I'll investigate if SEH or the frame pointer changes have anything to do with this issue.
Comment From: qmuntal
Did some archeology and found the culprit: the program fails to load the WakeByAddressSingle, WakeByAddressAll and WaitOnAddress symbols from kernel32.dll. These symbols are required by the new race_windows_amd64.syso library, updated in go1.21 as part of #49761, specifically in CL 420197.
The issue is that the missing symbols are not part of kernel32.dll since Windows 10, they are exported in libsynchronization.a instead. Mingw-w64 fixed this loading issue in v8.0.0 (https://github.com/mingw-w64/mingw-w64/commit/68d0e7d551d2e2a5b8a15bdca3f8d15fc097ed01), but tdm-gcc (which repackage mingw-w64) didn't port that change till v10 (https://github.com/jmeubank/mingw-w64/commit/68d0e7d551d2e2a5b8a15bdca3f8d15fc097ed01). Therefore, tdm-gcc v9 can't be used with go1.21.
That said, if we know of specific compilers that do or do not work with the race detector, or with Go in general, then probably we should add that detail to https://go.dev/wiki/MinimumRequirements and/or https://go.dev/doc/articles/race_detector#Requirements.
This was discussed and rejected in https://github.com/golang/go/issues/35006. This comment made by @thanm gives a good reasoning: https://github.com/golang/go/issues/35006#issuecomment-1344522618.
Comment From: jameshartig
@qmuntal thank you for the in-depth analysis and explanation! Does it make sense to mention this in the release notes at all if GCC 9 is still fairly popular?
Comment From: qmuntal
Does it make sense to mention this in the release notes at all if GCC 9 is still fairly popular?
This is not a problem with GCC or Mingw-w64 9, they are both fine, but with tdm-gcc 9. I don't know how popular is tdm-gcc and if it worth adding this issue to the release notes, as add this kind of things in there.
Comment From: bcmills
This was discussed and rejected in https://github.com/golang/go/issues/35006. This comment made by @thanm gives a good reasoning: https://github.com/golang/go/issues/35006#issuecomment-1344522618.
I agree with that reasoning, but https://go.dev/wiki/MinimumRequirements#cgo is currently pretty unambiguous that “gcc 4.6 or newer is required”, implying that every version higher than that one is supported. I don't think it's accurate at this point.
I think what we really mean is “cmd/go will pass C compiler flags compatible with gcc 4.6 or newer”, which is a much weaker statement. I'm not sure exactly how to update the wiki page to reflect that.
Comment From: gopherbot
Change https://go.dev/cl/508461 mentions this issue: _content/doc/articles/race_detector: document windows C compiler version requirements
Comment From: JAicewizard
I am currently facing this on github CI with gcc 12.2 from mingw64, any tips on debugging this? @qmuntal do you remember how you found out what the issue was?