Go version
go version go1.23.11 darwin/amd64
Output of go env in your module/workspace:
GO111MODULE='on'
GOARCH='amd64'
GOBIN='/opt/s/w/ir/cache/builder/infra/go/bin'
GOCACHE='/opt/s/w/ir/cache/builder/infra/go/golang/gocache'
GOENV='/Users/chrome-bot/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS='-mod=readonly'
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/opt/s/w/ir/cache/builder/infra/go/golang/modcache'
GONOPROXY='*.googlesource.com,*.git.corp.google.com,google.com'
GONOSUMDB='*.googlesource.com,*.git.corp.google.com,google.com'
GOOS='darwin'
GOPATH='/Users/chrome-bot/go'
GOPRIVATE='*.googlesource.com,*.git.corp.google.com,google.com'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/s/w/ir/cache/builder/infra/go/golang/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/opt/s/w/ir/cache/builder/infra/go/golang/go/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.23.11'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/chrome-bot/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='0'
GOMOD='/opt/s/w/ir/cache/builder/infra/go/src/infra/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/opt/s/w/ir/x/t/go-build3729329483=/tmp/go-build -gno-record-gcc-switches -fno-common'
What did you do?
Just update from go 1.23.10 to go 1.23.11 https://chromium-review.googlesource.com/c/infra/infra/+/6716762
What did you see happen?
test fail on mac
https://ci.chromium.org/ui/p/infra/builders/try/infra-try-mac/b8709864579993950241/overview
signal: segmentation fault
FAIL go.chromium.org/infra/build/bench 0.013s
What did you expect to see?
no test fail
This is go 1.23.10 https://ci.chromium.org/ui/p/infra/builders/try/infra-try-mac/b8709863512656524721/overview
Comment From: gabyhelp
Related Issues
- Test #67906 (closed)
- Test #70447 (closed)
- affected/package: #51201 (closed)
- test #43061 (closed)
- go #49423 (closed)
- cmd/compile: seg fault with generics, channels and closures #51355 (closed)
- go #64466 (closed)
- Go test breaks on flags in go 1.13 #34078 (closed)
- cmd/link: go test -c on darwin/arm64 produces invalid code signature in combination with cgo #63576 (closed)
- bb #71176 (closed)
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)
Comment From: randall77
We're going to need more instructions to reproduce. I tried:
git clone https://chromium.googlesource.com/infra/infra
cd infra/go/src/infra
~/go1.23.10/bin/go test ./build/bench
~/go1.23.11/bin/go test ./build/bench
Both passed on darwin/arm64. I will try again on darwin/amd64, but am I doing the right dance? It isn't easy to tell from your logs.
Comment From: randall77
Same results on darwin/amd64. Both Go 1.23.10 and 1.23.11 pass that package's tests.
Comment From: ukai
I can reproduce it locally.
ukai-macbookpro3:infra ukai$ uname -a
Darwin ukai-macbookpro3.roam.corp.google.com 24.5.0 Darwin Kernel Version 24.5.0: Tue Apr 22 19:53:26 PDT 2025; root:xnu-11417.121.6~2/RELEASE_X86_64 x86_64
ukai-macbookpro3:infra ukai$ GOTOOLCHAIN=go1.23.10 go test -count=1 ./build/bench
ok go.chromium.org/infra/build/bench 1.124s
ukai-macbookpro3:infra ukai$ GOTOOLCHAIN=go1.23.11 go test -count=1 ./build/bench
signal: segmentation fault
FAIL go.chromium.org/infra/build/bench 0.288s
FAIL
Comment From: ukai
attach bench.test generated by GOTOOLCHAIN=go1.23.11 go test -c ./build/bench
ukai-macbookpro3:infra ukai$ GOTOOLCHAIN=go1.23.11 go test -c ./build/bench
ukai-macbookpro3:infra ukai$ ./bench.test
Segmentation fault: 11
Comment From: Jorropo
Would you have a (program counter / instruction pointer) for the segmentation fault ?
Comment From: ukai
ukai-macbookpro3:infra ukai$ lldb ./bench.test
(lldb) target create "./bench.test"
Current executable set to '/Users/ukai/src/infra/infra/go/src/infra/bench.test' (x86_64).
(lldb) run
Process 15997 launched: '/Users/ukai/src/infra/infra/go/src/infra/bench.test' (x86_64)
Process 15997 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x0000000000000000
error: memory read failed for 0x0
Target 0: (bench.test) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000000000000
frame #1: 0x00000000011d5f72 bench.test`github.com/ebitengine/purego/internal/fakecgo.call5.abi0 + 50
(lldb)
?
Comment From: ukai
ah, removing dependency on github.com/shirou/gopsutil/v4/mem fixes the issue...
Comment From: Jorropo
The mov rsp, rbp is the return instruction pointer for frame #1.
The frame #0: 0x0000000000000000 means it is trying to jump a nil pointer,
So the call right above is trying to call into a nil function; usually it would lead to a panic but here I guess it does not because the fakecgo package breaks out the rules of go.
Comment From: dmitshur
CC @golang/runtime.
Comment From: randall77
It seems odd that call5 gets a zero function argument. All its callsites use an obviously nonzero argument (the address of some global).
I would add some print statements to make sure all the addresses being passed to call5 are nonzero.
All the symbols starting at https://github.com/ebitengine/purego/blob/cdb93a81b8a32cc1f9884786e430d19b39c984b1/internal/fakecgo/symbols.go#L149
Just print those addresses out in an init function and check them.
This is the list of changes that went into 1.23.11. https://github.com/golang/go/issues?q=milestone%3AGo1.23.11+label%3ACherryPickApproved Nothing obvious, but the first one is tangentially related to linkname, so maybe?
Comment From: TotallyGamerJet
It was pointed out by @hajimehoshi that just updating psutil fixes the issue
https://github.com/ebitengine/purego/issues/327#issuecomment-3052988819
Comment From: randall77
Sure, but I still think we'd like to understand what changed from 1.23.10 to 1.23.11 to cause this. The worry is that if it happened to one person, it might happen to others. That said, gopsutil is pretty scary unsafe code. I wouldn't be surprised if there's heap clobbering going on and minor, unrelated changes adjust positions of things to cause failures to appear or disappear. Whether that minor change is upgrading Go from 1.23.10 to 1.23.11, or upgrading psutil from v4.24.12 to v4.25.6.
Comment From: randall77
BTW, CGO_ENABLED=0 is the thing I was missing when trying to reproduce. Now reproduction instructions are:
git clone https://chromium.googlesource.com/infra/infra
cd infra/go/src/infra
CGO_ENABLED=0 GOARCH=amd64 ~/go1.23.10/bin/go test ./build/bench
CGO_ENABLED=0 GOARCH=amd64 ~/go1.23.11/bin/go test ./build/bench
(GOARCH=amd64 makes this reproduce even on an arm64 darwin machine.)
Comment From: randall77
This code drops into Go code super early during initialization (called from rt0_go). Nothing seems set up correctly to run any code. X15 isn't zeroed, which Go code requires. Trampolines aren't set up.
This failure is caused because when the code tries to call pthread_self, the trampoline (some shared library thing? not sure) isn't initialized.
In lldb, when we are here: https://github.com/ebitengine/purego/blob/cdb93a81b8a32cc1f9884786e430d19b39c984b1/internal/fakecgo/trampolines_amd64.s#L98
things look ~ok. ebx is initialized with the address of pthead_self. The other arg registers are junk because the compiler attempted to zero them by copying from a junk-containing x15 (but that's not the immediate problem, as pthread_self doesn't read its args). But step one more instruction, and we're now looking at:
frame #0: 0x00000000011d69c0 bench.test`_pthread_self
bench.test`:
-> 0x11d69c0 <+0>: jmp 0x0
0x11d69c5 <+5>: retq
That looks like a trampoline of some sort that never got initialized. On the next step, we fault at 0.
Not sure what changed in 1.23.11. In any case, I wouldn't expect ~any regular Go user to run into this.
Comment From: cherrymui
If it is in rt0_go, could it be cgo_init? We call cgo_init early on in rt0_go without initializing Go ABI. In regular cgo, cgo_init is a C function implemented in runtime/cgo, so that is not a problem. But in fakecgo it is a Go function, which does require Go ABI being initialized. Perhaps fakecgo's x_cgo_init_trampoline should set up Go internal ABI. But that is an internal ABI, which should not be depended in user code. Perhaps fakecgo should arrange it to call to the ABI0 symbol through an ABI wrapper.
cgo_init does call pthread_self https://github.com/ebitengine/purego/blob/cdb93a81b8a32cc1f9884786e430d19b39c984b1/internal/fakecgo/go_darwin_amd64.go#L71 . So that could be it.
Comment From: TotallyGamerJet
Perhaps fakecgo should arrange it to call to the ABI0 symbol through an ABI wrapper.
I'm willing to update it to use whatever is the most stable and "supported" option. I know that term is very loose in this context of unsafe and linkname