Go version

go version go1.23.11 darwin/amd64

Output of go env in your module/workspace:

GO111MODULE='on'
GOARCH='amd64'
GOBIN='/opt/s/w/ir/cache/builder/infra/go/bin'
GOCACHE='/opt/s/w/ir/cache/builder/infra/go/golang/gocache'
GOENV='/Users/chrome-bot/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS='-mod=readonly'
GOHOSTARCH='amd64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/opt/s/w/ir/cache/builder/infra/go/golang/modcache'
GONOPROXY='*.googlesource.com,*.git.corp.google.com,google.com'
GONOSUMDB='*.googlesource.com,*.git.corp.google.com,google.com'
GOOS='darwin'
GOPATH='/Users/chrome-bot/go'
GOPRIVATE='*.googlesource.com,*.git.corp.google.com,google.com'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/s/w/ir/cache/builder/infra/go/golang/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/opt/s/w/ir/cache/builder/infra/go/golang/go/pkg/tool/darwin_amd64'
GOVCS=''
GOVERSION='go1.23.11'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/Users/chrome-bot/Library/Application Support/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='clang'
CXX='clang++'
CGO_ENABLED='0'
GOMOD='/opt/s/w/ir/cache/builder/infra/go/src/infra/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch x86_64 -m64 -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/opt/s/w/ir/x/t/go-build3729329483=/tmp/go-build -gno-record-gcc-switches -fno-common'

What did you do?

Just update from go 1.23.10 to go 1.23.11 https://chromium-review.googlesource.com/c/infra/infra/+/6716762

What did you see happen?

test fail on mac

https://ci.chromium.org/ui/p/infra/builders/try/infra-try-mac/b8709864579993950241/overview

signal: segmentation fault
FAIL    go.chromium.org/infra/build/bench   0.013s

What did you expect to see?

no test fail

This is go 1.23.10 https://ci.chromium.org/ui/p/infra/builders/try/infra-try-mac/b8709863512656524721/overview

Comment From: gabyhelp

Related Issues

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: randall77

We're going to need more instructions to reproduce. I tried:

git clone https://chromium.googlesource.com/infra/infra
cd infra/go/src/infra
~/go1.23.10/bin/go test ./build/bench
~/go1.23.11/bin/go test ./build/bench

Both passed on darwin/arm64. I will try again on darwin/amd64, but am I doing the right dance? It isn't easy to tell from your logs.

Comment From: randall77

Same results on darwin/amd64. Both Go 1.23.10 and 1.23.11 pass that package's tests.

Comment From: ukai

I can reproduce it locally.

ukai-macbookpro3:infra ukai$ uname -a
Darwin ukai-macbookpro3.roam.corp.google.com 24.5.0 Darwin Kernel Version 24.5.0: Tue Apr 22 19:53:26 PDT 2025; root:xnu-11417.121.6~2/RELEASE_X86_64 x86_64
ukai-macbookpro3:infra ukai$ GOTOOLCHAIN=go1.23.10 go test -count=1 ./build/bench
ok      go.chromium.org/infra/build/bench   1.124s
ukai-macbookpro3:infra ukai$ GOTOOLCHAIN=go1.23.11 go test -count=1 ./build/bench
signal: segmentation fault
FAIL    go.chromium.org/infra/build/bench   0.288s
FAIL

Comment From: ukai

attach bench.test generated by GOTOOLCHAIN=go1.23.11 go test -c ./build/bench

ukai-macbookpro3:infra ukai$ GOTOOLCHAIN=go1.23.11 go test -c ./build/bench
ukai-macbookpro3:infra ukai$ ./bench.test 
Segmentation fault: 11

test.tgz

Comment From: Jorropo

Would you have a (program counter / instruction pointer) for the segmentation fault ?

Comment From: ukai

ukai-macbookpro3:infra ukai$ lldb ./bench.test 
(lldb) target create "./bench.test"
Current executable set to '/Users/ukai/src/infra/infra/go/src/infra/bench.test' (x86_64).
(lldb) run
Process 15997 launched: '/Users/ukai/src/infra/infra/go/src/infra/bench.test' (x86_64)
Process 15997 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
Target 0: (bench.test) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x00000000011d5f72 bench.test`github.com/ebitengine/purego/internal/fakecgo.call5.abi0 + 50
(lldb) 

?

Comment From: ukai

ah, removing dependency on github.com/shirou/gopsutil/v4/mem fixes the issue...

Comment From: Jorropo

Image

The mov rsp, rbp is the return instruction pointer for frame #1. The frame #0: 0x0000000000000000 means it is trying to jump a nil pointer, So the call right above is trying to call into a nil function; usually it would lead to a panic but here I guess it does not because the fakecgo package breaks out the rules of go.

Comment From: dmitshur

CC @golang/runtime.

Comment From: randall77

It seems odd that call5 gets a zero function argument. All its callsites use an obviously nonzero argument (the address of some global).

I would add some print statements to make sure all the addresses being passed to call5 are nonzero.

All the symbols starting at https://github.com/ebitengine/purego/blob/cdb93a81b8a32cc1f9884786e430d19b39c984b1/internal/fakecgo/symbols.go#L149 Just print those addresses out in an init function and check them.

This is the list of changes that went into 1.23.11. https://github.com/golang/go/issues?q=milestone%3AGo1.23.11+label%3ACherryPickApproved Nothing obvious, but the first one is tangentially related to linkname, so maybe?

Comment From: TotallyGamerJet

It was pointed out by @hajimehoshi that just updating psutil fixes the issue

https://github.com/ebitengine/purego/issues/327#issuecomment-3052988819

Comment From: randall77

Sure, but I still think we'd like to understand what changed from 1.23.10 to 1.23.11 to cause this. The worry is that if it happened to one person, it might happen to others. That said, gopsutil is pretty scary unsafe code. I wouldn't be surprised if there's heap clobbering going on and minor, unrelated changes adjust positions of things to cause failures to appear or disappear. Whether that minor change is upgrading Go from 1.23.10 to 1.23.11, or upgrading psutil from v4.24.12 to v4.25.6.

Comment From: randall77

BTW, CGO_ENABLED=0 is the thing I was missing when trying to reproduce. Now reproduction instructions are:

git clone https://chromium.googlesource.com/infra/infra
cd infra/go/src/infra
CGO_ENABLED=0 GOARCH=amd64 ~/go1.23.10/bin/go test ./build/bench
CGO_ENABLED=0 GOARCH=amd64 ~/go1.23.11/bin/go test ./build/bench

(GOARCH=amd64 makes this reproduce even on an arm64 darwin machine.)

Comment From: randall77

This code drops into Go code super early during initialization (called from rt0_go). Nothing seems set up correctly to run any code. X15 isn't zeroed, which Go code requires. Trampolines aren't set up.

This failure is caused because when the code tries to call pthread_self, the trampoline (some shared library thing? not sure) isn't initialized.

In lldb, when we are here: https://github.com/ebitengine/purego/blob/cdb93a81b8a32cc1f9884786e430d19b39c984b1/internal/fakecgo/trampolines_amd64.s#L98

things look ~ok. ebx is initialized with the address of pthead_self. The other arg registers are junk because the compiler attempted to zero them by copying from a junk-containing x15 (but that's not the immediate problem, as pthread_self doesn't read its args). But step one more instruction, and we're now looking at:

    frame #0: 0x00000000011d69c0 bench.test`_pthread_self
bench.test`:
->  0x11d69c0 <+0>: jmp    0x0
    0x11d69c5 <+5>: retq   

That looks like a trampoline of some sort that never got initialized. On the next step, we fault at 0.

Not sure what changed in 1.23.11. In any case, I wouldn't expect ~any regular Go user to run into this.

Comment From: cherrymui

If it is in rt0_go, could it be cgo_init? We call cgo_init early on in rt0_go without initializing Go ABI. In regular cgo, cgo_init is a C function implemented in runtime/cgo, so that is not a problem. But in fakecgo it is a Go function, which does require Go ABI being initialized. Perhaps fakecgo's x_cgo_init_trampoline should set up Go internal ABI. But that is an internal ABI, which should not be depended in user code. Perhaps fakecgo should arrange it to call to the ABI0 symbol through an ABI wrapper.

cgo_init does call pthread_self https://github.com/ebitengine/purego/blob/cdb93a81b8a32cc1f9884786e430d19b39c984b1/internal/fakecgo/go_darwin_amd64.go#L71 . So that could be it.

Comment From: TotallyGamerJet

Perhaps fakecgo should arrange it to call to the ABI0 symbol through an ABI wrapper.

I'm willing to update it to use whatever is the most stable and "supported" option. I know that term is very loose in this context of unsafe and linkname