Go version

go version go1.25.0 linux/arm64

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT='nospinbitmutex'
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build767931946=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/go/src/xxxx.xxxx.xxxx/y/z/tools/abc/go.mod'
GOMODCACHE='/go/pkg/mod'
GONOPROXY='none'
GONOSUMDB='xxxx.xxxx.xxxx'
GOOS='linux'
GOPATH='/go'
GOPRIVATE='xxxx.xxxx.xxxx'
GOPROXY='xxxx.xxxx.xxxx|proxy.golang.org|direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='go1.25.0'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

/workspace # go version go version go1.25.0 linux/arm64 /workspace # export GOEXPERIMENT=nospinbitmutex /workspace # go version go: unknown GOEXPERIMENT spinbitmutex

What did you see happen?

go: unknown GOEXPERIMENT spinbitmutex

What did you expect to see?

Should be fine, as the 1.24 release notes mention the new GOEXPERIMENT=nospinbitmutex. In 1.25, there are no release notes that mention the removal of this experiment. However, in the goexperiment standard library, it has been removed - comparing 1.25 to 1.24. We are using this flag in our production builds for some services due to changes in latency behavior since the mutex implementation changes in 1.24.

Comment From: gabyhelp

Related Issues

Related Code Changes

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: ALTree

Looks like the GOEXPERIMENT spinbitmutex flag was removed in 7a177114df3e26f4362378e907a869c4fbbf38b7 but the change is indeed not mentioned in the 1.25 release notes.

due to changes in latency behavior since the mutex implementation changes in 1.24

Did you open an issue for this?

Comment From: itsalvinchris

I haven't raised any issues yet, but based on my observations and benchmarks, the problem is most likely related to tail latency. Our HTTP service is highly sensitive to the p99 maximum latency (p95 is simillar when using nospinbit versus spinbit). The delays primarily occur during network calls, such as those to Redis, MySQL, or external HTTP services.

Comment From: itsalvinchris

I conducted the benchmark myself on an arm64 system using an AWS c7g.4xlarge instance, which has 16 cores and 32 GB of memory. The results showed that overall performance is similar when using nospinbit versus spinbit. However, while the average latency is comparable, the variance in latency is noticeably higher.

Comment From: cherrymui

Generally, GOEXPERIMENTs are removed without notice, when they have been enabled by default for a release or longer without major issues. So we don't need to document the removal on the release notes. However, we probably should document GOEXPERIMENT better in general.

Perhaps you want to file a new issue for the performance regression with spinbitmutex, with more details about your workload and how the issue could be reproduced. Thanks.