Golang crypto/tls: segfault when calling tlsrsakex.IncNonDefault()

Go version

go version go1.22.0 linux/amd64

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN='/home/mike/.local/bin'
GOCACHE='/home/mike/.cache/go-build'
GOENV='/home/mike/.config/go/env'
GOEXE=''
GOEXPERIMENT='boringcrypto'
GOFLAGS='-trimpath -tags=opa_no_oci'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/mike/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/mike/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/mike/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.linux-amd64'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/mike/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.0.linux-amd64/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/home/mike/projects/project/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1264452044=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Tried to connect to a postgres DB over TLS using jackc/pgx (see stack trace).

What did you see happen?

A panic when crypto/tls calls internal/godebug.(*Setting).IncNonDefault:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x552252]
goroutine 1 [running]:
internal/godebug.(*Setting).IncNonDefault(0x32c3a60?)
    internal/godebug/godebug.go:102 +0x12
crypto/tls.(*clientHandshakeState).pickCipherSuite(0xc00103f450)
    crypto/tls/handshake_client.go:530 +0xbd
crypto/tls.(*clientHandshakeState).processServerHello(0xc00103f450)
    crypto/tls/handshake_client.go:755 +0x25
crypto/tls.(*clientHandshakeState).handshake(0xc00103f450)
    crypto/tls/handshake_client.go:442 +0x35
crypto/tls.(*Conn).clientHandshake(0xc000694e08, {0x475baf8, 0xc00054b1d0})
    crypto/tls/handshake_client.go:274 +0x685
crypto/tls.(*Conn).handshakeContext(0xc000694e08, {0x475b8c8, 0x6d0e820})
    crypto/tls/conn.go:1553 +0x3cb
crypto/tls.(*Conn).HandshakeContext(...)
    crypto/tls/conn.go:1493
crypto/tls.(*Conn).Handshake(...)
    crypto/tls/conn.go:1477
crypto/tls.(*Conn).Write(0x30d31c0?, {0xc001002e40, 0x21, 0x40})
    crypto/tls/conn.go:1194 +0xe5
github.com/jackc/pgx/v5/pgproto3.(*Frontend).Flush(0xc0007ca008)
    github.com/jackc/pgx/v5@v5.5.3/pgproto3/frontend.go:88 +0x3d
github.com/jackc/pgx/v5/pgconn.(*PgConn).flushWithPotentialWriteReadDeadlock(0xc001230008)
    github.com/jackc/pgx/v5@v5.5.3/pgconn/pgconn.go:1818 +0x73
github.com/jackc/pgx/v5/pgconn.connect({0x475bb68, 0xc000020070}, 0xc0007ab680, 0xc000b2c160, 0x0)
    github.com/jackc/pgx/v5@v5.5.3/pgconn/pgconn.go:337 +0xbd3
github.com/jackc/pgx/v5/pgconn.ConnectConfig({0x475b8c8, 0x6d0e820}, 0xc0007ab680)
    github.com/jackc/pgx/v5@v5.5.3/pgconn/pgconn.go:176 +0x4f3
github.com/jackc/pgx/v5.connect({0x475b8c8?, 0x6d0e820?}, 0xc0007ab680)
    github.com/jackc/pgx/v5@v5.5.3/conn.go:257 +0x34b
github.com/jackc/pgx/v5.ConnectConfig({0x475b8c8, 0x6d0e820}, 0xc0007ab560)
    github.com/jackc/pgx/v5@v5.5.3/conn.go:141 +0xf0
github.com/jackc/pgx/v5/stdlib.(*driverConnector).Connect(0xc000fd6b28, {0x475b8c8, 0x6d0e820})
    github.com/jackc/pgx/v5@v5.5.3/stdlib/sql.go:365 +0xcd
database/sql.(*DB).conn(0xc000c7d790, {0x475b8c8, 0x6d0e820}, 0x1)
    database/sql/sql.go:1415 +0x71e
database/sql.(*DB).PingContext.func1(0xc8?)
    database/sql/sql.go:883 +0x3a
database/sql.(*DB).retry(0xc001087288?, 0xc001087258)
    database/sql/sql.go:1566 +0x42
database/sql.(*DB).PingContext(0xc000c7d790, {0x475b8c8, 0x6d0e820})
    database/sql/sql.go:882 +0x89
database/sql.(*DB).Ping(...)
    database/sql/sql.go:900
...

What did you expect to see?

Expected no segfault/panic. Looking at crypto/tls, it's my suspicion that, due to having goboring enabled and thus needing FIPS, tlsrsakex.Value() has not been called when tlsrsakex.IncNonDefault() is called. This could probably be triggered by having non default .CipherSuites set as well. Perhaps the underlying issue is that RSA ciphers aren't filtered out of fipsCipherSuites() in the first place, which is what allows the branch with IncNonDefault() to even be taken.

Comment From: seankhliao

cc @golang/security

Comment From: FiloSottile

The good news is that the IncNonDefault call is gated by hs.c.config.CipherSuites == nil, so it's not reachable by having non default CipherSuites.

Definitely an oversight in Go+BoringCrypto mode.

Comment From: FiloSottile

@gopherbot please open a Go 1.22 backport issue. This is a severe regression in Go+BoringCrypto mode.

Comment From: gopherbot

Backport issue(s) opened: #65994 (for 1.22).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

Comment From: michaelbeaumont

@FiloSottile Not sure who to ping on this, would it be enough to just solve the segfault?

Comment From: gopherbot

Change https://go.dev/cl/582315 mentions this issue: crypto/tls: don't call tlsrsakex.IncNonDefault with FIPS

Comment From: gopherbot

Change https://go.dev/cl/586755 mentions this issue: crypto/tls: ensure GODEBUGs are initialized

Comment From: lyoung-confluent

FYI for anyone scrolling to the bottom of this the cherry-pick/backport of this patch into a 1.22 release is being tracked in #65994

Comment From: gopherbot

Change https://go.dev/cl/593395 mentions this issue: [release-branch.go1.22] crypto/tls: don't call tlsrsakex.IncNonDefault with FIPS