NOTE -- please read
If you see lines starting with ld: warning but not other outputs, they are just warnings, not errors. You should still get a working binary, if there is no other output. Please only report if you see a link error, or a broken binary, or a warning that is not mentioned in the "warnings" section below. Thanks.
See https://github.com/golang/go/issues/61229#issuecomment-1988965927 for more notes about malformed LC_DYSYMTAB warning. Thanks.
In Xcode 15 beta it comes with Apple's new linker, ld-prime. As of beta 3 it is enabled by default. I did some experiments and found some issues for Go (cgo) with the new Apple linker. Some of the issues are reported in #61190. I'm filing a new issue to track all related faulures.
warnings
Apparently the new linker is more picky. It issues some warnings that the old linker (ld64) doesn't. This includes
ld: warning: -bind_at_load is deprecated on macOS
ld: warning: search path '/usr/local/lib' not found
ld: warning: ignoring duplicate library '-lm'
ld: warning: '.../go.o' has malformed LC_DYSYMTAB, expected 92 undefined symbols to start at index 15983, found 102 undefined symbol
These are just warnings and don't affect the correctness of the build. I'm preparing CLs for fix/work around them.
offset/addend being dropped
Mostly for c-archive, c-shared, and plugin build mode, for some relocations it seems the offset/addend being dropped and the relocation is resolved to the beginning of the section. This may cause failures like SIGILL. E.g.
% go test cmd/cgo/internal/testcarchive
--- FAIL: TestInstall (1.10s)
carchive_test.go:487: [go tool cgo -objdir /var/folders/tf/nsc4z0p112vc_drrjn299ds80000gn/T/carchive_test3584575482/_obj472038228 -exportheader p.h p/p.go]
carchive_test.go:489: [go install -buildmode=c-archive ./libgo]
carchive_test.go:489: [clang -fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/tf/nsc4z0p112vc_drrjn299ds80000gn/T/go-build2775077316=/tmp/go-build -gno-record-gcc-switches -fno-common -I /var/folders/tf/nsc4z0p112vc_drrjn299ds80000gn/T/carchive_test3584575482/pkg/darwin_arm64_shared/testcarchive -o ./testp1 main.c main_unix.c /var/folders/tf/nsc4z0p112vc_drrjn299ds80000gn/T/carchive_test3584575482/pkg/darwin_arm64_shared/testcarchive/libgo.a]
carchive_test.go:489:
carchive_test.go:489: signal: illegal instruction
...
60694 is the same failure mode.
debug info issues
The new linker doesn't generate stab symbols for our functions, causing debug info being lost. Failure look like
--- FAIL: TestDWARF (0.00s)
--- FAIL: TestDWARF/testprogcgo (3.85s)
dwarf_test.go:171: ErrUnknownPC
FAIL
FAIL cmd/link 8.822s
There is also issues with combining DWARF into executable/shared object. Currently it is affecting c-shared build mode. ld-prime rejects the shared object if we add a DWARF segment to it.
non-PIE build doesn't work (Intel Mac only)
The new linker silently ignores the -no_pie flag and generates a PIE binary, which won't work because we assumed some addresses are not relocated. CL https://go.dev/cl/461697 changes the default to PIE and CL https://go.dev/cl/511355 contains a workaround if we are asked to build a non-PIE binary.
fixes/workarounds
CL https://golang.org/cl/505415 and the stack, as well as other CLs linked in this issue, contain tentative fixes and workarounds that address some of the issues. I'll also work with Apple to have them addressed on their side.
I'm hoping o have them addressed before Apple's new linker is finally released.
Comment From: ismail
I also have an ld assertion failure (reported to Apple as FB12550006):
❯ go install -ldflags "-s -w" github.com/rclone/rclone@master
# github.com/rclone/rclone
/opt/homebrew/Cellar/go/1.20.5/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: warning: -bind_at_load is deprecated on macOS
ld: warning: '/private/var/folders/68/1pjj14514sl3vj7lbt69lst40000gn/T/go-link-876362130/go.o' has malformed LC_DYSYMTAB, expected 152 undefined symbols to start at index 168240, found 170 undefined symbols starting at index 135
0 0x104a50380 __assert_rtn + 72
1 0x104a099bc ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 9552
2 0x184981950 _dispatch_client_callout2 + 20
3 0x184994ba0 _dispatch_apply_invoke + 176
4 0x184981910 _dispatch_client_callout + 20
5 0x1849933cc _dispatch_root_queue_drain + 864
6 0x184993a04 _dispatch_worker_thread2 + 156
7 0x184b2b0d8 _pthread_wqthread + 228
ld: Assertion failed: (addr + content.size() <= sectionEndAddr), function writeContentWithoutLinkEdit_block_invoke, file Layout.cpp, line 5689.
clang: error: linker command failed with exit code 1 (use -v to see invocation)
❯ go version
go version go1.20.5 darwin/arm64
❯ clang -v
Apple clang version 15.0.0 (clang-1500.0.34.3)
Target: arm64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Comment From: cherrymui
@ismail Thanks for the report. The CL stack mentioned above (https://golang.org/cl/505415 and dependencies) also avoids the assertion failure.
Comment From: gopherbot
Change https://go.dev/cl/503935 mentions this issue: cmd/link: always use symbol-targeted relocations on Mach-O
Comment From: gopherbot
Change https://go.dev/cl/503538 mentions this issue: cmd/link: fix up more zero-sized local symbols on darwin dynamic linking
Comment From: gopherbot
Change https://go.dev/cl/502616 mentions this issue: cmd/link: use symbol-targeted relocation for initializers on Mach-O
Comment From: gopherbot
Change https://go.dev/cl/502617 mentions this issue: cmd/link: don't generate DYSYMTAB when external linking on Mach-O
Comment From: gopherbot
Change https://go.dev/cl/503539 mentions this issue: cmd/link: sort HOSTOBJ and UNDEFEXT symbols with undefined symbols in Mach-O symbol table
Comment From: gopherbot
Change https://go.dev/cl/505415 mentions this issue: cmd/link: work around issues with ld-prime
Comment From: gopherbot
Change https://go.dev/cl/508696 mentions this issue: cmd/link: suppress some warnings for ld-prime
Comment From: gopherbot
Change https://go.dev/cl/508697 mentions this issue: cmd/cgo/internal/test: don't pass -lm on darwin
Comment From: cherrymui
Another failure mode: (AMD64 Mac only, not ARM64) as of Xcode 15 beta 4 (dyld-1009.6 BUILD 20:09:19 Jul 7 2023) non-PIE binary linked with ld-prime crashes at startup.
runtime: pcHeader: magic= 0xfffffff1 pad1= 0 pad2= 0 minLC= 1 ptrSize= 8 pcHeader.textStart= 0x0 text= 0x10b865000 pluginpath=
fatal error: invalid function symbol table
runtime: panic before malloc heap initialized
runtime stack:
runtime.throw({0x10b8d04de?, 0x7ff7b469e7a0?})
/Users/cherryyz/src/go/src/runtime/panic.go:1077 +0x5c fp=0x7ff7b469e420 sp=0x7ff7b469e3f0 pc=0x10b88ffdc
runtime.moduledataverify1(0x7ff7b469e560?)
/Users/cherryyz/src/go/src/runtime/symtab.go:533 +0x816 fp=0x7ff7b469e540 sp=0x7ff7b469e420 pc=0x10b8aa376
runtime.moduledataverify(...)
/Users/cherryyz/src/go/src/runtime/symtab.go:519
runtime.schedinit()
/Users/cherryyz/src/go/src/runtime/proc.go:724 +0x34 fp=0x7ff7b469e580 sp=0x7ff7b469e540 pc=0x10b8939b4
runtime.rt0_go()
/Users/cherryyz/src/go/src/runtime/asm_amd64.s:349 +0x11e fp=0x7ff7b469e588 sp=0x7ff7b469e580 pc=0x10b8b859e
And the CLs above are not enough to fix it. I'm looking into it. Possibly related to the crash in #61190, not investigated yet.
(PIE binaries are fine. Also fine on ARM64 as it is always PIE.)
Comment From: piusalfred
I am onMacOS Sonoma beta M1,Xcode 15 beta. I use go.120. When I try to install gopls this is what I get
``` go install golang.org/x/tools/cmd/goimports@latest
golang.org/x/tools/cmd/goimports
ld: warning: '/private/var/folders/jc/lq1rvy214tq4643flbh887xr0000gn/T/go-link-1689581912/go.o' has malformed LC_DYSYMTAB, expected 75 undefined symbols to start at index 6656, found 84 undefined symbols starting at index 53
**Comment From: gopherbot**
Change https://go.dev/cl/461697 mentions this issue: `cmd/go: default to PIE linking on darwin/amd64`
**Comment From: gopherbot**
Change https://go.dev/cl/511355 mentions this issue: `cmd/link: force ld64 when building non-PIE binary on macOS`
**Comment From: cherrymui**
@piusalfred that is just a warning, not an error. You should still get a binary. Does the binary run? If not, see #61190, and post on that issue if you have any update. Thanks!
**Comment From: cherrymui**
The non-PIE binary crash (https://github.com/golang/go/issues/61229#issuecomment-1636351613) is due to ld-prime silently ignores -no_pie flag and generates a PIE binary instead. But that doesn't work because in exe buildmode we assumed the binary is not relocated and so generated non-relocated addresses, which would not be correct if the binary is actually relocated.
The CLs above handle it: 1. switch to PIE by default, 2. force ld64 in exe buildmode.
(#61190 is not this. Still need to investigate.)
**Comment From: cobo-louis**
➜ node1 git:(master) go version
go version go1.20.4 darwin/arm64
➜ node1 git:(master) clang -v
Homebrew clang version 16.0.6
Target: x86_64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /usr/local/opt/llvm/bin
ld: warning: '/private/var/folders/7n/lt8klzz95mj0kn2fx631dt740000gp/T/go-link-717398196/go.o' has malformed LC_DYSYMTAB, expected 129 undefined symbols to start at index 33979, found 210 undefined symbols starting at index 75
**Comment From: cherrymui**
@cobo-louis that is just a warning, not an error. You should still get a binary, if there is no other output. Please report if there is an actual link error or the binary doesn't work as expected. Thanks.
**Comment From: cobo-louis**
>
/usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
ld: warning: '/private/var/folders/7n/lt8klzz95mj0kn2fx631dt740000gp/T/go-link-2991864128/go.o' has malformed LC_DYSYMTAB, expected 125 undefined symbols to start at index 56533, found 206 undefined symbols starting at index 142
0 0x100ccee8a __assert_rtn + 64
1 0x100c7693c ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 10316
2 0x7ff802dd6066 _dispatch_client_callout2 + 8
3 0x7ff802de918f _dispatch_apply_invoke_and_wait + 213
4 0x7ff802de8692 _dispatch_apply_with_attr_f + 1207
5 0x7ff802de8847 dispatch_apply + 45
6 0x100c76cf9 void mapReduce<ld::Atom const*, mach_o::Error>(std::__1::span<ld::Atom const*, 18446744073709551615ul>, unsigned long, void (unsigned long, mach_o::Error&, std::__1::span<ld::Atom const*, 18446744073709551615ul>) block_pointer, void (std::__1::span<mach_o::Error, 18446744073709551615ul>) block_pointer) + 745
7 0x100c7e218 ld::LayoutExecutable::writeToFile(char const*) + 22792
8 0x100c1d686 main + 12838
ld: Assertion failed: (0 && "Kind::arm64_adrp_ldr missing extra info"), function applyFixup, file Fixup.cpp, line 743.
clang-16: error: linker command failed with exit code 1 (use -v to see invocation)
**Comment From: cherrymui**
@cobo-louis could you share the program you are building and the steps to reproduce the failure? Then we can report it to Apple. This is a bug in Apple's new linker. Thanks.
**Comment From: gopherbot**
Change https://go.dev/cl/513757 mentions this issue: `[release-branch.go1.21] cmd/link: don't generate DYSYMTAB when external linking on Mach-O`
**Comment From: gopherbot**
Change https://go.dev/cl/513758 mentions this issue: `[release-branch.go1.21 cmd/cgo/internal/test: don't pass -lm on darwin`
**Comment From: lukasmalkmus**
@cherrymui Hope that helps.
`go.mod`:
```go.mod
module github.com/lukasmalkmus/linkerbug
go 1.19
require (
github.com/golangci/golangci-lint v1.53.3
)
tools.go:
//go:build tools
package bug
import (
_ "github.com/golangci/golangci-lint/cmd/golangci-lint"
)
go run github.com/golangci/golangci-lint/cmd/golangci-lint run:
# github.com/golangci/golangci-lint/cmd/golangci-lint
/opt/homebrew/Cellar/go/1.20.6/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: warning: -bind_at_load is deprecated on macOS
ld: warning: '/private/var/folders/4h/swcdv5xd7p53_33xlbfvzygc0000gn/T/go-link-77169989/go.o' has malformed LC_DYSYMTAB, expected 138 undefined symbols to start at index 81807, found 150 undefined symbols starting at index 112
0 0x1001c0130 __assert_rtn + 72
1 0x1001794f4 ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 9772
2 0x186179950 _dispatch_client_callout2 + 20
3 0x18618cba0 _dispatch_apply_invoke + 176
4 0x186179910 _dispatch_client_callout + 20
5 0x18618b3cc _dispatch_root_queue_drain + 864
6 0x18618ba04 _dispatch_worker_thread2 + 156
7 0x1863230d8 _pthread_wqthread + 228
ld: Assertion failed: (addr + content.size() <= sectionEndAddr), function writeContentWithoutLinkEdit_block_invoke, file Layout.cpp, line 5690.
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Comment From: lukasmalkmus
And another one but cannot provide an easily reproducible example just yet:
function symbol table not sorted by PC offset: 0xd > 0x4 , plugin:
0x0
0xd
fatal error: invalid runtime symbol table
runtime: panic before malloc heap initialized
runtime stack:
Comment From: cherrymui
@lukasmalkmus Thanks for the report! For the second case, could you try if the workaround CLs helps? A simple way to try is
% go install golang.org/dl/gotip@latest
% gotip download 511355
Then use gotip as the go command, such as gotip build, gotip run etc.
For the first one, the assertion failure looks similar to https://github.com/golang/go/issues/61229#issuecomment-1627026000 above. Could be the same issue. I think the CLs also avoid the failure.
Thanks.
Comment From: lukasmalkmus
For the first case, the CL produces a slightly different output but still errors:
$ go run github.com/golangci/golangci-lint/cmd/golangci-lint run
# github.com/golangci/golangci-lint/cmd/golangci-lint
/opt/homebrew/Cellar/go/1.20.6/libexec/pkg/tool/darwin_arm64/link: running cc failed: exit status 1
ld: warning: -bind_at_load is deprecated on macOS
ld: warning: '/private/var/folders/4h/swcdv5xd7p53_33xlbfvzygc0000gn/T/go-link-3108928878/go.o' has malformed LC_DYSYMTAB, expected 138 undefined symbols to start at index 81807, found 150 undefined symbols starting at index 112
0 0x100628130 __assert_rtn + 72
1 0x1005e14f4 ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 9772
2 0x186179950 _dispatch_client_callout2 + 20
3 0x18618cba0 _dispatch_apply_invoke + 176
4 0x186179910 _dispatch_client_callout + 20
5 0x18618b3cc _dispatch_root_queue_drain + 864
6 0x18618ba04 _dispatch_worker_thread2 + 156
7 0x1863230d8 _pthread_wqthread + 228
ld: Assertion failed: (addr + content.size() <= sectionEndAddr), function writeContentWithoutLinkEdit_block_invoke, file Layout.cpp, line 5690.
clang: error: linker command failed with exit code 1 (use -v to see invocation)
$ gotip run github.com/golangci/golangci-lint/cmd/golangci-lint run
# github.com/golangci/golangci-lint/cmd/golangci-lint
/Users/lukasmalkmus/sdk/gotip/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
ld: warning: -bind_at_load is deprecated on macOS
0 0x10425c130 __assert_rtn + 72
1 0x1041b5f74 mach_o::PointerFormat_DYLD_CHAINED_PTR_64_OFFSET::unauthRebaseIsVmAddr() const + 0
2 0x1041b7120 ___ZN6mach_o13ChainedFixups11buildFixupsENSt3__14spanIKNS_5Fixup10BindTargetELm18446744073709551615EEENS2_IKNS0_17SegmentFixupsInfoELm18446744073709551615EEEyRKNS0_13PointerFormatEjb_block_invoke_2 + 212
3 0x186179950 _dispatch_client_callout2 + 20
4 0x18618cba0 _dispatch_apply_invoke + 176
5 0x186179910 _dispatch_client_callout + 20
6 0x18618b3cc _dispatch_root_queue_drain + 864
7 0x18618ba04 _dispatch_worker_thread2 + 156
8 0x1863230d8 _pthread_wqthread + 228
ld: Assertion failed: (rebasePtr->target == low56), function writeChainEntry, file ChainedFixups.cpp, line 1218.
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Comment From: lukasmalkmus
For the second problem (function symbol table not sorted by PC offset), I wasn't able to fix that with the CL but I also wasn't able to extract an easily reproducible example. The failure is observed in a rather complex build process that also installs tools, lots of Makefiles and scripts... I try to extract something reproducible first before giving it another go with gotip.
Comment From: cherrymui
Thanks @lukasmalkmus . I can reproduce the new assert failure. It seems that it works with CLs on Intel Mac, but still fails on ARM64 Mac (the original assert failure is probably worked around by the CLs, but then we run into another one...) I'll see if I can find out anything, and report to Apple.
Comment From: gopherbot
Change https://go.dev/cl/514535 mentions this issue: [release-branch.go1.21] cmd/link: use symbol-targeted relocation for initializers on Mach-O
Comment From: lukasmalkmus
[...] It seems that it works with CLs on Intel Mac, but still fails on ARM64 Mac [...]
I should have probably mentioned that I'm on an ARM64 Mac after I took over this thread 🤦 But I guess you figured from the output.
Great to hear it is reproducible.
I'm now noticing failures like reported in #61676. But since all of those issues comes back to this one, I'll keep it cool.
Feel free to ping if you require something else.
Comment From: cherrymui
I'm now noticing failures like reported in https://github.com/golang/go/issues/61676.
See also #61190. Also, does the CLs linked in this issue help to work around that? The particular problem for #61676 and #61190 should be addressed by the CLs. Thanks.
Comment From: nezorflame
I'm getting the same error while trying to build new pre-release version of gopls:
~ go install golang.org/x/tools/gopls@v0.13.0-pre.3
go: downloading golang.org/x/tools/gopls v0.13.0-pre.3
# golang.org/x/tools/gopls
ld: warning: '/private/var/folders/pf/jygc38g14yg6jvv74fprbv7m0000gn/T/go-link-1881478201/go.o' has malformed LC_DYSYMTAB, expected 130 undefined symbols to start at index 26418, found 143 undefined symbols starting at index 78
Comment From: cherrymui
@nezorflame Please see the NOTE at the top of this issue: this is a warning, not an error. If you don't see other output, the build succeeds and you should get a binary. That said, for gopls specifically, the binary may not work, see #61190 . Thanks.
Comment From: Philipp-Plotnikov
@cherrymui am I right my problem related with this issue ? macOS Sonoma Xcode 15 beta
Build Error: go build -o /Users/philippplotnikov/go/src/github.com/argoproj/argo-rollouts/std -gcflags all=-N -l ./main.go
# command-line-arguments
/usr/local/go/pkg/tool/darwin_arm64/link: running clang failed: exit status 1
ld: warning: -bind_at_load is deprecated on macOS
ld: warning: '/private/var/folders/dp/24_52kmd5qbbkm3mdrjg1phr0000gn/T/go-link-1951400309/go.o' has malformed LC_DYSYMTAB, expected 123 undefined symbols to start at index 230505, found 143 undefined symbols starting at index 88
0 0x1030bc130 __assert_rtn + 72
1 0x1030754f4 ___ZN2ld16LayoutExecutable27writeContentWithoutLinkEditENSt3__14spanIhLm18446744073709551615EEEy_block_invoke + 9772
2 0x187909950 _dispatch_client_callout2 + 20
3 0x18791cba0 _dispatch_apply_invoke + 176
4 0x187909910 _dispatch_client_callout + 20
5 0x18791b3cc _dispatch_root_queue_drain + 864
6 0x18791ba04 _dispatch_worker_thread2 + 156
7 0x187ab30d8 _pthread_wqthread + 228
ld: Assertion failed: (addr + content.size() <= sectionEndAddr), function writeContentWithoutLinkEdit_block_invoke, file Layout.cpp, line 5690.
clang: error: linker command failed with exit code 1 (use -v to see invocation) (exit status 1)
Golang version is - go version go1.19.11 darwin/arm64
clang version is
Apple clang version 15.0.0 (clang-1500.0.38.1)
Target: arm64-apple-darwin23.0.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Comment From: Philipp-Plotnikov
Do we have some ways how we can solve this problem now ? Or we need to wait until PR with fix will be pushed in go ?
Comment From: troian
looks like go1.20.7 has added -ld64 as the default option during the link stage.
Comment From: Philipp-Plotnikov
it should also help with go install golang.org/x/tools/gopls@latest ?
Comment From: Philipp-Plotnikov
@troian I installed it and I have the same error. Or I need to add some flag ?
Comment From: troian
@Philipp-Plotnikov
export CGO_LDFLAGS=-Wl,-ld_classic -Wno-deprecated-declarations
Comment From: Philipp-Plotnikov
I put export CGO_LDFLAGS="-Wl,-ld_classic -Wno-deprecated-declarations" in ~/.zshrc but it didnt help. The same problem
Comment From: ismail
Building rclone with go 1.21.0 works out of the box with the new Xcode 15 beta 6.
Comment From: cobo-louis
It seems that the latest xcode update has already fix this bug, works well for me.
Comment From: cherrymui
Thanks for the update. Yes, it seems Xcode 15 beta 6 fixes a number of bugs. With beta 6, rclone and golangci-lint both build successfully with Go tip, 1.21.0, and 1.20.7. Most of the tests in all.bash are also passing, except for some linker warnings and with debug info. So we are in a good shape now.
@Philipp-Plotnikov could you try Xcode 15 beta 6 and see if it still fails (without using -Wl,-ld_classic)? If that still fails, could you share the step of how to reproduce the build failure? Thanks.
Comment From: gopherbot
Change https://go.dev/cl/520039 mentions this issue: cmd/link: suppress search path not found warning
Comment From: gopherbot
Change https://go.dev/cl/527415 mentions this issue: cmd/link: disable DWARF by default in c-shared mode on darwin
Comment From: cherrymui
With all the fix/workaround CLs in for the Go side, and Apple fixing bugs in the new linker, as of Xcode 15 beta 7 (and 8), I'm not aware of any failures. So I think this is done.
I'll prepare CLs for backporting to Go 1.20 and 1.21.
If you see a new failure related to Apple's new linker, feel free to post here and reopen the issue. Thanks.
Comment From: cherrymui
@gopherbot please backport this to Go 1.20 and 1.21 releases. I'll fill in the details in the corresponding issues. Thanks.
Comment From: gopherbot
Backport issue(s) opened: #62597 (for 1.20), #62598 (for 1.21).
Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.
Comment From: gopherbot
Change https://go.dev/cl/527816 mentions this issue: [release-branch.go1.21] cmd/link: disable DWARF by default in c-shared mode on darwin
Comment From: gopherbot
Change https://go.dev/cl/527815 mentions this issue: [release-branch.go1.21] cmd/link: force old Apple linker in plugin mode
Comment From: gopherbot
Change https://go.dev/cl/527817 mentions this issue: [release-branch.go1.21] cmd/link: suppress -bind_at_load deprecation warning for ld-prime
Comment From: gopherbot
Change https://go.dev/cl/527798 mentions this issue: [release-branch.go1.20] cmd/link: suppress -bind_at_load deprecation warning for ld-prime
Comment From: gopherbot
Change https://go.dev/cl/527818 mentions this issue: [release-branch.go1.20] cmd/link: force old Apple linker in plugin mode
Comment From: gopherbot
Change https://go.dev/cl/527819 mentions this issue: [release-branch.go1.20] cmd/link: disable DWARF by default in c-shared mode on darwin
Comment From: lzap
Apple released "Sodoma" and new XCode Tools and indeed warnings are still there :-P
Comment From: gopherbot
Change https://go.dev/cl/547455 mentions this issue: doc/go1.22: document enabling PIE by default on darwin/amd64
Comment From: maraino
These warnings also appear on Go 1.22 (darwin/amd64) when the -race flag is used. Generating Non-PIE binaries with -buildmode=exe solves them.
Comment From: nebiros
These warnings also appear on Go 1.22 (darwin/amd64) when the
-raceflag is used. Generating Non-PIE binaries with-buildmode=exesolves them.
in my case it happens when I run unit tests and the -race flag:
ld: warning: '/private/var/folders/4c/9g_d3cwd55g_9nsp2qwlzc38jj5x9m/T/go-link-715828973/000034.o' has malformed LC_DYSYMTAB, expected 98 undefined symbols to start at index 1626, found 95 undefined symbols starting at index 1626
Comment From: cherrymui
Thanks for the information. Unfortunately I'm not sure if there is anything we can do. The warning seems to complain a C object file from the race detector library (LLVM ThreadSanitizer), not a Go object. It might be possible to change ThreadSanitizer or how it is built to avoid warning? I'm not sure.
As mentioned above, it is just a warning and does not affect the correctness of the program. Feel free to ignore.
Generating Non-PIE binaries with -buildmode=exe solves them.
Because that forces the old Apple linker. (The new linker does not support non-PIE binary.)
Comment From: nebiros
Thanks for the information. Unfortunately I'm not sure if there is anything we can do. The warning seems to complain a C object file from the race detector library (LLVM ThreadSanitizer), not a Go object. It might be possible to change ThreadSanitizer or how it is built to avoid warning? I'm not sure.
As mentioned above, it is just a warning and does not affect the correctness of the program. Feel free to ignore.
Generating Non-PIE binaries with -buildmode=exe solves them.
Because that forces the old Apple linker. (The new linker does not support non-PIE binary.)
thank you for your response. Yeah, seems like Goland takes these warnings as errors, :\
Comment From: nebiros
Thanks for the information. Unfortunately I'm not sure if there is anything we can do. The warning seems to complain a C object file from the race detector library (LLVM ThreadSanitizer), not a Go object. It might be possible to change ThreadSanitizer or how it is built to avoid warning? I'm not sure. As mentioned above, it is just a warning and does not affect the correctness of the program. Feel free to ignore.
Generating Non-PIE binaries with -buildmode=exe solves them.
Because that forces the old Apple linker. (The new linker does not support non-PIE binary.)
thank you for your response. Yeah, seems like Goland takes these warnings as errors, :\
anyway, if anybody wants to silence this "errors" on Goland, try adding this env var: GOFLAGS=-ldflags=-extldflags=-Wl,-ld_classic;
Comment From: cherrymui
thank you for your response. Yeah, seems like Goland takes these warnings as errors, :\
They should not do that. I would consider this is a bug/unexpected behavior of Goland.
Comment From: cherrymui
The warning on the race C object (the syso file) is due to this code in LLVM TSAN: https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/sanitizer_common/sanitizer_redefine_builtins.h#L20-L22
asm("memcpy = __sanitizer_internal_memcpy");
asm("memmove = __sanitizer_internal_memmove");
asm("memset = __sanitizer_internal_memset");
Apparently, on Darwin, that caused the object to include weird symbols like (objdump -t output)
00000000000000f3 *UND* memcpy
0000000000006331 *UND* memmove
0000000000002b25 *UND* memset
They are "undefined" symbols but have values (unlike all other UND symbols). And apparently the new Apple linker doesn't like them and warns.
If I comment those three lines off, it no longer generates those symbols and no more warnings from the linker.
On Darwin, at assembler level, all C symbols are prefixed by _. So those lines are probably already not doing anything useful on Darwin. Perhaps those lines should be
asm("_memcpy = ___sanitizer_internal_memcpy");
asm("_memmove = ___sanitizer_internal_memmove");
asm("_memset = ___sanitizer_internal_memset");
on Darwin? That also makes the weird symbols go away, as well as the warning. cc @dvyukov (These symbols are from https://reviews.llvm.org/D151152 . You probably have some context. Thanks!)
Comment From: gabkov
Hi,
I am trying to install a cli package but i receive the following error and the package is not installed:
➜ ~ go install github.com/ethereum/go-ethereum/cmd/abigen@latest
# github.com/ethereum/go-ethereum/cmd/abigen
ld: warning: '/private/var/folders/nh/78bdkps97rv6pdws74j1xy0w0000gn/T/go-link-1841817092/go.o' has malformed LC_DYSYMTAB, expected 121 undefined symbols to start at index 30647, found 165 undefined symbols starting at index 96
I am not sure how this is xcode related however I used xcode before. Any help appreciated.
Comment From: nebiros
Hi,
I am trying to install a cli package but i receive the following error and the package is not installed:
``` ➜ ~ go install github.com/ethereum/go-ethereum/cmd/abigen@latest
github.com/ethereum/go-ethereum/cmd/abigen
ld: warning: '/private/var/folders/nh/78bdkps97rv6pdws74j1xy0w0000gn/T/go-link-1841817092/go.o' has malformed LC_DYSYMTAB, expected 121 undefined symbols to start at index 30647, found 165 undefined symbols starting at index 96 ```
I am not sure how this is xcode related however I used xcode before. Any help appreciated.
Try adding that env var: https://github.com/golang/go/issues/61229#issuecomment-1952798326 before running go install
Comment From: cherrymui
@gabkov your build should have succeeded and the binary installed. As mentioned above, it is just a warning, not an error.
That said, which version of Go are you using? I would not expect a warning like this for this particular build with recent versions of Go.
Comment From: tkgalk
Still seeing those warnings on 1.22 installed today on a Sonoma 14.3.1 M1.
Comment From: gabkov
Hi, I am trying to install a cli package but i receive the following error and the package is not installed: ``` ➜ ~ go install github.com/ethereum/go-ethereum/cmd/abigen@latest
github.com/ethereum/go-ethereum/cmd/abigen
ld: warning: '/private/var/folders/nh/78bdkps97rv6pdws74j1xy0w0000gn/T/go-link-1841817092/go.o' has malformed LC_DYSYMTAB, expected 121 undefined symbols to start at index 30647, found 165 undefined symbols starting at index 96 ```
I am not sure how this is xcode related however I used xcode before. Any help appreciated.
Try adding that env var: #61229 (comment) before running
go install
Thank you for the suggestion @nebiros, but unfortunately setting the env var only hid the warning and the package was still not installed.
@gabkov your build should have succeeded and the binary installed. As mentioned above, it is just a warning, not an error.
That said, which version of Go are you using? I would not expect a warning like this for this particular build with recent versions of Go.
@cherrymui i was using 1.20.6, but today i upgraded to 1.22.0 and the warning is gone and i was able to install the package as well. Not sure if it's just a local issue or related to the actual go version. Thanks anyway.
Comment From: wendorf
I've been able to reproduce the "malformed LC_DYSYMTAB" warning on my Macbook Pro M2 Max running Sonoma 14.3.1 and go 1.22.1. I do not see the warning at all when using go 1.21.8.
The best reproducible test case I've been able to come up with is running tests on https://github.com/DataDog/zstd , ref 869dae002e5efb372a0b09cd7d99390ca2089cc1:
❯ go test -race .
# github.com/DataDog/zstd.test
ld: warning: '/private/var/folders/mf/75msnn554t940_ky9yrfzfcw0000gn/T/go-link-98474360/000055.o' has malformed LC_DYSYMTAB, expected 98 undefined symbols to start at index 1626, found 95 undefined symbols starting at index 1626
ok github.com/DataDog/zstd (cached)
When I fully remove the C code and edit the Go code to not use the C package, the warning is not present. Unfortunately, I'm unable to figure out what about the C code is causing this problem.
Comment From: cherrymui
@wendorf as mentioned above https://github.com/golang/go/issues/61229#issuecomment-1954706803 , the warning is for the C code in the race detector (LLVM TSAN) itself. The only difference that user C code made is that it changes the default link mode. Without user C code, the Go linker by default does all the linking itself, without using the C linker (Apple's linker). When user C code is present, it uses the C linker by default. You can try passing -ldflags=-linkmode=internal to force using the Go linker to do the entire linking (although it is not guaranteed to be able to handle all C objects).
Comment From: cherrymui
A note about the malformed LC_DYSYMTAB warning: take a look at the object file name that is warned on:
- if it is .../go.o, it is the object of the Go code. This is already fixed in recent versions of Go 1.20.x, 1.21.x, and 1.22.x, and should not appear with an up-to-date version of Go.
- if it is .../00NNNN.o with some 6-digit number, it is a C object.
- if you pass -race and you're using Go 1.22.x or newer, it is probably the race detector's C code, as mentioned above https://github.com/golang/go/issues/61229#issuecomment-1954706803
- otherwise it is from user C code
The warning on the race detector C object is not really a bug in Go. Maybe we can fix in the upstream TSAN (also see https://github.com/golang/go/issues/61229#issuecomment-1954706803). To work around, you could try
- passing -ldflags=-extldflags=-Wl,-ld_classic to force the old Apple linker
- passing -ldflags=-linkmode=internal to force using Go linker to do the whole linking (although it is not guaranteed to be able to handle all C objects)
- also, an IDE should NOT treat warning as error by default.
Thanks.
Comment From: h31
Hello @nebiros,
I reproduced your problem with the linker warning highlighted as an error in GoLand. The reason why the warning is displayed in red is that the linker message is being written to stderr. This is a highlighting problem and it shouldn't affect the build process in any way. I've created an issue about this in our bug tracker: GO-16873.
Comment From: tkgalk
The reason why the warning is displayed in red is that the linker message is being written to stderr.
That would explain why GoLand marked those tests as failed. I don't think this is a nice behaviour, warnings shouldn't be posted to stderr, it's reasonable for other applications to assume anything on stderr is an error.
Comment From: h31
That would explain why GoLand marked those tests as failed.
I think this is a different problem, GoLand shouldn't mark tests as failed due to extra output. Can you please describe how to reproduce this issue? Here or in the comments to GO-16873.
Comment From: nebiros
Hello @nebiros,
I reproduced your problem with the linker warning highlighted as an error in GoLand. The reason why the warning is displayed in red is that the linker message is being written to stderr. This is a highlighting problem and it shouldn't affect the build process in any way. I've created an issue about this in our bug tracker: GO-16873.
@h31 I know it doesn't affect the build process, but I wasn't talking about that, I was talking about testing and running unit tests:https://github.com/golang/go/issues/61229#issuecomment-1949442357, those warnings mark tests as failed, so it breaks all tests
Comment From: VarusHsu
I resolved it by disabling cgo.
go env -w CGO_ENABLED='0'
Comment From: tkgalk
That would explain why GoLand marked those tests as failed.
I think this is a different problem, GoLand shouldn't mark tests as failed due to extra output. Can you please describe how to reproduce this issue? Here or in the comments to GO-16873.
I wouldn't agree it's a different problem. GoLand, or any other app, that sees things on stderr instead of stdout should be reasonably able to assume it's an error and treat it as a failure.
Comment From: edaniels
https://github.com/golang/go/issues/61229#issuecomment-1954706803
@cherrymui is it possible the .set directives now being used https://github.com/llvm/llvm-project/blame/main/compiler-rt/lib/sanitizer_common/sanitizer_redefine_builtins.h#L20-L22 fix the issue? If so, would it be possible to update the syso files race uses? They're two years old at this point so it seems like this isn't really a common operation. Forgive me if I'm misunderstanding how the internal mechanics of utilizing TSAN works.
Comment From: cherrymui
I tried that a while ago and it appears that changing to .set doesn't help. The warning is still there. We'll need to think more on what the solution is.
Comment From: AlekSi
We'll need to think more on what the solution is
Then maybe the issue could be re-opened and updated? Started seeing has malformed LC_DYSYMTAB with 1.24 out of the blue, and it is weird that the relevant issue is closed (and also mentions beta in the title, etc)
Comment From: cherrymui
Removed "beta."
The output binary is not malformed and is correct.
Comment From: AlekSi
Yes, it works great for me. It caused some other problems due to 1.24's go build -json being called from go test -json, and my consumer could not handle those build warnings. That's how I found that issue.
But – why are those warnings visible at all? Normally, go build does not produce warnings and hides output from system tools. So why make this one warning visible, especially if it is harmless and the user can't do anything about it anyway? It is just scary-looking and confusing.
Comment From: AlekSi
if you pass -race and you're using Go 1.22.x, it is probably the race detector's C code
Is it still the case for 1.24?
Comment From: cherrymui
Is it still the case for 1.24?
Yes. Edited to "Go 1.22.x or newer"
Comment From: cherrymui
But – why are those warnings visible at all?
These warnings are from the Apple C linker, not the Go linker. In general it is possible that the C linker warns on some other user C object file, not necessarily the race detector one. I guess it is possible to parse the warning and the object file it warns on and see if it is the race detector one and hide that one if so. Not sure if this is really a good idea.
That said, if it doesn't work well with go build -json, we should fix.
Comment From: AlekSi
That said, if it doesn't work well with
go build -json, we should fix.
No-no, it works fine. It is just that my go test -json parsing code uses DisallowUnknownFields, and 1.24's go test -json calls go build -json that generates this warning with ImportPath field (as documented).
Thank you so much for your answers!
Comment From: sbp-bvanb
FYI: Still occurs while using 1.24.1.
Comment From: ejain
Seen on go version go1.24.2 darwin/arm64, Xcode 16.3.
Comment From: anacrolix
Definitely still seeing this.
Comment From: maniroo
Seeing this still
Comment From: alexpls
https://github.com/golang/go/issues/61229#issuecomment-1988965927
On ld-1167.4.1, forcing the classic linker with -ldflags=-extldflags=-Wl,-ld_classic now comes with its own warnings which get sent to stderr:
ld: warning: -ld_classic is deprecated and will be removed in a future release
And -ldflags=-linkmode=internal is a no-go (😉) on my build as it results in some undefined objects.
In the end, if you're comfortable with suppressing all warnings from the macOS linker, that can be done by setting the -w flag. So setting CGO_LDFLAGS="-w" env var does the trick. If your build runs on platforms other than macOS, you'll wanna make sure to set it conditionally.
Comment From: anacrolix
Let me translate the above comments for whoever is marking them duplicate: Reopen.
Comment From: gopherbot
Change https://go.dev/cl/692975 mentions this issue: cmd/racebuild: pass the syso through ld -r
Comment From: gopherbot
Change https://go.dev/cl/692996 mentions this issue: runtime/race: update darwin race syso
Comment From: gopherbot
Change https://go.dev/cl/692995 mentions this issue: debug/macho: support reading imported symbols without LC_DYSYMTAB
Comment From: cherrymui
For the warning on the race detector object files (https://github.com/golang/go/issues/61229#issuecomment-1988965927), I found a workaround: pass the object through ld -r, which removes the unnecessary LC_DYSYMTAB. The CLs above do so.