As reported via the golang-dev mailing list by @FiloSottile:
Looks like watchflakes opened over 500 duplicate issues a few hours ago.
Can they be deleted (as opposed to closed as duplicates), so they won't pollute search results and gabyhelp lookups?
Maybe relatedly, I noticed that recently watchflakes was adding a "Found new dashboard test flakes" comment with the exact same flake as the one that opened the issue, which is briefly confusing.
@golang/release
Comment From: prattmic
Maybe relatedly, I noticed that recently watchflakes was adding a "Found new dashboard test flakes" comment with the exact same flake as the one that opened the issue, which is briefly confusing.
FWIW, I believe this has always been the case, though I agree it is a bit confusing.
Comment From: dmitshur
The problem doesn't reproduce when running watchflakes locally (in dry-run mode) now:
[…]
watchflakes: 2025/03/07 11:24:53 readComments 60085
watchflakes: 2025/03/07 11:24:53 readComments 70602
watchflakes: 2025/03/07 11:24:54 readComments 72151
- new for #49338 all: plan9-arm builder failing frequently with a variety of errors
- 2025-03-06 16:06 gotip-plan9-arm go@e7f9e760 crypto/cipher:gofips140
https://ci.chromium.org/b/8721046369747702737
- new for #65293 runtime: TestPingPongHog failures
- 2025-03-06 16:07 gotip-darwin-amd64-longtest go@5e0dbec7 runtime.TestPingPongHog
https://ci.chromium.org/b/8721049635990102081
watchflakes: 2025/03/07 11:24:54 Done. 106 boards, 1153 failures, 1136 issues, 2 posts, in 10m24.100820666s
Looking at just the end-of-run stats from logs gives a pretty good hint as to what went wrong, highlighted in green:
[…]
watchflakes: 2025/03/06 12:27:43 Done. 106 boards, 1162 failures, 834 issues, 1 posts, in 11m47.540717124s
watchflakes: 2025/03/06 13:27:42 Done. 106 boards, 1162 failures, 835 issues, 1 posts, in 11m46.850392254s
watchflakes: 2025/03/06 14:28:06 Done. 106 boards, 1162 failures, 835 issues, 0 posts, in 12m10.414244599s
watchflakes: 2025/03/06 15:28:07 Done. 106 boards, 1163 failures, 836 issues, 2 posts, in 12m11.211220873s
watchflakes: 2025/03/06 16:28:27 Done. 106 boards, 1165 failures, 836 issues, 2 posts, in 12m31.410462304s
watchflakes: 2025/03/06 17:27:56 Done. 106 boards, 1166 failures, 836 issues, 2 posts, in 12m0.843599754s
watchflakes: 2025/03/06 18:38:54 Done. 106 boards, 1166 failures, 837 issues, 2 posts, in 12m21.476020521s
watchflakes: 2025/03/06 19:39:09 Done. 106 boards, 1173 failures, 837 issues, 7 posts, in 12m37.004481974s
watchflakes: 2025/03/06 20:38:51 Done. 106 boards, 1175 failures, 837 issues, 7 posts, in 12m19.092738108s
watchflakes: 2025/03/06 21:38:27 Done. 106 boards, 1176 failures, 837 issues, 3 posts, in 11m55.033018151s
watchflakes: 2025/03/06 22:38:11 Done. 106 boards, 1140 failures, 836 issues, 7 posts, in 11m38.225585236s
watchflakes: 2025/03/06 23:38:01 Done. 106 boards, 1140 failures, 836 issues, 2 posts, in 11m28.207521902s
watchflakes: 2025/03/07 00:38:14 Done. 106 boards, 1144 failures, 837 issues, 4 posts, in 11m41.719187756s
watchflakes: 2025/03/07 01:38:55 Done. 106 boards, 1145 failures, 837 issues, 2 posts, in 12m22.376084966s
watchflakes: 2025/03/07 02:38:09 Done. 106 boards, 1145 failures, 837 issues, 2 posts, in 11m36.695677355s
watchflakes: 2025/03/07 03:42:23 Done. 106 boards, 1146 failures, 837 issues, 3 posts, in 11m36.522389042s
watchflakes: 2025/03/07 04:42:01 Done. 106 boards, 1147 failures, 837 issues, 3 posts, in 11m14.457386912s
watchflakes: 2025/03/07 05:42:26 Done. 106 boards, 1148 failures, 837 issues, 3 posts, in 11m38.971656091s
watchflakes: 2025/03/07 06:42:23 Done. 106 boards, 1149 failures, 837 issues, 1 posts, in 11m35.762629696s
watchflakes: 2025/03/07 07:42:08 Done. 106 boards, 1149 failures, 837 issues, 0 posts, in 11m21.213304922s
+watchflakes: 2025/03/07 08:42:53 Done. 106 boards, 1149 failures, 837 issues, 0 posts, in 12m5.76321574s
+watchflakes: 2025/03/07 10:20:42 Done. 106 boards, 1151 failures, 386 issues, 288 posts, in 49m54.968927068s
+watchflakes: 2025/03/07 11:19:53 Done. 106 boards, 1151 failures, 386 issues, 286 posts, in 49m5.961277611s
+watchflakes: 2025/03/07 11:42:35 Done. 106 boards, 1153 failures, 1408 issues, 4 posts, in 11m48.038100906s
watchflakes: 2025/03/07 12:42:48 Done. 106 boards, 1153 failures, 1408 issues, 0 posts, in 12m1.480930352s
watchflakes: 2025/03/07 13:42:42 Done. 106 boards, 1152 failures, 1408 issues, 1 posts, in 11m54.857192706s
Notably, watchflakes was tracking around 834-837 issues, then all of a sudden it only saw 386-286 = 100 issues, causing it to make 280+ new posts, then it went back to seeing all of the 837 + new posts = 1408 issues.
Comment From: FiloSottile
As reported via the golang-dev mailing list by @FiloSottile:
(Not sure why I emailed golang-dev instead of opening an ~~email~~ issue, FWIW. I think my brain went "problem with the issue tracker" → "report it through other channel" but it doesn't really make sense. 😅 Thanks for opening the issue.)
Comment From: dmitshur
Timing seems to align with https://www.githubstatus.com/incidents/lb0d8kp99f2v which spanned Mar 07, 2025 10:03 UTC to Mar 07, 2025 - 11:24 UTC:
Issues is experiencing degraded performance.
Searches for issues and pull-requests may be slower than normal and timeout for some users
It might be that the github client has a problem that caused it not to report a timeout as an error, or maybe GitHub's servers happened to return an incomplete issue listing without reporting an error during the incident.
Comment From: aarzilli
I don't know if this is the cause (I don't know how watchflakes works) however this morning I noticed that github issues were having issues. Once in a while projects would report having 0 issues or the number of issues would load but not the issues themselves. Also happened to golang/go.
Comment From: prattmic
Offline @cherrymui suggested that if watchflakes finds many new issues at once (e.g., 100) that it doesn't create them and instead files an issue that something is likely wrong. That should catch issues like this (depending on the threshold).
Comment From: dmitshur
I didn't spot problems in the client code, and based on what @aarzilli observed, it seems we can't dismiss the possibility of GitHub returning an incomplete issue list without also reporting an error when it's having an incident. So a heuristic like suggested above is probably all we can do. I'll send a CL and resume watchflakes after that.
For posterity, the duplicate issues were deleted with:
/*
bulkdeleteissue was used to delete 572 duplicate issues watchflakes made. See go.dev/issue/72731.
The query
https://github.com/golang/go/issues?q=is%3Aissue%20author%3Agopherbot%20created%3A%40today%20closed%3A%40today%20watchflakes%20%22Issue%20created%20automatically%20to%20collect%20these%20failures.%22%20&page=1
used to match 572 issues, and it's zero after the deletion took place.
Scrollback
bulkdeleteissue $ export GH_TOKEN_ISSUE_72731="$(pbpaste)"
bulkdeleteissue $ go run . -delete-on-remote-server
72608 go/importer: TestForCompiler failures
72322 go/importer: TestForCompiler failures
72622 net/http: Test304Responses failures
72587 runtime:cpu4: TestChanSendBarrier failures
72336 net/http: Test304Responses failures
72301 runtime:cpu4: TestChanSendBarrier failures
72701 os: TestDirFSPathsValid failures
72632 os: TestPipeIOCloseRace failures
72415 os: TestDirFSPathsValid failures
72346 os: TestPipeIOCloseRace failures
72648 x/tools/go/ssa: TestGenericAliases failures
72462 os/exec: TestWaitid failures
72362 x/tools/go/ssa: TestGenericAliases failures
72176 os/exec: TestWaitid failures
72716 x/tools/go/loader: TestLoad2 failures [consistent failure]
72713 x/tools/go/loader: TestLoad1 failures
72430 x/tools/go/loader: TestLoad2 failures [consistent failure]
72427 x/tools/go/loader: TestLoad1 failures
72639 x/tools/gopls/internal/util/safetoken: TestGoplsSourceDoesNotCallTokenFileMethods failures
72563 net/http: TestClientWriteShutdown/h2 failures
72353 x/tools/gopls/internal/util/safetoken: TestGoplsSourceDoesNotCallTokenFileMethods failures
72277 net/http: TestClientWriteShutdown/h2 failures
72700 math/big: TestAliasing/Exp-XY failures
72696 math/big: TestAliasing/Exp-YZ failures
72686 math/big: TestAliasing/Exp-XZ failures
72414 math/big: TestAliasing/Exp-XY failures
72410 math/big: TestAliasing/Exp-YZ failures
72400 math/big: TestAliasing/Exp-XZ failures
72717 x/tools/gopls/internal/test/integration/workspace: TestCriticalErrorsInOrphanedFiles/default failures
72714 x/tools/gopls/internal/test/integration/misc: TestImplementationsInVendor/default failures
72707 x/tools/gopls/internal/test/integration/misc: TestImplementationsOfError/default failures
72697 x/tools/gopls/internal/test/integration/workspace: TestGoworkMutation/default failures
72690 x/tools/gopls/internal/test/integration/workspace: TestOpenAndClosePorts/default failures
72673 x/tools/gopls/internal/test/integration/diagnostics: TestUndeclaredDiagnostics/default failures
72653 x/tools/gopls/internal/test/integration/misc: TestPrepareRenameWithNoPackageDeclaration/default failures
72649 x/tools/gopls/internal/test/integration/workspace: TestGoModReplace/default failures
72642 x/tools/gopls/internal/test/integration/diagnostics: TestDiagnosticsAreResentAfterCloseOrOpen/default failures
72617 x/tools/gopls/internal/test/integration/workspace: TestAddAndRemoveGoWork/default failures
72627 x/tools/gopls/internal/test/integration/misc: TestRenamePackageWithAlias/default failures
72431 x/tools/gopls/internal/test/integration/workspace: TestCriticalErrorsInOrphanedFiles/default failures
72421 x/tools/gopls/internal/test/integration/misc: TestImplementationsOfError/default failures
72428 x/tools/gopls/internal/test/integration/misc: TestImplementationsInVendor/default failures
72411 x/tools/gopls/internal/test/integration/workspace: TestGoworkMutation/default failures
72404 x/tools/gopls/internal/test/integration/workspace: TestOpenAndClosePorts/default failures
72387 x/tools/gopls/internal/test/integration/diagnostics: TestUndeclaredDiagnostics/default failures
72367 x/tools/gopls/internal/test/integration/misc: TestPrepareRenameWithNoPackageDeclaration/default failures
72363 x/tools/gopls/internal/test/integration/workspace: TestGoModReplace/default failures
72356 x/tools/gopls/internal/test/integration/diagnostics: TestDiagnosticsAreResentAfterCloseOrOpen/default failures
72341 x/tools/gopls/internal/test/integration/misc: TestRenamePackageWithAlias/default failures
72331 x/tools/gopls/internal/test/integration/workspace: TestAddAndRemoveGoWork/default failures
72671 crypto/sha3: TestCSHAKELargeS failures
72660 runtime: TestGdbAutotmpTypes failures
72538 os/signal: TestStress failures
72535 log/syslog: TestConcurrentReconnect failures
72385 crypto/sha3: TestCSHAKELargeS failures
72374 runtime: TestGdbAutotmpTypes failures
72249 log/syslog: TestConcurrentReconnect failures
72252 os/signal: TestStress failures
72650 x/tools/gopls/internal/test/integration/diagnostics: TestValidImportCheck_GoPackagesDriver/default failures
72643 x/tools/gopls/internal/test/integration/workspace: TestChangeNonFileWorkspaceFolders/add/default failures
72638 x/tools/gopls/internal/test/integration/workspace: TestClearNonWorkspaceDiagnostics_NoView/default failures
72636 runtime:cpu4: TestGdbAutotmpTypes failures
72624 x/tools/gopls/internal/test/integration/diagnostics: TestBrokenWorkspace_GOPACKAGESDRIVER/default failures [consistent failure]
72628 x/tools/gopls/internal/test/integration/diagnostics: TestCreatingPackageInvalidatesDiagnostics_Issue66384/default failures
72615 x/tools/gopls/internal/test/integration/misc: TestRenamePackage_InternalPackage/default failures
72364 x/tools/gopls/internal/test/integration/diagnostics: TestValidImportCheck_GoPackagesDriver/default failures
72350 runtime:cpu4: TestGdbAutotmpTypes failures
72357 x/tools/gopls/internal/test/integration/workspace: TestChangeNonFileWorkspaceFolders/add/default failures
72342 x/tools/gopls/internal/test/integration/diagnostics: TestCreatingPackageInvalidatesDiagnostics_Issue66384/default failures
72352 x/tools/gopls/internal/test/integration/workspace: TestClearNonWorkspaceDiagnostics_NoView/default failures
72338 x/tools/gopls/internal/test/integration/diagnostics: TestBrokenWorkspace_GOPACKAGESDRIVER/default failures [consistent failure]
72329 x/tools/gopls/internal/test/integration/misc: TestRenamePackage_InternalPackage/default failures
72524 runtime: TestUserArena failures
72238 runtime: TestUserArena failures
72719 x/tools/go/analysis/checker: Example failures
72651 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/no/default failures
72625 x/tools/gopls/internal/test/integration/workspace: TestInitializeWithNonFileWorkspaceFolders/real,virtual/default failures
72582 archive/tar: unrecognized failures
72505 runtime:cpu2: TestUserArena failures
72492 time: TestLongAdjustTimers failures
72433 x/tools/go/analysis/checker: Example failures
72365 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/no/default failures
72339 x/tools/gopls/internal/test/integration/workspace: TestInitializeWithNonFileWorkspaceFolders/real,virtual/default failures
72296 archive/tar: unrecognized failures
72219 runtime:cpu2: TestUserArena failures
72206 time: TestLongAdjustTimers failures
72721 runtime:cpu1: TestPreemptionAfterSyscall/10µs failures
72435 runtime:cpu1: TestPreemptionAfterSyscall/10µs failures
72692 x/tools/gopls/internal/test/marker: Test/selectionrange/selectionrange.txt failures
72619 x/tools/gopls/internal/test/marker: Test/callhierarchy/issue64451.txt failures [consistent failure]
72613 net/http: TestTransportMaxConnsPerHostDialCancellation/h1 failures
72588 x/tools/gopls/internal/test/marker: Test/callhierarchy/callhierarchy.txt failures
72406 x/tools/gopls/internal/test/marker: Test/selectionrange/selectionrange.txt failures
72333 x/tools/gopls/internal/test/marker: Test/callhierarchy/issue64451.txt failures [consistent failure]
72327 net/http: TestTransportMaxConnsPerHostDialCancellation/h1 failures
72302 x/tools/gopls/internal/test/marker: Test/callhierarchy/callhierarchy.txt failures
72504 x/build: unrecognized failures
72218 x/build: unrecognized failures
72475 runtime: TestPingPongHog failures
72452 time: TestChan/asynctimerchan=1/Ticker failures
72189 runtime: TestPingPongHog failures
72166 time: TestChan/asynctimerchan=1/Ticker failures
72682 x/tools/gopls/internal/test/marker: Test/codeaction/functionextraction_issue44813.txt failures
72614 x/tools/gopls/internal/test/marker: Test/rename/prepare_func.txt failures
72443 x/tools/gopls/internal/test/integration/inlayhints: TestEnablingInlayHints/enable_const/default failures
72396 x/tools/gopls/internal/test/marker: Test/codeaction/functionextraction_issue44813.txt failures
72328 x/tools/gopls/internal/test/marker: Test/rename/prepare_func.txt failures
72157 x/tools/gopls/internal/test/integration/inlayhints: TestEnablingInlayHints/enable_const/default failures
72712 x/tools/gopls/internal/test/integration/workspace: TestClearNonWorkspaceDiagnostics/default failures
72634 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_PromptFile/v0.16-format/pPending/default failures
72618 x/tools/gopls/doc/generate: unrecognized failures [consistent failure]
72616 x/tools/gopls/internal/test/marker: Test/rename/issue57479.txt failures
72567 x/tools/gopls/internal/test/marker: Test/codeaction/addtest.txt failures
72426 x/tools/gopls/internal/test/integration/workspace: TestClearNonWorkspaceDiagnostics/default failures
72348 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_PromptFile/v0.16-format/pPending/default failures
72332 x/tools/gopls/doc/generate: unrecognized failures [consistent failure]
72330 x/tools/gopls/internal/test/marker: Test/rename/issue57479.txt failures
72281 x/tools/gopls/internal/test/marker: Test/codeaction/addtest.txt failures
72715 x/tools/gopls/internal/analysis/fillreturns: unrecognized failures [consistent failure]
72676 x/net/html: TestParser failures
72623 x/tools/gopls/internal/analysis/embeddirective: unrecognized failures [consistent failure]
72620 x/tools/gopls/internal/analysis/deprecated: unrecognized failures [consistent failure]
72525 cmd/compile/internal/ssagen: unrecognized failures
72463 net/http: TestTimeoutHandlerEmptyResponse/h2 failures
72445 x/tools/gopls/internal/test/marker: Test/codeaction/fill_struct_resolve.txt failures
72429 x/tools/gopls/internal/analysis/fillreturns: unrecognized failures [consistent failure]
72390 x/net/html: TestParser failures
72337 x/tools/gopls/internal/analysis/embeddirective: unrecognized failures [consistent failure]
72334 x/tools/gopls/internal/analysis/deprecated: unrecognized failures [consistent failure]
72239 cmd/compile/internal/ssagen: unrecognized failures
72177 net/http: TestTimeoutHandlerEmptyResponse/h2 failures
72159 x/tools/gopls/internal/test/marker: Test/codeaction/fill_struct_resolve.txt failures
72637 cmd/compile/internal/amd64: unrecognized failures
72600 x/tools/gopls/internal/cmd: TestStats failures
72589 x/tools/gopls/internal/test/marker: Test/codeaction/change_quote.txt failures
72483 cmd/internal/testdir: Test/initexp.go failures
72351 cmd/compile/internal/amd64: unrecognized failures
72314 x/tools/gopls/internal/cmd: TestStats failures
72303 x/tools/gopls/internal/test/marker: Test/codeaction/change_quote.txt failures
72197 cmd/internal/testdir: Test/initexp.go failures
72482 crypto/internal/fips140test:gofips140-v1.0.0: TestACVP failures
72479 crypto/internal/fips140/v1.0.0/rsa:gofips140-v1.0.0: TestTotient failures
72196 crypto/internal/fips140test:gofips140-v1.0.0: TestACVP failures
72193 crypto/internal/fips140/v1.0.0/rsa:gofips140-v1.0.0: TestTotient failures
72704 cmd/compile/internal/test: TestCode/FP,softfloat failures
72418 cmd/compile/internal/test: TestCode/FP,softfloat failures
72485 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/yes failures
72460 cmd/internal/testdir: Test/fixedbugs/issue46234.go failures
72444 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/empty failures
72454 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/no failures
72199 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/yes failures
72174 cmd/internal/testdir: Test/fixedbugs/issue46234.go failures
72168 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/no failures
72158 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/empty failures
72672 cmd/compile/internal/types2: TestFixedbugs/issue60434.go failures
72468 bufio: unrecognized failures
72386 cmd/compile/internal/types2: TestFixedbugs/issue60434.go failures
72182 bufio: unrecognized failures
72668 net/http: TestOptions/h1 failures
72647 archive/tar_test [archive/tar.test]: unrecognized failures
72382 net/http: TestOptions/h1 failures
72361 archive/tar_test [archive/tar.test]: unrecognized failures
72705 cmd/cgo/internal/testsanitizers: TestTSAN/tsan12 failures
72702 net/http/httputil: TestReverseProxyWebSocketHalfTCP/server_close_read failures
72658 cmd/addr2line: unrecognized failures
72657 cmd/cgo/internal/testsanitizers: TestTSAN/tsan13 failures
72644 internal/trace: TestTraceGOMAXPROCS/Stress failures
72611 internal/trace: TestTraceCgoCallback/Stress failures
72573 internal/trace: TestTraceStressStartStop/AllocFree failures
72570 runtime:runtime:gccheckmark: TestSpuriousWakeupsNeverHangSemasleep failures
72509 runtime:runtime:gcstoptheworld2: TestSpuriousWakeupsNeverHangSemasleep failures
72419 cmd/cgo/internal/testsanitizers: TestTSAN/tsan12 failures
72416 net/http/httputil: TestReverseProxyWebSocketHalfTCP/server_close_read failures
72372 cmd/addr2line: unrecognized failures
72371 cmd/cgo/internal/testsanitizers: TestTSAN/tsan13 failures
72358 internal/trace: TestTraceGOMAXPROCS/Stress failures
72325 internal/trace: TestTraceCgoCallback/Stress failures
72287 internal/trace: TestTraceStressStartStop/AllocFree failures
72284 runtime:runtime:gccheckmark: TestSpuriousWakeupsNeverHangSemasleep failures
72223 runtime:runtime:gcstoptheworld2: TestSpuriousWakeupsNeverHangSemasleep failures
72711 x/vulndb: TestLintReports/data/reports/GO-2024-3188.yaml failures
72710 x/vulndb: TestLintReports/data/reports/GO-2024-3181.yaml failures
72663 cmd/api: unrecognized failures
72656 internal/trace: TestTraceStacks/Default failures
72654 cmd/cgo/internal/testsanitizers: TestShared/tsan_shared failures
72597 cmd/link: TestIssue38554 failures
72515 net/http: TestServerKeepAlivesEnabled/h2 failures
72469 cmd/internal/testdir: Test/rangegen.go failures
72425 x/vulndb: TestLintReports/data/reports/GO-2024-3188.yaml failures
72424 x/vulndb: TestLintReports/data/reports/GO-2024-3181.yaml failures
72370 internal/trace: TestTraceStacks/Default failures
72368 cmd/cgo/internal/testsanitizers: TestShared/tsan_shared failures
72377 cmd/api: unrecognized failures
72311 cmd/link: TestIssue38554 failures
72229 net/http: TestServerKeepAlivesEnabled/h2 failures
72183 cmd/internal/testdir: Test/rangegen.go failures
72693 crypto/cipher: TestGCMAEAD/AES-NI/AES-128/NonceSize-1/AppendDst/Plaintext-Length=8208,AddData-Length=8193/Open failures
72665 cmd/asm/internal/asm: unrecognized failures
72521 cmd/internal/obj/x86: TestVexEvexPCrelative failures
72517 x/vulndb: TestLintReports/data/reports/GO-2025-3451.yaml failures
72407 crypto/cipher: TestGCMAEAD/AES-NI/AES-128/NonceSize-1/AppendDst/Plaintext-Length=8208,AddData-Length=8193/Open failures
72379 cmd/asm/internal/asm: unrecognized failures
72235 cmd/internal/obj/x86: TestVexEvexPCrelative failures
72231 x/vulndb: TestLintReports/data/reports/GO-2025-3451.yaml failures
72703 net/http/httputil: TestReverseProxyWebSocketHalfTCP/server_close_write failures
72674 build: build failure on x_build-gotip-darwin-amd64_14
72661 cmd/link/internal/ld: TestRISCVTrampolines failures
72578 cmd/internal/testdir: Test/fixedbugs/issue21576.go failures
72506 cmd/link: TestDWARF/testprogcgo failures
72488 crypto/internal/sysrand:gofips140-v1.0.0: unrecognized failures
72417 net/http/httputil: TestReverseProxyWebSocketHalfTCP/server_close_write failures
72388 build: build failure on x_build-gotip-darwin-amd64_14
72375 cmd/link/internal/ld: TestRISCVTrampolines failures
72292 cmd/internal/testdir: Test/fixedbugs/issue21576.go failures
72220 cmd/link: TestDWARF/testprogcgo failures
72202 crypto/internal/sysrand:gofips140-v1.0.0: unrecognized failures
72652 crypto/tls: TestBogoSuite failures
72458 x/tools/go/analysis/passes/loopclosure: unrecognized failures
72366 crypto/tls: TestBogoSuite failures
72172 x/tools/go/analysis/passes/loopclosure: unrecognized failures
72667 cmd/link/internal/ld: TestStackCheckOutput failures
72590 cmd/link: TestContentAddressableSymbols failures
72574 runtime:cpu1: TestLockRankGenerated failures
72514 cmd/vet: TestVet/appends failures
72381 cmd/link/internal/ld: TestStackCheckOutput failures
72304 cmd/link: TestContentAddressableSymbols failures
72288 runtime:cpu1: TestLockRankGenerated failures
72228 cmd/vet: TestVet/appends failures
72612 cmd/link: TestCGOLTO/0 failures
72577 cmd/link: TestCheckLinkname/push.go failures
72565 cmd/link/internal/ld: TestSubprogramDeclFileLine/line-directive failures
72562 cmd/link: TestGNUBuildID/specific failures
72560 cmd/vet: TestVet/slog failures
72543 cmd/link: TestCheckLinkname/ok.go failures
72540 cmd/link/internal/ld: TestDeadcode/ifacemethod3 failures
72490 crypto/ecdsa:gofips140-v1.0.0: unrecognized failures
72477 cmd/internal/testdir: Test/fixedbugs/issue70189.go failures
72326 cmd/link: TestCGOLTO/0 failures
72291 cmd/link: TestCheckLinkname/push.go failures
72274 cmd/vet: TestVet/slog failures
72279 cmd/link/internal/ld: TestSubprogramDeclFileLine/line-directive failures
72276 cmd/link: TestGNUBuildID/specific failures
72257 cmd/link: TestCheckLinkname/ok.go failures
72254 cmd/link/internal/ld: TestDeadcode/ifacemethod3 failures
72204 crypto/ecdsa:gofips140-v1.0.0: unrecognized failures
72191 cmd/internal/testdir: Test/fixedbugs/issue70189.go failures
72547 cmd/link: TestUnresolved failures
72545 runtime:mayMoreStackPreempt: TestCrashWhileTracing failures
72491 crypto/ecdh:gofips140-v1.0.0: TestLinker failures
72480 crypto/internal/fips140deps:gofips140-v1.0.0: TestImports failures
72261 cmd/link: TestUnresolved failures
72259 runtime:mayMoreStackPreempt: TestCrashWhileTracing failures
72194 crypto/internal/fips140deps:gofips140-v1.0.0: TestImports failures
72205 crypto/ecdh:gofips140-v1.0.0: TestLinker failures
72675 x/tools/gopls/internal/telemetry: TestSettingTelemetry failures
72664 cmd/link: TestDWARFiOS/c-archive/testprogcgo failures
72640 cmd/link: TestGNUBuildID/default failures
72630 cmd/link: TestDWARF/testprog failures
72537 cmd/link/internal/ld: TestVarDeclLine/line-directive failures
72389 x/tools/gopls/internal/telemetry: TestSettingTelemetry failures
72378 cmd/link: TestDWARFiOS/c-archive/testprogcgo failures
72344 cmd/link: TestDWARF/testprog failures
72354 cmd/link: TestGNUBuildID/default failures
72251 cmd/link/internal/ld: TestVarDeclLine/line-directive failures
72724 runtime:cpu4: TestRuntimeLockMetricsAndProfile/runtime.lock/sample-1 failures
72558 cmd/internal/moddeps: TestAllDependencies/cmd(quick) failures
72500 cmd/internal/testdir: Test/fixedbugs/issue71226.go failures
72497 runtime: TestRuntimeLockMetricsAndProfile/runtime.lock/sample-1 failures
72489 cmd/link: TestMachOBuildVersion failures
72451 x/net/http2: TestServerWritesTrailers_WithoutFlush failures
72438 runtime:cpu4: TestRuntimeLockMetricsAndProfile/runtime.lock/sample-1 failures
72272 cmd/internal/moddeps: TestAllDependencies/cmd(quick) failures
72214 cmd/internal/testdir: Test/fixedbugs/issue71226.go failures
72211 runtime: TestRuntimeLockMetricsAndProfile/runtime.lock/sample-1 failures
72203 cmd/link: TestMachOBuildVersion failures
72165 x/net/http2: TestServerWritesTrailers_WithoutFlush failures
72723 cmd/cgo/internal/testsanitizers: TestLSAN/lsan3 failures
72722 cmd/cgo/internal/testsanitizers: TestLSAN/lsan2 failures
72646 cmd/vet: TestVet/lostcancel failures
72583 cmd/link/internal/ld: TestDeadcode/ifacemethod2 failures
72557 cmd/link: TestSectionsWithSameName failures
72516 cmd/link/internal/ld: TestDeadcode/ifacemethod5 failures
72437 cmd/cgo/internal/testsanitizers: TestLSAN/lsan3 failures
72436 cmd/cgo/internal/testsanitizers: TestLSAN/lsan2 failures
72360 cmd/vet: TestVet/lostcancel failures
72297 cmd/link/internal/ld: TestDeadcode/ifacemethod2 failures
72271 cmd/link: TestSectionsWithSameName failures
72230 cmd/link/internal/ld: TestDeadcode/ifacemethod5 failures
72659 os: TestGetwdDeepWithPWDSet failures
72493 make.bash: unrecognized failures
72373 os: TestGetwdDeepWithPWDSet failures
72207 make.bash: unrecognized failures
72645 cmd/link/internal/ld: TestArchiveBuildInvokeWithExec failures
72568 cmd/link: TestCheckLinkname/fastrand.go failures
72550 cmd/link/internal/ld: TestElfBindNow/bindnow-linkmode-internal failures
72539 internal/trace: TestTraceStressStartStop/Default failures
72531 x/tools/gopls/internal/filecache: TestBasics failures
72461 cmd/go/internal/vcweb/vcstest: TestScripts/bzr/hello.txt failures
72359 cmd/link/internal/ld: TestArchiveBuildInvokeWithExec failures
72282 cmd/link: TestCheckLinkname/fastrand.go failures
72264 cmd/link/internal/ld: TestElfBindNow/bindnow-linkmode-internal failures
72245 x/tools/gopls/internal/filecache: TestBasics failures
72253 internal/trace: TestTraceStressStartStop/Default failures
72175 cmd/go/internal/vcweb/vcstest: TestScripts/bzr/hello.txt failures
72666 internal/trace: TestTraceStress/AllocFree failures
72599 build: build failure on x_tools-go1.24-linux-riscv64
72594 build: build failure on x_telemetry-go1.24-linux-riscv64
72487 build: build failure on x_tools-gotip-linux-riscv64
72481 crypto/x509:gofips140-v1.0.0: TestImports failures
72380 internal/trace: TestTraceStress/AllocFree failures
72313 build: build failure on x_tools-go1.24-linux-riscv64
72308 build: build failure on x_telemetry-go1.24-linux-riscv64
72195 crypto/x509:gofips140-v1.0.0: TestImports failures
72201 build: build failure on x_tools-gotip-linux-riscv64
72680 build: build failure on x_tools-gotip-darwin-amd64_14
72679 build: build failure on x_tools-go1.24-darwin-amd64-longtest
72677 build: build failure on x_build-gotip-darwin-amd64-race
72586 cmd/go/internal/vcweb/vcstest: TestScripts/git/v2sub.txt failures
72552 build: build failure on x_website-go1.24-darwin-amd64_13
72541 cmd/go/internal/vcweb/vcstest: TestScripts/git/querytest.txt failures
72536 cmd/internal/testdir: Test/fixedbugs/issue19699.go failures
72520 cmd/internal/testdir: Test/fixedbugs/issue66663.go failures
72528 cmd/internal/testdir: Test/typeparam/issue51836.go failures
72518 cmd/internal/testdir: Test/fixedbugs/issue20174.go failures
72513 cmd/link: TestStrictDup failures
72476 cmd/internal/testdir: Test/fixedbugs/issue16037_run.go failures
72464 cmd/go/internal/vcweb/vcstest: TestScripts/git/gitrepo1.txt failures
72447 cmd/go/internal/vcweb/vcstest: TestScripts/git/vgotest1.txt failures
72446 cmd/vet: TestVet/method failures
72441 cmd/go/internal/vcweb/vcstest: TestScripts/git/tagtests.txt failures
72394 build: build failure on x_tools-gotip-darwin-amd64_14
72393 build: build failure on x_tools-go1.24-darwin-amd64-longtest
72391 build: build failure on x_build-gotip-darwin-amd64-race
72300 cmd/go/internal/vcweb/vcstest: TestScripts/git/v2sub.txt failures
72266 build: build failure on x_website-go1.24-darwin-amd64_13
72250 cmd/internal/testdir: Test/fixedbugs/issue19699.go failures
72255 cmd/go/internal/vcweb/vcstest: TestScripts/git/querytest.txt failures
72242 cmd/internal/testdir: Test/typeparam/issue51836.go failures
72234 cmd/internal/testdir: Test/fixedbugs/issue66663.go failures
72232 cmd/internal/testdir: Test/fixedbugs/issue20174.go failures
72227 cmd/link: TestStrictDup failures
72190 cmd/internal/testdir: Test/fixedbugs/issue16037_run.go failures
72178 cmd/go/internal/vcweb/vcstest: TestScripts/git/gitrepo1.txt failures
72161 cmd/go/internal/vcweb/vcstest: TestScripts/git/vgotest1.txt failures
72160 cmd/vet: TestVet/method failures
72155 cmd/go/internal/vcweb/vcstest: TestScripts/git/tagtests.txt failures
72678 build: build failure on x_build-go1.24-darwin-amd64-longtest
72605 cmd/link: TestScript/randlayout_option failures
72602 cmd/link/internal/ld: TestDeadcode/typedesc failures
72566 cmd/go/internal/vcweb/vcstest: TestScripts/git/no-tags.txt failures
72546 cmd/go/internal/vcweb/vcstest: TestScripts/git/issue61415.txt failures
72496 cmd/go/internal/vcweb/vcstest: TestScripts/git/prefixtagtests.txt failures
72503 cmd/go/internal/vcweb/vcstest: TestScripts/git/semver-branch.txt failures
72471 cmd/go/internal/vcweb/vcstest: TestScripts/git/odd-tags.txt failures
72392 build: build failure on x_build-go1.24-darwin-amd64-longtest
72319 cmd/link: TestScript/randlayout_option failures
72316 cmd/link/internal/ld: TestDeadcode/typedesc failures
72280 cmd/go/internal/vcweb/vcstest: TestScripts/git/no-tags.txt failures
72260 cmd/go/internal/vcweb/vcstest: TestScripts/git/issue61415.txt failures
72217 cmd/go/internal/vcweb/vcstest: TestScripts/git/semver-branch.txt failures
72210 cmd/go/internal/vcweb/vcstest: TestScripts/git/prefixtagtests.txt failures
72185 cmd/go/internal/vcweb/vcstest: TestScripts/git/odd-tags.txt failures
72689 x/tools/gopls/internal/telemetry/cmd/stacks: TestUpdateIssues/should_be_reopened failures
72662 cmd/link: TestBuildForTvOS failures
72629 cmd/link: TestGNUBuildID/default_external failures
72607 x/tools/gopls/internal/test/integration/misc: TestRenamePackage/default failures
72585 internal/trace: TestTraceCPUProfile/Default failures
72519 cmd/link: TestScript/script_test_basics failures
72527 x/tools/gopls/internal/telemetry/cmd/stacks: TestUpdateIssues/open_issue failures
72502 cmd/go/internal/vcweb/vcstest: TestScripts/git/commit-after-tag.txt failures
72457 x/tools/go/analysis/internal/checker: unrecognized failures
72403 x/tools/gopls/internal/telemetry/cmd/stacks: TestUpdateIssues/should_be_reopened failures
72376 cmd/link: TestBuildForTvOS failures
72343 cmd/link: TestGNUBuildID/default_external failures
72321 x/tools/gopls/internal/test/integration/misc: TestRenamePackage/default failures
72299 internal/trace: TestTraceCPUProfile/Default failures
72241 x/tools/gopls/internal/telemetry/cmd/stacks: TestUpdateIssues/open_issue failures
72233 cmd/link: TestScript/script_test_basics failures
72216 cmd/go/internal/vcweb/vcstest: TestScripts/git/commit-after-tag.txt failures
72171 x/tools/go/analysis/internal/checker: unrecognized failures
72683 x/net/http2: TestServerMaxHandlerGoroutines failures [consistent failure]
72681 cmd/compile: TestScript/issue70173 failures
72610 cmd/link: TestLargeReloc failures
72572 cmd/vet: TestVet/asm failures
72553 internal/platform: TestGenerated failures
72507 cmd/go/internal/modfetch: TestCodeRepo/swtch.com_testmod/v1.0.0 failures
72459 x/tools/gopls/internal/telemetry/cmd/stacks: TestReadPCLineTable/gopls failures
72397 x/net/http2: TestServerMaxHandlerGoroutines failures [consistent failure]
72395 cmd/compile: TestScript/issue70173 failures
72324 cmd/link: TestLargeReloc failures
72286 cmd/vet: TestVet/asm failures
72267 internal/platform: TestGenerated failures
72221 cmd/go/internal/modfetch: TestCodeRepo/swtch.com_testmod/v1.0.0 failures
72173 x/tools/gopls/internal/telemetry/cmd/stacks: TestReadPCLineTable/gopls failures
72720 cmd/cgo/internal/testsanitizers: TestLSAN/lsan1 failures
72718 x/tools/go/loader: TestStdlib failures
72598 cmd/go: TestScript/gotoolchain_net failures
72595 cmd/compile: TestScript/script_test_basics failures
72603 cmd/cgo/internal/test: TestCthread failures
72495 os/exec: TestContextCancel failures
72455 x/net/http2: TestServer_Rejects_Too_Many_Streams failures
72434 cmd/cgo/internal/testsanitizers: TestLSAN/lsan1 failures
72432 x/tools/go/loader: TestStdlib failures
72317 cmd/cgo/internal/test: TestCthread failures
72312 cmd/go: TestScript/gotoolchain_net failures
72309 cmd/compile: TestScript/script_test_basics failures
72209 os/exec: TestContextCancel failures
72169 x/net/http2: TestServer_Rejects_Too_Many_Streams failures
72694 x/crypto/ssh/test: TestRunCommandSuccess failures
72691 x/tools/gopls/internal/cmd: TestImplementations failures
72606 strings: unrecognized failures
72523 crypto/tls: TestFIPSCertAlgs failures
72511 x/crypto/ssh/test: TestDialUnix failures
72408 x/crypto/ssh/test: TestRunCommandSuccess failures
72405 x/tools/gopls/internal/cmd: TestImplementations failures
72320 strings: unrecognized failures
72237 crypto/tls: TestFIPSCertAlgs failures
72225 x/crypto/ssh/test: TestDialUnix failures
72592 runtime: TestMemPprof failures
72575 reflect:mayMoreStackMove: TestChanOfGC failures
72306 runtime: TestMemPprof failures
72289 reflect:mayMoreStackMove: TestChanOfGC failures
72709 internal/coverage/cfile: TestCoverageApis/emitWithCounterClear failures
72593 x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd failures
72604 cmd/link: TestFlagS failures
72522 x/tools/gopls/internal/test/integration/misc: TestRenamePackage_TestVariant/default failures
72423 internal/coverage/cfile: TestCoverageApis/emitWithCounterClear failures
72318 cmd/link: TestFlagS failures
72307 x/benchmarks/sweet/cmd/sweet: TestSweetEndToEnd failures
72236 x/tools/gopls/internal/test/integration/misc: TestRenamePackage_TestVariant/default failures
72725 cmd/go: TestScript/test_fuzz_match failures
72687 cmd/go: TestScript/toolexec failures
72655 build: build failure on go1.23-linux-arm64_c4ah72-perf_vs_release
72609 build: build failure on go1.24-linux-arm64_c4ah72-perf_vs_release
72571 cmd/go: TestScript/mod_gonoproxy failures
72561 cmd/go: TestScript/mod_get_direct failures
72556 cmd/objdump: TestDisasm failures
72508 cmd/go: TestScript/cover_statements failures
72474 build: build failure on gotip-linux-arm64_c4as16-perf_vs_release
72473 build: build failure on gotip-linux-amd64_c3h88-perf_vs_release
72472 build: build failure on gotip-linux-amd64_c2s16-perf_vs_release
72448 build: build failure on gotip-linux-arm64_c4ah72-perf_vs_release
72439 cmd/go: TestScript/test_fuzz_match failures
72401 cmd/go: TestScript/toolexec failures
72369 build: build failure on go1.23-linux-arm64_c4ah72-perf_vs_release
72323 build: build failure on go1.24-linux-arm64_c4ah72-perf_vs_release
72285 cmd/go: TestScript/mod_gonoproxy failures
72275 cmd/go: TestScript/mod_get_direct failures
72270 cmd/objdump: TestDisasm failures
72222 cmd/go: TestScript/cover_statements failures
72188 build: build failure on gotip-linux-arm64_c4as16-perf_vs_release
72187 build: build failure on gotip-linux-amd64_c3h88-perf_vs_release
72186 build: build failure on gotip-linux-amd64_c2s16-perf_vs_release
72162 build: build failure on gotip-linux-arm64_c4ah72-perf_vs_release
72706 cmd/go: TestScript/test_fuzz_fuzztime failures
72669 cmd/internal/bootstrap_test: TestRepeatBootstrap failures
72626 x/tools/go/packages: TestNewPackagesInOverlay/Modules/multiple_files_three_packages failures
72526 x/tools/cmd/signature-fuzzer/fuzz-runner: TestRunner/Minimization2 failures
72494 cmd/go: TestScript/mod_cache_dir failures
72470 cmd/go/internal/modfetch/codehost: TestReadZip/localGitRepo/aaaaaaaaab/ failures
72420 cmd/go: TestScript/test_fuzz_fuzztime failures
72383 cmd/internal/bootstrap_test: TestRepeatBootstrap failures
72340 x/tools/go/packages: TestNewPackagesInOverlay/Modules/multiple_files_three_packages failures
72240 x/tools/cmd/signature-fuzzer/fuzz-runner: TestRunner/Minimization2 failures
72208 cmd/go: TestScript/mod_cache_dir failures
72184 cmd/go/internal/modfetch/codehost: TestReadZip/localGitRepo/aaaaaaaaab/ failures
72698 x/tools/cmd/stringer: TestMultifileStringer/x_test_package_only failures
72695 crypto/x509:gofips140: TestConstraintCases/#0 failures
72688 cmd/compile/internal/test: TestPGODevirtualize failures
72670 cmd/go: TestScript/mod_pseudo_cache failures
72633 cmd/go: TestScript/modfile_flag failures
72569 cmd/go: TestScript/get_404_meta failures
72548 cmd/go: TestScript/mod_sumdb_golang failures
72544 cmd/go/internal/modfetch/codehost: TestReadFile/localGitRepo/latest/README failures
72551 cmd/link: TestCheckLinkname/badlinkname.go failures
72534 x/tools/cmd/stringer: TestTestFiles failures
72510 x/tools/gopls/internal/test/integration/workspace: TestMultiView_LocalReplace/default failures
72478 cmd/link: TestMinusRSymsWithSameName failures
72467 cmd/go: TestScript/version_buildvcs_bzr failures
72466 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/good/default/default failures
72450 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/bad/nested/default failures
72453 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/bad/default/default failures
72412 x/tools/cmd/stringer: TestMultifileStringer/x_test_package_only failures
72409 crypto/x509:gofips140: TestConstraintCases/#0 failures
72402 cmd/compile/internal/test: TestPGODevirtualize failures
72384 cmd/go: TestScript/mod_pseudo_cache failures
72347 cmd/go: TestScript/modfile_flag failures
72283 cmd/go: TestScript/get_404_meta failures
72265 cmd/link: TestCheckLinkname/badlinkname.go failures
72262 cmd/go: TestScript/mod_sumdb_golang failures
72248 x/tools/cmd/stringer: TestTestFiles failures
72258 cmd/go/internal/modfetch/codehost: TestReadFile/localGitRepo/latest/README failures
72224 x/tools/gopls/internal/test/integration/workspace: TestMultiView_LocalReplace/default failures
72192 cmd/link: TestMinusRSymsWithSameName failures
72181 cmd/go: TestScript/version_buildvcs_bzr failures
72180 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/good/default/default failures
72167 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/bad/default/default failures
72164 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/bad/nested/default failures
72699 cmd/go: TestScript/test_fuzz_deadline failures
72621 cmd/go: TestScript/mod_get_exec_toolchain failures
72631 build: build failure on gotip-linux-ppc64_power8
72591 net/http: TestServerCancelsReadTimeoutWhenIdle/h2 failures
72581 cmd/link: TestIndexMismatch failures
72576 cmd/go: TestScript/mod_download_exec_toolchain failures
72555 cmd/go: TestScript/mod_sumdb_file_path failures
72501 x/tools/gopls/internal/test/integration/codelens: TestUpgradeCodelens_Workspace/Upgrade_individual_dependency_vendoring=false/default failures
72442 x/build/maintner/godata: TestCorpusCheck failures
72413 cmd/go: TestScript/test_fuzz_deadline failures
72345 build: build failure on gotip-linux-ppc64_power8
72335 cmd/go: TestScript/mod_get_exec_toolchain failures
72305 net/http: TestServerCancelsReadTimeoutWhenIdle/h2 failures
72295 cmd/link: TestIndexMismatch failures
72290 cmd/go: TestScript/mod_download_exec_toolchain failures
72269 cmd/go: TestScript/mod_sumdb_file_path failures
72215 x/tools/gopls/internal/test/integration/codelens: TestUpgradeCodelens_Workspace/Upgrade_individual_dependency_vendoring=false/default failures
72156 x/build/maintner/godata: TestCorpusCheck failures
72708 x/tools/cmd/stringer: TestEndToEnd/conv2.go failures
72685 x/tools/gopls/internal/test/integration/codelens: TestUpgradeCodelens_Workspace/Upgrade_transitive_dependencies/default failures
72635 x/tools/gopls/internal/test/integration/codelens: TestUnusedDependenciesCodelens/default failures
72530 x/tools/gopls/internal/cmd: TestCodeAction failures
72486 x/tools/gopls/internal/test/integration/codelens: TestUpgradeCodelens_Workspace/Upgrade_individual_dependency_vendoring=true/default failures
72484 x/tools/gopls/internal/test/integration/modfile: TestBadlyVersionedModule/default/default failures
72449 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/good/nested/default failures
72456 x/tools/gopls/internal/test/integration/modfile: TestGovimModReadonly/default failures
72440 x/tools/gopls/internal/test/integration/modfile: TestBadlyVersionedModule/nested/default failures
72422 x/tools/cmd/stringer: TestEndToEnd/conv2.go failures
72399 x/tools/gopls/internal/test/integration/codelens: TestUpgradeCodelens_Workspace/Upgrade_transitive_dependencies/default failures
72349 x/tools/gopls/internal/test/integration/codelens: TestUnusedDependenciesCodelens/default failures
72244 x/tools/gopls/internal/cmd: TestCodeAction failures
72200 x/tools/gopls/internal/test/integration/codelens: TestUpgradeCodelens_Workspace/Upgrade_individual_dependency_vendoring=true/default failures
72198 x/tools/gopls/internal/test/integration/modfile: TestBadlyVersionedModule/default/default failures
72170 x/tools/gopls/internal/test/integration/modfile: TestGovimModReadonly/default failures
72163 x/tools/gopls/internal/test/integration/modfile: TestUnknownRevision/good/nested/default failures
72154 x/tools/gopls/internal/test/integration/modfile: TestBadlyVersionedModule/nested/default failures
72641 cmd/go/internal/modfetch/codehost: TestReadZip/gitrepo1/v2.3.4/ failures
72601 x/tools/gopls/internal/test/integration/codelens: TestDisablingCodeLens/default/default failures [consistent failure]
72584 cmd/go: TestScript/mod_tidy_diff failures
72580 crypto/internal/fips140test:gofips140: TestACVP failures
72579 crypto/internal/fips140test: TestACVP failures
72564 internal/coverage/cfile: TestCoverageApis/emitToDir failures
72355 cmd/go/internal/modfetch/codehost: TestReadZip/gitrepo1/v2.3.4/ failures
72315 x/tools/gopls/internal/test/integration/codelens: TestDisablingCodeLens/default/default failures [consistent failure]
72298 cmd/go: TestScript/mod_tidy_diff failures
72294 crypto/internal/fips140test:gofips140: TestACVP failures
72293 crypto/internal/fips140test: TestACVP failures
72278 internal/coverage/cfile: TestCoverageApis/emitToDir failures
72596 cmd/go/internal/modfetch/codehost: TestReadFile/localGitRepo/v2.3.4/another.txt failures
72532 cmd/go: TestScript/test_cleanup_failnow failures
72529 cmd/go: TestScript/build_cwd_newline failures
72512 cmd/link: TestExtLinkCmdlineDeterminism failures
72498 cmd/go/internal/modfetch/codehost: TestReadZip/vgotest1/submod/v1.0.4/submod failures
72465 cmd/go/internal/modfetch/codehost: TestReadZip/localGitRepo/v3/v3/sub/dir failures
72310 cmd/go/internal/modfetch/codehost: TestReadFile/localGitRepo/v2.3.4/another.txt failures
72246 cmd/go: TestScript/test_cleanup_failnow failures
72243 cmd/go: TestScript/build_cwd_newline failures
72226 cmd/link: TestExtLinkCmdlineDeterminism failures
72212 cmd/go/internal/modfetch/codehost: TestReadZip/vgotest1/submod/v1.0.4/submod failures
72179 cmd/go/internal/modfetch/codehost: TestReadZip/localGitRepo/v3/v3/sub/dir failures
72533 cmd/go: TestScript/test_fuzz_return failures
72247 cmd/go: TestScript/test_fuzz_return failures
72542 cmd/go/internal/modfetch/codehost: TestReadFile/gitrepo1/latest/README failures
72256 cmd/go/internal/modfetch/codehost: TestReadFile/gitrepo1/latest/README failures
72499 cmd/go/internal/modfetch/codehost: TestReadFile/localGitRepo/v2/another.txt failures
72213 cmd/go/internal/modfetch/codehost: TestReadFile/localGitRepo/v2/another.txt failures
72559 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/empty/default failures
72273 x/tools/gopls/internal/test/integration/misc: TestTelemetryPrompt_Response/empty/default failures
72554 cmd/go/internal/modfetch/codehost: TestReadZip/localGitRepo/v2.3.4/ failures
72268 cmd/go/internal/modfetch/codehost: TestReadZip/localGitRepo/v2.3.4/ failures
72684 x/build/internal/relui: TestRelease/rc failures
72398 x/build/internal/relui: TestRelease/rc failures
72549 cmd/link: TestResponseFile failures
72263 cmd/link: TestResponseFile failures
deleted 572 issues
bulkdeleteissue $ unset GH_TOKEN_ISSUE_72731
bulkdeleteissue $ echo $GH_TOKEN_ISSUE_72731
bulkdeleteissue $
*/
package main
import (
"context"
"flag"
"fmt"
"log"
"os"
"time"
"github.com/google/go-github/v69/github"
"github.com/shurcooL/githubv4"
"golang.org/x/oauth2"
)
var deleteOnRemoteServer = flag.Bool("delete-on-remote-server", false, "actually delete issues from the remote server")
func main() {
flag.Parse()
if flag.NArg() != 0 {
flag.Usage()
os.Exit(2)
}
err := run(context.Background())
if err != nil {
log.Fatalln(err)
}
}
func run(ctx context.Context) error {
tokSrc := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: os.Getenv("GH_TOKEN_ISSUE_72731")})
httpCl := oauth2.NewClient(ctx, tokSrc)
ghV3, ghV4 := github.NewClient(httpCl), githubv4.NewClient(httpCl)
const query = `is:issue author:gopherbot created:2025-03-07 closed:2025-03-07 watchflakes "Issue created automatically to collect these failures."`
const wantTotal = 572
start := time.Date(2025, 3, 7, 9, 40, 0, 0, time.UTC)
end := start.Add(2 * time.Hour)
var issues []*github.Issue
var opt = github.SearchOptions{ListOptions: github.ListOptions{PerPage: 100}}
for i := range 6 {
opt.Page = i + 1
r, _, err := ghV3.Search.Issues(ctx, query, &opt)
if err != nil {
return err
}
if r.GetTotal() != wantTotal {
return fmt.Errorf("got %d total, want %d", r.GetTotal(), wantTotal)
} else if r.GetIncompleteResults() {
return fmt.Errorf("got incomplete results")
}
issues = append(issues, r.Issues...)
}
if len(issues) != wantTotal {
return fmt.Errorf("got %d issues, want %d", len(issues), wantTotal)
}
var deleted []int
for _, i := range issues {
const gopherbotUserID = int64(8566911)
if got, want := i.GetUser().GetID(), gopherbotUserID; got != want {
return fmt.Errorf("issue %d opened by user ID %d, want %d", i.GetNumber(), got, want)
}
created := i.GetCreatedAt().Time
if created.Before(start) || created.After(end) {
return fmt.Errorf("issue %d created at %v, want between %v and %v", i.GetNumber(), created, start, end)
}
closedAt := i.GetClosedAt().Time
if closedAt.Before(start.Add(5*time.Hour)) || closedAt.After(end.Add(5*time.Hour)) {
return fmt.Errorf("issue %d closed at %v, want between %v and %v", i.GetNumber(), closedAt, start.Add(5*time.Hour), end.Add(5*time.Hour))
}
fmt.Println(i.GetNumber(), i.GetTitle())
if *deleteOnRemoteServer {
if len(deleted) < 3 {
time.Sleep(30 * time.Second)
}
err := ghV4.Mutate(ctx, new(struct {
DeleteIssue struct {
ClientMutationID string // Unused; GraphQL doesn't allow for mutations to return nothing.
} `graphql:"deleteIssue(input: $input)"`
}), githubv4.DeleteIssueInput{
IssueID: i.GetNodeID(),
}, nil)
if err != nil {
return err
}
deleted = append(deleted, i.GetNumber())
time.Sleep(time.Second)
}
}
fmt.Printf("deleted %d issues\n", len(deleted))
return nil
}
Comment From: gopherbot
Change https://go.dev/cl/655996 mentions this issue: cmd/watchflakes: add a simple heuristic to not post too many new issues
Comment From: dmitshur
There's a GitHub incident involving issues today (https://www.githubstatus.com/incidents/9rfydl2xdqqj), and in the logs it's clear the small measure we took above is being helpful:
watchflakes: 2025/08/12 14:08:47 Backing off and then crashing out of abundance of caution: need to create 387 new issues, which might be a recurrence of go.dev/issue/72731; 141 boards, 1199 failures, 387 issues, in 13m4.626475163s
watchflakes: 2025/08/12 14:38:47 Now crashing out of abundance of caution: need to create 387 new issues, which might be a recurrence of go.dev/issue/72731; 141 boards, 1199 failures, 387 issues, in 13m4.626475163s