What version of Go are you using (go version)?
$ go version go version go1.18.5 linux/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
go env Output
$ go env GO111MODULE="" GOARCH="amd64" GOBIN="" GOCACHE="/home/mlaventure/.cache/go-build" GOENV="/home/mlaventure/.config/go/env" GOEXE="" GOEXPERIMENT="" GOFLAGS="" GOHOSTARCH="amd64" GOHOSTOS="linux" GOINSECURE="" GOMODCACHE="/home/mlaventure/go/pkg/mod" GONOPROXY="" GONOSUMDB="" GOOS="linux" GOPATH="/home/mlaventure/go" GOPRIVATE="" GOPROXY="direct" GOROOT="/usr/local/go" GOSUMDB="sum.golang.org" GOTMPDIR="" GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64" GOVCS="" GOVERSION="go1.18.5" GCCGO="gccgo" GOAMD64="v1" AR="ar" CC="gcc" CXX="g++" CGO_ENABLED="1" GOMOD="/home/mlaventure/tmp/go/workspace/realdeal/go.mod" GOWORK="/home/mlaventure/tmp/go/workspace/go.work" CGO_CFLAGS="-g -O2" CGO_CPPFLAGS="" CGO_CXXFLAGS="-g -O2" CGO_FFLAGS="-g -O2" CGO_LDFLAGS="-g -O2" PKG_CONFIG="pkg-config" GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2178506260=/tmp/go-build -gno-record-gcc-switches"
What did you do?
Given the following workspace structure:
~/tmp/go/workspace $ tree
.
├── fubar
│ ├── fubar.go
│ └── go.mod
├── go.work
└── realdeal
├── go.mod
├── realdeal
└── realdeal.go
2 directories, 6 files
With the following go.work:
~/tmp/go/workspace $ cat go.work
go 1.18
use ./fubar
use ./realdeal
replace github.com/foo/fubar => ./fubar
Build realdeal which depends on fubar:
~/tmp/go/workspace/realdeal $ go build .
What did you expect to see?
Nothing, the build should succeed
What did you see instead?
go: workspace module github.com/foo/fubar is replaced at all versions in the go.work file. To fix, remove the replacement from the go.work file or specify the version at which to replace the module.
Found workaround
The only way I've found to fix this is to update the go.work to be as follow:
~/tmp/go/workspace $ cat go.work
go 1.18
use ./fubar
use ./realdeal
replace github.com/foo/fubar v0.0.0-00010101000000-000000000000 => ./fubar
This is not needed if a workspace is not used. It also seems to contradict the example from the documentation (https://go.dev/ref/mod#workspaces):
replace (
golang.org/x/net v1.2.3 => example.com/fork/net v1.4.5
golang.org/x/net => example.com/fork/net v1.4.5
golang.org/x/net v1.2.3 => ./fork/net
golang.org/x/net => ./fork/net
)
Comment From: dmitshur
CC @bcmills, @matloob.
Comment From: BourgeoisBear
workspace replace lines also cause problems with go mod tidy
go: downloading mydomain.com/module v0.0.0-00010101000000-000000000000
github.com/acct/testcmd imports
mydomain.com/module: unrecognized import path "mydomain.com/module": https fetch: Get "https://mydomain.com/module?go-get=1": ...
Comment From: wangmir
@dmitshur @bcmills Is there any progress in this? I also have similar problem, when I trying to migrate existing project to use go workspace. It seems replace in go.work is not available to me too. I cannot run go mod tidy
Comment From: intel352
Same problem for me.
I presume this could be getting caused by the fact that I'm specifying the same local module in the use directive as in the replace directive, which I suspect is not intended behavior, but considering Go keeps trying to remotely fetch workspace modules during build rather than respecting the go.work, I was hoping the replace would solve the situation.
Frankly I'm surprised, most of the go tooling is generally high quality, but go.work seems quite error prone...
Comment From: bcmills
In general we would expect modules that are included in a workspace to not also require a replace directive in the go.work file. Can you explain more about the circumstance that led you to that arrangement?
Comment From: sudo-bmitch
The reason I was looking to do this was to handle an environment where I cannot directly connect to the upstream git repo. I'm seeing go execute git ls-remote -q origin ... (which fails in my environment) for the modules referenced with use but not with replace.
I can remove the use and only have a replace, which works, but only for libraries where I'm not running a go build.
Comment From: theaog
don't know where you came up w/ that long versioning string, fyi: v0.0.0 also works to satisfy go work complaints.
use ./cmd/fubar
replace github.com/foo/fubar v0.0.0 => ./cmd/fubar
to note that you shouldn't need the replace directive when in a workspace managed by go.work. it should automatically resolve your remote module github.com/foo/fubar to the local version available at ./cmd/fubar.
Comment From: sudo-bmitch
it should automatically resolve your remote module github.com/foo/fubar to the local version available at ./cmd/fubar.
It appears to do that by running git ls-remote -q origin ... which requires access to the remote git server.
Comment From: wangmir
@bcmills Please check this issue. I think it's related.
https://github.com/golang/go/issues/50750
go.work should support replace, when we think about monorepo situation. Placing replace on all of the modules of microservices in a monorepo is not ideal case i think.
Comment From: independentid
I am having the same problems in 1.19. Once I put in go.work everything goes haywire.
* Replace works in go.mod but can cause problems elsewhere in the workspace (Goland Issues suggestions to create a workspace)
* go.work - using use block causes go work sync to issue errors about missing versions for modules in the use clause. Using the long version number, a new number, or using v.0.0.0 will still generate the same errors
* Replace in go.work sort of solves some problems but the versioning issue remains if any module is an indirect requirement in another module.
Thinking about trying the latest go to see if the behavior changes.
Comment From: folays
The reason I was looking to do this was to handle an environment where I cannot directly connect to the upstream git repo. I'm seeing go execute
git ls-remote -q origin ...(which fails in my environment) for the modules referenced withusebut not withreplace.
As said above, since the beginning of go workspaces, and at least up to current Go 1.21.5 ;
Tree structure :
./module-a/go.mod
./module-a/module-a.go
./main-project/go.mod
./main-project/main.go
./main-project/go.work
CONTENT of module-A (no dependencies) :
$ cat module-a/go.mod
module gitlab.unpublished.private/modules/module-a.git
go 1.21.5
----8<----8<----8<----8<----8<----8<----8<----8<----8<
$ cat module-a/module-a.go
package module_a
import "fmt"
func ModuleA() {
fmt.Println("ModuleA()")
}
CONTENT of mains-project (depends on moduleA) :
$ cat main-project/go.mod
module main-project
go 1.21.5
require (
gitlab.unpublished.private/modules/module-a.git v0.9.9
)
----8<----8<----8<----8<----8<----8<----8<----8<----8<
$ cat main-project/main.go
package main
import "fmt"
import "gitlab.unpublished.private/modules/module-a.git"
func main() {
fmt.Println("main project")
module_a.ModuleA()
}
$ cat main-project/go.work
----8<----8<----8<----8<----8<----8<----8<----8<----8<
go 1.21.5
use (
.
../module-a
)
So, they are manually-crafted files, and the repos/gitlab doesn't event exists, go.mod.sum are missing, the v0.9.9 doesn't really exists, etc etc... but it reflects the SAME PROBLEM than really existing private git-cloned repos.
Inside ./main-project/, upon try to go build -v or go run -v, I encounter :
$ go build -v .
# cd /Users/folays/go/pkg/mod; git ls-remote https://gitlab.unpublished.private/modules/module-a
fatal: unable to access 'https://gitlab.unpublished.private/modules/module-a/': Could not resolve host: gitlab.unpublished.private
# cd /Users/folays/go/pkg/mod; git ls-remote git+ssh://gitlab.unpublished.private/modules/module-a
ssh: Could not resolve hostname gitlab.unpublished.private: nodename nor servname provided, or not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
# cd /Users/folays/go/pkg/mod; git ls-remote ssh://gitlab.unpublished.private/modules/module-a
ssh: Could not resolve hostname gitlab.unpublished.private: nodename nor servname provided, or not known
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
So the full text above is not really interesting. The MAIN POINT is that go it trying to git ls-remote unjoignable repos, which in real scenarios, I would myself git clone with my own means, and which I would only be able to git pull / push with my own means.
I would also like to specify, in aim to alleviate unneedy discussions, that for a real scenarios, my $HOME/.gitconfig would alias insteadOf = https://gitlab.unpublished.private/ to [url "ssh://git@gitlab.unpublished.private/"]
So, for the described scenario above, using only go.work's use ../PATH-to-immediate-dependency, Go WILL NOT build the main module, because it wants to git ls-remote the dep.
This WONT happen for even-dummy-er testbed scenarios where you would omit this from main-project/go.mod :
require (
gitlab.unpublished.private/modules/module-a.git v0.9.9
)
Indeed, if you only use local-only never-git-pushed tagless/unversionned dependencies, the problem would not occur.
If for some reasons (which seems valid to me), your main-project has go.mod containing requiring versioned dependencies (because, from time to time, you want to publish a new version of main-project to your org), then only using use ../PATH directives from go.work without replace directives won't work at all.
----8<----8<----8<----8<----8<----8<----8<----8<
I severely miss a simple method to having this workflow working seamlessly :
- fetch all org-private Go modules for which I may want to modify some of their code
- some of them having dependencies on some others
- fetch the org-private main-project which is use those dependencies
- having anybody able to just go get https://org-private/url/to/the/main-project.git
- having a local, private git-uncommited go.work which would just use . and use ../EACH/DEPENDENCIES and use ../EACH/SUBDEPENDENCIES
I mean, all the above works seamlessly, except for me, the dev, which needs to constantly jungle with some go.work, use, and replace directives.
This just doesn't work seamlessly, go.work use ../each-dependency does not suffice, I need to add replace everywhere, and when I finally want to commit everything I may have modified, I have to JUNGLE BACK to remove those replace directive for making the tools able to find and fill the go.mod files with coherent requires ... vVERSION before git push'ing everything.
----8<----8<----8<----8<----8<----8<----8<----8<
To me, it feels like the go.work stuff has been made with a majority of its usage being in spirit of using a "monorepo", but there is lot of juggling needing to be done to use it in a lot of cases.
Googling around on "go.work", "git ls-remote", "use", "ignored", "without replace", etc... yields a lot of results of confused people which seems to have a bad time figuring out a workflow which would suits to them.
I would just like a simple way to :
- locally fetch (OUTSIDE of Go handling) every go repos I may be willing to modify locally (on which I have authority to git push),
- have Go finds them the local directories to where I fetched them, with go.work
- being able to LOCALLY git-commit new versioned versions of dependencies which I modify
- being able to LOCALLY git-commit a new version of the main project, using the modified dependencies, WITHOUT FORCING ME beforehand to push the deps, WITHOUT TRYING to git ls-remote them at any time, since they are ALL POINTED BY go work's use directives
- when I'm satisfied on all the work done locally, I myself git push everything to the remote (sub dependencies, dependencies, main project).
That's just hard in Go, having to constantly jungle with replaces directives.
I cannot seem to have a fixed-once-figured-out-do-not-constantly-modify-replaces set of go.mod/go.work files where I would just git clone everything I need, modify my .go files locally, then update the dependencies in go.mod, and git push everything.
No, Go is willing to git ls-remote constantly, OR requires me to have some replace directives in go. files, which then would prevent go.mod to contains coherent requires DEPENDENCY vVERSION versions, which would make those go.mod (and the whole project/dependencies) unpublishable without first juggling with the go. directives, to make the Go tool produce coherent go.mod vVERSION requires.
Comment From: mweibel
I'd like to chime in too with the same issue and an additional potential bug (not sure if I should open a separate issue for that though). I felt it's most appropriate in this issue, but it's certainly also related to #50750 .
FTR we're using Go 1.21.
In our monorepo we gitignored go.work until recently. We made the switch because updating a dependent module was very cumbersome.
Imagine the following repo structure:
.
├── go.work # gitignored so just local
├── lib
│ ├── go.mod
│ └── lib.go
├── moduletwo # depends on lib
│ ├── go.mod
│ └── moduletwo.go
├── README.md
└── test
└── moduleone # depends on lib
├── go.mod
└── moduleone.go
A local Go workspace enabled us to efficiently make updates to multiple modules in one change and test them without having to update versions. Push to CI/CD however, requires an update to the versions (e.g. if module lib has an update the other modules depending on it need an updated version). This requires a separate commit just to update the revision in go.mod and making sure we don't need to rebase after that, since otherwise commit hashes change.
To avoid this, we decided to add go.work (not go.work.sum) to the repo. This seems to work only in some cases. In one module we encountered issues like these:
# linux CI runner
cmd/cmd.go:9:2: path/to/module/wrappers@v0.0.0-20240202141310-9ed56a5b95df: invalid version: unknown revision 9ed56a5b95df
# windows CI runner
path\cli\root.go:26:2: path/to/lib@v0.0.0-20231207112259-11b3c6dc08c9: invalid version: git ls-remote -q origin in C:\Windows\system32\config\systemprofile\go\pkg\mod\cache\vcs\927cbc5a0bf57c000feeddfdd18d1586382b2a626237337481dc6f6b33425a3a: exit status 128:
Unhandled Exception: System.ComponentModel.Win32Exception: The directory name is invalid
Removing go.work for this change in CI fixed the issue.
To get around this I tried to use replace directives just as outlined in this issue. A commited go.work plus each go.mod containing replace directives to replace the modules in the same folder structure with one above.
Repository with demo: https://github.com/mweibel/gomodreplace/tree/fail (please note, this is only a partial reproduction because we have an additional issue in our monorepo due to the fact that we are on an on-premise GitLab instance without a go module proxy in front).
The issue with this approach is visible when running go vet ./...:
$ go vet ./...
conflicting replacements found for github.com/mweibel/gomodreplace/lib@v0.0.0 in workspace modules defined by /tmp/gomodreplace/moduletwo/go.mod and /tmp/gomodreplace/test/moduleone/go.mod
Because moduletwo replaces lib as ../lib and moduleone replaces lib as ../../lib, go vet errors.
"Fixed" monorepo structure in main branch: https://github.com/mweibel/gomodreplace
This works because both moduleone and moduletwo replace lib in the same way.
I certainly might be doing something wrong and would love to get some help/documentation if that's the case.
Thank you!
Comment From: bcmills
There are a couple of solutions we could consider here:
- We could allow the replace directive as long as it names the same directory as what is in the use directive, so that the workspace module is “replaced with itself” but all prior versions are also replaced with what is in the workspace (so replace github.com/foo/fubar => ./fubar is allowed but must be a no-op).
- We could define that wildcard replace directives don't apply to the version of the module that is injected via the workspace (so replace github.com/foo/fubar => ./fubar is allowed but does not apply to the workspace module).
(CC @matloob @samthanawalla)
Comment From: rselph-tibco
Truly, this behavior makes go.work useless in a mono repo, and contradicts the documentation. Spent half the day trying to use the feature as documented, and getting progressively more frustrated. Found this thread and realized I should just abandon the effort, and go back to a maze of replace directives in the individual modules.
Could we at least ask for something in the documentation that clarifies what's required to make workspaces functional? That is to say, at least describe the behavior as it is to save this kind of frustration for others?
Comment From: matloob
@rselph-tibco Could you clarify what part of the documentation is inaccurate? I'd like to correct it to clarify that this isn't supported. Workspaces were not built to support modules that can't be built outside a workspace context.
Comment From: rselph-tibco
Hi @matloob, this issue isn't about modules that don't work outside of a workspace. It's about handling them reasonably in more complex situations such as a mono-repo.
The workspace doc is pretty short and sweet on the subject of the replace directive:
Similar to a replace directive in a go.mod file, a replace directive in a go.work file replaces the contents of a specific version of a module, or all versions of a module, with contents found elsewhere. A wildcard replace in go.work overrides a version-specific replace in a go.mod file.
replace directives in go.work files override any replaces of the same module or module version in workspace modules.
But the behavior discussed in this issue is an exception to this. Following the documentation in this situation leads to difficult to understand errors that seem to be in direct contradiction to the doc. For instance, I have a number of modules in one repo that build using replace directives pointing to another part of the source tree. They build fine outside of a workspace. The doc promises that I could instead have a single replace directive in my go.work that would avoid many different relative replace directives in the various go.mod files. But that doesn't work due to the issues outlined here.
Comment From: matloob
@rselph-tibco Okay, I can see a documentation problem: we should more clearly document that a workspace module can not be replaced using a replace directive in a go.work file (replacing at specific versions is supported for the purposes of tweaking the dependency graph, but should almost never need to be used. blanket replaces that apply to all versions are explicitly disallowed). The reason for this is that it's not clear what the behavior should be if a workspace module (which is at the root at the module graph) is replaced with a different module. (There is an analogue to this in single module mode: There is also weird behavior in a single-module context if that module replaces itself. We don't report an error in that case but we probably should.)
So I can add a line to the modules reference explicitly clarifying that workspaces can not be replaced using wildcard replaces.
But I don't think that solves your core problem: "handling [modules] reasonably in more complex situations such as a mono-repo". I might still be misunderstanding this, so bear with me, but it seems to me that the main case I've seen in this bug for replacing a workspace module in a go.work file is to circumvent fetching the module when it appears as a dependency of other workspace modules. My understanding of when this would happen (please correct me if I'm wrong here) is because the module is not able to be fetched either using git or a local module proxy. If that is the case the module would not be buildable outside of a workspace context (unless it replaces all of its dependencies!). Does that sound right to you?
If not, I would like to hear more about your use case.
Comment From: dpifke
One use case I have—which I think is similar to the one @rselph-tibco described—is when I have a series of modules that depend on each other, and I want to test a change to a module and those that depend on it locally, before pushing everything to a central Git server.
It'd be useful in that case to check out the module and a handful of packages that depend on it into a "workspace," and enforce that everything in that workspace should always use the local copy, without having to edit X individual go.mod files.
Using the present system, I've missed a file when adding my replace directives, and wondered why my change seemingly had no effect. I've also frequently forgotten to remove the temporary "replace" directive before pushing up my changes. Having a mechanism (go.work replace or otherwise) that took precedence over go.mod would make this workflow much easier and less error-prone.
Comment From: matloob
@dpifke After you push your modules to a central git server, is it possible to build the modules individually outside of a workspace? That is, are the modules able to be downloaded from git or a private proxy? If so, then your use case falls squarely under the set of use cases that go.work is meant to support. Your workspace should work without needing to replace any of the workspace modules. If the modules do work individually after pushing your changes, but you still need replace directives I'd like to know what issues you're running into.
Comment From: dpifke
~~My point is that I want go.work to work before I push my changes to the central Git server.~~
~~I want to use workspaces as follows:~~
~~1. I have workspace1 with a fix for bug1 in my module, which has been submitted as a PR to Github. There's a lengthy and contentious PR review, and I want to start working on a fix for bug2 in parallel.~~
~~2. I create a new workspace2, and do some local development which can't be merged until after the fix for bug1 lands.~~
~~3. I want to ensure the bug2 fix doesn't break any of a handful of downstream modules that import my package, possibly indirectly.~~
~~4. Right now, that means downloading each downstream module and editing its go.mod to point at my module in workspace2, then running its test suite.~~
~~5. I'd prefer to just download the downstream users of my module into workspace2, and have go.work handle the replacement for me—no copying-and-pasting of replace clauses into a bunch of separate go.mod files individually.~~
~~Maybe the fix for bug2 is experimental, and if it turns out to break downstream users, I plan to go back to the drawing board. In that case, I don't want it to ever be published in Git at all, because someone might accidentally pick a rejected work-in-progress!~~
Edit: re-reading the docs, I think it's clear this scenario is supposed to work. It's been almost two years since I subscribed to this bug, and my recollection is that replace in go.work didn't do anything and I had to edit a bunch of go.mod files instead, but maybe I was holding it wrong or there was another issue at play like #50750. When I have more time, I'll try to reproduce; until then, sorry for the noise.
Comment From: matloob
@dpifke If I understand your use case directly it should work. What doesn't work is replacing the workspace modules (the modules that are used in your workspace, but you shouldn't need to do that.
Comment From: Kentzo
I was also confused after reading the tutorial at https://go.dev/doc/tutorial/workspaces
I wanted to make some changes to a dependency of my module and set up a workspace. Since my module required a particular version and I wanted to test my changes with a newer version I though I had to add the replace statement to go.work.
Apparently that was not necessary as go disregarded the version mismatch and picked up my local version of the dependency from the workspace. I verified via go list -deps -f '{{.Dir}}' <workspace>/<main-module>/....
Comment From: Cidan
Hi, Xoogler (with go readability!) chiming in here on the outside now building out a monorepo.
All the complaints stated here are pretty much spot on across the board. go.work breaks in the sense that adding a new module forces the Go tooling to use git and try to get the remote, even if you've defined it locally in go.work.
It's even worse if you try to rename a module from one folder to another -- literally everything breaks until you push, meaning that you need to blindly push a major change before you can test it.
This has been incredibly frustrating to the point that I'm half considering moving to blaze/bazel just because I know this workflow is supported cleanly, even if it comes with all the pain of self managing bazel.
edit:
The fix here is much simpler than it seems I believe: when go.work has a use directive, always use that local folder and never call git to try to determine the module version and/or if the module exists, at all, under any circumstance. This fixes new and renamed modules.
That's it, and in-line with expected behavior.
edit 2:
Looks like #73654 does exactly this. Will wait for release!