gopls version

I'd like to propose a discussion on adding support for an alternative method of loading the index for "Organize Imports" and unimported completions in gopls.

Background: The pre-gopls v0.19.0 implementation, ImportsSourceGoimports, relies on the packages.Load("./...") query executed by gopls at startup. While this works with third-party build systems, it is inefficient for large monorepos. Our gopackagesdriver is intentionally restricted from scanning the entire monorepo due to its size, as it would not complete in a reasonable time.

Current Situation: With the release of gopls v0.19.0, ImportsSourceGopls was introduced to scan the GOMODCACHE directory. However, this approach is incompatible with build systems like Buck or Bazel, which manage dependencies differently.

Proposal: I propose introducing ImportsSourceExternal, which would allow users to provide an external binary to generate the imports index. This would enable support for unimported symbol completions in our monorepo and could be particularly beneficial for Bazel users, as the deprecation of ImportsSourceGoimports may disrupt these features for them. The current index-file format is straightforward for an external tool to produce.

Implementation: To run gopls with a custom build system, users would need to set two environment variables: GOPACKAGESDRIVER and GOPLS_IMPORTS_SOURCE (example name).

Questions: - How does this idea resonate with you? - Have you considered similar approaches?

Looking forward to your thoughts and any additional insights you might have!

go env

~

What did you do?

~

What did you see happen?

~

What did you expect to see?

~

Editor and settings

No response

Logs

No response

Comment From: podtserkovskiy

cc @JamyDev You might be interested in using this with Bazel

Comment From: adonovan

I think the idea is reasonable, but we should definitely not expose the current index file format as an interface. It would be better to define a JSON request/response protocol over a pipe, with one message type per method of imports.Source. Then gopls would execute the command as a long-lived child process, allowing it to amortize any start-up overheads.

Comment From: podtserkovskiy

Yeah, this approach sounds better. We can try to implement a prototype later this year to test how it works in practice.

BTW, are you planning to deprecate packages.Load("./...") query after deprecation of ImportsSourceGoimports or this data is used for something else? Gopls does additional query when users open specific .go files, so it seems packages.Load("./...") isn't required for that, that's why I'm asking 😀

Comment From: adonovan

Yeah, this approach sounds better. We can try to implement a prototype later this year to test how it works in practice.

Great! I look forward to it.

BTW, are you planning to deprecate packages.Load("./...") query after deprecation of ImportsSourceGoimports or this data is used for something else?

This metadata query is fundamental to gopls: it's how it learns which files belong to which packages, and the dependency graph over them. It's not going away.

Comment From: podtserkovskiy

This metadata query is fundamental to gopls: it's how it learns which files belong to which packages, and the dependency graph over them. It's not going away.

That's interesting gopls still works on our repo without this data (but some features are unabailable, like unimported completions and find-all-references).

Comment From: adonovan

That's interesting gopls still works on our repo without this data

gopls only uses the ./... query when there are no go.mod files in the workspace, or when using a GOPACKAGESDRIVER. If there are go.mod files, it never executes this query.