Golang x/tools/gopls: cannot choose the "right" value for completionBudget

What version of Go are you using (`go version`)?

$ go version
go version go1.21.1 linux/arm64
$ go list -m golang.org/x/tools
golang.org/x/tools v0.14.1-0.20231008020826-a3b5082fb05e
$ go list -m golang.org/x/tools/gopls
golang.org/x/tools/gopls v0.0.0-20231008020826-a3b5082fb05e

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

go env Output

$ go env
GO111MODULE='on'
GOARCH='arm64'
GOBIN=''
GOCACHE='/home/myitcv/.cache/go-build'
GOENV='/home/myitcv/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/myitcv/gostuff/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/myitcv/gostuff'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/myitcv/gos'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/myitcv/gos/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='go1.21.1'
GCCGO='gccgo'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/myitcv/gostuff/src/github.com/myitcv/govim/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3782877243=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Picking up the conversation from https://gophers.slack.com/archives/CRWSN9NCD/p1696429330059839

For the purposes of the discussion below, here are two example completions:

Example 1: a qualified identifier, unimported completion:

package main

func main() {
    load.Ins_
}

This should resolve to cuelang.org/go/cue/load.Instances.

Example 2: an unqualified completion:

package main

func main() {
    str_
}

My understanding based on the exchange with @findleyr on Slack of the current semantics around completionBudget is as follows:

completionBudget needs to be set to a "high" value in order that unimported completions (of the form taken in Example 1) don't time out, especially after a load from a cold cache but not limited to that. A large module cache in effect "forces" us to set a "high" value in this instance, because of the cost of (scanning and) searching the module cache.
completionBudget needs to be set to a "low" value in order that unqualified completions (of the form taken in Example 2) don't take a long time. Per @findleyr:

The problem is that this trivial completion is bundled together with a very non-trivial completion that scans the module cache.

The issue: if I choose a "high" value then Example 2 style completions can take ~200-250ms... which is very noticeable and annoying especially when the editor requests multiple in sequence as a completion candidate is refined. If I choose a "low" value, then I don't get unimported completions of Example 1 style in all situations.

I therefore cannot choose the "right" value.

What did you expect to see?

Being able to set a completionBudget that satisfies both cases, Examples 1 and 2.

What did you see instead?

The above.

Per @findleyr on Slack:

I think for starters we need to treat those two completions differently, so that you always get the trivial package name completion in identifier context, no matter how short your completionBudget is.

cc @findleyr

Comment From: gopherbot

Change https://go.dev/cl/560457 mentions this issue: gopls/internal/test/integration/bench: improve completion benchmarks

Comment From: gopherbot

Change https://go.dev/cl/560458 mentions this issue: internal/imports: cache canonizalization

Comment From: pjweinb

Unimported completions out of the module cache now use an index, so are pretty fast. Is this still an issue?

Comment From: myitcv

Please feel free to close if you wish. I've just flipped from Vim (using govim) to Neovim on the basis that we have largely archived govim now (https://github.com/govim/govim?tab=readme-ov-file#%EF%B8%8F-project-status-no-longer-actively-maintained), so I'm in the process of trying to re-establish a working configuration right now!

FYI @findleyr

cc @leitzler re govim