Viper Nested . separated flags with common prefix not working with pflag in Viper >= 1.18.0

Preflight Checklist

[X] I have searched the issue tracker for an issue that matches the one I want to file, without success.
[X] I am not looking for support or already pursued the available support channels without success.
[X] I have checked the troubleshooting guide for my problem, without success.

Viper Version

1.18.0

Go Version

1.22.6

Config Source

Flags

Format

Other (specify below)

Repl.it link

No response

Code reproducing the issue

package main

import (
    "fmt"

    "github.com/spf13/pflag"
    "github.com/spf13/viper"
)

var (
    _ = pflag.String("gateway.endpoint.v1", "some/path/v1", "endpoint of gateway server v2")
    _ = pflag.String("gateway.endpoint", "some/path/", "endpoint of gateway server v3")
)

func init() {
    pflag.Parse()
    viper.BindPFlags(pflag.CommandLine)
    viper.AutomaticEnv()
}

func main() {
    fmt.Println("viper.GetString(\"gateway.endpoint\"):", viper.GetString("gateway.endpoint"))
    fmt.Println("viper.GetString(\"gateway.endpoint.v1\"):", viper.GetString("gateway.endpoint.v1")) // getting empty 
}

got output for this:
viper.GetString("gateway.endpoint"): some/path/
viper.GetString("gateway.endpoint.v1"):

Expected Behavior

it should print:

~ go run test.go                                                                    
viper.GetString("gateway.endpoint"): some/path/
viper.GetString("gateway.endpoint.v1"): some/path/v1

Actual Behavior

in v1.17.0 it is working as expected returning output as

~ go run test.go                                                                    
viper.GetString("gateway.endpoint"): some/path/
viper.GetString("gateway.endpoint.v1"): some/path/v1

Steps To Reproduce

 ✗ go env                                                                                  
GO111MODULE=''
GOARCH='arm64'
GOBIN=''
GOCACHE='/Users/spahk/Library/Caches/go-build'
GOENV='/Users/spahk/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMODCACHE='/Users/spahk/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/spahk/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/opt/homebrew/Cellar/go/1.22.4/libexec'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/opt/homebrew/Cellar/go/1.22.4/libexec/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.22.4'
GCCGO='gccgo'
AR='ar'
CC='cc'
CXX='c++'
CGO_ENABLED='1'
GOMOD='/Users/spahk/projects/test/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/jx/dpklpdss291clqs9t6xrh0c00000gp/T/go-build851702384=/tmp/go-build -gno-record-gcc-switches -fno-common'

go mod

module test

go 1.22.4

require (
    github.com/spf13/pflag v1.0.5
    github.com/spf13/viper v1.18.0
)

require (
    github.com/fsnotify/fsnotify v1.7.0 // indirect
    github.com/hashicorp/hcl v1.0.0 // indirect
    github.com/magiconair/properties v1.8.7 // indirect
    github.com/mitchellh/mapstructure v1.5.0 // indirect
    github.com/pelletier/go-toml/v2 v2.2.2 // indirect
    github.com/sagikazarmark/locafero v0.4.0 // indirect
    github.com/sagikazarmark/slog-shim v0.1.0 // indirect
    github.com/sourcegraph/conc v0.3.0 // indirect
    github.com/spf13/afero v1.11.0 // indirect
    github.com/spf13/cast v1.6.0 // indirect
    github.com/subosito/gotenv v1.6.0 // indirect
    go.uber.org/atomic v1.9.0 // indirect
    go.uber.org/multierr v1.9.0 // indirect
    golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
    golang.org/x/sys v0.18.0 // indirect
    golang.org/x/text v0.14.0 // indirect
    gopkg.in/ini.v1 v1.67.0 // indirect
    gopkg.in/yaml.v3 v3.0.1 // indirect
)

Additional Information

Updated: I can see it is working with another nested flags like _ = pflag.String("a.b.c.d.v1", "some/path/abc", "endpoint of gateway server v4") but i am wondering what is the special with key gateway.endpoint.*

Comment From: github-actions[bot]

👋 Thanks for reporting!

A maintainer will take a look at your issue shortly. 👀

In the meantime: We are working on Viper v2 and we would love to hear your thoughts about what you like or don't like about Viper, so we can improve or fix those issues.

⏰ If you have a couple minutes, please take some time and share your thoughts: https://forms.gle/R6faU74qPRPAzchZ9

📣 If you've already given us your feedback, you can still help by spreading the news, either by sharing the above link or telling people about this on Twitter:

https://twitter.com/sagikazarmark/status/1306904078967074816

Thank you! ❤️

Comment From: akshay8

I am also facing the same issue. I found a pattern where in two flags registered with same common prefix, then it is returning empty for the child flags. For example:

    _ = pflag.String("akshay.barik.v1", "akshay barik v1 value", "some v1 description")
    _ = pflag.String("akshay.barik", "akshay barik value", "some description")

    fmt.Println(viper.GetString("akshay.barik.v1"))         // getting empty
    fmt.Println( viper.GetString("akshay.barik"))

Comment From: shani1998

I am also facing the same issue. I found a pattern where in two flags registered with same common prefix, then it is returning empty for the child flags. For example:

``` _ = pflag.String("akshay.barik.v1", "akshay barik v1 value", "some v1 description") _ = pflag.String("akshay.barik", "akshay barik value", "some description")

fmt.Println(viper.GetString("akshay.barik.v1")) // getting empty fmt.Println( viper.GetString("akshay.barik"))
```

Thanks, Akshay, for noticing this pattern. I see the same pattern now. I initially thought it was specific to gateway.endpoint, but it seems like multiple nested flags with a common prefix are having issues in Viper 1.18.0. Hopefully, we’ll get an update on this soon.

Comment From: chengxilo

Maybe read this part would be helpful. "gateway.endpoint.v1" was shadowed by "gateway.endpoint" Viper Nested . separated flags with common prefix not working with pflag in Viper >= 1.18.0

Here is the source code cause this 'problem'. Pay attention to the v.isPathShadowedInFlatMap(path, v.pflags) would be helpful. https://github.com/spf13/viper/blob/8b223a45cef1badfd02317591a316095fb15a5d2/viper.go#L1202-L1204 https://github.com/spf13/viper/blob/8b223a45cef1badfd02317591a316095fb15a5d2/viper.go#L633-L675

It should not be treated as a bug.

Comment From: chengxilo

But I do think there should be some error when we use Pflag.String to set those keys, telling us that we should not use shadowed key.

Comment From: github-actions[bot]

Issues with no activity for 30 days are marked stale and subject to being closed.

Comment From: shani1998

But I do think there should be some error when we use Pflag.String to set those keys, telling us that we should not use shadowed key.

Agree, that would helpful to user to know upfront.