Viper Possibility to delete keys

Sometimes it might be necessary to delete/unset existing keys. For instance in my case, I want to use viper.WriteConfig to dump config to file, but I do want to filter out certain keys: I've tried the following:

viper.SetDefault("config", nil)
viper.Set("config", nil)

But it is interpreted as empty string in resulted config file. I also ran into https://stackoverflow.com/questions/52339336/removal-of-key-value-pair-from-viper-config-file but I wasn't able to make it work for a root level keys.

Comment From: sagikazarmark

The reason why it is not working is that there is no such key as config, but something like config.key, config.key2, etc.

What you can do is fetching all keys from a viper instance using AllKeys and setting all keys starting with config. to null, but I think that wouldn't work either.

Another option is getting all configuration with AllSettings and writing it to a file manually.

Please note that neither options will work with AutomaticEnv and env vars as it only works with direct access (using Get* functions).

Unfortunately implementing the feature you want is not trivial, because "unsetting" would trigger the next configuration source.

Comment From: dee-kryvenko

config is a valid root key in my case, something like:

config: /foo/bar
xxx: yyy

Basically I'm trying to maintain in-memory config based on inputs from viper and cobra, i.e. from CLI arguments, env variables and multiple config files. As part of CLI arguments or env variables you would be able to pass a path to the config file that later needs to be considered as part of the rest of the input.

When I want to change something in a config, such as similar to kubectl config use-context - I want to take what's in-memory, filter out some keys (such as why config path needs to be in the config itself?), and dump it to the file.

I were able to set the key to null, but when I try to save the file - the key ends up in the yaml with an empty value. Expected behavior would be to not to have that key at all.

Writing to file manually is the only option atm but it would be nice if I can reuse what's already in the library and not reinventing the wheel.

Theres two potential ways I can see this can be implemented. One is to go through every map in memory (config, override, defaults, etc...) and delete a key from there. Second is to improve AllSettings and interpret null value as "no key". It seems to break backward compatibility so might require a feature flag / option to be introduced.

Comment From: sagikazarmark

Writing to file manually is the only option atm but it would be nice if I can reuse what's already in the library and not reinventing the wheel.

Personally I think the current writing mechanism is flawed in so many ways, so I avoid using it whenever I can. I can only suggest doing the same. 😕

Comment From: dee-kryvenko

Could you explain please why do you think so? Any suggestions/alternatives? I'm pretty new to viper, well actually I'm pretty new to golang...

Comment From: anjannath

Yay!! There's already a patch for this, see https://github.com/spf13/viper/pull/519

Comment From: sagikazarmark

@llibicpep sorry, missed your answer.

Viper reads configuration from a number of sources. When you write the configuration, everything is written to a file. This poses several issues in itself.

For example, if you configure Viper to read secrets from a secret store and the rest from file, your secrets would be written too file as well. That's not what you usually want.

So using Viper for writing to file only makes sense, of you only use a file as source as well. Even then, you have limited access to the configuration itself (the issue itself proves that).

Based on what the use case is, I usually prefer using Viper when I need to configure an application, and use custom logic when I need to write files. Eg. use a separate Viper instance for file config. Or just read the config file manually, merge in viper, and write back the original config manually.

Take a look at my suggestions in my first comment as well. My advice: avoid writing with Viper. I'm actually going to propose removing config writing if a v2 Viper ever becomes a thing.

Comment From: anjannath

@llibicpep As a hack (if you are only using a config file) if you really want to use viper.WriteConfig you could do something as following:

func Unset(key string) error {
    configMap := viper.AllSettings()
    delete(configMap, key)
    encodedConfig, _ := json.MarshalIndent(configMap, "", " ")
    err := viper.ReadConfig(encodedConfig)
    if err != nil {
        return err
    }
    viper.WriteConfig()
}

Note: do proper error handling

Comment From: Jazun713

@llibicpep As a hack (if you are only using a config file) if you really want to use viper.WriteConfig you could do something as following:

go func Unset(key string) error { configMap := viper.AllSettings() delete(configMap, key) encodedConfig, _ := json.MarshalIndent(configMap, "", " ") err := viper.ReadConfig(encodedConfig) if err != nil { return err } viper.WriteConfig() }

Note: do proper error handling

I had to convert the byte[] here to a reader, otherwise, you receive: cannot use (type []byte) as type io.Reader in argument to viper.ReadConfig Here's the modified hack:

func Unset(key string) error {
    configMap := viper.AllSettings()
    delete(configMap, key)
    encodedConfig, _ := json.MarshalIndent(configMap, "", " ")
    err := viper.ReadConfig(bytes.NewReader(encodedConfig))
    if err != nil {
        return err
    }
    viper.WriteConfig()
}

Comment From: stevenh

A more complete version of Unset which deals with more than just root level items.

It's still not perfect as it will save default's and has no concept of where values came from so could save values from ENV to the config including secure vars, which isn't desireable.

func Unset(vars ...string) error {
        cfg := viper.AllSettings()
        vals := cfg

        for _, v := range vars {
                parts := strings.Split(v, ".")
                for i, k := range parts {
                        v, ok := vals[k]
                        if !ok {
                                // Doesn't exist no action needed
                                break
                        }

                        switch len(parts) {
                        case i + 1:
                                // Last part so delete.
                                delete(vals, k)
                        default:
                                m, ok := v.(map[string]interface{})
                                if !ok {
                                        return fmt.Errorf("unsupported type: %T for %q", v, strings.Join(parts[0:i], "."))
                                }
                                vals = m
                        }
                }
        }

        b, err := json.MarshalIndent(cfg, "", " ")
        if err != nil {
                return err
        }

        if err = viper.ReadConfig(bytes.NewReader(b)); err != nil {
                return err
        }

        return viper.WriteConfig()
}

Comment From: gusega

this is a really annoying thing, say I have a key which is a map, I want to remove it. How do I do that? What worked for me is when deleting to write map[string]string{} value and also create a custom getValue function that does this check to see if the value is there !viper.IsSet(key) || len(maps.Keys(viper.GetStringMapString(key))) == 0

Comment From: tangxinfa

A hack method:

    delABC := viper.New()
    delABC.MergeConfigMap(source.AllSettings())
    delABC.Set("a.b.c", struct{}{})
    noABC := viper.New()
    noABC.MergeConfigMap(delABC.AllSettings())

Comment From: linux019

I faced up with the same problem, this library is a pure trash, tons of issues