Viper viper.Sub() not extracting nested default values

I noticed an issue where viper.Sub() fails to extract default values of a nested key. I attached demo code to reproduce the issue using latest viper and Go 1.12.6 under x64_linux.

main.go:

package main

import (
    "fmt"

    "github.com/davecgh/go-spew/spew"
    "github.com/spf13/viper"
)

func main() {
    viper.SetDefault("config.value2.internal", 3)
    viper.SetConfigFile("./demo.yaml")
    err := viper.ReadInConfig()
    if err != nil {
        panic(fmt.Errorf("Fatal error config file: %s ", err))
    }
    // Until now, everything works as expected
    fmt.Printf("config.value1: %d\n", viper.GetInt("config.value1"))
    fmt.Printf("config.value2: %d\n", viper.GetInt("config.value2"))
    fmt.Printf("config.value2.internal: %d\n", viper.GetInt("config.value2.internal"))

    // Extract the config subtree
    v := viper.Sub("config")
    // This doesn't work if value2 node is missing in configuration
    fmt.Printf("value2.internal: %d\n", v.GetInt("value2.internal"))

    spew.Dump(v.AllSettings())
}

demo.yaml (works as inteded, but defaults are useless since everything is explicitly set):

config:
  value1: 1
  value2: 
    internal: 3

demo.yaml (broken):

config:
  value1: 1
  #value2: 
  #  internal: 3

Output for the broken example:

config.value1: 1
config.value2: 0
config.value2.internal: 3
value2.internal: 0
(map[string]interface {}) (len=1) {
 (string) (len=6) "value1": (int) 1
}

Whereas I would expect to have value2.internal: 3

The issue seems somehow related / similar to issue #71 as well as PR #195

Comment From: aquincum

pretty impressive this hasn't been fixed in 3 years and it seems like it doesn't impact many...

Comment From: setrofim

I ran into related issue when Get()'ing a sub-tree that defaults specified for some of its keys. The issue is that when a value is specified for a path in the config, it effectively shadows the corresponding path in the defaults. But if the nested deeper path is not present in the set value, path resolution falls back to looking in the defaults. This results in potentially inconsistent behaviour when accessing a configuration point via its full path results in one value (the default), while accessing it via a relative path from a sub-tree results an that type's null value (nil, 0, "", etc).

Potential fix in https://github.com/spf13/viper/pull/1439

(note: this technically changes existing expected behaviour -- see the updated overrides_test.go; however, given this issue, and the referenced previous discussions, merging nested keys from defaults seems like desirable behaviour?)

Comment From: dp1140a

Well Im late to the party but this bit me in the ass. I have a function that sets defaults:

func (lc *LoggerConfig) RegisterDefaults(v *viper.Viper) {
    v.SetDefault("logging.loglevel", "INFO")
    v.SetDefault("logging.logFile", fmt.Sprintf("log/%v.log", pkg.APP_NAME))
    v.SetDefault("logging.stdOut", true)
    v.SetDefault("logging.fileOut", false)
    v.SetDefault("logging.format", "json")
    // no defaults for id/name here (they’re dynamic)
}

But at runtime I want these overriden from a toml config file.

[logging]
format="text"
loglevel="DEBUG"

That works . . . Somewhat. I can see the proper settings from my config file in Viper.config and I can see my defaults in Viper.defaults and if I do Viper.AllSettings() it seems to merge them:

logging:map[fileout:false format:text logfile:log/swarmos.log loglevel:DEBUG stdout:true]

But if I try Viper.Sub("logging"):

map[format:text loglevel:DEBUG]

And Viper.Unmarshal:

type LoggerConfig struct {
    LogFile  string `json:"logFile" toml:"logFile"`
    LogLevel string `mapstructure:"loglevel" json:"loglevel" toml:"loglevel"`
    StdOut   bool   `json:"stdOut" toml:"stdOut"`
    FileOut  bool   `json:"fileOut" toml:"fileOut"`
    Format   string `json:"format" toml:"format"`
}

cfg := &LoggerConfig{}
sub := viper.Sub(_MODULE)
if sub == nil {
    // If this happens, something is off with key names
    log.Error("viper.Sub(\"logging\") returned nil")
    return cfg
}
if err := sub.Unmarshal(cfg); err != nil {
    log.WithError(err).Error("failed to unmarshal logger config")
}

And then print cfg:

{"stdOut":true,"fileOut":false,"format":""}