Preflight Checklist
- [X] I have searched the issue tracker for an issue that matches the one I want to file, without success.
- [X] I am not looking for support or already pursued the available support channels without success.
- [X] I have checked the troubleshooting guide for my problem, without success.
Viper Version
1.19.0
Go Version
1.21
Config Source
Environment variables, Files
Format
INI
Repl.it link
https://replit.com/@jamestrotter4/Go
Code reproducing the issue
package main
import (
"errors"
"fmt"
"log"
"os"
"path/filepath"
"reflect"
"runtime"
"strings"
"github.com/go-playground/validator"
"github.com/spf13/viper"
)
// config struct
type Config struct {
Name string `mapstructure:"name" validate:"required"`
Nested Nested `mapstructure:"nested"`
}
type Nested struct {
Val string `mapstructure:"value" validate:"required"`
}
func main() {
var config Config
// set env for test
os.Setenv("NAME", "test")
os.Setenv("NESTED_VALUE", "test")
// get the path of the current file
_, b, _, _ := runtime.Caller(0)
basepath := filepath.Dir(b)
err := GetConfig(&config, basepath)
log.Println(err)
}
func GetConfig(config *Config, cwd string) error {
if reflect.ValueOf(config).Kind() != reflect.Ptr {
return errors.New("config must be a pointer")
}
//config should be in the same folder that config.go is called from
viper.AddConfigPath(cwd)
viper.SetConfigName("config")
viper.SetConfigType("ini")
viper.SetEnvKeyReplacer(strings.NewReplacer(`.`, `_`))
viper.AutomaticEnv()
// Will not load the config file in development, staging, and production environment. Instead will use configmap
if err := viper.ReadInConfig(); err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); ok {
log.Println(err)
log.Println("error loading server env config file (expected)")
}
}
err := viper.Unmarshal(config)
if err != nil {
return errors.New("unable to decode into struct")
}
// should log values
log.Println("name: ", config.Name)
log.Println("nested field: ", config.Nested.Val)
validate := validator.New()
if err := validate.Struct(config); err != nil {
fmt.Print(fmt.Sprintf("%v", err))
return errors.New("missing required attributes")
}
return nil
}
Expected Behavior
Expected to not throw error on validation and for name and nested.val to have a value in logs.
Actual Behavior
config in the end does not have any value, env seems to not be read.
Steps To Reproduce
- use version 1.19.0 of viper
- use above code (or similar, with AutomaticEnv)
- Expect to see no values added
Additional Information
Provided code works with viper version 1.18.0. Ideally I would not want to add BindEnv for every field I want to use. I've tried prefixing the env and other things like that thinking maybe it was missing something. In the end I was not able to get it working.
2024/07/31 12:18:16 Config File "config" Not Found in "[/home/runner/Go]"
2024/07/31 12:18:16 error loading server env config file (expected)
these logs are expected when there is no file. I expect to then get from ENV
Actual
2024/07/31 12:18:16 name:
2024/07/31 12:18:16 nested field:
Key: 'Config.Name' Error:Field validation for 'Name' failed on the 'required' tag
Key: 'Config.Nested.Val' Error:Field validation for 'Val' failed on the 'required' tag2024/07/31 12:18:16 missing required attributes
Expected:
2024/07/31 12:18:16 Config File "config" Not Found in "[/home/runner/Go]"
2024/07/31 12:18:16 error loading server env config file (expected)
2024/07/31 12:18:16 name: test
2024/07/31 12:18:16 nested field: test
Comment From: github-actions[bot]
👋 Thanks for reporting!
A maintainer will take a look at your issue shortly. 👀
In the meantime: We are working on Viper v2 and we would love to hear your thoughts about what you like or don't like about Viper, so we can improve or fix those issues.
⏰ If you have a couple minutes, please take some time and share your thoughts: https://forms.gle/R6faU74qPRPAzchZ9
📣 If you've already given us your feedback, you can still help by spreading the news, either by sharing the above link or telling people about this on Twitter:
https://twitter.com/sagikazarmark/status/1306904078967074816
Thank you! ❤️
Comment From: tandem97
Same problem. Downgrade to 1.18 solves this problem.
Comment From: vyas-git
Same issue in 1.19 unable to map env to struct.
I checked in changelog after v1.18.2 we need to use viper_bind_struct in tags .
try go run --tags=viper_bind_struct main.go . It works !!
Comment From: parnic
Confirmed, same thing happening here. Starting with version 1.18.2 and continuing into 1.19.0, the only way an environment variable override gets applied in my case is if a config file exists with the target key in it. If there's no config file, or it doesn't have the key I'm trying to override via env, its value isn't being pulled from the env var.
type AlertConfig struct {
NotificationType string `mapstructure:"NOTIFICATION_TYPE"`
NotificationURL string `mapstructure:"NOTIFICATION_URL"`
}
func loadConfig(path string) (config AlertConfig, err error) {
viper.AddConfigPath(path)
viper.SetConfigName("alerts")
viper.SetConfigType("env")
viper.AutomaticEnv()
err = viper.ReadInConfig()
if err != nil {
if _, ok := err.(viper.ConfigFileNotFoundError); !ok {
return
}
}
err = viper.Unmarshal(&config)
return
}
version <1.18.2: whether an alerts.env exists or not, the final value of config.NotificationType is whatever is in the env var.
version >= 1.18.2: env var override only works if alerts.env exists containing a NOTIFICATION_TYPE=something entry.
Comment From: florianrusch
I found a workaround to avoid the config file:
// The following is required to introduce the paths to viper. Else viper
// would not be able to fetch these configurations from environment
// variables as it does not know the key to search for. Only configs which
// don't have any default value (`viper.SetDefault`) needs to be mentioned
// here.
var configsWithoutDefaultValues = []byte(`
nested:
var:
name:
`)
viper.SetConfigType("yaml")
viper.ReadConfig(bytes.NewBuffer(configsWithoutDefaultValues))
Comment From: florianrusch
Or another solution would be to set the default e.g. to an empty string:
viper.SetDefault("nested.var", "")
Comment From: kt1755
I workaround this issue by parsing empty json object config to viper, to init viper key as follow
var cfg Config
viper.SetConfigType("json")
b, err := json.Marshal(cfg)
if err != nil {
return cfg, err
}
reader := bytes.NewReader(b)
err = viper.ReadConfig(reader)
if err != nil {
return cfg, err
}
...
Comment From: sadegh-ij
Hey, I tested the following similar scenario, It works without a problem.
Version of used packages:
go v1.23.5 viper v1.20.0 godotenv v1.5.1
Files inside directory: (Assuming you are managing the path of the files correctly)
.env config.yml main.go
main.go
package main
import (
"fmt"
"github.com/joho/godotenv"
"github.com/spf13/viper"
)
type AppConfig struct {
// hiding crucial information inside a .env file
// Variables should be visible to viper
Password string `mapstructure:"password"`
SecretKey string `mapstructure:"secret_key"`
}
type Config struct {
App AppConfig `mapstructure:"app"`
}
func main() {
// Load environment variables from .env file
err := godotenv.Load()
if err != nil {
log.Fatalf("Error loading .env file")
}
// Set the file name and type for Viper
viper.SetConfigName("config") // name of config file (without extension)
viper.SetConfigType("yaml") // required if the config file does not have the extension
viper.AddConfigPath(".") // optionally look for config in the working directory
viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
// This will enable Viper to read environment variables
viper.AutomaticEnv()
// You can also set your default vaiables too.
viper.SetDefault("app.password", "default-password")
viper.SetDefault("app.secret_key", "default-secret-key")
app_password := viper.GetString("app.password")
app_secret_key := viper.GetString("app.secret_key")
fmt.Println("app->password: ", app_password)
fmt.Println("app->secret_key: ", app_secret_key)
// In order to `Unmarshal` viper inside variable `config`,
// config fields should be visible to viper,
// so you should name them starting with an Uppercase.
var config Config
if err := viper.Unmarshal(&config); err != nil {
return
}
fmt.Println(config)
}
config.yml
app:
password: ${APP_PASSWORD}
secret_key: ${APP_SECRET_KEY}
.env
APP_PASSWORD=password
APP_SECRET_KEY=secret-key
Notes
- Make sure that your environment variables are set correctly. If you are using a
.envfile, ensure that you load it properly using packages likegodotenv. - Viper converts the keys in your configuration file to uppercase and replaces dots (.) with underscores (_). So you need to declare your environmental variables by considering this.
- Make sure you call
viper.AutomaticEnv()before you try to access any configuration values. - (IMPORTANT) In order to call
viper.Unmarshal(&config)and assign values loaded by viper inside variableconfig, config fields should be visible to viper, so you should name them starting with an Uppercase.
Comment From: burik666
@sadegh-ij it works for you because config.yaml is not empty.
Comment From: OliverWangData
Confirming this issue is still present in v1.20.X.
The automatic environment variable mapping configured like this:
v.SetEnvPrefix("PREPEND")
v.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
v.AutomaticEnv()
When attempting to unmarshal into a struct such as:
type AppConfig struct {
Proxy struct {
Url string
}
}
...
var cfg AppConfig
if err := v.Unmarshal(&cfg); err != nil {
....
}
...
Does not correctly populate cfg.Proxy.Url from the PREPEND_PROXY_URL environment variable.
The environment variable is only picked up if an explicit BindEnv call is made, for example:
v.BindEnv("proxy.url", "PREPEND_PROXY_URL")
This workaround defeats the purpose of AutomaticEnv() for struct unmarshaling. Downgrading to v1.18.0 resolves the issue.