Proposal Details

Today, Go supports on Unices (not darwin) the ability to override the system roots through the use of the environment variables SSL_CERT_FILE and SSL_CERT_DIR.

Having this ability to change the system roots without modifying the target Go binary is great for testing and integration into larger programs that invoke other Go programs. In particular, while it is trivial to modify a Go program to specify an alternate source for the system roots, not all Go programs that open TLS connections provide an option to do so.

Unfortunately, SSL_CERT_FILE and SSL_CERT_DIR are not honored on every system. This can be understandable since both are, originally, OpenSSL-specific, and Windows and macOS have their own ways to manage their system certificate store.

For these reasons, I'd like to propose a mechanism to override the system roots on platforms that do not support SSL_CERT_FILE/DIR. Today, these platforms are Windows, macOS, and Plan9 (and wasm, but that's probably not applicable?). I can mostly see two options here:

Option 1: extend SSL_CERT_FILE and SSL_CERT_DIR to other platforms.

This one is straightforward: move the logic that loads the roots from these variables out of roots_unix.go and in the SystemCertPool function.

The benefit is that quite a lot of tooling that set SSL_CERT_FILE would now work with no modification on more platforms. There is precedent in some languages/libraries doing this (Rust TLS for instance), too.

The drawback however is that this sudden support might not be desired, or lead to surprising behavior (for instance, tools that set the variable by mistake on Windows and relied on the actual system roots being loaded would suddenly honor the variable).

Option 2: introduce Go-specific environment variables GO_SSL_CERT_FILE and GO_SSL_CERT_DIR

This is less disruptive; it would mostly entail copying the code handling SSL_CERT_FILE/DIR and duplicate it into SystemCertPool, but using the environment variables GO_SSL_CERT_FILE/DIR instead. There is precedent in other languages and/or libraries doing this (NODE_EXTRA_CA_CERTS, REQUESTS_CA_BUNDLE, ...), so it wouldn't be too surprising to users.

Overall, option 1 would probably be the most seamless w.r.t. integration, but realistically I think the only real option is option 2 to avoid any potential breakage.

Comment From: gabyhelp

Related Issues

Related Code Changes

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

Comment From: ianlancetaylor

CC @golang/security

Comment From: mateusz834

FYI: I think that as a workaround you can make this work with x509.SetFallbackRoots and setting the GODEBUG x509usefallbackroots=1

Comment From: Snaipe

I don't think this is a proper workaround, because it still requires every application to opt-in, right? The goal of this proposal is to have a unified way for users and system administrators to override the system roots with no modifications of the vendor application

Comment From: apparentlymart

This request seems like another example of the ongoing tension between which behaviors are under the control of the application developer and which are under the control of the end-user in spite of the application developer.

While I certainly sympathize with the problem this is trying to solve, I'm always nervous about unilaterally introducing new ways that end-users can vary the runtime behavior of an application outside of what the application developer explicitly allowed, because it adds extra hidden backward-compatibility surface area for applications that they may not be aware of and thus may not actually be testing for, despite them having end-users relying on it.

The existing support on Unix platforms is in a grey area where these environment variables are not technically "part of the platform" but have become treated as such de-facto, similar to the proxy-related environment variables implemented by net/http. These do potentially have the same risks but at least application developers are more likely to be familiar with these mechanisms due to them being broadly supported across most software on a given target platform.

I would personally draw the line at Go-specific environment variables outside of the application developers control. I concede that some already exist (GODEBUG, for example) but I have already expressed elsewhere that I find those unfortunate because I work on an application where the fact that it's implemented in Go is an implementation detail rather than something end-users should ever be relying on directly.

I would support a variant of this proposal that makes it easier for application developers to provide such a mechanism on an opt-in basis, with the external-facing interface to it under the application developer's control.

The "hard part" of adding that support today is that TLS certificate verification is a cross-cutting concern that may affect many separate parts of an application, but arranging for every separate TLS client in a program to be aware of an additional set of trusted certificates is challenging.

The Unix-only SSL_CERT_FILE/SSL_CERT_DIR are convenient in that they affect the behavior of all callers unilaterally (unless they have explicitly changed the TLS configuration), but having a standard-library-based mechanism to add additional certificates that are trusted for all future verifications with a nil VerifyOptions.Roots would allow application developers to provide their own cross-platform environment variables (or similar) without significantly redesigning how their application uses TLS.

For the application I'm most concerned about I expect we'd like to offer a configuration-file-based mechanism to achieve the same result, either in addition to or instead of environment variables, because that is how other similar settings tend to be exposed in our application. So far we've declined to offer our end-users a cross-platform way to change the certificate validation rules because our application relies on various third-party libraries, some of which construct their own HTTP clients without exposing a convenient way for us to alter the certificate pool they use, and so the resulting behavior would appear inconsistent.

If there were a way to add new trusted certs unilaterally for everything that's using the default certificate pool then I expect that would handle it well enough for us to be able to offer such a feature.