Problem Description:
After upgrading our Spring Boot application from version 3.3.x to 3.4.6, we're consistently encountering PropertyNotFoundException
errors during application startup. Our application relies on spring.config.import
with sm://
or sm@
prefixes to dynamically fetch secrets from a cloud Secret Manager (e.g., GCP Secret Manager) and inject them into the Spring Environment.
Observed Behavior in Spring Boot 3.4.6:
The application fails to start because certain components or beans attempt to access properties (e.g., spring.data.database.uri
, spring.data.database.username
, spring.data.database.password
) that are expected to be resolved from sm://
or sm@
placeholders. However, these properties are not yet available in the Spring Environment at the time of access. This indicates that the spring.config.import
mechanism for these external secrets is completing after or too late for some early bean initialization phases.
Expected Behavior in Spring Boot 3.3.x:
In Spring Boot 3.3.x, this setup worked reliably. The spring.config.import
mechanism for sm://
or sm@
prefixes would consistently load all necessary secrets into the Spring Environment before any beans that depended on these secrets were initialized. No PropertyNotFoundException
or similar startup failures were observed related to secret resolution timing.
Context / Suspected Change:
This change in behavior suggests a shift in the timing or order of property loading and spring.config.import
processing relative to bean initialization between Spring Boot 3.3.x and 3.4.x.
Workaround Implemented:
To mitigate this issue and allow the application to start, we've implemented a custom EnvironmentPostProcessor
to fetch and inject secrets very early in the Spring Boot startup process.
-
Declare
EnvironmentPostProcessor
:src/main/resources/META-INF/spring.factories
:properties org.springframework.boot.env.EnvironmentPostProcessor=\ com.example.app.config.CustomSecretManagerEnvProcessor
-
Implement
CustomSecretManagerEnvProcessor
: This class is ordered with@Order(Ordered.HIGHEST_PRECEDENCE + 11)
to ensure early execution. It reads a mapping fromapplication.yml
, fetches the corresponding secrets from our Secret Manager, and then adds them directly to Spring'sConfigurableEnvironment
usingenv.propertySources.addFirst()
.```java package com.example.app.config;
import org.springframework.boot.SpringApplication; import org.springframework.boot.env.EnvironmentPostProcessor; import org.springframework.core.Ordered; import org.springframework.core.annotation.Order; import org.springframework.core.env.ConfigurableEnvironment; import org.springframework.core.env.MapPropertySource; import java.util.HashMap; import java.util.Map; // Potentially import org.springframework.boot.env.YamlPropertySourceLoader; // Or similar to read the secrets.mapping if it's complex
@Order(Ordered.HIGHEST_PRECEDENCE + 11) // Ensures very early execution public class CustomSecretManagerEnvProcessor implements EnvironmentPostProcessor {
private static final String CUSTOM_PROPERTY_SOURCE_NAME = "custom-secret-properties"; // private static final String YAML_SECRETS_MAPPING_PATH = "secrets.mapping"; // If reading from env @Override public void postProcessEnvironment(ConfigurableEnvironment env, SpringApplication app) { final Map<String, Object> loadedSecretProps = new HashMap<>(); // MASKED: Logic to read 'secrets.mapping' from YAML // Example: Access properties like env.getProperty("secrets.mapping.secret-name") // MASKED: Logic to initialize Secret Manager client (e.g., GCP Secret Manager client) // and fetch secrets based on the 'secrets.mapping' configuration. // Example structure: // List<Map<String, String>> mappings = (List<Map<String, String>>) env.getProperty("secrets.mapping", List.class); // if (mappings != null) { // SecretManagerClient client = initializeSecretManagerClient(); // Your client init // for (Map<String, String> mapping : mappings) { // String secretId = mapping.get("secret-name"); // String propertyKey = mapping.get("property-key"); // if (secretId != null && propertyKey != null) { // String rawSecret = client.getSecret(secretId); // Your secret fetch logic // loadedSecretProps.put(propertyKey, rawSecret); // } // } // } // Example hardcoded for demonstration (replace with actual fetching based on mapping) loadedSecretProps.put("masked.db.password", "fetchedPassword123"); loadedSecretProps.put("masked.azure.client-id", "fetchedClientId"); if (!loadedSecretProps.isEmpty()) { env.propertySources.addFirst( new MapPropertySource(CUSTOM_PROPERTY_SOURCE_NAME, loadedSecretProps) ); } }
} ```
-
Define Secret Mapping in
application.yml
: This section specifies which secrets theEnvironmentPostProcessor
should fetch and map to Spring properties.```yaml
application.yml snippet
spring: cloud: gcp: secretmanager: enabled: false # Optional: Disable default if it conflicts/duplicates # ... other configurations
This section drives the CustomSecretManagerEnvProcessor
secrets: mapping: - secret-name: "your-gcp-secret-id-1" # Name in Secret Manager property-key: "masked.db.password" # Property key in Spring env - secret-name: "your-gcp-secret-id-2" property-key: "masked.azure.client-id" # ... more mappings
Example of consuming the properties after the EnvironmentPostProcessor has run
app: datasource: password: ${masked.db.password} azure: clientId: ${masked.azure.client-id} ```
This workaround ensures that critical secrets are loaded and available in the Spring Environment at a very early stage, bypassing the timing issues experienced with spring.config.import
in Spring Boot 3.4.x.
Question for the Spring Boot Team:
Is this change in the timing of spring.config.import
for external secrets an intentional architectural shift in Spring Boot 3.4.x, or could it be an unintended regression? Understanding the intended behavior would help us determine if our EnvironmentPostProcessor
workaround is the recommended approach for such scenarios or if there's a more idiomatic/built-in way to handle early secret loading in 3.4.x.
Example application.yml
(original, problematic setup):
# application.yml
spring:
config:
import: sm://
data:
database:
uri: ${sm://<PATH_TO_SECRET_FOR_URI>}
username: ${sm://<PATH_TO_SECRET_FOR_USERNAME>}
password: ${sm://<PATH_TO_SECRET_FOR_PASSWORD>}
**Comment From: wilkinsona**
`spring.config.import` is processed as part of preparing the environment. That has not (intentionally) changed in Spring Boot 3.4. None of the properties that you have mentioned (`spring.data.database.uri` etc) are standard properties so we don't know how they're being consumed. Without knowing that, we can't really offer any more guidance here. On the face of it, given that your own environment post-processor fixes the problem, it would appear to be a problem with the Spring Cloud GCP secret manager integration.
If you would like us to spend some more time investigating, please spend some time providing a complete yet minimal sample that reproduces the problem. You can share it with us by pushing it to a separate repository on GitHub or by zipping it up and attaching it to this issue.
**Comment From: spring-projects-issues**
If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.
**Comment From: spring-projects-issues**
Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.
**Comment From: Raja-shekaran**
## 1 | Executive summary
* Upgrading from **Spring Boot 3.3.x → 3.4.x** breaks our Secret-Manager-based configuration.
* In Boot 3.4.6 the secret injected via either
• `sm://some-secret` **or**
• `sm@{some-secret}`
arrives in the environment as
<ByteString@76c5599b size=36 contents="d720c960-d79b-44dc-86ec-0dd077389f24">
instead of the expected decoded `String`.
* Because the database password is now the `ByteString.toString()` representation, **Flyway fails** during startup.
* Custom `Converter<ByteString,String>` not invoked; manual `SecretManagerTemplate` works but defeats `spring.config.import`.
---
## 2 | Minimal configuration that reproduces the issue
### 2.1 `application.yml`
```yaml
spring:
config:
import: sm:// # also tried sm@ → same result
cloud:
gcp:
secretmanager:
enabled: true # set to false → bug disappears when fetched manually
datasource:
url: jdbc:postgresql:///db?socketFactory=com.google.cloud.sql.postgres.SocketFactory
username: pps-psql-preprod
password: ${sm://psql-preprod}
# password: ${sm@pps-psql-preprod}
A bean with @Value("${spring.datasource.password}")
already logs the ByteString@…
before Flyway runs.
2.2 build.gradle.kts
plugins {
id("org.springframework.boot") version "3.4.6"
id("io.spring.dependency-management") version "1.1.7"
kotlin("jvm") version "1.9.25"
kotlin("plugin.spring") version "1.9.25"
}
group = "example"
version = "0.0.1-SNAPSHOT"
java.sourceCompatibility = JavaVersion.VERSION_17
dependencies {
implementation(platform("org.springframework.cloud:spring-cloud-dependencies:2024.0.1"))
implementation("org.springframework.boot:spring-boot-starter-web")
implementation("com.google.cloud:spring-cloud-gcp-starter-secretmanager")
testImplementation("org.springframework.boot:spring-boot-starter-test")
}
Run:
./gradlew bootRun # fails with Boot 3.4.6
./gradlew -PbootVersion=3.3.7 bootRun # succeeds
3 | Log excerpt (Boot 3.4.6)
... DEBUG [com.google.cloud.spring.secretmanager.SecretManagerPropertySource] - Found secret psql-preprod-...
... DEBUG [org.springframework.core.env.PropertySourcesPropertyResolver] - Could not convert value of type 'com.google.protobuf.ByteString' to required type 'java.lang.String'
spring.datasource.password = <ByteString@76c5599b size=36 contents="">
...
org.postgresql.util.PSQLException: FATAL: password authentication failed for user "pps-psql-preprod"
...
org.flywaydb.core.internal.exception.FlywaySqlException: Unable to obtain connection from database
4 | Version manifest
Component | Version |
---|---|
Java | 17 |
Spring Boot | 3.4.6 ❌ (fails) 3.4.5 ✔ (decodes locally, blocked by CVE scan) 3.3.7 ✔ |
Spring Cloud BOM | 2024.0.1 |
spring-cloud-gcp-secretmanager | 6.2.1 |
google-cloud-secretmanager | 2.44.0 |
5 | What we already tried
- Boot 3.4.5 – decodes locally, CI fails CVE scan.
- Override CVEs under 3.4.5 – CVEs gone, still ByteString.
- Custom Converter – not invoked.
- Disable autoconfig + manual SecretManagerTemplate – works.
- Switch
sm://
→sm@
notation – same ByteString behaviour.
6 | What we can (and cannot) share
- ✅ Sanitised build & YAML (above)
- ✅ Full DEBUG logs on request
- ❌ Proprietary source code or internal Maven URLs (policy)
If more is needed, let us know the minimum requirement and we will provide a clean-room repro.
7 | Questions
- Is the
ByteString@…
value in Boot 3.4.6 a known regression or expected change? - Any property-binding changes between 3.4.5 → 3.4.6 that explain this?
- Will a stripped-down sample (Boot + Secret Manager + one bean) be sufficient for investigation?
Thanks in advance for your help.
Comment From: philwebb
Spring Boot doesn't have a ByteString
class so I suspect you'll need to raise this with whatever project is providing your sm:
import support. I would suggest putting together an actual sample together that folks can actually run (a GitHub project or zip file).