Spring Cloud Config AwsS3EnvironmentRepository does not comply with Git

Describe the bug Using version 3.1.9 due to restrictions, but looking in the code of AwsS3EnvironmentRepository I see the newest version has the same issue. Our configuration is sorted into folders for each application with the name of that given application. The implementation does not handle this like the Git version - tho I see that the newest can use folders, but with hardcoded "application" as the name of the file.

Sample Works in Git, but not in AwsS3EnvironmentRepository :

app1/app1-profile.yml app1/app1.yml

What looks to work in AwsS3EnvironmentRepository in 4.3.0-SNAPSHOT:

app1/application-profile.yml app1/application.yml

Comment From: ryanjbaxter

Please provide a sample that reproduces the error with a supported release (4.1.x or greater).

Comment From: Husted-1

This is from the newest AwsS3EnvironmentRepository where you generate the S3 key:

    private String buildObjectKeyPrefix(String application, String profile, String label) {
        StringBuilder objectKeyPrefix = new StringBuilder();
        if (!ObjectUtils.isEmpty(label)) {
            objectKeyPrefix.append(label).append(PATH_SEPARATOR);
        }
        objectKeyPrefix.append(application);
        if (this.useApplicationAsDirectory) {
            objectKeyPrefix.append(PATH_SEPARATOR).append("application");
        }
        if (!ObjectUtils.isEmpty(profile)) {
            objectKeyPrefix.append("-").append(profile);
        }
        return objectKeyPrefix.toString();
    }

    private void addPropertySource(Environment environment, String app, String profile, String label) {
        String objectKeyPrefix = buildObjectKeyPrefix(app, profile, label);
        S3ConfigFile s3ConfigFile = getS3ConfigFile(objectKeyPrefix);
        if (s3ConfigFile != null) {
            environment.setVersion(s3ConfigFile.getVersion());

            final Properties config = s3ConfigFile.read();
            config.putAll(serverProperties.getOverrides());
            PropertySource propertySource = new PropertySource("s3:" + objectKeyPrefix, config);
            if (LOG.isDebugEnabled()) {
                LOG.debug("Adding property source to environment " + propertySource);
            }
            environment.add(propertySource);
        }
    }

As one can see, it doesn't cover my provided example of what the git version can.

Code found here: https://github.com/spring-cloud/spring-cloud-config/blob/main/spring-cloud-config-server/src/main/java/org/springframework/cloud/config/server/environment/AwsS3EnvironmentRepository.java

Comment From: ryanjbaxter

A concrete sample that reproduces your use case would help here to make sure we are on the same page.

It sounds like what you are suggesting is that AWS S3 Environment Respository support the pattern matching functionality that we have in Git. https://docs.spring.io/spring-cloud-config/reference/server/environment-repository/git-backend.html#pattern-matching-and-multiple-repositories

Comment From: Husted-1

Yes, it could be a good thing if one could move ones config to a new repository without needing to change it.

This is the setup from application.yml on the cloud config server:

spring:
  profiles:
    active: default
  cloud:
    config:
      server:
        defaultLabel: master
        git:
          uri: git@bitbucket.org:####/cloud-config.git
          searchPaths: '/*'
          cloneOnStart: false

File setup like this works in git, and native, but not in awsS3:

app1/app1-profile.yml
app1/app1.yml
application-profile.yml
application.yml

The code for awsS3 wants the files to be named "application" and will not accept "app1", as it is hardcoded, like I wrote.

Comment From: ryanjbaxter

OK this is what I thought, this would be a worthwhile enhancement

Comment From: tomy8964

Hi @ryanjbaxter! I'd like to take on this enhancement and contribute a PR for S3 pattern matching support, similar to the Git backend.
I'll start working on this and will share a draft or questions here if I run into issues. Can you assign this to me? Thanks!

Comment From: ryanjbaxter

Great thanks!