Expected Behavior

I would like to be able to dynamically configure ChatModel's Options, including the apiKey, at runtime during SpringAI's startup, without requiring these options to be initialized at startup.

Current Behavior

Currently, SpringAI initializes all imported spring-ai-starter-model-xxx at startup, which requires providing Options (like the apiKey) in the configuration. Sometimes our requirements are dynamic, and the model configurations might come from other sources (like a database). In such cases, I have to manually construct all the necessary ChatModels one by one, which involves a considerable amount of effort. Therefore, I wonder if there is a way to make ChatOptions runtime-validated instead of being required at startup. It feels somewhat unreasonable to have an invalid apiKey configured in the yml file and then override it at runtime.

Context

I have reviewed the official documentation, but I couldn’t find a suitable solution for my needs. I acknowledge that I may have a misunderstanding, and I would greatly appreciate any suggestions or guidance you might have.

Comment From: dev-jonghoonpark

what about disable spring boot autio configuration?

Enabling and disabling of the chat auto-configurations are now configured via top level properties with the prefix spring.ai.model.chat.

To enable, spring.ai.model.chat=anthropic (It is enabled by default)

To disable, spring.ai.model.chat=none (or any value which doesn’t match anthropic)

This change is done to allow configuration of multiple models.

Comment From: FFatTiger

能否禁用 Spring Boot 的自动配置？

聊天自动配置的启用和禁用现在通过以 spring.ai.model.chat 为前缀的顶级属性进行配置。 要启用，spring.ai.model.chat=anthropic（默认情况下已启用） 要禁用，请设置 spring.ai.model.chat=none（或任何与 anthropic 不匹配的值） 进行此更改是为了允许配置多个模型。

Thank you for your response.

While reviewing the source code, I discovered this functionality, but it only meets the requirement for the project to start normally; the ChatModel still needs to be initialized. For example, in the case of DeepSeek's AutoConfiguration, it injects several dependencies by default, which I prefer not to handle manually. I want to achieve runtime dynamic configuration for multiple models, but if each ChatModel needs to be manually initialized, I would have to be concerned with too many dependencies.

@Bean
@ConditionalOnMissingBean
public DeepSeekChatModel deepSeekChatModel(DeepSeekConnectionProperties commonProperties,
        DeepSeekChatProperties chatProperties, ObjectProvider<RestClient.Builder> restClientBuilderProvider,
        ObjectProvider<WebClient.Builder> webClientBuilderProvider, ToolCallingManager toolCallingManager,
        RetryTemplate retryTemplate, ResponseErrorHandler responseErrorHandler,
        ObjectProvider<ObservationRegistry> observationRegistry,
        ObjectProvider<ChatModelObservationConvention> observationConvention,
        ObjectProvider<ToolExecutionEligibilityPredicate> deepseekToolExecutionEligibilityPredicate) {

    var deepSeekApi = deepSeekApi(chatProperties, commonProperties,
            restClientBuilderProvider.getIfAvailable(RestClient::builder),
            webClientBuilderProvider.getIfAvailable(WebClient::builder), responseErrorHandler);

    var chatModel = DeepSeekChatModel.builder()
        .deepSeekApi(deepSeekApi)
        .defaultOptions(chatProperties.getOptions())
        .toolCallingManager(toolCallingManager)
        .toolExecutionEligibilityPredicate(deepseekToolExecutionEligibilityPredicate
            .getIfUnique(DefaultToolExecutionEligibilityPredicate::new))
        .retryTemplate(retryTemplate)
        .observationRegistry(observationRegistry.getIfUnique(() -> ObservationRegistry.NOOP))
        .build();

    observationConvention.ifAvailable(chatModel::setObservationConvention);

    return chatModel;
}

Therefore, it might be more convenient if these options could be initialized dynamically at runtime.

Comment From: markpollack

There was a design to address this, but it looks like I didn't take it all the way to completion - at least for OpenAI.

There is an interface ApiKey

public interface ApiKey {

    /**
     * Returns an api key to use for a making request. Users of this method should NOT
     * cache the returned api key, instead call this method whenever you need an api key.
     * Implementors of this method MUST ensure that the returned key is not expired.
     * @return the current value of the api key
     */
    String getValue();

}

So basically whenever a request at runtime is made, this "API Key Provider' is called to get an API key.

The current problem is that in OpenAiChatAutoConfiguration one can't provide an alternate implementation of this interface.

    private OpenAiApi openAiApi(OpenAiChatProperties chatProperties, OpenAiConnectionProperties commonProperties,
            RestClient.Builder restClientBuilder, WebClient.Builder webClientBuilder,
            ResponseErrorHandler responseErrorHandler, String modelType) {

        OpenAIAutoConfigurationUtil.ResolvedConnectionProperties resolved = resolveConnectionProperties(
                commonProperties, chatProperties, modelType);

        return OpenAiApi.builder()
            .baseUrl(resolved.baseUrl())
            .apiKey(new SimpleApiKey(resolved.apiKey()))
            .headers(resolved.headers())
            .completionsPath(chatProperties.getCompletionsPath())
            .embeddingsPath(OpenAiEmbeddingProperties.DEFAULT_EMBEDDINGS_PATH)
            .restClientBuilder(restClientBuilder)
            .webClientBuilder(webClientBuilder)
            .responseErrorHandler(responseErrorHandler)
            .build();
    }

.apiKey(new SimpleApiKey(resolved.apiKey())) is hardcoded.

I can make a PR so that you can provide an implementation of this interface. The assertion in the builder is only that there is an instance of this class, the underlying value.

        public OpenAiApi build() {
            Assert.notNull(this.apiKey, "apiKey must be set");
            return new OpenAiApi(this.baseUrl, this.apiKey, this.headers, this.completionsPath, this.embeddingsPath,
                    this.restClientBuilder, this.webClientBuilder, this.responseErrorHandler);
        }

I'll post back and you can try this out. Also, this pattern should be done across all the other models. i believe though AWS already has a awskeyresolver or something like that.