Currently, proprietary or open models running in Google Vertex can be accessed through the Vertex Gemini module, respectively using the OpenAI API.

#################################
# Google Vertex AI Gemini
#################################
spring.ai.vertex.ai.gemini.project-id=<project>
spring.ai.vertex.ai.gemini.location=us-central1
spring.ai.vertex.ai.gemini.chat.options.model=gemini-2.0-flash-001
spring.ai.vertex.ai.gemini.transport=grpc

#################################
# OpenAI API VertexAI
#################################
spring.ai.openai.api-key=abc123
spring.ai.openai.vertex.ai.gemini.project-id=<project>
spring.ai.openai.vertex.ai.gemini.location=us-central1
spring.ai.openai.vertex.ai.chat.options.model=meta/llama3-405b-instruct-maas
spring.ai.openai.vertex.ai.chat.base-url=<baseURL>
spring.ai.openai.vertex.ai.chat.completions-path=/chat/completions
spring.ai.openai.vertex.ai.chat.options.max-tokens=1024

The problem - there is no ability to auto-config more than 1 Gemini or open model in Vertex at this time.

The proposal - introduce the ability to add a in the auto-config, to allow for multiple configurations

spring.ai.vertex.ai.gemini.<gemini-flash>.project-id=<project>
spring.ai.vertex.ai.gemini.<gemini-flash>.location=us-central1
spring.ai.vertex.ai.gemini.<gemini-flash>.chat.options.model=gemini-2.0-flash-001
spring.ai.vertex.ai.gemini.<gemini-flash>.transport=grpc

spring.ai.vertex.ai.gemini.<gemini-pro>.project-id=<project>
spring.ai.vertex.ai.gemini.<gemini-pro>.location=us-central1
spring.ai.vertex.ai.gemini.<gemini-pro>.chat.options.model=gemini-1.5-001
spring.ai.vertex.ai.gemini.<gemini-pro>.transport=grpc

Workaround - not using auto-config and manually configuring each model in use

CC @tzolov

Comment From: ThomasVitale

I would really like to have this possibility in Spring AI. Unfortunately, I'm afraid it's up to the Spring Boot project to support registering multiple beans of the same type via auto-configuration, a capability that so far has not been supported by Spring Boot.

I hope things will change in Spring Boot 4, based on the new changes introduced in Spring Framework 7 for programmatically registering beans, that would enable such functionality (https://docs.spring.io/spring-framework/reference/7.0/core/beans/java/programmatic-bean-registration.html). See https://github.com/spring-projects/spring-boot/issues/15732.

Comment From: markpollack

I think that isn't really a deal breaker, there can be work around. Need an epic to research and discuss.

Comment From: ThomasVitale

@markpollack I created this issue to be used as an epic to track the discussion: https://github.com/spring-projects/spring-ai/issues/3518

Comment From: paulbakker

I would really like to see this supported as well. I think this could be done by adding a qualifier to each bean created, so that, if there's more than a single bean, the user can selectively inject the correct one based on the qualifier. The qualifier value could be the model name. This likely means the beans need to be created through a BeanFactoryPostProcessor instead of regular auto configuration.

Comment From: paulbakker

I found a workaround until this is fully supported. It's easy to mutate an existing ChatModel instance to switch the underlying model:

Let's assume I have:

@Autowired
OpenAiChatModel openAiChatModel;

I can create another instance with a different model by mutating a builder based on the original:

var evalModel = openAiChatModel.mutate().defaultOptions(OpenAiChatOptions.builder().model("another-model").build()).build();