Spring AI Support Foundry local https://github.com/microsoft/Foundry-Local

Expected Behavior

Spring AI should provide a dedicated Foundry Local starter that auto-configures an OpenAI-compatible chat model for Microsoft's Foundry Local inference engine.

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-foundry-local</artifactId>
    <version>1.1.0</version>
</dependency>

spring:
  ai:
    foundry-local:
      base-url: http://localhost:8081
      chat:
        options:
          model: phi-3.5-mini-instruct-generic-gpu
          temperature: 0.7

@Service
@RequiredArgsConstructor
public class ChatService {
   private final ChatModel model;  // Auto-injected FoundryLocal chat model
   public String reply(String msg) { return model.call(msg); }
}

The starter should surface Foundry-specific capabilities like hardware acceleration hints, model lifecycle management, and provide seamless migration to Azure AI Foundry in production.

Current Behavior

Foundry Local can only be used by manually configuring the generic OpenAI starter with overridden base URL and dummy API key:

spring:
  ai:
    openai:
      base-url: http://localhost:8081
      api-key: dummy-key  # Required but unused

This approach: - Hides Foundry-specific configuration options (hardware acceleration, model management) - Forces dependency on spring-ai-starter-openai, obscuring intent in BOMs - Lacks proper Spring Boot integration (health checks, metrics, conditional beans) - Provides no migration path for Azure AI Foundry deployment

Context

Problem: Foundry Local delivers privacy-first, offline LLM inference with OpenAI-compatible APIs, but Spring AI lacks first-class support despite being ideal for Spring's "laptop to production" philosophy.

Impact: - Java developers must hand-roll configuration in every project - Missing hardware acceleration features (NPU/GPU optimization) - No clean development → production migration story - Enterprise teams can't leverage data sovereignty benefits

Alternatives Considered: - Manual OpenAI configuration (current workaround, but loses Foundry features) - Custom auto-configuration per project (duplicated effort across teams)

Use Case: Enable zero-friction local development with sensitive data that never leaves the device, then seamlessly deploy to Azure AI Foundry by changing one property. Critical for enterprises with compliance requirements and development teams evaluating local-first LLM workflows.

Comment From: sunyuhan1998

Does it look a bit like a Microsoft version of Ollama?

Comment From: MaanavD

Hey @roryp ! I'm a PM on the Foundry Local team and super supportive of this endeavor. Not sure we have the resources to contribute first class support right away, but if you have ideas on how we can help, would love to make this a reality.