Spring AI Support of Guardrails in Spring AI

Can we implement some guardrails features in built in Spring AI so we no need to fine tune the prompt:

Gurard rails features:

• Should not be baiosed when ansering questions
• Does not answer political questions
• Avoid content related to violence, hate speech, self-harm, or illegal activities.
• Do not share or ask for personal identifiable information (PII).
• Use respectful and professional language.
• Ensure responses do not cause harm or distress to users.

This should also be configurable , like developer can add custom also.

Comment From: bntv-config

@markpollack Could you please let me know if this is in Roadmap and by when it will be available ?

Comment From: RRajdev

@markpollack any update on this. It is crucial for our use case.

Comment From: viralpatel

For those who are using AWS Bedrock and want to integrate Bedrock Guardrails:

Until the Guardrail support is added to the Spring AI project, here's an alternate way of overriding the bedrock client and inject guardrail Id and version.

import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.bedrockruntime.BedrockRuntimeClient;
import software.amazon.awssdk.services.bedrockruntime.model.ConverseRequest;
import software.amazon.awssdk.services.bedrockruntime.model.ConverseResponse;
import software.amazon.awssdk.services.bedrockruntime.model.GuardrailConfiguration;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class BedrockGuardrailConfiguration {

    @Bean
    public BedrockRuntimeClient bedrockRuntimeClient() {
        BedrockRuntimeClient bedrockRuntimeClient = BedrockRuntimeClient.builder()
                .region(Region.US_EAST_1)
                .credentialsProvider(DefaultCredentialsProvider.create())
                .build();

        return new GuardrailBedrockClient(bedrockRuntimeClient, "guardrailId", "guardrailVersion");
    }
}

class GuardrailBedrockClient implements BedrockRuntimeClient {
    private final BedrockRuntimeClient instance;
    private final String guardrailId;
    private final String guardrailVersion;

    public GuardrailBedrockClient(BedrockRuntimeClient instance, String guardrailId, String guardrailVersion) {
        this.instance = instance;
        this.guardrailId = guardrailId;
        this.guardrailVersion = guardrailVersion;
    }

    @Override
    public void close() {
        instance.close();
    }

    @Override
    public String serviceName() {
        return instance.serviceName();
    }

    @Override
    public ConverseResponse converse(ConverseRequest converseRequest) {
        ConverseRequest newRequest = converseRequest.toBuilder()
                .guardrailConfig(
                    GuardrailConfiguration.builder()
                        .guardrailIdentifier(guardrailId)
                        .guardrailVersion(guardrailVersion)
                        .build()
                )
                .build();

        return instance.converse(newRequest);
    }
}

Comment From: bntv-config

We are more intersted to use some open source capabilities integrated with Spring AI. In my current code where I am using Spring AI M8 version, I have build a SDK for my company use case and using https://github.com/protectai/llm-guard for specific use cases (It is affective and solving my purpose) . I have used advisiors for this.

Just wanted to know if Spring AI community is doing something in this area and can we have some another approach to solve this. Means can we get in built support or user can extend by themself.

Comment From: ThomasVitale

I include here my answer from https://github.com/spring-projects/spring-ai/issues/1011#issuecomment-2891260724:

It should probably be included in the documentation, but the Advisor API is how you can specify guardrails in a ChatClient workflow. Advisors intercept an LLM request before and after its execution, giving you the hooks to define input and output guardrails. Here's some examples of an input guardrail and an output guardrail.

It would be nice to get some specialisation of the Advisor API tailored towards guardrails, providing convenient utils for doing things like deny LLM requests, retry, or re-prompt. You can do all that today with the Advisor API, but less conveniently.