Spring AI Advisor context

Description

In the latest updates of SpringAI the tools approach has been changed and now you can define context dependent tools (tools with context available only on invocation) as a default tools during chat client creation. To provide context on a call you can use toolContext method:

ChatResponse response = chatClient.prompt(new Prompt(message))
    .toolContext(Map.of("foo", foo))
    .call().chatResponse();

But we doesn't have the same for the advisors. In case the advisor is context dependent we have no choice but to create it on a per call basis passing the context via the constructor:

ChatResponse response = chatClient.prompt(new Prompt(message))
    .advisors(new FooAdvisor(foo))
    .call().chatResponse();

That would be nice to provide a method advisorContext similar to toolContext to be able to pass some context related data to the advisors as well. This method can be used to prepopulate context of the ChatClientRequest to make this data available in the advisor, or alternatively can be passed as an additional external context parameter to the advisors methods.

Workaround

You can create a dedicated advisor that will populate internal advisor context with an external advisor context:

public class ExternalContextAdvisor implements BaseAdvisor {

    private final Map<String, Object> externalContext;

    public ExternalContextChatAdvisor(Map<String, Object> externalContext) {
        this.externalContext = externalContext;
    }

    @Override
    public int getOrder() {
        return Ordered.HIGHEST_PRECEDENCE;
    }

    @Override
    public ChatClientRequest before(ChatClientRequest request, AdvisorChain chain) {
        request.context().put("external-context", externalContext);
        return request;
    }

    @Override
    public ChatClientResponse after(ChatClientResponse response, AdvisorChain chain) {
        return response;
    }
}

And use it to pass an external context to advisors during the client call:

Map<String, Object> externalContext = new HashMap<>();

externalContext.push("foo", foo);
externalContext.push("bar", bar);

ChatResponse response = chatClient.prompt(new Prompt(message))
        .advisors(new ExternalContextAdvisor(externalContext)
        .call().chatResponse();

Using this external context advisor you can now register all other context dependent advisors as a default advisors during chat client creation:

ChatClient chatClient = ChatClient.builder(chatModel)
        .defaultAdvisors(fooAdvisor, barAdvisor)
        .build();

Inside any of the context dependent advisors you can now extract external context to get necessary parameters:

public class FooAdvisor implements BaseAdvisor {

    @Override
    public int getOrder() {
        return 0;
    }

    @Override
    public ChatClientRequest before(ChatClientRequest request, AdvisorChain chain) {

        Map<String, Object> externalContext = (Map<String, Object>) request.context()
                .get("external-context");

        Foo foo = (Foo) externalContext.get("foo");

        // Do something with foo

        return request;
    }

    @Override
    public ChatClientResponse after(ChatClientResponse response, AdvisorChain chain) {
        return response;
    }
}

With the above approach you can also place some results to the external context that will be available from outside

Comment From: YunKuiLu

You can use advisors(Consumer<AdvisorSpec> consumer) method to set advisor context. eg:

defaultChatClient.prompt()
      .advisors(adv -> adv.param(<you advisor context key>, <you advisor context value>)
                          .param(<you advisor context key>, <you advisor context value>))
      .user(message)
      .stream()
      .content()

Comment From: ijioio

You can use advisors(Consumer<AdvisorSpec> consumer) method to set advisor context. eg:

Thank you for pointing this out! It was not that obvious to me, I would prefer more clear signature 😅 But it seems that similar approach is applied in other builders as well, so I just need to get used to that API style.

Comment From: YunKuiLu

It was not that obvious to me, I would prefer more clear signature

Yes, and the param method is not easy to associate with advisor context. Maybe it will be improved in the future. But maintainers are cautious about changes to ChatClient.

That would be nice to provide a method advisorContext similar to toolContext to be able to pass some context related data to the advisors as well. This method can be used to prepopulate context of the ChatClientRequest to make this data available in the advisor, or alternatively can be passed as an additional external context parameter to the advisors methods.

You can keep this issue open, maybe they will consider your suggestion.