Bug description When attempting to use a tools along with streaming, I am using a java.util.function.Supplier as my FunctionCallback (vs a java.util.function.Function); I am using BedrockProxyChatModel as my ChatModel.

The initial bedrock LLM response correctly selects my Supplier as the tool choice, however to tool is never called by Spring AI. This appears to only happen with Suppliers; java.util.function.Function works as expected.

While debugging, the issue a manifested in the isToolCall of the org.springframework.ai.chat.model.AbstractToolCallSupport class when after the following debug message.

DEBUG 34272 ---  [tyEventLoop-2-2] o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:MessageStopEvent(StopReason=tool_use)

The chatResponse contains no generations, therefor the tool is not executed.

    protected boolean isToolCall(ChatResponse chatResponse, Set<String> toolCallFinishReasons) {
        Assert.isTrue(!CollectionUtils.isEmpty(toolCallFinishReasons), "Tool call finish reasons cannot be empty!");
        if (chatResponse == null) {
            return false;
        }
        var generations = chatResponse.getResults();
        if (CollectionUtils.isEmpty(generations)) {
            return false;
        }
        return generations.stream().anyMatch(g -> isToolCall(g, toolCallFinishReasons));
    }

Environment This was encountered using spring-ai M4 along with Java 21.

Steps to reproduce The following is an excerpt from a trivial SpringBoot application that can reproduce the problem when Spring AI is configured use the Bedrock/Converse chat models. It includes a very simple System.out to view the output.

    @Bean
    public CommandLineRunner commandLineRunner(ChatClient chatClient) {

        return (args) -> {

            FunctionCallback.Builder builder =
                    FunctionCallback.builder();

            FunctionCallback functionCallBack = builder.description("Get the weather")
            .function("weather", new WeatherService())
            .build();


            ChatClientRequestSpec spec = chatClient.prompt();

            spec.functions(functionCallBack).user("What's the weather like").stream().chatResponse()
            .doOnNext(resp -> {
                if (resp.getResult() != null)
                    System.out.println(resp.getResult().getOutput().getContent());
                }).collectList().block();           

        };
    }

    public static class WeatherService implements Supplier<WeatherService.Response> {

        public record Response(double temp) {}

        public Response get() {
            return new Response(30.0);
        }

    }

Expected behavior I expect an output something similar to "The current weather is 30 degrees".

Minimal Complete Reproducible example See the steps to reproduce for a code sample.

Comment From: stardif101

Hello,

Same problem here, but i've an explanation. When the tool don't have argument then bedrock is not sending a empty json in input...

Here the log with the problem (no json body for the tools):

o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockStartEvent(Start=ContentBlockStart(ToolUse=ToolUseBlockStart(ToolUseId=tooluse_hMRVZ4BSQousFl3fbd32Bw, Name=getTime)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockDeltaEvent(Delta=ContentBlockDelta(ToolUse=ToolUseBlockDelta(Input=)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockStopEvent(ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:MessageStopEvent(StopReason=tool_use)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ConverseStreamMetadataEvent(Usage=TokenUsage(InputTokens=667, OutputTokens=97, TotalTokens=764), Metrics=ConverseStreamMetrics(LatencyMs=2482))

After my workaround:

o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockStartEvent(Start=ContentBlockStart(ToolUse=ToolUseBlockStart(ToolUseId=tooluse_Ew5eUe0ZSh-eIMdKJlm4Bw, Name=getTime)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockDeltaEvent(Delta=ContentBlockDelta(ToolUse=ToolUseBlockDelta(Input=)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockDeltaEvent(Delta=ContentBlockDelta(ToolUse=ToolUseBlockDelta(Input={"messa)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockDeltaEvent(Delta=ContentBlockDelta(ToolUse=ToolUseBlockDelta(Input=ge": "Obte)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockDeltaEvent(Delta=ContentBlockDelta(ToolUse=ToolUseBlockDelta(Input=nir l'heure)), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockDeltaEvent(Delta=ContentBlockDelta(ToolUse=ToolUseBlockDelta(Input= actuelle"})), ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ContentBlockStopEvent(ContentBlockIndex=1)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:MessageStopEvent(StopReason=tool_use)
o.s.a.b.converse.BedrockProxyChatModel   : Received converse stream output:ConverseStreamMetadataEvent(Usage=TokenUsage(InputTokens=694, OutputTokens=95, TotalTokens=789), Metrics=ConverseStreamMetrics(LatencyMs=2720))
o.s.a.m.tool.DefaultToolCallingManager   : Executing tool call: getTime

Workaround: just add a dummy parameter in order to have a valid json for input.

Example:

@Description("Get current time as a message with dummy input (in order too prevent a bug in spring ai)")
    Function<GetTime.Request, GetTime.Response> getTime() {
        return new GetTime();
    }

Comment From: maschnetwork

Got the same issue and resolved by using a dummy parameter.

```java
@Tool(description = "Get the current date and time in the user's timezone") String getCurrentDateTime(String dummyParam) { return LocalDateTime.now().atZone(LocaleContextHolder.getTimeZone().toZoneId()).toString(); } ````

Comment From: tzolov

I've added those tests that show that both the Supplier and the tool methods without input arguments work fine with post M8 1.0.0-SNAPSHOT https://github.com/spring-projects/spring-ai/commit/d5bbb131bf759a4b61c000e053c57a9f3add0784

Comment From: tzolov

Added some additional (streaming) tests related to the same issue. All pass https://github.com/spring-projects/spring-ai/commit/ad783d9867fb0d105ff25f779f9615accc4b1635