Question:

If I use DeepSeekChatModel to configure Qwen3, when hitting ToolCall, an exception ava.lang.IllegalArgumentException: toolInput cannot be null or empty will be thrown. This is because Qwen3 ToolCall Response is streaming, and we have not handled this situation.

Example：

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904475,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[{"index":0,"delta":{"role":"assistant","content":"","reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904476,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":"call_0d94d57a4ec94353a4eaa6d7","index":0,"type":"function","function":{"name":"getweather","arguments":""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904476,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"{\"city\": \""}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904476,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"Be"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904476,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":[{"id":null,"index":0,"type":"function","function":{"name":null,"arguments":"ijing\"}"}}]},"logprobs":null,"finish_reason":null,"matched_stop":null}],"usage":null}

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904476,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[{"index":0,"delta":{"role":null,"content":null,"reasoning_content":null,"tool_calls":null},"logprobs":null,"finish_reason":"tool_calls","matched_stop":151645}],"usage":null}

data: {"id":"91afc9cbf3a347f987df10e8e96f2495","object":"chat.completion.chunk","created":1754904476,"model":"qwen3-235b-a22b-instruct-2507-fp8","choices":[],"usage":{"prompt_tokens":124,"total_tokens":145,"completion_tokens":21,"prompt_tokens_details":null}}

data: [DONE]

Resolve

We need merge those chunks , possible suggestions like this

Plan A

Add merging logic to DeepSeekChatModel

  • Before https://github.com/spring-projects/spring-ai/blob/c7f7b68063aa8dd12335e1109e969297258c7ca4/models/spring-ai-deepseek/src/main/java/org/springframework/ai/deepseek/DeepSeekChatModel.java#L327-L345

  • After

    private List<AssistantMessage.ToolCall> merge(List<AssistantMessage.ToolCall> tools) {
        if (tools.isEmpty()) {
            return tools;
        }


        Map<String, AssistantMessage.ToolCall> completedTools = new HashMap<>();
        AssistantMessage.ToolCall last = null;


        for (AssistantMessage.ToolCall t : tools) {
            if (!t.id().isEmpty()) {
                last = t;
            } else {
                last = mergeTool(last, t);
            }


            completedTools.put(last.id(), last);
        }


        return completedTools.values().stream().toList();
    }


    private AssistantMessage.ToolCall mergeTool(AssistantMessage.ToolCall target, AssistantMessage.ToolCall from) {
        if (target == null) {
            return from;
        }


        if (from == null) {
            return target;
        }


        String argument = from.arguments() == null || from.arguments().isEmpty() ? target.arguments() : target.arguments() + from.arguments();
        return new AssistantMessage.ToolCall(target.id(), target.type(), target.name(), argument);
    }
  ```
And use it in buildGeneration method
  ```
    private Generation buildGeneration(Choice choice, Map<String, Object> metadata) {
        List<AssistantMessage.ToolCall> toolCalls = choice.message().toolCalls() == null ? List.of()
                : merge(choice.message()
                .toolCalls()
                .stream()
                .map(toolCall -> new AssistantMessage.ToolCall(toolCall.id(), "function",
                        toolCall.function().name(), toolCall.function().arguments()))
                .toList());


        String finishReason = (choice.finishReason() != null ? choice.finishReason().name() : "");
        var generationMetadataBuilder = ChatGenerationMetadata.builder().finishReason(finishReason);


        String textContent = choice.message().content();
        String reasoningContent = choice.message().reasoningContent();


        QwenAssistantMessage assistantMessage = new QwenAssistantMessage(textContent, reasoningContent,
                metadata, toolCalls);
        return new Generation(assistantMessage, generationMetadataBuilder.build());
    }

Plan B

add QwenChatModel module

https://github.com/shenweijiekdel/spring-ai/blob/d4c84429bbeb996747b40bd30ab2ebe8c29bf2d1/models/spring-ai-qwen/src/main/java/org/springframework/ai/qwen/QwenChatModel.java#L312-L364

This fork contains this fix, currently works fine.

How should we choose a solution? Thank you.