Bug description
Similar to: https://github.com/spring-projects/spring-ai/issues/2857 (but this is the MethodToolCallback
impl)
When LLM intent to call a tool, there is possibility that the provided argument is invalid. For example: - The tool argument is an enum - The LLM provide invalid enum
When this occur it should throw ToolExecutionException
instead of IllegalArgumentException
. Or it might even be a good idea to change ToolExecutionExceptionProcessor
process to all Exception instead of ToolExecutionException
so other possible edge cases can be handled gracefully easily
Stack trace:
java.lang.IllegalArgumentException: No enum constant __INVALID_ENUM_NAME__
at java.base/java.lang.Enum.valueOf(Enum.java:293)
at org.springframework.ai.util.json.JsonParser.toTypedObject(JsonParser.java:152)
at org.springframework.ai.tool.method.MethodToolCallback.buildTypedArgument(MethodToolCallback.java:150)
at org.springframework.ai.tool.method.MethodToolCallback.lambda$buildMethodArguments$1(MethodToolCallback.java:139)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:575)
Environment Spring AI 1.0.0 Java 17 Azure OpenAI gpt-4o
Steps to reproduce Create a tool that accepts an enum as a parameter, let's say: - ONE_DAY - TWO_DAY - THREE_DAY
If you ask LLM for FOUR_DAY or similar, it might try to call the tool, even if the input schema doesn't list it so
Expected behavior
It should throw ToolExecutionException
so it can be sent back to model, or handle gracefully at ToolExecutionExceptionProcessor
.
Workaround (create a custom MethodToolCallback
with a delegate):
@RequiredArgsConstructor
public class CustomMethodToolCallback implements ToolCallback {
@Delegate
private final MethodToolCallback delegate;
@Override
public String call(String toolInput) {
return call(toolInput, null);
}
@Override
public String call(String toolInput, @Nullable ToolContext toolContext) {
try {
return delegate.call(toolInput, toolContext); // Not perfect, but acceptable
} catch (IllegalArgumentException iae) {
throw new ToolExecutionException(getToolDefinition(), iae);
}
}
}
Comment From: wilocu
take