The current Spring AI tool calling mechanism (as documented here) is a powerful way to extend the capabilities of AI models by allowing them to interact with external APIs and services.

However, there are two key features currently missing that would significantly improve the security, flexibility, and usability of tool calling: - Parameter Binding: The ability to pre-define or "bind" values to specific tool parameters before the model has a chance to interact with them. - Authentication: A standardized way to securely authenticate tool calls, especially when those tools access sensitive data or require user-specific authorization.

Expected Behavior

  • Non-Breaking Changes: The proposed additions should not require changes to existing core interfaces like ChatClient, ChatModel, or ToolCallingManager.
  • Extensibility: The design should be flexible enough to accommodate different authentication mechanisms (OAuth, API keys, etc.) and custom parameter binding logic.
  • Ease of Use: The API should be intuitive and easy to use for developers defining and calling tools.
  • Consistency: The implementation should align with existing Spring AI design principles and coding style.
  • Security: Sensitive information (e.g., API keys) should never be exposed to the LLM.
  • Correct Schema Handling: Parameter binding must correctly modify the JSON schema presented to the LLM, preventing it from attempting to provide values for already-bound parameters.
  • Tool Context Integration: Authentication should leverage the existing ToolContext mechanism for passing data to tools.

Drawing inspiration from the toolbox-langchain Python SDK (link to toolbox-langchain SDK documentation). The goal is to make Spring AI's tool calling more robust, secure, and suitable for production environments.

  • Code: https://github.com/googleapis/genai-toolbox-langchain-python/blob/main/src/toolbox_langchain/tools.py
  • Announcement https://cloud.google.com/blog/products/ai-machine-learning/announcing-gen-ai-toolbox-for-databases-get-started-today

Comment From: poo0054

Repeat the question and record it #2757