Bug description While using WatsonxAiChatModel I see the request is sent to the following endpoint: /ml/v1/text/generation?version=2023-05-29 https://cloud.ibm.com/apidocs/watsonx-ai#text-generation

while I expect it should use the following one: /ml/v1/text/chat?version=2023-05-29 https://cloud.ibm.com/apidocs/watsonx-ai#text-chat

and consequently all the parameters associated with the second API (such as tools)

watsonx-ai.text-chat is very close to opeai.completions

Environment Spring AI 1.0.0-M6, Java version 21

Steps to reproduce add the following fragment to the pom.xml

        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-watsonx-ai-spring-boot-starter</artifactId>
        </dependency>

Expected behavior I think that the WatsonxAiChatModel should be changed and be more close to the OpenAiChatModel

Comment From: apappascs

Hi @FabioAng71,

Both the /ml/v1/text/generation and /ml/v1/text/chat endpoints are valid Watsonx AI APIs. The current WatsonxAiChatModel in Spring AI is designed to use the text generation API /ml/v1/text/generation as you mentioned already. WatsonxAi Connection Properties

If your use case requires OpenAI API-compatible behavior the recommended approach is to use the spring-ai-openai-spring-boot-starter instead. You can configure it to point to Watsonx AI’s text-chat API by setting:

spring.ai.openai.chat.base-url=https://your-watsonx-instance-url
spring.ai.openai.chat.completions-path=/ml/v1/text/chat?version=2023-05-29

same approach as Deepseek and Groq

Let me know if this works for you or if you have any further questions!?

Comment From: FabioAng71

Hi @apappascs.

I followed you seggustion but it doesn't work for several reasons: 1)spring.ai.openai.chat.base-url doesn't override for the spring.ai.openai.base-url, so I used spring.ai.openai.base-url

2) the autentication token for watsonx.ai is generated from the IAM Token using the following api: https://cloud.ibm.com/docs/account?topic=account-iamtoken_from_apikey&code=curl#iamtoken so we need to generate a Bearer using any watsonx.ai endpoint

3) in the openai chat endpoint (https://platform.openai.com/docs/guides/text-generation) the model is specified in the 'model' field while in the watsonx.ai chat endpoint (https://cloud.ibm.com/apidocs/watsonx-ai#text-chat) the model is specified in the 'model_id' field

4) in the openai chat endpoint project_id is optional and it is in the header (https://platform.openai.com/docs/api-reference/authentication) while in the watsonx.ai chat endpoint the project_id is mandatory and it is in the body as 'project_id' field

5) most important, if WatsonxAiChatModeluse https://cloud.ibm.com/apidocs/watsonx-ai#text-generation it is not possibile to use Tools (https://docs.spring.io/spring-ai/reference/api/tools.html )and Advisors (https://docs.spring.io/spring-ai/reference/api/advisors.html) beacuse that api doesn't support them.

The right solution should be to re-implement WatsonxAiChatModel using https://cloud.ibm.com/apidocs/watsonx-ai#text-chat to fully satisfy the ChatClient interface (https://docs.spring.io/spring-ai/reference/api/chatclient.html).

Let me know your thoughts

Comment From: apappascs

@FabioAng71 Thank you for looking into this. Based on your findings, it seems that it's not compatible with the OpenAI API. However, to be honest, I'm not sure which of these endpoints is preferable or why.

I couldn't find any documentation comparing the two:

Comment From: FabioAng71

@apappascs If you look at WatsonxAi Chat API you will see it is very close to OpenAi Chat API apart the differences I mentioned at point 2) 3) 4) of my previous post. The request has the same structure as for the response.

On the other hand WatsonxAi Text Generation API is just a simple text request and a simple text response.

Looking into IBM documentaion for Text Generation API it is cleary stated that the correct API is WatsonxAi Chat API For chat use cases, use the Chat API. See Adding generative chat function to your applications with the chat API.

Comment From: FabioAng71

@apappascs Let me know if you need help for the implementation or for the testing :-)

Comment From: aichemzee

Hi @FabioAng71 / @apappascs , I am also interested in using spring ai with watsonx, along with tool calling support. Just curious if you were able to move past this and able to get the wiring right?

Right now I'm leaning towards re writing WatsonxAiChatModel for my specific needs. But wanted to check if you were able to proceed somehow?

Comment From: FabioAng71

@aichemzee I didn't receive any news about the issue. I am still thinking to rewrite WatsonxAiChatModel starting from OpenAiChatModel but I would like to hava a sign from SpringAI community to understand if it is in their plan or not.

Comment From: aichemzee

@FabioAng71 I think the implementation for watsonx is dated and would require a re write. Maybe we can raise a PR and have it reviewed. What do you think?

Comment From: markpollack

Hi folks. This is very timely as we are going to have an RC and finally a GA release next month. I do not have access to the watson models and this presents an issue in terms of providing support for Watson.

It very much sounds like we need a new WatsonAiChatModel implementation. I don't think we will have the time to make the current OpenAiChatModel more reusable by other OpenAI API like models. There are always some differences in request/response data formats or other nuanced. We accomodated deepseek request options in the openaimodel already but that wasn't enough.

So in terms of going forward.

  1. Is it possible to get access to Watson for us to develop against and to run our CI/CD process against.
  2. If we can get access and the timeline for RC (May 13) is ok for you to contribute a new module, we can incorporate it into spring ai. Otherwise, we have created a new spring-ai-community organization on github. You can ask to create a project there and then develop it at your own pace. More information is at https://github.com/spring-ai-community/community/wiki

Comment From: aichemzee

@FabioAng71 so this is the sign off we needed. I'm still working towards trying to get access to a watsonx env and it might take some time for that to go through. In the meantime could we finalize a spec for request response models based on the documentation? Also would you be interested in bringing out a PR? If not maybe I can take a stab at it.

Also @markpollack . What would be the process for getting access to a watson env for ci/cd? And would that be a hard requirement before a PR is accepted?

Comment From: markpollack

@aichemzee I've been trying to get access to the models for a long time, it hasn't worked out yet. I'm in an email exchange with someone at IBM who may be able to help. Yes, we can't accept any PRs if we can't test. We were pretty lose on this in the beginning, with contributors who worked at various companies that could test internally, but as we approach GA we can't continue doing that. I suggest you create a new community project for watson chat at https://github.com/spring-ai-community/community/issues

I'm going to be removing the older model support now.

Comment From: rjtmahinay

@apappascs Tools and advisors shall be created under the abstraction as per https://github.com/spring-projects/spring-ai/issues/1811

This is the issue where I commented if watsonx Tool Calling and Advisors can be supported https://github.com/spring-projects/spring-ai/issues/1366#issuecomment-2563097023

Hi @apappascs.

I followed you seggustion but it doesn't work for several reasons: 1)spring.ai.openai.chat.base-url doesn't override for the spring.ai.openai.base-url, so I used spring.ai.openai.base-url

  1. the autentication token for watsonx.ai is generated from the IAM Token using the following api: https://cloud.ibm.com/docs/account?topic=account-iamtoken_from_apikey&code=curl#iamtoken so we need to generate a Bearer using any watsonx.ai endpoint
  2. in the openai chat endpoint (https://platform.openai.com/docs/guides/text-generation) the model is specified in the 'model' field while in the watsonx.ai chat endpoint (https://cloud.ibm.com/apidocs/watsonx-ai#text-chat) the model is specified in the 'model_id' field
  3. in the openai chat endpoint project_id is optional and it is in the header (https://platform.openai.com/docs/api-reference/authentication) while in the watsonx.ai chat endpoint the project_id is mandatory and it is in the body as 'project_id' field
  4. most important, if WatsonxAiChatModeluse https://cloud.ibm.com/apidocs/watsonx-ai#text-generation it is not possibile to use Tools (https://docs.spring.io/spring-ai/reference/api/tools.html )and Advisors (https://docs.spring.io/spring-ai/reference/api/advisors.html) beacuse that api doesn't support them.

The right solution should be to re-implement WatsonxAiChatModel using https://cloud.ibm.com/apidocs/watsonx-ai#text-chat to fully satisfy the ChatClient interface (https://docs.spring.io/spring-ai/reference/api/chatclient.html).

Let me know your thoughts