Hi Team, First of all thank you so much for your effort on creating this Spring AI module. I am currently exploring Structured output entity feature to get the structured out in the format I wanted using few a Few Shot JSON example in the prompt itself.
Its throwing an errorjava.lang.IllegalArgumentException: The template string is not valid
Complete error details are given below.
Environment
Local Environment
Prompt
Extract the key information from the following text delimited by triple backticks and format it in JSON.
I need details like name, booking date, flight information (flight number, origin, destination, departure/arrival times),s
luggage details, ticket price, and seat number.
Here is an example output of the JSON format:\n
{
"name": "John Doe",
"booking_date": "January 11, 2024",
"flight_info": {
"flight_number": "123",
"origin_airport_code": "JFK",
"origin_city": "New York",
"destination_airport_code": "LAX",
"destination_city": "Los Angeles",
"departure_time": "8:00 AM",
"arrival_time": "11:30 AM"
},
"luggage": {
"carry_on": "1",
"checked_bag": "1"
},
"ticket_price": {
"value": "450.00",
"currency": "DOLLAR"
},
"seat_number": "14A"
}
Text: ```Emily Thompson booked a flight on October 10, 2024. She will be flying from New York (JFK) to Los Angeles (LAX) on flight number AA123.The departure time is 8:00 AM, and the arrival time is 11:30 AM. She has a carry-on bag and a checked bag. Her ticket price was $450.00, and she will be seated in 14A.```'
String template file : flight_details_fewshot.st
Extract the key information from the following text delimited by triple backticks and format it in JSON.
I need details like name, booking date, flight information (flight number, origin, destination, departure/arrival times),s
luggage details, ticket price, and seat number.
Here is an example output of the JSON format:\n
{jsonexample}
Text: ```{input}```
Controller:
@Value("classpath:/prompt-templates/structured_outputs/flight_details_fewshot.st")
private Resource flightBookingFewShot;
@PostMapping("/v1/structured_outputs/entity/fewshot")
public Object entityFewShot(@RequestBody @Valid UserInput userInput) {
log.info("userInput message : {} ", userInput);
String jsonExample = "{\n" +
" \"name\": \"John Doe\",\n" +
" \"booking_date\": \"January 11, 2024\",\n" +
" \"flight_info\": {\n" +
" \"flight_number\": \"123\",\n" +
" \"origin_airport_code\": \"JFK\",\n" +
" \"origin_city\": \"New York\",\n" +
" \"destination_airport_code\": \"LAX\",\n" +
" \"destination_city\": \"Los Angeles\",\n" +
" \"departure_time\": \"8:00 AM\",\n" +
" \"arrival_time\": \"11:30 AM\"\n" +
" },\n" +
" \"luggage\": {\n" +
" \"carry_on\": \"1\",\n" +
" \"checked_bag\": \"1\"\n" +
" },\n" +
" \"ticket_price\": {\n" +
" \"value\": \"450.00\",\n" +
" \"currency\": \"DOLLAR\"\n" +
" },\n" +
" \"seat_number\": \"14A\"\n" +
"}";
var promptTemplate = new PromptTemplate(flightBookingFewShot);
var message = promptTemplate.createMessage(Map.of("input", userInput.prompt(), "jsonexample", jsonExample));
var promptMessage = new Prompt(List.of(message));
log.info("Prompt : \n {}", promptMessage);
var requestSpec = chatClient.prompt(promptMessage);
var booking = requestSpec.call().entity(FlightBooking.class);
log.info("booking : {} ", booking);
return booking;
}
Error
{
"timestamp": "2025-03-21T09:49:51.580+00:00",
"status": 500,
"error": "Internal Server Error",
"trace": "java.lang.IllegalArgumentException: The template string is not valid.\n\tat org.springframework.ai.chat.prompt.PromptTemplate.<init>(PromptTemplate.java:86)\n\tat org.springframework.ai.chat.client.advisor.api.AdvisedRequest.toPrompt(AdvisedRequest.java:171)\n\tat org.springframework.ai.chat.client.DefaultChatClient$DefaultChatClientRequestSpec$1.aroundCall(DefaultChatClient.java:680)\n\tat org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.lambda$nextAroundCall$1(DefaultAroundAdvisorChain.java:98)\n\tat io.micrometer.observation.Observation.observe(Observation.java:565)\n\tat org.springframework.ai.chat.client.advisor.DefaultAroundAdvisorChain.nextAroundCall(DefaultAroundAdvisorChain.java:98)\n\tat org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doGetChatResponse(DefaultChatClient.java:493)\n\tat org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.lambda$doGetObservableChatResponse$1(DefaultChatClient.java:482)\n\tat io.micrometer.observation.Observation.observe(Observation.java:565)\n\tat org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doGetObservableChatResponse(DefaultChatClient.java:482)\n\tat org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.doSingleWithBeanOutputConverter(DefaultChatClient.java:456)\n\tat org.springframework.ai.chat.client.DefaultChatClient$DefaultCallResponseSpec.entity(DefaultChatClient.java:451)\n\tat com.llm.structuredoutputs.StructuredOutputsController.entityFewShot(StructuredOutputsController.java:147)\n\tat java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)\n\tat java.base/java.lang.reflect.Method.invoke(Method.java:580)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188)\n\tat org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)\n\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926)\n\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831)\n\tat org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)\n\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)\n\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979)\n\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)\n\tat org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914)\n\tat jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590)\n\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)\n\tat jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)\n\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)\n\tat org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)\n\tat org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)\n\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)\n\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)\n\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)\n\tat org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:384)\n\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)\n\tat org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:905)\n\tat org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)\n\tat org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)\n\tat org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190)\n\tat org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)\n\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: org.stringtemplate.v4.compiler.STException\n\tat org.stringtemplate.v4.compiler.Compiler.reportMessageAndThrowSTException(Compiler.java:224)\n\tat org.stringtemplate.v4.compiler.Compiler.compile(Compiler.java:154)\n\tat org.stringtemplate.v4.STGroup.compile(STGroup.java:514)\n\tat org.stringtemplate.v4.ST.<init>(ST.java:162)\n\tat org.stringtemplate.v4.ST.<init>(ST.java:156)\n\tat org.springframework.ai.chat.prompt.PromptTemplate.<init>(PromptTemplate.java:80)\n\t... 60 more\n",
"message": "The template string is not valid.",
"path": "/springai/v1/structured_outputs/entity/fewshot"
}
Spring AI Version
set('springAiVersion', "1.0.0-M6")
Expected output
{
"name": "Li Wei",
"booking_date": "November 5, 2024",
"flight_info": {
"flight_number": "CA456",
"origin_airport_code": "PEK",
"origin_city": "Beijing",
"destination_airport_code": "PVG",
"destination_city": "Shanghai",
"departure_time": "10:00 AM",
"arrival_time": "12:30 PM"
},
"luggage": {
"carry_on": "1",
"checked_bag": "1"
},
"ticket_price": {
"value": "3200.00",
"currency": "YUAN"
},
"seat_number": "22C"
}
Working example without the entity() function call.
When I run the same example but just use the chatClient.prompt(promptMessage).call.content(), its working.
@PostMapping("/v1/structured_outputs/fewshot")
public Object chat1(@RequestBody @Valid UserInput userInput) {
log.info("userInput message : {} ", userInput);
var promptTemplate = new PromptTemplate(flightBookingFewShot);
var message = promptTemplate.createMessage(Map.of("input", userInput.prompt(), "jsonexample", CommonUtil.flightJson()));
var promptMessage = new Prompt(List.of(message));
var requestSpec = chatClient.prompt(promptMessage);
log.info("requestSpec : {} ", requestSpec);
return requestSpec.call().content();
// var responseSpec = requestSpec.call().entity(FlightBooking.class);
// return responseSpec;
}
This kind of interaction is pretty common to drive the LLM to map the right values into JSON properties so that the application can take necessary action on them.
Fixing this would be a really helpful in dealing with Structured outputs.
Thanks, Dilip Sundarraj
Comment From: danilalisichkin
I also encountered this problem. The problem occures in org.springframework.ai.chat.client.advisor.api.AdvisedRequest.toPrompt() method:
public Prompt toPrompt() {
ArrayList<Message> messages = new ArrayList(this.messages());
String processedSystemText = this.systemText();
if (StringUtils.hasText(processedSystemText)) {
if (!CollectionUtils.isEmpty(this.systemParams())) {
processedSystemText = (new PromptTemplate(processedSystemText, this.systemParams())).render();
}
messages.add(new SystemMessage(processedSystemText));
}
String formatParam = (String)this.adviseContext().get("formatParam");
String processedUserText = StringUtils.hasText(formatParam) ? this.userText() + System.lineSeparator() + "{spring_ai_soc_format}" : this.userText();
if (StringUtils.hasText(processedUserText)) {
Map<String, Object> userParams = new HashMap(this.userParams());
if (StringUtils.hasText(formatParam)) {
userParams.put("spring_ai_soc_format", formatParam);
}
if (!CollectionUtils.isEmpty(userParams)) {
processedUserText = (new PromptTemplate(processedUserText, userParams)).render();
}
messages.add(new UserMessage(processedUserText, this.media()));
}
ChatOptions var6 = this.chatOptions();
if (var6 instanceof FunctionCallingOptions functionCallingOptions) {
if (!this.functionNames().isEmpty()) {
functionCallingOptions.setFunctions(new HashSet(this.functionNames()));
}
if (!this.functionCallbacks().isEmpty()) {
functionCallingOptions.setFunctionCallbacks(this.functionCallbacks());
}
if (!CollectionUtils.isEmpty(this.toolContext())) {
functionCallingOptions.setToolContext(this.toolContext());
}
}
return new Prompt(messages, this.chatOptions());
}
Under the hood .entity() method uses AdvisorsApi, adding "formatParam" Adviser to AdviseContext for ChatClient. We have:
String processedUserText = StringUtils.hasText(formatParam) ? this.userText() + System.lineSeparator() + "{spring_ai_soc_format}" : this.userText();
formatParam- helper message, obtained from AdviseContext by the "formatParam" keythis.userText()- your user's message: your JSON;{spring_ai_soc_format}- a special template used to substitute a message about how LLM should structure its answer, stored in userParams As a result, we get text that contains our JSON along with {}, as well as a line-glue{spring_ai_soc_format}.
Then
processedUserText = (new PromptTemplate(processedUserText, userParams)).render();
is called and PromptTemplate tries to substitute the string, but our JSON is treated as a placeholder.
Well, I think that the main problem of this method is userText concatenation before the format message substitution. Is there any reason why it is made this way? I think it would be better to allow the client to configure how the template is applied to userText.
As stated in other open issues, you can escape { with {{ and } with }} (or \} etc.) , but then LLM may report that the data is in an invalid format: invalid JSON structure. So you have to add another helper message about treating {{}} as {}, but that's a crutch. Maybe this problem can be solved by using ModelApi and Prompt natively, but I'd like to use ChatClientApi. I think it makes sense that I use structured input to get structured output...