Expected Behavior When using JdbcChatMemoryRepository it is expected that you will be able to continue a chat conversation keeping all the information that the User passed to the LLM model such as the text and also the Media resources (i.e.: for multimodal vision capable LLM models).
Current Behavior JdbcChatMemoryRepository, as it is designed in version 1.0.0, persists only the following information: - conversation_id (not an intrinsic field of the Message, passed in the saveAll method to group the messages pertaining to a given conversation) - content (the text part of the message) - type (the message type as a string representation of the Enum org.springframework.ai.chat.messages.MessageType) - timestamp (a database generated field, for ordered retrieval purpose)
This behaviour lack a way to persist the Media attached to a UserMessage.
Context With the use of JdbcChatMemoryRepository the LLM lose the capability to continue the conversation about a previously attached Media.
InMemoryChatMemory does not have any issues about keeping the Media attached to the Chat History.
I understand that the ChatMemory is not to be intended as a strategy to persist a Chat History but keeping Media attached to the UserMessage for the purpose of continuing a conversation about a given context (Media included) should be the standard behaviour of every ChatMemory implementation.
The only workaround I found is to write a custom implementation of ChatMemoryRepository capable to persist and retrieve the chat memory (at least for the UserMessage and AssistantMessage message types) serializing the content of the Media field if present.
Comment From: lovely-little-pineapple
I think it is not a good idea to save pictures, videos, and audio data in a database. If you want to save the above data, it is best to introduce it into OBS.