Expected Behavior
I would like to be able to support multiple indexes or collections for vector data instead of being limited to configuring just one index. For example, in Redis or Qdrant implementations, it would be beneficial to have the ability to add vectors to different collections or indexes based on the business requirements.
An example method signature could be:
void add(List<Document> documents, String collectionName);
Current Behavior
https://github.com/spring-projects/spring-ai/blob/7634c6b78087a708a1dff8592b1485806da8ed4d/spring-ai-core/src/main/java/org/springframework/ai/vectorstore/VectorStore.java#L32-L73
Currently, the system only allows for configuring a single index or collection for storing vector data. This limitation restricts the flexibility and scalability of the system, especially when there is a need to store vectors in multiple separate collections or indexes.
Context
This feature request is essential for scenarios where there is a requirement to store and retrieve vector data from different collections or indexes based on specific use cases or business logic. Without the ability to support multiple indexes or collections, the current system falls short in meeting the diverse needs of various applications that rely on vector data storage.
I have considered the workaround of creating separate instances of the vector storage system for each collection needed, but this approach is not scalable and creates unnecessary complexity in managing multiple instances.
Having an add method that supports specifying the collection name or index where the documents should be added would greatly enhance the flexibility and usability of the system.
Comment From: markpollack
this is an important request. We will be reviewing all core APIs and will take this into account. Thanks for taking the time to raise an issue.
Comment From: jaruizes
As a possible work around, using metadata would be a temporal solution? For instance, adding a metadata "collection=XXXX" when we are going to store the information and then, using that metadata as a filter when we are querying data
Comment From: humanry
+1
Comment From: zachary-zhaoqi
+1