Azure AI Search has the following feature along with Vector Similarity Search (which is already present in Spring AI).

  • Hybrid Search (Vector Search + Text Search)
  • Semantic Hybrid Search (Hybrid search + re-ranking)

In the JavaScript Langchain project they basically have the 3 search models in 3 functions next to each other:

  1. similaritySearchVectorWithScore - this is the current vector search Spring AI already has
  2. hybridSearchVectorWithScore - this is vector search + text search
  3. semanticHybridSearchVectorWithScore - this is hybrid search + reranking

I see something similar was recently introduced in the Langchain4j project too. Would be good to have this feature in Spring AI as well.

Comment From: markpollack

Hi. Yes, this is a good idea.

Comment From: markpollack

related issue - https://github.com/spring-projects/spring-ai/issues/579

Comment From: alessiobertazzo88

Hello everyone, I have developed this feature to enable users to choose their preferred search mode. Within the SearchRequest, it is possible to use similarity search in line with what is currently available (vector-based, enabled by default), full-text search, or both (hybrid search). Additionally, it provides the option to use reranking approaches as well.

This implementation is applied to Azure SearchAI as first vector store, but new methods defaults can be overridden to enhance the feature for all the others compatible.

This is the PR #1227

Comment From: iAMSagar44

Hi @markpollack / @csterwa / @alessiobertazzo88

I have done some analysis on using the Hybrid Search with Semantic Ranking in Azure AI Search. The existing code in Spring AI for filtering the search results (response from Azure AI Search) based on the result score and similarity threshold will not work. This is due to a different algorithm used by AI Search when semantic ranking is used (please refer to the last section named ‘Ranking’ in this link - Hybrid Search). In short, with the Hybrid Search, the result.getScore() can appear quite low, even with a high similarity match and the following code, which is currently present in the Spring AI Azure Vector Store class will always return false - .filter(result -> result.getScore() >= request.getSimilarityThreshold())

With the standard vector search, the result.getScore()could return a value ‘0.8399121’ for a high similarity match for a user query. When tested with the same user query, but with Hybrid Search + Semantic Ranking, the result.getScore() could return a value of ‘0.03278688341379’ for a high similarity match.

To filter the documents appropriately when Hybrid + Semantic Ranking is used, a better option is to use the @search.rerankerScore which is returned in the search results. This score ranges between 0 - 4 (Semantic Ranking Scores). I tried the following code and was able to get satisfactory results - .filter(result -> result.getSemanticSearch().getRerankerScore() >= 2)

We might need an additional property for Azure AI Search, like the ’similarity threshold’ property which is specifically used for Hybrid Search + Semantic Ranking.

Also, with my analysis and testing so far, I noticed that the default Top K parameter and Similarity Threshold has no impact to the total of the Search Results returned when Hybrid Search + Semantic Ranking is used. We will need to use a different parameter (called as Top) when defining the SearchOptions to restrict the amount of results returned. See code below for an example -

var searchOptions = new SearchOptions()
                .setVectorSearchOptions(new VectorSearchOptions().setQueries(vectorQuery))
                .setQueryType(QueryType.SEMANTIC)
                .setSemanticSearchOptions(
                        new SemanticSearchOptions().setSemanticConfigurationName("semantic-config-nid")
                                .setQueryAnswer(new QueryAnswer(QueryAnswerType.EXTRACTIVE))
                                .setQueryCaption(new QueryCaption(QueryCaptionType.EXTRACTIVE)))
                .setIncludeTotalCount(true)
                .setTop(4); // This needs to be set to limit the results returned

        final var searchResults = getSearchClient().search(query, searchOptions, Context.NONE);

And below is the code to filter on the search results based on semantic ranking score

return searchResults.stream()
                .filter(result -> result.getSemanticSearch().getRerankerScore() >= 2)
                .map(result -> {
                    logger.info("Re-rank score: {}", result.getSemanticSearch().getRerankerScore());

                    final AzureSearchDocument entry = result.getDocument(AzureSearchDocument.class);

                    Map<String, Object> metadata = (StringUtils.hasText(entry.metadata()))
                            ? JSONObject.parseObject(entry.metadata(), new TypeReference<Map<String, Object>>() {

                            })
                            : Map.of();

                    metadata.put("re-rank_score", result.getSemanticSearch().getRerankerScore());

                    final Document doc = new Document(entry.id(), entry.content(), metadata);
                    doc.setEmbedding(EmbeddingUtils.toPrimitive(entry.embedding()));

                    return doc;

                })
                .collect(Collectors.toList());

Hope this helps. Can you also please confirm if this feature is planned to be introduced in the M5 Release?

Comment From: sobychacko

@iAMSagar44 Based on your analysis, the directions taken in this PR are incomplete (apart from the other reviews about re-ranking agent, etc.). Do you agree with that? We are trying to work on these sets of features and trying to establish a baseline. For various vector stores, these implementation details are going to be different, but we can come up with a unified API that supports full-text and hybrid search.