Hello! I am having trouble with hybrid search at t...
# community-help
j
Hello! I am having trouble with hybrid search at the moment. It appears that i get incorrect ranking in certain circumstances. In particular, when there is a tie in the text matching, i expect to get the document with the lowest vector distance on top, but this is not happening From the documentation we have
Copy code
K = rank of document in keyword search
S = rank of document in semantic search

rank_fusion_score = 0.7 * K + 0.3 * S
It appears that if we have several hits with the same text match, they will get an arbitrary keyword search rank. This arbitrary rank will then be weighted together with the semantic search rank. The result seems to be arbitrary in the end, even if semantically closer documents clearly should be on top. This is something that we experience as random ordering for many searches, which is not great. Perhaps this algorithm can be adjusted to allow several documents with the same keyword search rank, in order to make the semantic search rank the tie breaker.