Hybrid Search with Keyword and Vector Search
TLDR R.A.I.D asked about controlling weights between keyword-based and vector-based search results, with emphasis on personal customization. Kishore Nallan confirmed this option will be possible in the 0.25 release.
Jul 04, 2023 (5 months ago)
R.A.I.D
10:22 AMHow can I control the weights/balance between the keyword based search result (and item popularity sorting) and the contribution from the vector search part?
Kishore Nallan
01:07 PMJul 05, 2023 (5 months ago)
R.A.I.D
08:54 AMRegarding the gist on hybrid search:
https://gist.github.com/jasonbosco/f4187f6b4f585d2dc8902af85408994a
When reading the gist it looks like we have to provide a semantic model and let Typesense generate embeddings for each item using this model.
We are not aiming for a hybrid of keyword and /semantic search/. We have our own local model for generating embeddings. We would like to enrich our index with one embeddings for each item. And then manually provide another embedding at query-time.
We then want the hybrid search result to be a mix of the keyword search and vector search (based on similarity on embeddings).
Is this possible?
And can we control the weights between the keyword- and the vector-search? Or is it fixed to 0.7 vs 0.3?
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Questions on Hybrid Search Feature in Upcoming Release
Chetan inquired about the compatibility and parameter control of hybrid search feature. Jason clarified its functionality, indicating its compatibility with manually created vec field and parameters affecting keyword vs semantic search results.
Discussion on Calculating Vector Distance in Hybrid Search Results
Narayan raised concerns on the `vector_distance` results of keyword hits in hybrid search, suggesting it could be manually calculated. Jason explained the algorithm's design, where `vector_distance` does not apply to keyword searches. They agreed on future considerations for scenarios relying on vector distance.
Utilizing Vector Search and Word Embeddings for Comprehensive Search in Typesense
Bill sought clarification on using vector search with multiple word embeddings in Typesense and using them instead of OpenAI's embedding. Kishore Nallan and Jason informed him that their development version 0.25 supports open source embedding models. They also resolved Bill's concerns regarding search performance, language support, and limitations in the search parameters.