Enhancing Typesense Search for Multiple Indexes in CRM Data
TLDR William was facing issues with Typesense's search performance on CRM data. JinW and Kishore Nallan suggested strategies, such as adjusting typesense tokens and creating a "concatenated" field for better search results.
Nov 08, 2021 (24 months ago)
William
01:35 AMBecause I am using it for searching CRM data across multiple indexes (customers, invoices, etc), it’s been critical to set drop_tokens_threshold=0.. otherwise ‘bob smith 7910’ brings back anyone who might be smith, or bob, or live at 7910, and this crowds out the other results and pushes them off-screen)
However, the inverse is true. Typesense doesn’t understand we want an ‘and’ search. I haven’t found great answers on how people have solved this previously.. the two approaches I can think of would be to:
1. create an additional ‘giant column of all text’ so that typesense can find all words. However, I presume I would then be giving up any kind of ranking control (i.e customer name ‘bob smith’ is a more relevant result than a customer whose notes mention they have a cat named ‘bob smith’)
2. Leave drop_tokens_threshold off the query so that we get the ‘OR’ results, and then filter out any hit where the ‘matchedWords’ array inside highlightResult doesn’t == the number of words we sent in the query.
Has anyone seen or heard of better results with one approach or the other?
JinW
01:58 AMKishore Nallan
02:06 AMTypesense uses a per-field index so it's difficult to do a global AND across fields like this. See this earlier discussion: https://typesense-community.slack.com/archives/C01P749MET0/p1636246311146500
Kishore Nallan
02:08 AMWilliam
02:15 AMTypesense
Indexed 2776 threads (79% resolved)
Similar Threads
Troubleshooting "drop_tokens_threshold" and Typo Tolerance in Typesense
Joe had issues with "drop_tokens_threshold" = 0 and typo tolerance in Typesense, after which Kishore Nallan provided solutions and clarifications on feature functionality. Their issues with the search result limit and tokens were resolved after discussion and testing.
Resolving Typesense Result Issue in Document Collection Queries
Mike was encountering errors when searching for specific query in their Typesense document collection. Jason suggested it may be due to the `drop_tokens_threshold` setting. There was a misunderstanding but after further explanation from Jason, Mike understood and decided to continue the conversation via email.
Adjusting Text Match Score Calculation in TypeSense
Johannes wanted to modify the Text Match Score calculation in TypeSense to improve search results returns. With counsel from Jason and Kishore Nallan, various solutions were proposed, including creating a Github issue, attempting different parameters, and updating Docker to a new version to resolve the matter.