Troubleshooting "drop_tokens_threshold" and Typo Tolerance in Typesense
TLDR Joe had issues with "drop_tokens_threshold" = 0 and typo tolerance in Typesense, after which Kishore Nallan provided solutions and clarifications on feature functionality. Their issues with the search result limit and tokens were resolved after discussion and testing.
3
Nov 07, 2021 (26 months ago)
Joe
12:51 AMHow can I limit results, such that the matched document must (fuzzy) contain all terms in the query (not necessarily in order, or on the same attributes)?
Joe
12:52 AMdavis lorum ipsum
With the example query, all documents that include "davis", are being matched, despite having no mention of "lorum" or "ipsum", in any of the document attributes. No matter how many non-matching words I add at the end of the query, it still returns documents that only match the first word.
How can I limit results, such that the matched document must (fuzzy) contain all terms in the query (not necessarily in order, or on the same attributes)? e.g.
{name: 'davis', class: 'lorum etc...', 'notes': 'call ipsum'}
should match.Kishore Nallan
03:04 AMtypesense/typesense:0.22.0.rcs25
Docker build.Joe
03:28 AMJinW
03:37 AMKishore Nallan
03:38 AMhttps://dl.typesense.org/releases/0.22.0.rcs25/typesense-server-0.22.0.rcs25-amd64.deb
We keep fixing some small edge cases that we encounter and last mile performance regressions as we head to the final GA build.
1
Joe
03:40 AMKishore Nallan
03:41 AM1
Joe
03:43 AMKishore Nallan
03:45 AMKishore Nallan
03:46 AMJoe
03:58 AMJoe
04:00 AMKishore Nallan
04:03 AMdrop_tokens_thresold: 0
you are saying that don't drop any tokens from the query string. So Typesense will look for fields that contain both tokens jim
and baker
.Kishore Nallan
04:03 AMJoe
04:06 AMdavis lorum ipsum
)Kishore Nallan
04:08 AMJoe
04:08 AMKishore Nallan
04:09 AMKishore Nallan
04:09 AMhighlight_fields
parameter during search.Joe
04:09 AMKishore Nallan
04:11 AMJoe
04:13 AMKishore Nallan
04:17 AMJoe
04:18 AM_text_match"
will be higher for document that has more matching tokens?, if so I could just terminate the search as soon as first result in encountered with insufficient tokens.Kishore Nallan
04:19 AMJoe
04:19 AMKishore Nallan
04:20 AM_text_match
will be higher for documents with better match, both is number of tokens found and how near they are found to each other in terms of proximity.Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Phrase Search Relevancy and Weights Fix
Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.
Issue with Query Expectations on Typesense Search
Sean was having an issue with their search query on Typesense. Kishore Nallan suggested adjusting the 'drop_tokens_threshold' parameter. After making the adjustment, Sean found an issue with the order of the results, which was resolved by updating Typesense version.
Issues with Repeated Words and Hyphen Queries in Typesense API
JinW discusses issues with repeated word queries and hyphen-containing queries in Typesense. Kishore Nallan offers possible solutions. During the discussion, Mr seeks advice on `token_separators` and how to send custom headers. Issues remain with repeated word queries.
Resolving Typesense Result Issue in Document Collection Queries
Mike was encountering errors when searching for specific query in their Typesense document collection. Jason suggested it may be due to the `drop_tokens_threshold` setting. There was a misunderstanding but after further explanation from Jason, Mike understood and decided to continue the conversation via email.
Resolving Typesense Search Issues
Conversation started by Maximilian about Typesense search behavior led to Users Kishore Nallan and Mike discussing and suggesting workaround, with Kishore Nallan promising an official solution soon. No final confirmation of resolution provided.