Understanding Typesense's `drop_tokens_threshold` and `typo_tokens_threshold`
TLDR em1nos sought clarification on Typesense's
typo_tokens_threshold. Kishore Nallan defined them, emphasizing that they depend on the number of documents found, not tokens or typos;
num_typos configures the typo allowance.
Aug 01, 2021 (25 months ago)
typo_tokens_thresholdwork. I've read the docs about it but I'm not sure I understand it fully.
Kishore Nallan03:13 PM
typo_tokens_threshold: Xyou are telling Typesense to continue generating alternative tokens from the tokens in the query that are within an edit distance of
num_typosuntil you find atleast X results. You want to stop at some point, because you can keep modifying the query tokens to generate a lot of alternative tokens.
Similarly, there might be no documents that contains all tokens in the query. In that case, Typesense tries to drop tokens in the query, for e.g. searching only for "beta gamma" to find relevant documents. When you set
drop_tokens_threshold: Xyou are telling Typesense to continue dropping tokens from the query until X results are found.
Kishore Nallan03:34 PM
Kishore Nallan03:34 PM
Kishore Nallan03:35 PM
num_typosplay together with these previous settings?
Kishore Nallan03:36 PM
num_typosis the maximum number of typos (0, 1, 2) allowed.
Kishore Nallan03:37 PM
Aug 02, 2021 (25 months ago)
Kishore Nallan01:19 AM
Indexed 2764 threads (79% resolved)
Understanding Typesense Query Fuzziness and Thresholds
Ashraful was confused about different query results when applying filters in Typesense. Jason clarified the function of `drop_tokens_threshold` and `typo_tokens_threshold` options, explaining their effect on search results and their precedence.
Understanding Typo Tolerance in Search Queries
gab sought clarity on typo tolerance settings in search operations, specifically on the discrepancy in document returns when typos are involved. Kishore Nallan explained the "num_typos" and "typo_tokens_threshold" parameters within search queries, and how they dictate typo allowance during searches.
Issue with Search Term Results in Typesense
Dipankar had issues with specific search terms returning unexpected results. Kishore Nallan clarified why this may occur and how to fine-tune the behavior using the 'drop_tokens_threshold' parameter in Typesense.
Typesense Search Solution Issues
Rolando faced incorrect search results using Typesense. Kishore Nallan suggested changing typo parameters and upgrading Typesense version. However, undesired results persisted and need further investigation.
Issue with Typo Correction/Prefix Search and the Role of max_candidates
John noticed inconsistent search results based on max_candidates settings, and Kishore Nallan clarified its role for multi-word queries. They resolved that increasing max_candidates ensures the query isn't prematurely limited.