#community-help

Extracting Typos & Dropped Tokens Info from Search Results

TLDR Dima wants to extract info about applied typos/dropped tokens from search results. Jason offers suggestions and encourages creating a GitHub issue for more structured information. Dima creates the issue.

3mo

Join the chat

Jun 14, 2023 (3 months ago)

Photo of md5-1b62114a658b760944aa7d2b4c274460

Dima

08:07 PM

And another question: how can I extract info about applied typos / dropped tokens in the results? We want to collect this information in query analytics + maybe show user some notice about possible irrelevant results

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

08:10 PM

The search results don’t give additional information about typos / dropped tokens at the moment, but one way you could deduce this information is using the matched_tokens key in the response and compare it to the query that the user typed

08:11

Jason

08:11 PM

matched_tokens is also an array, so if you count the number of items in the array and compare it to the number of words in the query, you’ll be able to tell if drop tokens kicked in

Dima

08:12 PM

What about prefix search? I thought matched_tokens may contain more elements than query tokens

Jason

08:13 PM

You’re right, the last word in the query will do a prefix search

08:14

Jason

08:14 PM

So you’d have to consider partial substring matches just for the last word

08:14

Jason

08:14 PM

In any case, I think it would be good to expose this information in a more structured way in the response.

08:15

Jason

08:15 PM

Could you create another GitHub issue for this, describing the ideal type of information (the JSON structure) that would help to power your use-case?

Dima

08:15 PM

Of course 🤓

08:22

Dima

08:22 PM

https://github.com/typesense/typesense/issues/1059

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Troubleshooting "drop_tokens_threshold" and Typo Tolerance in Typesense

Joe had issues with "drop_tokens_threshold" = 0 and typo tolerance in Typesense, after which Kishore Nallan provided solutions and clarifications on feature functionality. Their issues with the search result limit and tokens were resolved after discussion and testing.

24mo

Phrase Search Relevancy and Weights Fix

Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.

111

6mo

"Query By" Parameters Resulting in Fewer Search Results

Jon encountered unexpected behavior with "query by" parameters. Upon checking the post request body by Jason, Kishore Nallan noted the issue originates from the typo match with "full stack". Suggested solution was setting `split_joins_tokens: always`.

15mo

Issue with Typo Correction/Prefix Search and the Role of max_candidates

John noticed inconsistent search results based on max_candidates settings, and Kishore Nallan clarified its role for multi-word queries. They resolved that increasing max_candidates ensures the query isn't prematurely limited.

16mo

Resolving Typesense Search Issues

Conversation started by Maximilian about Typesense search behavior led to Users Kishore Nallan and Mike discussing and suggesting workaround, with Kishore Nallan promising an official solution soon. No final confirmation of resolution provided.

19mo