Hi I am having a hard time understand how drop tok...
# community-help
j
Hi I am having a hard time understand how drop tokens work. I have this q
<http://orangetheoryfitness.com|orangetheoryfitness.com> fl boca raton 6000 broken sound pkwy nw
and
drop_token_mode:right_to_left
. In the response I see
"num_tokens_dropped": 7
so I would expect that the query matched on
<http://orangetheoryfitness.com|orangetheoryfitness.com> fl
but when I look at the highlights and the results it seems to have actually matched on
<http://orangetheoryfitness.com|orangetheoryfitness.com> pkwy
instead. How are tokens dropped and what does
drop_token_mode
actually do then? Thank you!!
r
I could be mistaken, but my theory would be that initially there were not enough matches, so it dropped
nw
. then it searched again, and there were some matches (so
pkwy
matched?) but not enough so it drops it and proceeds this process until enough matches are found
thankyou 1
the documentation is a bit brief on this subject. there is some mention of the process here, but unclear if that is the whole story or not; https://typesense.org/docs/guide/faqs.html#can-i-implement-boolean-search-with-typesense
k
but when I look at the highlights and the results it seems to have actually matched on
<http://orangetheoryfitness.com|orangetheoryfitness.com> pkwy
Note that we could highlight a keyword from the query that is present in the result even if the actual search keywords that found that query is a different set.
And you could even highlight a field that's not part of
query_by
(we just try to match the content on the stored field with query tokens). So hard to say what exactly is happening. If you are able to share a simple example in this format, I can maybe debug: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b