Hi, I'm facing a bit of an issue with q and query...
# community-help
o
Hi, I'm facing a bit of an issue with q and query_by parameters. I have a set of records in my collection with "Sample dispenser" in the description field and I've been experiencing varying match patterns, in relation to the content of query_by. • When I use query_by="description", everything works as expected. • When I use query_by="*" , the number of matches becomes quite random
Here are my stats with query_by="*" : • "q": "dispenser sample" -> 1349 • "q": "dispense sample" -> 1349 • "q": "dispens sample" -> 1349 • "q": "dispen sample" -> 30 • "q": "dispe sample" -> 1349 • "q": "disp sample" -> 127 • "q": "dis sample" -> 1349 I would expect that by using shorter words and with prefix matching coming into play, the number of results would grow as the prefix to search is shorter... But this is quite puzzling to me and I would like to understand the logic that impacts this matching...
I went further on this, and I understand that query_by="*" may not be supported... I've experimented with different number of fields present in the query_by (including nested objects), and it seems that at a certain point i come into this situation. So, it looks like when "too many fields" are specified in query_by, then some get "dropped" from the search.
a
Hey Olivier, There is quite a few things in play here. 1. When you use query_by=*, it will query on all fields, which mean that as soon as it matches the amount of tokens of max_candidates it will return the results. 2. Prefix is only performed on the last phrase of your query 3. Depending on num_typos,
dispense
,
dispens
,
dispen
can generate very different typo-corrected tokens, which will lead to different answers, specially across all fields. 4. When
query_by="*"
is used, Typesense tokenizes the query once using the settings from the most weighted field However, the matching happens across ALL fields, each of which may have: - Different
prefix
settings (enabled/disabled) - Different
token_separators
and
symbols_to_index
- Different
locale
settings - Different tokenization results for the same text
👀 1
o
Thank you @Alan Martini, that will enable me to dig deeper into this.
Thank again @Alan Martini! It seems this was also related to
drop_tokens_threshold
in my case which had been set to 0. I've been able to get more predictable behavior now!