#community-help

Synonym Issue with Multiple Word Query

TLDR John observed inconsistent search results involving synonyms and raised the issue. Kishore Nallan admitted the bug and asked John to create Github issues, promising to prioritize the multi-synonym bug.

Powered by Struct AI

1

ty

1

11
11mo
Solved
Join the chat
Nov 15, 2022 (11 months ago)
John
Photo of md5-21545f1facb7836c149bc4c70752bd2b
John
01:07 PM
Having some issues where multiple synonyms in a query yields weird results. For example if I have synonyms blazer<->suit and male<->man and a document containing “blazer” and “male”, I find the document when I search for “suit man” but NOT when I search for “blazer man” which seems really weird.
Are there any known issues with multiple synonyms in a query?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:34 PM
No known issues, but this could be an unhandled edge case. Is this trivially reproduceable with a small test case of one/few documents and synonyms?
John
Photo of md5-21545f1facb7836c149bc4c70752bd2b
John
02:23 PM
Couldn’t reproduce exactly what I mentioned, but with this script https://pastebin.com/YnvMAGBV I get
> Got 1 results for “blazer male” (Should match immediately)
> Got 0 results for “blazer man” (Should match with synonym male<->man)
> Got 0 results for “suit male” (Should match with synonym blazer<->suit)
> Got 1 results for “suit man” (Should match with synonyms blazer<->suit and male<->man)
02:23
John
02:23 PM
This is with 0.24.0.rc20, will try with some other versions now as well. Same on 0.23.1 and 0.24.0.rcn32.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:40 PM
Ok will check and get back to you

1

Nov 16, 2022 (11 months ago)
John
Photo of md5-21545f1facb7836c149bc4c70752bd2b
John
09:21 AM
Somewhat related synonym issue here https://pastebin.com/LtjR4zHt where I get
> Got 1 results for “trousers” (Should match immediately)
> Got 1 results for “trouesrs” (Should match by allowing misspelled words <trouser>)
> Got 1 results for “pants” (Should match with synonym trousers<->pants)
> Got 0 results for “patns” (Should match misspelled <pants> synonym to <trousers>)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
12:38 PM
There is definitely an issue with both words in a query being eligible for a synonym substitution. Currently the engine ends up with a substituted query that matches no records because both words are swapped.

Regarding patns -- that's expected because we don't apply synonyms on typo corrected variations of the query. This is to prevent false positives from happening.
John
Photo of md5-21545f1facb7836c149bc4c70752bd2b
John
12:45 PM
Thanks! I see, do you think it could be fixed shortly or should I try to find a work-around?

> Regarding patns -- that’s expected because we don’t apply synonyms on typo corrected variations of the query. This is to prevent false positives from happening.
Understand, seems quite context-specific though, could be exposed as a flag!
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
12:48 PM
Can you please create a Github issue for both? Thank you in advance.

Will try to prioritize the multi-synonym bug fix first. I've to see what kind of effort is involved. Currently, both matching synonyms are swapped so you end up with a variation that doesn't exist. Need to see how to account for all variations (both swapped, first word swapped, second word swapped). Gets more complicated when query string is larger and several words match: might generate too many combinations.
John
Photo of md5-21545f1facb7836c149bc4c70752bd2b
John
12:49 PM
Will do, thanks! 🙌