#community-help

Typesense Filter Bug Involving Similar Facets

TLDR SamHendley reported a bug in Typesense where filtering by facet returns wrong documents, providing a reproduction case. Jason and Kishore Nallan recognized the issue, tracked it on GitHub, and implemented a fix in a new Docker build.

Powered by Struct AI

1

17
11mo
Solved
Join the chat
Nov 17, 2022 (11 months ago)
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
04:21 PM
I got another weird response case. Reproduction case in thread but basically a filter_by for a facet sometimes gives me the wrong documents if it has the right combination of similar facets.
04:22
SamHendley
04:22 PM
curl "" \
       -X DELETE \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"

curl "" \
       -X POST \
       -H "Content-Type: application/json" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
       -d '{
         "name": "companies",
         "fields": [
           {"name": "company_name", "type": "string" },
           {"name": "capability", "type": "string[]", "facet": true}
         ]
       }'
       
curl "" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -H "Content-Type: text/plain" \
        -X POST \
        -d '
{"id": "125","company_name": "Company1", "capability": ["Encoding capabilities for network communications", "Obfuscation capabilities"]}
{"id": "126","company_name": "Company2", "capability": ["Encoding capabilities for network communications"]}
{"id": "127","company_name": "Company3", "capability": ["Obfuscation capabilities"]}
{"id": "128","company_name": "Company4", "capability": ["Encoding capabilities"]}
' 
# for this search only Company4 that actually has our expected facet but this
# search returns Company1 as well despite it not having the filter value.
# It appears to need both of these incorrect facet values, removing either one stops the bug from occuring
curl "" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "companies",
                "query_by": "company_name",
              "q": "",
              "filter_by": "capability:=[`Encoding capabilities`]",
              "facet_by": "capability"
            }
          ]
        }'
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:24 PM
Hmmm, that looks like a bug.

Could you try "filter_by": "capability:=[Encoding capabilities]" just to see if the bug is with the backtick escaping mechanism?
04:24
Jason
04:24 PM
Also, may I know what version of Typesense this is on?
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
04:25 PM
same issue without backticks
04:25
SamHendley
04:25 PM
0.23.1
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:25 PM
Could you also try on 0.24.0.rcn34?
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
04:29 PM
yes happens there as well
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:33 PM
Will take a closer look. Thank you for the reproduction steps. Could you copy paste this into a Github issue, so we can track it?
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
05:27 PM
Sorry real pain for me to do anything on github out of my own companies properties
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:30 PM
I see, ok. No worries. Iโ€™ll create the issue
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
05:35 PM
Iโ€™m glad it reduced down to such a small repro case. I had hundred+ attributes in the case we noticed.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:36 PM
Yeah, reproducing an issue consistently on a small dataset is half the battle usually!
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
05:38 PM
I have another similar case that I didnโ€™t boil down like this so if you have an rc with a fix I can see if it fixes that case as well.

1

Nov 23, 2022 (11 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:16 AM
SamHendley This is fixed in typesense/typesense:0.24.0.rcn35 Docker build.
SamHendley
Photo of md5-a9a351e11d64f05b41fec183816a0cda
SamHendley
05:00 PM
thanks for the update!