Hello everyone, I need some help with a strange behaviour we are seeing when using group by, pinned hits and filter curated hits together. I have verified this behaviour on both v27 and v28.0.rc22. here's our data model: We have a collection called product_variants, which contains variant data. Each of them has a field called

productID

which we use as group by to group variants together. We use id of one of the variants of each of these groups as pinned hits. the issue: When applying a filter which the pinned hits documents don't match, we aren't getting all the results back if we use group by. For example, consider this query

curl --location --globoff '<http://localhost:8108/collections/product_variant/documents/search?q=*&filter_by=size%20%3A%3D%20%20[%600-3%20M%60]&pinned_hits=gid-shopify-productvariant-46208745865471%3A1%2Cgid-shopify-productvariant-46208735707391%3A2%2Cgid-shopify-productvariant-46244272931071%3A3%2Cgid-shopify-productvariant-46185030811903%3A4%2Cgid-shopify-productvariant-45021668737279%3A5%2Cgid-shopify-productvariant-46194895257855%3A6%2Cgid-shopify-productvariant-46209044644095%3A7%2Cgid-shopify-productvariant-45016400625919%3A8%2Cgid-shopify-productvariant-46211713040639%3A9%2Cgid-shopify-productvariant-45018106659071%3A10%2Cgid-shopify-productvariant-46212039704831%3A11%2Cgid-shopify-productvariant-46338788622591%3A12%2Cgid-shopify-productvariant-46208750616831%3A13%2Cgid-shopify-productvariant-46212083646719%3A14%2Cgid-shopify-productvariant-46295308009727%3A15%2Cgid-shopify-productvariant-46338787344639%3A16%2Cgid-shopify-productvariant-46204843786495%3A17%2Cgid-shopify-productvariant-46200236343551%3A18%2Cgid-shopify-productvariant-46245851332863%3A19%2Cgid-shopify-productvariant-46209235255551%3A20&filter_curated_hits=true&per_page=250>' \
--header 'x-typesense-api-key: xyz'

this has filter by and pinned hits and curate hits, but no group by. This returns 13 hits as expected.

"facet_counts": [],
    "found": 13,
    "hits": [...],
        "out_of": 124,
    "page": 1,
    "request_params": {
        "collection_name": "product_variant",
        "first_q": "*",
        "per_page": 250,
        "q": "*"
    },
    "search_cutoff": false,
    "search_time_ms": 3

Now in the following query, I just add a group by product Id and nothing else

curl --location --globoff '<http://localhost:8108/collections/product_variant/documents/search?q=*&filter_by=size%20%3A%3D%20%20[%600-3%20M%60]&pinned_hits=gid-shopify-productvariant-46208745865471%3A1%2Cgid-shopify-productvariant-46208735707391%3A2%2Cgid-shopify-productvariant-46244272931071%3A3%2Cgid-shopify-productvariant-46185030811903%3A4%2Cgid-shopify-productvariant-45021668737279%3A5%2Cgid-shopify-productvariant-46194895257855%3A6%2Cgid-shopify-productvariant-46209044644095%3A7%2Cgid-shopify-productvariant-45016400625919%3A8%2Cgid-shopify-productvariant-46211713040639%3A9%2Cgid-shopify-productvariant-45018106659071%3A10%2Cgid-shopify-productvariant-46212039704831%3A11%2Cgid-shopify-productvariant-46338788622591%3A12%2Cgid-shopify-productvariant-46208750616831%3A13%2Cgid-shopify-productvariant-46212083646719%3A14%2Cgid-shopify-productvariant-46295308009727%3A15%2Cgid-shopify-productvariant-46338787344639%3A16%2Cgid-shopify-productvariant-46204843786495%3A17%2Cgid-shopify-productvariant-46200236343551%3A18%2Cgid-shopify-productvariant-46245851332863%3A19%2Cgid-shopify-productvariant-46209235255551%3A20&group_by=productId&filter_curated_hits=true&per_page=250>' \
--header 'x-typesense-api-key: xyz'

now it returns only one grouped hit. It says found 1, found_docs 13

"facet_counts": [],
    "found": 1,
    "found_docs": 13,
    "out_of": 124,
    "page": 1,
    "request_params": {
        "collection_name": "product_variant",
        "first_q": "*",
        "per_page": 250,
        "q": "*"
    },
    "search_cutoff": false,
    "search_time_ms": 3

all the 13 documents that match this query don't have the same product Id. I have verified this by checking the product IDs and also the grouped hit only has one hit inside it. Unfortunately since I don't know what's causing this, the minimum reproducible example has 125 documents. I can share a zip of my typesense data folder if that helps, or I can export the collection and share the JSONL file as well. Please let me know what I should do next, thanks in advance.