Hi everyone! We are exploring using vector search with typesense and I am noticing some strange behaviour. It seems when I am using three fields in sort by (where one of them is

_vector_distance:asc

) typsense is not returning all the documents. For example, with sort by of

"sortBy": "_vector_distance:asc,genericFlField1:desc",

, I get found as 200 (which is the

k

I set). With sort by of

"_vector_distance:asc,genericFlField1:desc,genericFlField2:desc"

, I get found as 1. I am using typesense v28. It this a known issue? Thanks in advance!

Can you provide a set of curl commands so we can replicate this issue on our end?

Hi, can I send this to you in DM?

We'd prefer keeping things here in order for other users to be able to refer to previous issues if something similar comes up again in the future

I understand. I tried reproducing this locally but I am unable to do so with whatever few documents I am able to seed typesense with. It is happening in the typesense cloud cluster we have, and I can share the curls hitting that typesense cluster but I don't want to share the api key

If you can, share your cluster id, we'll check it on from there. Just paste the equivalent curl commands without the API keys

Great, I can do that. The cluster id is:

f1kdzbg6o7i5n2sxp

and I put both the curls in this gist: https://gist.github.com/vamshiaruru32/3287dee668d083fa6b187b40b8cf7b0f as they are too large to be sent on slack message. Please let me know if this is enough. Thank you.

I was able to replicate this behavior. Can we copy your dataset on one of our debug clusters in order to find what's going wrong?

Sure, you can do that. Thanks.

After debugging, I found that the last sorting field, is optional. If there isn't a match with that field, by defaut we assign it a super low score, and as results fill up, they will always be bumped down, removing them from the end result.For that, you will need to assign a behavior for the missing values. We have some docs for it: https://typesense.org/docs/28.0/api/search.html#sorting-null-empty-or-missing-values Adding this:

"sort_by": "_vector_distance:asc,genericFlField2(missing_values:last):asc",

Will return all 200 results

Hi, Just tried with this:

_vector_distance:asc,genericFlField1:desc,genericFlField2(missing_values:last):asc

it is still returning only 1 result. The issue happens when there are three sorting fields. When there are two sorting fields, it works without the missing_values as well

Hi Vamshi, this is fixed in

29.0.rc27

.

Thanks a lot!