Hello 👋 I've got a question about how

text_match

is calculated on search 🧵

May I know which version of Typesense this is on?

The weights are not added across fields but is used for prioritizing. If 2 records already match fully against the query on the highest weighted field, their text match score will be the same. Prior to 0.23 we were summing up weights across fields but it led to various edge cases leading to noisy results.

Is there any chance of getting this re-enabled? The way we need to rank our content would rely on making frequency of search term factor into the

text_match

score.

Can you post a real world example that shows exactly the behavior you want? For e.g. do you just want repeated tokens across field to be counted or do you have any other requirements?

sure so for example here:

Got it, thanks. Let me look into what kind of effort we need to make this configurable and get back to you. Right now there is now way to have the old behavior.

@Kishore Nallan is this current behavior really intended? how can they have the same text_match score when one of the results matches across more fields here's another example that highlights the problem. the two results have the same text_match score despite "description" designated to receive higher weight

do you mind setting up a zoom call for us all to sync about this issue? @Jason Bosco @Kishore Nallan? this is a pretty big blocker for us

Jason will contact Rebecca, but in the mean time, I will summarize the 3 different issues in discussion here: a) The very first example in the thread involving the

fox

query: Typesense does not count individual occurences of the tokens since that caused relevancy issues due to keyword stuffing in real-world data sets that can be noisy. b) The second example involving the

javascript

query: since Typesense derives a text match score from the best matched field of a record, the scores are same here. This is something that I agree is not always ideal so we've to see how we can support a more fine-grain scoring that considers additional fields that match. c) The

java

query: I've to check what's happening here as the category field is weighted lower so that record must appear ahead.

thanks for investigating 🙇

We have a potential fix that addresses b) and c). Would you be ok with us applying the patch on your cluster once it’s ready in an RC build?

yep, we haven’t been able to launch this to production users yet so a patch should be fine

Will keep you posted, RC should be ready in 24 hours

@Jason Bosco I'm not seeing a difference in behavior, did you patch production?

Yup production. Could you post screenshots of what results you now see for b and c? Along with the search params you're using?

so this search for "javascript" with

query_by=title,shortDescription,description

Can you please copy + paste the content of the description field of the first result here?

It looks like there might be some numeric overflow issues when assembling the text match score, which we’re looking into… But the results themselves should be sorted properly now.

this one is production and the one where 3 fields is ranked below one where 2 fields match

Could you open up the browser dev console, look at the network requests generated to the

.../api/...

endpoint as you search, copy as curl the last request and DM it to me?