#community-help

Discrepancy in Search Results Between Postman and Python Library

TLDR Md raised an issue about differing search results when using Postman and a Python library. Kishore Nallan suggested trying a multi_search request to compare values, and to set a distance threshold on the vector component. The issue was resolved.

Powered by Struct AI
12
1mo
Solved
Join the chat
Aug 22, 2023 (1 month ago)
Md
Photo of md5-4c155ba3e9038148c2637f40aa514679
Md
09:40 AM
I have got a problem where using postman I get different results for hybrid search and using python library it returns different results.
typesense==0.16.0 (python client)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:10 AM
Can you please post a small reproduceable example?
Md
Photo of md5-4c155ba3e9038148c2637f40aa514679
Md
10:17 AM
I am just getting the IDs for sequentially fetching from DB using django queryset

search_parameters1 = {
'q': remove_trailing_whitespace(search),
'query_by': 'embedding,title,job_responsibility,skills,industry,location,type,division,company',
'include_fields': 'id',
'per_page': 250,
'page': 1,
'sort_by': '_vector_distance:desc'

}
res = client.collections['alljobsupdated'].documents.search(search_parameters1)

newlist = [x['document']['id'] for x in res['hits']]
print(newlist)
10:19
Md
10:19 AM
Here the sequence and list are different than fetching directly from postman like
{server}/collections/alljobsupdated/documents/search?q=laravel&query_by=embedding,title,job_responsibility,skills,industry,location,type,division,company&sort_by=_vector_distance:desc&include_fields=id&page=250
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:27 AM
Can you try a multi_search request with both python client and with postman and compare those values? I wonder if there is some url encoding issue because of the GET params
10:28
Kishore Nallan
10:28 AM
Multi search uses POST so we can rule that out
Md
Photo of md5-4c155ba3e9038148c2637f40aa514679
Md
10:32 AM
okay thanks bhai, I will try it
02:29
Md
02:29 PM
it is working now but sometimes returns irrelevant data , do we need to remove stop words before inserting on typesense ?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
02:32 PM
Irrelevant data can happen because of semantic search.
Md
Photo of md5-4c155ba3e9038148c2637f40aa514679
Md
02:36 PM
is there any possibilities to tweak the fusion mechanism dynamically?
Aug 23, 2023 (1 month ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
07:32 AM
Can you try setting a distance threshold on the vector component:

vector_query=embedding:([], distance_threshold:0.30)

In the snippet above, embedding is the name of the vector field. Try adding the above to your queries to see if it helps.
Md
Photo of md5-2d52bdc6a037ca5676860972cd54817a
Md
09:44 AM
thank you for the support