Hi all, We have recently migrated from algolia to ...
# community-help
s
Hi all, We have recently migrated from algolia to typesense, with 22M docs in our index. We are using 32GB 8v 3node server for our use-case. We are facing issue with search terms: if searching "TRIVOLIB FORTE", search is fast and is within our timeout limit. Response time is 300ms. but if we add "TRIVOLIB FORTE 1", the search becomes very latent and causes CPU utilisation to go 100% with both write and read taking lot of time. Sharing the Preset Config we are using in our searches. Response time becomes 2.5s. Has anyone faced this before?? Any help would be appreciated. More details attached in below doc. https://docs.google.com/document/d/1tdjRe3RRJ8EtraqBFTaqjGARlYIQsZj1tC6zO7_jrE4/edit?tab=t.17s1n6bifxqm
k
I suspect the issue is the larger
max_candidates
value. Have you tried running that query without increasing it?
s
We tried playing around with this
max_candidates
value, it didnt help much.
@Prabhakar Burugupalli
k
Does the same query without the trailing "1" work fine?
Also please DM me the cluster ID
s
Yes, that works as expected even with prefix set as true
k
Thanks. Yes the issue I think that single letter prefix being very common is pulling a lot of words (associated with a lot of records) and that blows up the search processing time. Do you also have a filter associated with that query (it's not in the preset, but wanted to confirm).
s
Yes there are filters sharing sample curl below:
"searches": [ { "collection": "flatdistributorinventorysearch", "q": "TRIVOLIB FORTE {{id}}", "filter_by": "did:[21996]", "facet_by": "did,sl", "per_page": 50, "page": 1, "query_by": "dnm,prd.mfr.nm,prd.moc,itc,prd.mn,prd.sid,sp.nm", "prefix": true }, { "collection": "flatdistributorinventorysearch", "q": "TRIVOLIB FORTE {{id}}", "filter_by": "did:[119510,13915,14912,13213,14697,12943,14817,20477,9522,10584,13643,18589,13922,12926,14002,14973,12863,13302,15058,14942,9419,13191,14952,53814,13387,14014,16297,15495,3367]", "facet_by": "did,sl", "per_page": 50, "page": 1, "query_by": "dnm,prd.mfr.nm,prd.moc,itc,prd.mn,prd.sid,sp.nm", "prefix": true } ] }'
k
What happens when you just query
{{ id }}
independently? Also I see 2 searches: are the search time roughly the same for both or one is skewed?
s
• For both the quesries, search time is same • Searching only with {{id}} say 10, 40, 650 is normal, no cpu spikes, no latent results
k
Only with
1
the issue is there?
s
Happens with other int as well
So the issue we are seeing is mainly with {{string + any number}}.
k
So I suspect that all the single digit numbers have so many hits, when we merge them with the other tokens, the search time is now bounded by those large number of records matching those numbers. One work around I can think of is to disable prefix searching for queries that end with number to see if that helps.