Hi everyone, I’m running into performance issues a...
# community-help
a
Hi everyone, I’m running into performance issues and wanted to check if this is expected or if there are ways to improve it. I have a cluster with a collection of ~16 million products. Each product contains a nested field called
is
, with a structure like this:
Copy code
is: {
  "active": "true",
  "buyable": "true",
  "buyable_in_all_b2b_cm": "true",
  "buyable_in_all_b2b_ecom": "true",
  "buyable_in_all_b2b_lib_ecom": "true",
  "buyable_in_all_b2c_ecom": "true",
  "hidden": "false",
  "home_deliverable_in_all_b2b_cm": "true",
  "home_deliverable_in_all_b2b_ecom": "true",
  "home_deliverable_in_all_b2b_lib_ecom": "true",
  "home_deliverable_in_all_b2c_ecom": "true",
  "isactive": "true",
  "isbuyable": "true",
  "not_orderable_from_central_warehouse_in_all_b2b_cm": "false",
  "not_orderable_from_central_warehouse_in_all_b2b_ecom": "false",
  "not_orderable_from_central_warehouse_in_all_b2b_lib_ecom": "false",
  "not_orderable_from_central_warehouse_in_all_b2c_ecom": "false",
  "not_orderable_from_supplier_in_all_b2b_cm": "false",
  "not_orderable_from_supplier_in_all_b2b_ecom": "false",
  "not_orderable_from_supplier_in_all_b2b_lib_ecom": "false",
  "not_orderable_from_supplier_in_all_b2c_ecom": "false",
  "priced": "true",
  "product": "true",
  "visible": "true",
  "visible_in_all_b2b_cm": "true",
  "visible_in_all_b2b_ecom": "true",
  "visible_in_all_b2b_lib_ecom": "true",
  "visible_in_all_b2c_ecom": "false"
}
I’ve enabled nested fields so I can filter on values like: •
is.active:=true
•
is.buyable:=true
•
is.visible:=true
Observed performance Filtering on a single field (e.g.
is.active:=true
) takes ~1.1s Combining multiple filters (e.g. active + buyable + visible) takes up to ~2.2s • Filtering on a numeric field (
price:>0.46
) takes ~2.6s •
range_index: true
is enabled for this field Questions Are these response times expected for a dataset of this size (~16M documents)? Do nested fields significantly impact filter performance in this case? Are there recommended optimizations for: ◦ multiple boolean filters I also have a price field in the index that is a float, I have turned on the range_index on this field. But when I filter on it like price:>0.46 the response takes 2.22sec. Is this normal? Are there ways to improve? Any guidance or best practices would be greatly appreciated. Thanks! 🙏
f
Hi Antoon, Could you share an example
curl
request so we can debug?
a
I send you a message 😉
f
When you use q=* (also known as wildcard query), Typesense skips text search but still visits every document matching your filter to compute sort scores and report the found count. For
is.active:=true
matching ~14M of 16M documents, that means 14M iterations of scoring, ranking, and counting, only to return 10 results. Query time scales linearly with the number of matching documents, not the number of returned results. Multiple filters combined with
&&
add intersection cost and the result set is still large, explaining the roughly double query time. The numeric
price:>0.46
is slowest because the range trie must collect, merge, sort, and de-duplicate all matching IDs before the scoring loop even begins. I also took a look at your schema, and it's correctly configured. The
is.*
fields are typed as bool and I confirmed through testing that the filter engine uses the efficient boolean index path. The "true" strings you see in API responses are cosmetic, because when documents are ingested with string values, coercion to boolean applies to the index but not to the stored document. Filtering works correctly regardless. You could try reindexing everything as true JSON boolean values (
"is.active": true
vs
"is.active": "true"
) just for peace of mind though, as this is an edge case. There is a fast path in the code for q=* without any filter that returns results in O(page_size) by walking the sequence ID index in reverse and stopping early. This fast path explicitly requires no
filter_by
, so your queries cannot use it. If you want to provide a fast initial load, you can use
search_cutoff_ms
to cap latency. Typesense checks for timeout every ~32K documents and will return partial but correct top results. You should also check out
use_cache
for caching commonly used queries / pages on the server-side. When it comes down to reducing general response time, narrowing your filters to match fewer documents is the most direct way.
a
If I understand correctly the q=* is the best way to use my query? Or is there an other option? I will try and test the search_cutoff_ms. use_cashe is something we will turn on yes, but I believed the search should be able to be quicker then 2 seconds before using cashe so that is why I reached out already.
f
using
q=*
without any filters is going to be the fastest. By the way,
is.active:=true
matches ~14M out of the 16M total documents. If it's commonly applied, then breaking the collection into two, one for active and one for inactive documents will increase performance by a lot, since the filtering won't be needed during search time. The 1-2 second response time is a result of filters matching millions of documents, not of using
q=*
itself. A text query like
q=shoe
with the same filter would actually be faster because the text search narrows the candidate set before the filter is applied, meaning fewer documents go through the loop.
a
Yes I understand. The issue is that we need a few checks. is.active, is.buyable, is.visible,... are all filters that need to be applied before the products can be shown on the website. These filter values on the products change from time to time they don't always stay the same. And the value on the products also changes so putting them in different collections would be hard?
About the price would it be better or worse to use something like this: [0..0.46] instead of >0.46
f
Is there any specific condition under which products are shown? Is it a combination of
visible
,
active
,
buyable
? Or do they coexist to avoid data duplication?
Also, the range you showed is the opposite of >0.46, but <0.46
a
The combination changes, that is why there are multiple filters for it unfortionatly. the range is a type they should both mean the same😅
f
[0...0.46] means from zero up and until and including 0.46, not larger than 0.46
a
Yes, so the correct question would be if it is better to use >0.46 or [0.46..1000]?
f
As far as combinations go, if you can't shard the collection to active and inactive documents, then your best option is what I mentioned about caching and search cutoff. Wildcard searches with filtering are multiple times more computationally intensive than normal queries with filters.
Yes, so the correct question would be if it is better to use >0.46 or [0.46..1000]?
Depends, if your maximum price number is 1000, then it would be the equivalent. But using it will add an additional layer of filtering, like saying
price:>=0.46 && price:<=1000
. If 1000 is not the maximum, then it may be faster
👍 1
a
I understand that when you search you will first make the set smaller and then search. But if you need to filter on the entire set then you need the wildcard search. Cutoff seems to give some improvements. But I have noticed if you are to strict then facets might give issues?
f
Facets are computed from the matched documents, so an aggressive cutoff means fewer documents to count from. Try adding
facet_sample_percent=50
to your query. It computes facets from a sample instead of all matches, giving you faster results with representative counts that are automatically scaled up. You can combine both parameters and adjust to balance speed and accuracy.
a
After some testing, I still have some questions. When I filter with a filter that gives a smaller result set the filtering is faster, I mean when I filter with a specific filter that will give me 200.000 results of the 16mil it is rather fast. When I filter with a filter like the isActive I get 15mil results and this is slower. Now when I combine the 2 filters they are faster the the isActive alone but still slow. How can you explain this? I would like to understand the way the filters work in Typesense? I have also been thinking about creating a separate cluster for the isActive isBuyable and inCollection. Then I would no longer need these filters. The data would mostly if not all be the same is there an estimation what this would mean in RAM memory? Does it take a lot more or not? We are also thinking about making a bitmask filtering system meaning we will translate the different combinations of filters to numbers and filter on these instead. This would possibly create more memory, but might increase the filter speed? isActive and iBuyable and inCollection, 3 different filters on a bool would then become a single filter on an int and isActive and isBuyable would be a different int? What is your opinion on this?
f
A filter that matches 200k docs is faster because Typesense has a much smaller candidate set to work with after filtering. A filter like
is.active:=true
is slower because it still matches almost the whole collection, so even though the filter itself is cheap, the query still has to process a very large number of matching docs. When you combine the 200k filter with
is.active:=true
, the result is faster than is.active alone because the selective filter cuts the candidate set down a lot. It is still slower because 200k is still not a small set, just much smaller than 15M. TLDR, it's not about how many filters there are, but how many docs are retrieved when they're applied. Creating a separate cluster or collection for the active / buyable / in-collection subset will improve latency a lot, since it removes these very broad filters from the query path. But if the data is mostly duplicated, RAM usage will also go up a lot. It won't be a small overhead if you're planning to keep both indexed, rather than deleting the documents that match the "separate cluster" criteria. On the bitmask idea, I would not expect a big optimization, unless it also makes the matched set much smaller, which I presume it won't. Replacing 3 boolean filters with 1 integer filter can reduce some filter-combination work, but that is usually not the dominant problem here. If the bitmask value still matches millions of documents, the query will still be slow. So I’d see bitmasking as a modest optimization, not the thing that will change query times dramatically. I would start by testing it out first, before taking more drastic measures like duplicating the dataset in an "active" collection.
a
I understand what you are saying. What I notice is on our cluster, when I filter with the filter categoryCodes.sb_lvl3:=-1807844812 this gives 263847 results in 68ms, nice! But when I add the is.active filter it takes 486ms and 263452 results. When I add the is.buyable filter te respons takes 637ms and gives 263251 results. When i add the is.visible_in_all_b2b_ecom the respons takes 800ms and gives 209707 results. What I dont understand is that adding the filters adds all that delay? In this case the bitmask seems a good solution? The initial filter on the bools is.active, is.buyable will still take 1.2 sec, that I understand but the other ones would be faster? What I have also noticed is that when I only filter on is.active, response time 1.1sec 16.2mil results. When I add is.buyable, 1.4sec 16.2mil results. When I add is.visible_in_all_bcb_ecom, 1.36sec and 900k results. The adding of the filters seems to have a different result then the situation above? Could you give some more info on why the adding of the filters always adds 100ms on the smaller set, but on the bigger set it does not seem that bad?
@Alan Martini @Fanis Tharropoulos Some feedback on the reason for these response times?
j
Hi @Antoon Van Dessel. Alan and Fanis are off till tomorrow, still their weekend. I tried this example on your cluster myself and got 307ms, 585ms then 867ms which seems similar to your case. It seems like across both the smaller and larger sets the
&&
logic is adding about 300ms extra time just to process the intersections (if you look at the larger sets 1.1->1.4/1.36 is also around 300ms added); not sure if it should be taking this long but it makes sense that there is some overhead. Given this your bitwise mask idea may help the search times yes, it would be worth trying out.
a
We will test with the bitwise mask idea on our cluster. I will provide you with feedback as this seems interesting for you guys aswell. A concern I have is the combination of filters seems to add a lot of search time cost. Is this normal? Should they each add 300ms of intersection time? Can this be validated?
f
Hey Antoon, Adding filters to a search query will accumulate more search time, especially if the intersection is broad, since the filtering mechanism will go through all the possible candidates.