Hello everyone. We are having a critical performan...
# community-help
j
Hello everyone. We are having a critical performance issue with Typesense. • We have 254k products in the collection. Around 100k are Books. And this is more impactful when querying for Books category specifically. We are seeing multiple timeouts, and for some queries we even see the CPU spiking up to 100%. • We suspect this is related to the fact that InstantSearch adapter to Typesense is making 2 separate queries, one for the results, and one for faceting. How can we optimize it? Do you see any reason for this query to cause this? Is this an issue with our query or with typesense? Thanks a lot for the help!
Copy code
{
  "searches": [
    {
      "query_by": "embedding,navigationCategories_en,navigationCategories_pt,sellerName,brandName",
      "group_by": "productId",
      "include_fields": "productId,name,calculatedPrice,salePrice,price,image,reviewRatingAvg,reviewsCount,stock,image,preferredVariantId,sku,color_en,color_pt,rewardPercentageOnPublication",
      "group_limit": 1,
      "prefix": false,
      "highlight_full_fields": "none",
      "highlight_fields": "none",
      "query_by_weights": "0,50,50,10,10",
      "vector_query": "embedding:([], alpha: 0.5)",
      "rerank_hybrid_matches": true,
      "collection": "products",
      "q": "*",
      "facet_by": "brandName,calculatedPrice,hierarchicalNavigationCategories_en,navigationCategories_en,product_attributes_en.Color,product_attributes_en.Size",
      "filter_by": "stock:>0 && hierarchicalNavigationCategories_en:=[Entertainment > Books]",
      "max_facet_values": 20,
      "page": 1,
      "per_page": 20
    },
    {
      "query_by": "embedding,navigationCategories_en,navigationCategories_pt,sellerName,brandName",
      "group_by": "productId",
      "include_fields": "productId,name,calculatedPrice,salePrice,price,image,reviewRatingAvg,reviewsCount,stock,image,preferredVariantId,sku,color_en,color_pt,rewardPercentageOnPublication",
      "group_limit": 1,
      "prefix": false,
      "highlight_full_fields": "none",
      "highlight_fields": "none",
      "query_by_weights": "0,50,50,10,10",
      "vector_query": "embedding:([], alpha: 0.5)",
      "rerank_hybrid_matches": true,
      "collection": "products",
      "q": "*",
      "facet_by": "hierarchicalNavigationCategories_en",
      "filter_by": "stock:>0",
      "max_facet_values": 20,
      "page": 1
    }
  ]
}
a
Hey @Joao Neves, I noticed you're currently using version 29 on your cluster. In most of the recent RC builds of this version, we've introduced performance improvements for
group_by
operations. Because of that, we strongly recommend upgrading your cluster. Additionally, you're performing several CPU-intensive operations simultaneously: faceting on multiple fields, using
group_by
, running vector searches, and applying an
AND
condition in
filter_by
. Given this combination, high CPU usage is expected. One optimization you can try is setting the following parameter:
Copy code
facet_sample_threshold: 1000
Also, consider updating your collection schema by adding
range_index: true
to the
stock
field for better performance.
j
Thanks so much Alan. About the vector searches: Does it have any performance impact if we're using an "*" query?
a
When doing a "q=*" with vector search, you'll will be performing a Nearest-neighbor vector search specifically, but all vector searches are cpu intensive!
šŸ‘ 1
j
Got it. We are trying to enable cache to see if it helps. Do you think it would? However, we are not being able to do so. We are not figuring out how to add this url_parameter through the InstantSearch component. Any tip?
The problem we are facing with the cache is that the InstantSearch seems to send it as part of the body. Not as a query param. It would work with normal search. But since we are using multi_search, it does not.
a
As any cache, it helps if similar queries are being constantly made! But for embeddings, cache is only avaiable at the latest v29 version and the configuration is in fact applied at the server instead: https://github.com/typesense/typesense/pull/2260 If you'd like to use it, a rolling restart will be necessary.
thankyou 1
j
We are upgrading from 29.0.rc1 → 29.0.rc30 like you suggested, so I assume that includes a rolling restart already. Will it include server-side caching for embeddings?
a
@Joao Neves, Since the embeddings cache is only avaiable in the latests versions, you will first need to finish the current upgrade and only then add the parameter! For that, you'll want to go to Cluster Configuration -> scroll down to Typesense Server Parameters and add the
embedding-cache-num-entries
with an value like 5000. On the InstantSearch question, for using cache with InstantSearch you'll want to set it when defining the TypesenseAdapter instance, as explained here: https://github.com/typesense/typesense-instantsearch-adapter?tab=readme-ov-file#caching Finally, I missed saying that the
facet_sample_threshold
parameter also needs the
facet_sample_percent
to be set! So the parameters would be:
Copy code
facet_sample_threshold: 1000,
facet_sample_percent: 20
Let me know if you need help with any of these
j
Thanks a lot for the help Alan. We really appreciate it. Just the upgrade mitigated the issue significantly. And your tips and some tests we did based on it helped us identify a lot of improvements we should add to have more optimized queries. We will work on it and let you know if we struggle with anything. I'll keep you updated. Thanks again. Typesense rocks šŸ‘
a
Glad to help!! Anything else please let us know