Hello, we have a decently large cluster (Multi-nod...
# community-help
t
Hello, we have a decently large cluster (Multi-node HA / 64 GB Ram / 8 vCPU), and we’re starting to bump the ceiling on search performance again. Anyone had any decent success with specific techniques to reduce consumption? We’re looking at using Redis as like a five minute cache, but first wanted to source other peoples experience and advice before hand rolling anything. Thank you!!
j
If the searches are cacheable, then you can use Typesense's built-in use_cache feature
😮 1
We made some caching fixes in latest v29.0.rc builds, so you want to use that
t
🏁 Lightning fast, just like your search. Thank you @Jason Bosco!
👍🏽 1
😄 1
m
Fast disks are also important. Not all data are stored in memory. Only indexes. And it's all about the amount of data too. Sometimes split you data into different collections and use the join feature is better than keep all flat data into one collection. I have made some changes in my project changing 1 big fat collection into 2 smaller collections using joins and I reduced the response time from 2-3 seconds to sub-100ms.
thankyou 1
t
Thank you!
@Jason Bosco @Matheus Bombonato - Okay, one more question here. We’re finding some of our more expensive facet queries take quite a while. Currently, the pattern we use for these is something like:
Copy code
{
  facet_by: 'int32_field',
  filter_by: 'int32_field:[1,2,3,4,5,6,7,8]',
}
However, in our collection, which is around 6.5 million records, this can take upwards of 10 seconds. Would we significant speed ups or decrease in RAM if we instead had this pattern as advised elsewhere?
Copy code
{
  facet_by: 'int32_field',
  filter_by: 'boolean_for_int32_field:true',
}
@Jason Bosco @Matheus Bombonato - Okay one more and then I’m done. We have been hitting severe performance issues due to our group_by queries. Which would be a better direction to go here? (We are planning to upgrade to the latest v29 RC tonight) • Just upgrading and letting the latest version take care of things. • Moving our group by logic to instead perform a one-to-many join against a second collection and derive the grouped counts that way?
j
I would recommend first trying with the group_by perf improvements in v29.0.rc31
👍 2
Before doing a more extensive refactor
for facet_by, I'd highly recommend adding the following two parameters: facet_sample_threshold: 1000 facet_sample_percent: 10 on large collections
t
Ah, thank you! We’ll try those flags. I think we do the threshold, but we don’t do the percent.
j
Oh both of them are needed for either to take effect!
On a side note, another thing you might want to consider for your specific use-case is to move the largest collection into its own cluster, so that any issues with that collection don't affect the other collections / users due to resource availability
t
@Jason Bosco - We are using those flags and are onto 29, and holy cow! Things are so much faster, thank you!!! What a fix 🏥
j
Awesome! 🙌
t
@Jason Bosco - Things are looking A LOT better today, but we’re still seeing really long timeouts on high volume requests with this pattern: https://typesense-community.slack.com/archives/C01P749MET0/p1750884768220089?thread_ts=1750792526.678749&cid=C01P749MET0 Would there be any more extensive refactors we can do here to improve query / facet performance time?
j
Could you share the full curl request, minus your hostname and API key?
t
@Jason Bosco - Honestly, I could be wrong. With the litany of changes we’ve made this week (going up cluster versions, sampling facets, enabling caching), we might be on the other side of these performance concerns. It’s a multi search request in JS, and this is my options array:
Copy code
[
  {
    collection: 'org-search-694',
    q: '*',
    facet_by: undefined,
    filter_by: '',
    per_page: 0,
    max_facet_values: 1000,
    group_by: undefined,
    validate_field_names: false,
    facet_sample_percent: 10,
    facet_sample_threshold: 1000,
    use_cache: true
  },
  {
    collection: 'org-search-694',
    q: '*',
    facet_by: 'customer_marketing_model_cluster_1',
    filter_by: 'customer_marketing_model_cluster_1:=[1,2,3,4,5,6,7,8,397,398]',
    per_page: 0,
    max_facet_values: 1000,
    group_by: undefined,
    validate_field_names: false,
    facet_sample_percent: 10,
    facet_sample_threshold: 1000,
    use_cache: true
  }
]
j
This completes in about 200ms:
Copy code
curl -s -X POST "${TYPESENSE_HOST}/multi_search" \
  -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
  "searches": [
    {
      "collection": "org-search-694",
      "q": "*",
      "filter_by": "",
      "per_page": 0,
      "max_facet_values": 1000,
      "validate_field_names": false,
      "facet_sample_percent": 10,
      "facet_sample_threshold": 1000
    },
    {
      "collection": "org-search-694",
      "q": "*",
      "facet_by": "customer_marketing_model_cluster_1",
      "filter_by": "customer_marketing_model_cluster_1:=[1,2,3,4,5,6,7,8,397,398]",
      "per_page": 0,
      "max_facet_values": 1000,
      "validate_field_names": false,
      "facet_sample_percent": 10,
      "facet_sample_threshold": 1000
    }
  ]
}' | jq '.results[].search_time_ms'
184
20
On a side note
use_cache
only works when you use it as a URL GET parameter, not inside the POST body for multi_search
POST "${TYPESENSE_HOST}/multi_search?use_cache=true"
t
Thank you so much for checking this @Jason Bosco - I will check, but I think the js sdk takes care of that abtraction for me. Yeah, I think all of the fixes you’ve helped guide us to are sort of making the resources to handle that request really swiftly now. I won’t ask anything else this week, thank you for an immense amount of time and help this week. Please have a lovely weekend!
j
Happy to help! In typesense-js, you want to use this parameter called
useServerSideSearchCache
when instantiating it, which will take care of sending the use_cache param as a URL query parameter https://github.com/typesense/typesense-js/blob/a24e53afd4c8d4b032c2aaad0fe9fff073c14b3f/src/Typesense/Configuration.ts#L199
t
Oh, nice! I saw that a little bit but wasn’t sure where to do that at. I just made a PR locally adding use_cache as part of the second argument, but I like how clear that is!