Hello <@U01NZ4D0LDD> <@U01PL2YSG8L> - Good Morning...
# community-help
r
Hello @Jason Bosco @Kishore Nallan - Good Morning/Evening. Thank you for your wonderful search engine product. And good to know that, now the new version supports facet and query features in the joined collection which I would love to explore and implement it in our projects. I need your advise on implementing the best way for facets to display the count for unselected values for the large volume dataset (around 30 millions). Currently we are using the angular typesense components. And it is adding the additional query for each filter widget, that removes the particular filter for that widget, but keeps all other filters in the multi-search end-point. I know that, this is the expected behavior, as per the current design. Currently the requests are being timed-out after 5 seconds. Understand that, since it's performing OR for each facet widget, the computation is taking time. We can increase the timeout to address it temporarily. However, I would like to know, is there any way we can defer the facet loading lazily, so that, the main hits can populate the results within the timeframe. And the facet widgets can be loaded lazily. Your advice on this would be really appreciated. Looking forward to hear from you. FY: I've referred the below FAQ. https://typesense.org/docs/guide/faqs.html#faceting Also, referred the below LIVE DEMO https://songs-search.typesense.org/ Even in Live Demo, I've seen the same timeout errors after applying multiple facets. Sample Payload Request (of Demo) which exceeds > 5 seconds and got Timeout
Copy code
{
  "searches": [
    {
      "query_by": "primary_artist_name,title",
      "query_by_weights": "2,2",
      "sort_by": "_text_match(buckets: 10):desc,release_date:asc",
      "facet_sample_threshold": 1000,
      "facet_sample_percent": 20,
      "highlight_full_fields": "primary_artist_name,title",
      "collection": "s",
      "q": "ann",
      "facet_by": "country,genres,primary_artist_name,release_decade,release_group_types",
      "filter_by": "genres:[`Classical`,`Pop`,`Rock`] && primary_artist_name:[`Ann Peebles`,`Anne Sofie von Otter`,`Various Artists`] && release_group_types:[`Album`] && release_decade:[`2000s`]",
      "max_facet_values": 20,
      "page": 1,
      "per_page": 15
    },
    {
      "query_by": "primary_artist_name,title",
      "query_by_weights": "2,2",
      "sort_by": "_text_match(buckets: 10):desc,release_date:asc",
      "facet_sample_threshold": 1000,
      "facet_sample_percent": 20,
      "highlight_full_fields": "primary_artist_name,title",
      "collection": "s",
      "q": "ann",
      "facet_by": "genres",
      "filter_by": "primary_artist_name:[`Ann Peebles`,`Anne Sofie von Otter`,`Various Artists`] && release_group_types:[`Album`] && release_decade:[`2000s`]",
      "max_facet_values": 20,
      "page": 1
    },
    {
      "query_by": "primary_artist_name,title",
      "query_by_weights": "2,2",
      "sort_by": "_text_match(buckets: 10):desc,release_date:asc",
      "facet_sample_threshold": 1000,
      "facet_sample_percent": 20,
      "highlight_full_fields": "primary_artist_name,title",
      "collection": "s",
      "q": "ann",
      "facet_by": "primary_artist_name",
      "filter_by": "genres:[`Classical`,`Pop`,`Rock`] && release_group_types:[`Album`] && release_decade:[`2000s`]",
      "max_facet_values": 20,
      "page": 1
    },
    {
      "query_by": "primary_artist_name,title",
      "query_by_weights": "2,2",
      "sort_by": "_text_match(buckets: 10):desc,release_date:asc",
      "facet_sample_threshold": 1000,
      "facet_sample_percent": 20,
      "highlight_full_fields": "primary_artist_name,title",
      "collection": "s",
      "q": "ann",
      "facet_by": "release_decade",
      "filter_by": "genres:[`Classical`,`Pop`,`Rock`] && primary_artist_name:[`Ann Peebles`,`Anne Sofie von Otter`,`Various Artists`] && release_group_types:[`Album`]",
      "max_facet_values": 20,
      "page": 1
    },
    {
      "query_by": "primary_artist_name,title",
      "query_by_weights": "2,2",
      "sort_by": "_text_match(buckets: 10):desc,release_date:asc",
      "facet_sample_threshold": 1000,
      "facet_sample_percent": 20,
      "highlight_full_fields": "primary_artist_name,title",
      "collection": "s",
      "q": "ann",
      "facet_by": "release_group_types",
      "filter_by": "genres:[`Classical`,`Pop`,`Rock`] && primary_artist_name:[`Ann Peebles`,`Anne Sofie von Otter`,`Various Artists`] && release_decade:[`2000s`]",
      "max_facet_values": 20,
      "page": 1
    }
  ]
}
k
Faceting is an exhaustive operation. For large datasets, you have to look into sampling. Look into the the
facet_sample_percent
parameter here: https://typesense.org/docs/29.0/api/search.html#faceting-parameters
r
Hello @Kishore Nallan - Thank you for your response. If you checked the given sample payload of Song Search, It does include both the sampling parameters facet_sample_threshold and facet_sample_percent. But it still exceeds 5 seconds and got timed out. We have also tried in our collections too and got the same timeouts. Please advise. Thanks "facet_sample_threshold":1000,"facet_sample_percent":20,
Hello @Kishore Nallan - Please let me know your advise on this ? Looking forward to hear from you. Thanks
k
For large datasets, faceting is an exhaustive and slow process. Especially if you have large number of facets. You have to lower facet sample to 3%-5% and check.
r
Thank you @Kishore Nallan. Will try that. However, it would be great, if you could able to optimize the performance in subsequent releases. Looking forward to hear from you. Thanks again. Cheers !
👍 1