Typesense Performance with Large Datasets & Custom Sort
TLDR krok inquires about Typesense's performance on large datasets and custom sorting. Kishore Nallan explains that Typesense is optimized for this scenario using pagination and text relevance.
1
Feb 05, 2023 (8 months ago)
krok
03:19 PMsort_by
price. I heard about a hard limit of 1000 candidates on which Typesense does a sort to keep good performances. How does it work?Kishore Nallan
04:18 PMkrok
04:49 PMFeb 06, 2023 (8 months ago)
Kishore Nallan
04:26 AMkrok
07:56 AMquery = *
too? Does Typesense always sort the 250 first random documents only?Feb 07, 2023 (8 months ago)
krok
08:44 AMKishore Nallan
08:46 AMkrok
08:48 AMquery = *
the candidates is the whole set and therefore if only the first 250 candidates are sorted, the lowest ones are not shown first, right?Kishore Nallan
08:50 AMkrok
08:52 AMquery = *
all the documents must be eventually returned, right? They are stored internally in a non specific order, right? But you told me that only the first 250 documents are sorted when returned, it means that if the lowest document (the one that must be returned first) is very deep, it will not be returned first, as it is not in the first 250 candidates, right?Kishore Nallan
08:55 AMkrok
09:14 AMsort_by = price asc
in the settings.Kishore Nallan
09:40 AMFeb 08, 2023 (8 months ago)
krok
07:49 AMKishore Nallan
08:03 AMKishore Nallan
08:04 AM"sort_by": "_text_match(buckets: 10):desc"
krok
08:05 AMkrok
08:05 AMKishore Nallan
08:10 AMkrok
08:22 AMKishore Nallan
08:26 AMkrok
08:29 AMq = *
and hiding the left bar (filters and facets).Feb 09, 2023 (8 months ago)
krok
09:18 AMq = *
on the songs demo and it is very slow (7 secondes or 4 secondes when filtering). I understand why there is a front script preventing people to use it that way. Can you tell me the machine spec that is used for this demo?Kishore Nallan
12:47 PMMachine spec I think is 2 GB / 2 cpu.
Feb 13, 2023 (8 months ago)
krok
08:28 AMq = *
requests to the server?Kishore Nallan
08:34 AMkrok
05:52 PMkrok
05:52 PMFeb 14, 2023 (8 months ago)
Kishore Nallan
12:15 PMkrok
12:29 PMq = *
, i.e, the whole dataset?Feb 15, 2023 (8 months ago)
Kishore Nallan
11:16 AMkrok
11:18 AMFeb 17, 2023 (8 months ago)
krok
08:26 AMkrok
08:26 AMTypesense
Indexed 2779 threads (79% resolved)
Similar Threads
Phrase Search Relevancy and Weights Fix
Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.
Issues and Improvements in Typesense with 14 Million Records
Miguel experienced performance issues when using Typesense for large datasets. Jason suggested performance improvements made to Typesense since then and directed them to specific server-side parameters for better handling. Miguel agreed to try again.
Resolve Facets and Sorting Issues with Typesense
Ethan needed assistance with getting all facet values and sorting results by date using Typesense. Jason provided guidance on how to use Typesense properties to accomplish these tasks, and resolved issues related to specific use-cases provided by Ethan and Rushil.
Discussing Typesense Search Request Performance
Al experienced longer-than-reported times for Typesense search requests, sparking a detailed examination of json parsing, response times and data transfer. Jason and Kishore Nallan helped solve the issue.
Understanding Indexing and Search-As-You-Type In Typesense
Steven had queries about indexing and search-as-you-type in Typesense. Jason clarified that bulk updates are faster and search-as-you-type is resource intensive but worth it. The discussion also included querying benchmarks and Typesense's drop_tokens_threshold parameter, with participation from bnfd.