Discussing Large JSON File's Latency in Typesense
TLDR Daniel asked about handling a large JSON file in Typesense. Kishore Nallan explained the slowdown is due to wildcard queries hitting and sorting entire records. He suggested enabling caching to improve performance.
1Dec 01, 2021 (22 months ago)
Daniel
12:26 PM
Kishore Nallan
12:30 PMDaniel
12:38 PMDaniel
12:38 PMKishore Nallan
12:40 PM* ) on 0.22 RC builds (0.22 GA release is right around the corner)Daniel
12:42 PMKishore Nallan
12:45 PMIn the mean time, on 0.22 we also have a way for you to enable caching, so you can use that to handle this. Set
use_cache=true parameter. You can also set a cache_ttl parameter in seconds (default is 60 seconds) as a scoped API key parameter if you want to cache for longer duration.1Daniel
12:49 PMDaniel
12:50 PMKishore Nallan
12:54 PM
Typesense
Indexed 2764 threads (79% resolved)
Similar Threads
Debugging Search Issue with Typesense Server Caching
Jameshwart reported experiencing caching issues with typesense server, despite the server's default setting of not caching. Through troubleshooting steps with Kishore Nallan, they were able to confirm an issue with the Typesense Javascript client and resolved it by adding `cacheSearchResultsForSeconds: 0` after `apiKey` in the initialisation. Laura also participated in discussion, learning about server and client level caching.
Large JSONL Documents Import Issue & Resolution
Suraj was having trouble loading large JSONL documents into Typesense server. After several discussions and attempts, it was discovered that the issue was due to data quality. Once the team extracted the data again, the upload process worked smoothly.
4Issue with Search Duration on Typesense Database
Robert reported an issue about query time delay when adding a `filter_by` constraint in a large Typesense database. Kishore Nallan explained that this happens due to the order of operation and also promised to look into this issue further. Robert withdrew his interest in sponsoring the improvement due to moving from the project.
Slow, High CPU Write Operations After Collection Drop in Typesense
Himank discussed an issue in Typesense where deleting and recreating a collection led to slow write operations and high CPU usage. Kishore Nallan suggested using an alias to avoid this issue. Numerous tests and debugging was conducted as pboros contributed with local testing. Kishore Nallan aimed to start implementing a range delete and full db compaction after deletion to potentially solve the issue.
20Issues and Improvements in Typesense with 14 Million Records
Miguel experienced performance issues when using Typesense for large datasets. Jason suggested performance improvements made to Typesense since then and directed them to specific server-side parameters for better handling. Miguel agreed to try again.
2