Finding and Processing 6GB Typesense Book Search Data
TLDR satish inquired about obtaining 6GB data similar to Typesense Book search for a POC. Kishore Nallan directed to Open Library data and processing scripts, while Jason suggested running scripts on the downloaded Open Library dataset to extract necessary fields.
Jun 07, 2022 (17 months ago)
satish
09:52 AMKishore Nallan
09:56 AMKishore Nallan
09:57 AMsatish
10:09 AMJason
01:39 PMsatish
01:40 PMsatish
01:41 PMJason
01:41 PMTypesense
Indexed 2779 threads (79% resolved)
Similar Threads
Troubleshooting Typesense Document Import Error
Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.
Revisiting Typesense for Efficient DB Indexing and Querying
kopach experienced slow indexing and crashes with Typesense. The community suggested to use batch import and check the server's resources. Improvements were made but additional support was needed for special characters and multi-search queries.
Understanding Dataset Sizes and Data Types for Typesense
Ethan questioned about dataset size limits and data types for Typesense. Jason clarified that as long as the dataset fits the RAM, Typesense works, also adding that Typesense supports only JSONL.
Issues and Improvements in Typesense with 14 Million Records
Miguel experienced performance issues when using Typesense for large datasets. Jason suggested performance improvements made to Typesense since then and directed them to specific server-side parameters for better handling. Miguel agreed to try again.
Discussing Typesense Cloud's SSDs, NVMe, and Resources Needed
A asked about Typesense's storage type and configuration possibilities. Jason shared that they use SSDs and suggested NVMe SSDs for high-availability instances. They discussed server resources needed for specific user cases and briefly touched on DDoS protection via Cloudflare.