Understanding Disk Space Requirements for Designs
TLDR Pavan asked why their design requires so much disk space. Kishore Nallan explained that disk space needs depend on the average document size and how many documents are being indexed. They further clarified that the term "disk" refers to the
--data-dir provided to Typesense.
Sep 08, 2022 (16 months ago)
Kishore Nallan10:17 AM
Kishore Nallan10:18 AM
Kishore Nallan11:50 AM
Indexed 3015 threads (79% resolved)
Understanding Data Storage in Typesense
satish asked about typesense's data storage and RAM size considerations. Kishore Nallan explained on-disk storage, in-memory index reconstruction, and gave guidance on RAM sizing based on data shape.
Troubleshooting Typesense Document Import Error
Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.
Discussing Document Indexing Speeds and Typesense Features
Thomas asks about the speed of indexing and associated factors. The conversation reveals that larger batch sizes and NVMe disk usage can improve speed, but the index size is limited by RAM. Jason shares plans on supporting nested fields, and they explore a solution for products in multiple categories and catalogs.