Planning Capacity with Average Document Size on Typesense Cloud
TLDR Viji inquired how to gauge the average size of documents on Typesense Cloud for capacity planning. Jason explained to first evaluate the RAM needed per document and then suggested how to apply that knowledge to the pricing model.
Aug 29, 2022 (16 months ago)
Indexed 3015 threads (79% resolved)
Typesense Memory Issues and Scaling Concerns
David raised concerns about Typesense's memory usage in their project. Jason provided guidelines for calculating RAM requirements.
Troubleshooting Typesense Document Import Error
Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.
Understanding Data Storage in Typesense
satish asked about typesense's data storage and RAM size considerations. Kishore Nallan explained on-disk storage, in-memory index reconstruction, and gave guidance on RAM sizing based on data shape.
Estimating RAM Requirements for Indexing Documents
Epi asked about index sizes in relation to document sizes and RAM requirements for their dataset. Kishore Nallan suggested indexing a sample and extrapolating results, and confirmed suitability for indexing large documents like Wikipedia articles in Typesense.
Discussing Document Indexing Speeds and Typesense Features
Thomas asks about the speed of indexing and associated factors. The conversation reveals that larger batch sizes and NVMe disk usage can improve speed, but the index size is limited by RAM. Jason shares plans on supporting nested fields, and they explore a solution for products in multiple categories and catalogs.