Understanding Memory Usage Breakdown for Collections
TLDR Chetan asked for details on determining memory usage in collections. Kishore Nallan explained that it wasn't feasible by collection but dependant on the shape of data, and other factors and once 100k docs are present, extrapolation can be attempted.
Jul 20, 2023 (2 months ago)
Kishore Nallan01:43 PM
Jul 21, 2023 (2 months ago)
Kishore Nallan01:52 AM
Indexed 2764 threads (79% resolved)
Reducing Memory Usage in Large Dataset Indexing
Alan asked for an article on reducing memory usage when indexing large datasets. Jason provided general tips without sharing an article.
Discussing Document Indexing Speeds and Typesense Features
Thomas asks about the speed of indexing and associated factors. The conversation reveals that larger batch sizes and NVMe disk usage can improve speed, but the index size is limited by RAM. Jason shares plans on supporting nested fields, and they explore a solution for products in multiple categories and catalogs.
Optimizing Bulk Indexing and Reducing RAM Usage in Typesense
Timon experienced issues with Typesense becoming unresponsive during bulk indexing and sought advice. Jason recommended larger import requests and adjusting the client-side timeout allowance, revealing a need to increase RAM allocation for Docker. Kishore Nallan undertook to find ways to optimize memory usage, particularly for geopoint indexing.