Understanding Dataset Sizes and Data Types for Typesense

Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.

sonu asked about Typesense dataset management in a React project. Nick shared their solution using a postgres database, cron, and the import endpoint. Kishore Nallan suggested updating Typesense whenever MySQL data changes and explained primary databases' role. The dataset file would reside with Typesense, not affecting the front-end load time.

kopach experienced slow indexing and crashes with Typesense. The community suggested to use batch import and check the server's resources. Improvements were made but additional support was needed for special characters and multi-search queries.

Miguel experienced performance issues when using Typesense for large datasets. Jason suggested performance improvements made to Typesense since then and directed them to specific server-side parameters for better handling. Miguel agreed to try again.

TJ asked if Typesense would move to a model using memory-mapped files for datasets. Kishore Nallan said they have no concrete plans but appreciated the idea and would consider community feedback.