Handling Data Index in Memory
TLDR Adrian asked what happens when memory runs out while indexing. Kishore Nallan explained the process either results in latency due to swapping to disk or killing of the program by the OS.
Sep 17, 2021 (26 months ago)
Adrian
06:42 AMKishore Nallan
07:55 AMAdrian
08:38 AMTypesense
Indexed 2779 threads (79% resolved)
Similar Threads
Understanding Typesense's Internal Operation and Memory Requirement
vikram asked about Typesense's functionality and memory. Harrison, Jason explained that Typesense uses an inverted index stored in memory, with recommendations on how to calculate required memory. Jason warned about possible crashes if inadequate memory is provided.
Understanding Data Storage in Typesense
satish asked about typesense's data storage and RAM size considerations. Kishore Nallan explained on-disk storage, in-memory index reconstruction, and gave guidance on RAM sizing based on data shape.
Handling Growing Datasets Without Running Out of Memory
Steffen asks about handling growing datasets without running out of memory. Kishore Nallan says they don't have plans for using disk indexing and have chosen the current approach for specific use cases.