Efficiency in Key Name Sizing for Large Document Collections
TLDR Imtiaz asks if shorter key names impact storage and memory efficiency. Kishore Nallan explains key names don't impact memory but could slightly affect disk storage and retrieval times.
1
Oct 30, 2021 (27 months ago)
Imtiaz
04:37 PMMy question is regarding the size of the key names. Would there be any benefit in calling a key
n instead of full_name for example, and handle the mappings appropriately on the frontend. (interested in impact to storage and memory - as the network impacts are obvious but not important in my use case)?Thanks!
1
Oct 31, 2021 (27 months ago)
Kishore Nallan
01:46 AM
Typesense
Indexed 3011 threads (79% resolved)
Similar Threads
Discussing Document Indexing Speeds and Typesense Features
Thomas asks about the speed of indexing and associated factors. The conversation reveals that larger batch sizes and NVMe disk usage can improve speed, but the index size is limited by RAM. Jason shares plans on supporting nested fields, and they explore a solution for products in multiple categories and catalogs.
Offloading Idle Keys to Disk for Efficient Memory Usage in Typesense
Thomas proposed offloading idle keys to disk in data-rich typesense systems. Although Kishore Nallan explains that this conflicts with their in-memory design philosophy, Thomas suggests deleting idle collections from RAM and re-indexing when requested, and shares that they have implemented a similar serverless solution at their company.
Optimizing Bulk Indexing and Reducing RAM Usage in Typesense
Timon experienced issues with Typesense becoming unresponsive during bulk indexing and sought advice. Jason recommended larger import requests and adjusting the client-side timeout allowance, revealing a need to increase RAM allocation for Docker. Kishore Nallan undertook to find ways to optimize memory usage, particularly for geopoint indexing.