#community-help

Efficiency in Key Name Sizing for Large Document Collections

TLDR Imtiaz asks if shorter key names impact storage and memory efficiency. Kishore Nallan explains key names don't impact memory but could slightly affect disk storage and retrieval times.

24mo

Join the chat

Oct 30, 2021 (25 months ago)

Photo of md5-5c86b23fcf7d52313d7a346d8cc3e86b

Imtiaz

04:37 PM

Hi there, I have a question regarding key names in a document. I have a large collection and I am attempting to scale efficiently. I have stripped the keys down to just the minimum subset already.

My question is regarding the size of the key names. Would there be any benefit in calling a key n instead of full_name for example, and handle the mappings appropriately on the frontend. (interested in impact to storage and memory - as the network impacts are obvious but not important in my use case)?

Thanks!

Oct 31, 2021 (24 months ago)

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:46 AM

👋 No impact on memory since key names don't repeat in memory. But we store the document on disk so disk storage will be lesser and so will be retrieval times. However, I don't see this making too much of a difference unless you have 100s of keys.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Discussing Document Indexing Speeds and Typesense Features

Thomas asks about the speed of indexing and associated factors. The conversation reveals that larger batch sizes and NVMe disk usage can improve speed, but the index size is limited by RAM. Jason shares plans on supporting nested fields, and they explore a solution for products in multiple categories and catalogs.

20mo

Offloading Idle Keys to Disk for Efficient Memory Usage in Typesense

Thomas proposed offloading idle keys to disk in data-rich typesense systems. Although Kishore Nallan explains that this conflicts with their in-memory design philosophy, Thomas suggests deleting idle collections from RAM and re-indexing when requested, and shares that they have implemented a similar serverless solution at their company.

20mo

Optimizing Bulk Indexing and Reducing RAM Usage in Typesense

Timon experienced issues with Typesense becoming unresponsive during bulk indexing and sought advice. Jason recommended larger import requests and adjusting the client-side timeout allowance, revealing a need to increase RAM allocation for Docker. Kishore Nallan undertook to find ways to optimize memory usage, particularly for geopoint indexing.

22mo