Estimating Memory Space for Embedding Records
TLDR Jerry wanted help estimating memory space for embedding. Jason provided the formula to calculate this using Typesense.
Powered by Struct AI
1
2
4mo
Jul 31, 2023 (4 months ago)
Jerry
Jerry
06:11 PMQQ, how much how much memory space does 1 record of embedding (eg. dim 1536) need? I'm trying to do some memory capacity estimation. I think it largely depends on how much space each element of the embedding needs, are they 32bit float or 64bit double?
Jason
Jason
06:13 PMEach record with vector data takes: 7 bytes * number of dimensions to index in Typesense.
So for 1M records with vector data, you’d need 7 bytes * 1536 * 1M = ~10.8GB of RAM
So for 1M records with vector data, you’d need 7 bytes * 1536 * 1M = ~10.8GB of RAM
1
Typesense
Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI
Indexed 3015 threads (79% resolved)
Similar Threads
Configuring Typesense Server for Large Record Embedding
Akash asked for typesense server config for 2.5lakh records. Jason and Kishore Nallan gave advice and resource links.
3
1mo
Memory Usage Difference between Typesense 0.23 and 0.25.1
Six was inquiring about the change in memory usage when moving from typesense 0.23 to 0.25.1. Jason confirmed that vectors do use additional memory.
3
2mo
Optimum Cluster for 1M Documents with OpenAI Embedding
Denny inquired about the ideal cluster configuration for handling 1M documents with openAI embedding. Jason recommended a specific configuration, explained record size calculation, and clarified embedding generation speed factors and the conditions that trigger openAI.
12
3mo