#community-help

Understanding Memory Usage Breakdown for Collections

TLDR Chetan asked for details on determining memory usage in collections. Kishore Nallan explained that it wasn't feasible by collection but dependant on the shape of data, and other factors and once 100k docs are present, extrapolation can be attempted.

Powered by Struct AI
+11
white_check_mark1
5
2mo
Solved
Join the chat
Jul 20, 2023 (2 months ago)
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
01:39 PM
is there a way to see memory break down with respect to the collection used or even the columns in each collection?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:43 PM
It's not possible to see breakdown by collection.
+11
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
05:49 PM
how should we approach answering the question "where is my X GB of ram being used"? is it as simple as "it will be size of average record X number of records"? i assume there's extra overhead of indexing, faceting, etc.
05:52
Chetan
05:52 PM
context: i inserted 1.2M records yesterday in two new collections. one seemed to consume 1.2gb ram, the other seemed to consume 5 gb ram. it's possible i recall incorrectly. i'm trying to figure out why... 🤔
Jul 21, 2023 (2 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:52 AM
Memory usage depends on shape of data. For e.g. a lot of unique tokens == more memory. Type of field (text vs integer), whether facets are enabled etc. also contribute. Once you have about 100K docs, you can typically linearly extrapolate.
white_check_mark1