I’m trying to figure out if my node’s resting memo...
# community-help
s
I’m trying to figure out if my node’s resting memory makes sense. It’s in Typesense Cloud. The reason I’m confused is that it appears to be using 2.3 gb of memory with no collections at all. Details in the thread.
metrics after 12 hours at rest with no collections/documents { “system_cpu1_active_percentage”: “0.00", “system_cpu2_active_percentage”: “0.00", “system_cpu3_active_percentage”: “0.00", “system_cpu4_active_percentage”: “0.00", “system_cpu5_active_percentage”: “0.00", “system_cpu6_active_percentage”: “0.00", “system_cpu7_active_percentage”: “0.00", “system_cpu8_active_percentage”: “10.00", “system_cpu_active_percentage”: “2.47", “system_disk_total_bytes”: “85878374400", “system_disk_used_bytes”: “656809984", “system_memory_total_bytes”: “16439255040", “system_memory_total_swap_bytes”: “2147479552", “system_memory_used_bytes”: “1292021760", “system_memory_used_swap_bytes”: “0", “system_network_received_bytes”: “187676672", “system_network_sent_bytes”: “28823607", “typesense_memory_active_bytes”: “2478309376", “typesense_memory_allocated_bytes”: “2355328416", “typesense_memory_fragmentation_ratio”: “0.05", “typesense_memory_mapped_bytes”: “2558328832", “typesense_memory_metadata_bytes”: “162131080", “typesense_memory_resident_bytes”: “2478309376", “typesense_memory_retained_bytes”: “283312128" }%
it’s a node with 16gb of memory, 8 vcpu
I did have a collection in it previously, but after I deleted it and the memory refused to fully release I downgraded and then re-upgraded the node capacity. The metrics above are after the node was re-upgraded and at rest with no collections for >12 hours. Also, interestingly, the metrics above don’t seem to line up with the memory usage shown in the Cloud graph screenshot here, though maybe I’m mistaken 🤔 I’m using typesense version 27.1
when I create a brand new instance with the same configuration, it uses significantly less memory { “system_cpu1_active_percentage”: “0.00", “system_cpu2_active_percentage”: “0.00", “system_cpu3_active_percentage”: “0.00", “system_cpu4_active_percentage”: “0.00", “system_cpu5_active_percentage”: “0.00", “system_cpu6_active_percentage”: “0.00", “system_cpu7_active_percentage”: “0.00", “system_cpu8_active_percentage”: “0.00", “system_cpu_active_percentage”: “0.00", “system_disk_total_bytes”: “85857402880", “system_disk_used_bytes”: “650887168", “system_memory_total_bytes”: “16439259136", “system_memory_total_swap_bytes”: “2147479552", “system_memory_used_bytes”: “984768512", “system_memory_used_swap_bytes”: “0", “system_network_received_bytes”: “113035983", “system_network_sent_bytes”: “1761903", “typesense_memory_active_bytes”: “350879744", “typesense_memory_allocated_bytes”: “243142208", “typesense_memory_fragmentation_ratio”: “0.31", “typesense_memory_mapped_bytes”: “424411136", “typesense_memory_metadata_bytes”: “34095648", “typesense_memory_resident_bytes”: “350879744", “typesense_memory_retained_bytes”: “43778048" }%
f
Hi there. Are there no collections present in your cluster? So calling out to the
/collections
endpoint produces an empty array result?
[]
s
exactly - an empty array
f
And has that issue been the case for a long time (more than a couple of hours), or is it something that occurred more recently?
s
it’s been at rest (no activity) for >12 hours
at the time of the metrics/screenshot
with no change over time (that I can see)
k
Did you use an embedding model in any of your collections?
s
good question - openai. The configuration is here
ah actually, it’s slightly different - it’s using the default
num_dim
and
openai/text-embedding-3-small
. Otherwise it’s the same
k
This is super strange. The only thing I can think of is the memory allocator not releasing the memory reserved back to the OS so it can reuse the memory.
s
would it help if I shared the ID of the node? I’ve created a new one to use, so at this point I’m just keeping the old one around in case someone wants to debug it
k
Yes, please DM me.
s
thanks! Sent 🙏