Hey peeps, wanted to ask for a Cloud cluster, what...
# community-help
n
Hey peeps, wanted to ask for a Cloud cluster, what should I expect the RAM behaviour to be if I drop a collection? In order to facilitate some schema changes, fitting the modified schema required upgrading the cluster twice. I was hoping to downgrade after that but it seems the RAM usage hasn't gone down enough where I could reasonably downgrade. Before starting, we were on a 2GB cluster. Now on 8 because the data juuuuuust just went over 4GB. I've dropped the old collection and migrated usage to the new one, but haven't yet seen a significant RAM drop in ~30min. Just want to make sure I'm not being impatient with it. Collection is roughly 900k records
a
It should me freed soon enough. But Typesense holds allocated memory so use in next searches, so I wonder if there is something to do with that? Have you confirmed that the collection has indeed dropped? This that you can try to do: 1. Run a snapshot 2. Run a DB compaction 3. Restart the node
n
Well, not seeing it anywhere in the dashboard anymore. It is a HA cluster so not sure how much that affects how long it may take. I'll give DB compaction a go after peak hours 👍 Snapshot - We're on cloud, so looks like that's not applicable in the same way?
a
Ah You're already on cloud. Could you share your cluster id?
👀 1
Yeah on cloud we run hourly snapshots
🙌 1
n
To confirm: Seeing on metrics around 5GB disk space used, 4.1GB RAM still in use. Fetching collections via the API shows the old one has been deleted.
Compaction didn't seem to do anything. Initiated a cluster restart and the drop was miniscule. Again, if this was just because of the size of the new collection that took over the old one it would make sense on RAM use, but since this is the same resource usage from before deleting the old one, I'm confused as to what to try next.
I've been testing the old schema and new one locally and it seems the data set really is just that big now 😕 Specifically when adding
sort: true
to some text-based fields the size balloons. Old collection: ~500mb Old collection, with added sort on a
title
field: ~2GB New collection (added sort + second locale-less
title_en
with sort): just under 4GB
a
Nik, Yeah, the more "features" you add, the more data structures we create an index together, therefore increasing RAM usage. On the restart, I can see a ~500 MB dip in RAM, though, which you mentioned is the old collection RAM? Since the value is so small, it seems that Typesense is keeping the RAM allocated rather than releasing it. Have you tried dropping the almost 4GB to see if the RAM drops?
n
Yep, it was just surprising that the previous collection was that miniscule compared to the new one. Didn't figure adding sorting would 8x the storage like that.
Have you tried dropping the almost 4GB to see if the RAM drops?
Yeah I wanted to confirm any other methods of dropping resource use before trying that, as I don't want a production system to go down if I immediately hit OOM the next time I need to do a big update. It's also that, I definitely plan on adding a lot more kinds of data to this system over time (though not all this size), so being efficient with the storage & cost is important.