Already got

{"success":true}{"message": "Rejecting write: running out of resource type: OUT_OF_MEMORY"}

after the first 150 documents As far I see I did not hit a single limit (lowest package of .5GB) Dataset complete size is merely 80mb

Sometimes, if you keep recreating collections, the allocator is lazy to return memory back to OS in anticipation of future allocations. So we have improved this memory checking logic in recent builds. Would you be able to try on

28.0.rc23

to see if it helps?

Uhm, but this is the first collection I see it was clearly a

⚠ Total Memory (RAM + Swap) Usage exceeded available capacity at least once during this time period, given the amount of data you've indexed. We highly recommend upgrading your cluster's RAM capacity to handle the data you've indexed.

So my only concern is, how we can better estimate it For now, we have upgraded to 5gb just to do the initial upsert and will observe the stat to see what happens I feel the estimate of "size of JSONL/CSV for ram" is inadeqate, since we are so massively under .5 gb and it eat up all with a few upserts

Are you using a local emebdding model?

GPT

DM me the cluster id

Oh, ok - so it is expected to be slow-ish Thanks for confirming.

Automated cluster rotation rotates node by node to be safe.