#community-help

Resolving Typesense Server Upload Issues Due to Insufficient Memory

TLDR Daniel had issues uploading data to a Typesense server. After recreating the server and encountering the same issue, Jason identified the problem as insufficient memory capacity.

Powered by Struct AI
13
22mo
Solved
Join the chat
Nov 16, 2021 (22 months ago)
Daniel
Photo of md5-3e862430ae06b87120e3640d3f9f8061
Daniel
03:00 AM
I'm using the correct key

./typesense-server --data-dir=/tmp/typesense-data --api-key=samplekey95

client = typesense.Client({
  'nodes': [{ 'host': 'localhost', 'port': '8108', 'protocol': 'http' }],
  'api_key': 'samplekey95'
});
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:08 AM
Are you able to access the API via curl?
Daniel
Photo of md5-3e862430ae06b87120e3640d3f9f8061
Daniel
05:59 PM
Didn't try
06:00
Daniel
06:00 PM
I deleted the server and created a new one
06:00
Daniel
06:00 PM
Now it works, currently waiting for 7GB of 44 million rows to be uploaded
06:01
Daniel
06:01 PM
Sadly, it just got Killed by the OS
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:03 PM
It sounds like you might be running out of memory...
Daniel
Photo of md5-3e862430ae06b87120e3640d3f9f8061
Daniel
06:03 PM
06:03
Daniel
06:03 PM
Yeah, it has 8GB of RAM only
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:03 PM
You'd usually need 2x-3x RAM compared to the size of your dataset, since Typesense is an in-memory store: https://typesense.org/docs/guide/system-requirements.html#choosing-ram
Daniel
Photo of md5-3e862430ae06b87120e3640d3f9f8061
Daniel
06:05 PM
Thanks
06:06
Daniel
06:06 PM
I will try with another dataset of 2.6GB
06:06
Daniel
06:06 PM
Not sure if it will hold. but I'll try