Hi All, I need to understand how typesense work in...
# community-help
v
Hi All, I need to understand how typesense work internally, like how it store data and retrive data. As in apache lucense it uses inverted index.from this i will get to know how much memory will required.
h
I believe the recommended calculation to do is
memory = 2 * size_of_data
Unlike Lucene, etc... Typesense stores the actual indexes in memory entirely. There's also https://cloud.typesense.org/pricing/calculator where you can input the number of docs and the average size of each doc to get an estimate:
j
Typesense also uses an inverted index, but like Harrison said the entire index is stored in memory. Here’s how to choose RAM: https://typesense.org/docs/guide/system-requirements.html#choosing-ram
v
it is for one server right? or we can this ram into multiple server.
j
Correct, it’s for one server. When you use Typesense in a clustered mode on multiple servers, the entire dataset is fully replicated to each server, so each server should have enough RAM to hold the full dataset
s
What happens if your server does not provide enough memory? Will typesense crash or will it then also disk?
j
Typesese is an in-memory store, so without enough RAM it will first start using swap space which will slow it down significantly, and then most likely the OS will kill the Typesense process to preserve itself. If not, if the memory usage grows fast enough, the OS itself might become unstable.