We are piloting typesense and one of the concern c...
# community-help
a
We are piloting typesense and one of the concern coming from team is if concurrency would be a bottleneck or not https://typesense.org/docs/overview/benchmarks.html We have roughly 1 billion rows and we want to keep 30million in search -> i.e. in Typesense schema Does it get slow for upserts when ids are same or it will remain as fast ? O(1) or O(n)
We want to upsert 5k rows per minute i guess
Does concurrency number go up with bigger RAM and higher CPU ?
The data is real and it’s not logs
j
Concurrency is dependent on the amount of CPU capacity you have. So scaling up CPU cores is the way to get more concurrency in reads and writes.
Upserts are not O(1), may be O(log n), but then it depends on the number of fields you have in the docs, etc.
For eg, from that link you shared:
Took 3.6mins to index all 2.2M records
That was with 4 indexed fields per recipe record. So 5K upserts per minute should be fine, as long as you have sufficient CPU capacity and you’re using bulk import endpoint