Hi all, I know this is a loaded question but we ar...
# community-help
t
Hi all, I know this is a loaded question but we are trying to figure out how to best model our data for Typesense. We have the concept of a user(client or professional), posts, and services. In our search experience, we want to display users, services, and posts as different search categories. Services need to have some data from the user model(profile pic, name, username, and location), and posts also need to have some user data(name, username, and profile pic). With that context in mind, is it better to create one collection for users with services and posts stored inside each user document or have separate collections for users, services, and posts and duplicate any user data needed for searching across those collections, which means we have to update every service and post document whenever user data changes?
j
I would recommend creating separate collections for each of users, posts and services and then duplicating any display fields between the documents in each collection
👍🏽 1
t
@Jason Bosco Is there a limit to how many document updates you can batch at a given time?
Also, is there per-client rate limiting in place for the cloud version or do we need to proxy requests and implement our own rate limiting?
j
Is there a limit to how many document updates you can batch at a given time?
Yes, but it depends on the number of CPU cores you have, the size of each document and the total number of documents in the collection. For eg, I’ve been able to update 2.2M documents, with 5-6 fields each, in about 4 minutes on a 4vCPU server
👍🏽 1
re: rate limiting, we have built-in rate limiting in the next version of Typesense (v0.26) need to put together docs for it. But in the meantime, you could put something like Cloudflare in front of each Typesense endpoint for DDOS protection + rate limiting: https://typesense.org/docs/guide/data-access-control.html#ddos-protection
👍🏽 1
t
Awesome thank you!
👍 1