#community-help

Handling 1 Million Documents and User Permissions

TLDR Hariharan asked about storing 5k user IDs in documents and filtering. Jason suggested it's the right approach and provided additional best practices.

Powered by Struct AI
Feb 20, 2023 (10 months ago)
Hariharan
Photo of md5-18c41204e6fd9bb8436d758a90a0e5ef
Hariharan
02:24 PM
Hi...i have 1 million documents to index and search. Each document will have access to only few users in the organisation. Currently i have around 10k users in the organisation. I include of list_of_userids in each document and filter the document based on this field for all logged in users. Is it the right approach?

For each document, minimum 5k users will have permission. Is it advisable to store 5k user IDs in every document field or any other best practices?

If we include this fields, do we need to set the index=true for this field or not?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:12 PM
> Is it the right approach?
Yup that’s the right approach

> Is it advisable to store 5k user IDs in every document field or any other best practices?
Yup should be fine. You want to make sure you use exclude_fields (embedded inside a scoped search api key) and exclude this user_ids field from the search API response, so it doesn’t affect network performance

> If we include this fields, do we need to set the index=true for this field or not?
Yup, it has to be an indexed field