Nikhil Agarwal
03/17/2023, 5:35 PMlists
. We are capturing this fact by making an array of objects. A sample item would look like:-
{
"sku": "123123",
"name": "Broccoli",
"lists": {
"L1":{"p": 1, "sort": 500, "custItemName": "Green Veg Broccoli"},
"L2": {"p": 1, "sort": 300, "custItemName": "Powering Hulk"},
.
.
"L30000": {"p": 1, "sort": 2}
}
}
• Do note that under lists
, the key is the list id and items needs to be filtered by lists[id].p = 1
, sorted by lists[id].sort
and searched by lists[id].custItemName
as well as name
◦ There can be approx 30,000 lists
◦ The search is happening by item name as well as customer's version of item name in their list
◦ The items have to be sorted by sort position as per each list
Question - Is this an efficient data structure for querying where we are sorting and searching on nested parameters? Is it scalable to have 30,000 or more lists as map?Jason Bosco
03/17/2023, 6:05 PM{"p": 1, "sort": 500, "custItemName": "Green Veg Broccoli", "parentName": "Broccoli", "sku": "123123"},
Jason Bosco
03/17/2023, 6:05 PMNikhil Agarwal
03/17/2023, 7:17 PMlistIds
? Would it be okay to have an array of 30,000 ids and we essentially just check if the item belongs to a particular list?Nikhil Agarwal
03/17/2023, 7:19 PMJason Bosco
03/17/2023, 7:25 PMWould it be okay to have an array of 30,000 ids and we essentially just check if the item belongs to a particular list?Yup, that should be fine
Jason Bosco
03/17/2023, 7:26 PMIn our case, each list would contain 5,000 items with several properties. Hence maintaining 30,000 lists could be a challenge in terms of required memory if we had to flatten to multiple documents (edited)I’ve heard users store 100s of millions of doc in Typesense, so this shouldn’t be a problem as well
Nikhil Agarwal
03/17/2023, 8:11 PM