Hi, we are trying to search for items as per the f...
# community-help
n
Hi, we are trying to search for items as per the following constraints:- • Each item will be present in multiple
lists
. We are capturing this fact by making an array of objects. A sample item would look like:-
Copy code
{
	"sku": "123123",
	"name": "Broccoli",
	"lists": {
		"L1":{"p": 1, "sort": 500, "custItemName": "Green Veg Broccoli"}, 
		"L2": {"p": 1, "sort": 300, "custItemName": "Powering Hulk"},
         .
         .
        "L30000": {"p": 1, "sort": 2}

	}
}
• Do note that under
lists
, the key is the list id and items needs to be filtered by
lists[id].p = 1
, sorted by
lists[id].sort
and searched by
lists[id].custItemName
as well as
name
◦ There can be approx 30,000 lists ◦ The search is happening by item name as well as customer's version of item name in their list ◦ The items have to be sorted by sort position as per each list Question - Is this an efficient data structure for querying where we are sorting and searching on nested parameters? Is it scalable to have 30,000 or more lists as map?
j
Typesense does not support the type of nested filtering you’re referring to: https://github.com/typesense/typesense/issues/867 So you would have to change the data structure like this:
Copy code
{"p": 1, "sort": 500, "custItemName": "Green Veg Broccoli", "parentName": "Broccoli", "sku": "123123"},
Essentially flatten to multiple documents
n
Thanks @Jason Bosco. What if we didn't care about other properties and just had an array of
listIds
? Would it be okay to have an array of 30,000 ids and we essentially just check if the item belongs to a particular list?
In our case, each list would contain 5,000 items with several properties. Hence maintaining 30,000 lists could be a challenge in terms of required memory if we had to flatten to multiple documents
j
Would it be okay to have an array of 30,000 ids and we essentially just check if the item belongs to a particular list?
Yup, that should be fine
In our case, each list would contain 5,000 items with several properties. Hence maintaining 30,000 lists could be a challenge in terms of required memory if we had to flatten to multiple documents (edited)
I’ve heard users store 100s of millions of doc in Typesense, so this shouldn’t be a problem as well
n
Thanks so much for the help @Jason Bosco!
👍 1