#community-help

Optimizing Product Search by Category in Typsense Collection

TLDR Jacob is inquiring about the impacts of filtering based on category hierarchy in Typsense. Kishore Nallan suggests creating separate fields for various category levels and optimizing for search, despite possible frequent updates.

Powered by Struct AI

1

Oct 21, 2022 (14 months ago)
Jacob
Photo of md5-c60456339dc3724ba99c3f7ca170d671
Jacob
01:40 PM
Hi guys! I have a performance question that maybe you can give me hint on.

I have this category hierarchy (in memory, not in typsense) which might consist of a maximum of 1k categories. Depth wise maybe max 10.

In my typsense collection of products, every document holds a single category_id value. What I want to do is to search for products that belong to a specific category_id AND all of its subcategories. I can easily traverse the category tree and identify all category_ids that I need to filter for.

My question is, how bad is it, performance wise, to potentially filter on hundreds of values?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:43 PM
The most performant way to address this is to have 10 fields, e.g. category.level0, category.level1 etc. so that when you want to filter you can do that at any level in the category hierarchy with a single filter by field.
Jacob
Photo of md5-c60456339dc3724ba99c3f7ca170d671
Jacob
01:47 PM
That would be a nice solution. The only problem I see with that is when there are updates to the category tree, a lot of documents might need to be updated (there will be millions of products in the collection)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:49 PM
But those tend to be rare as well right? Updating those fields should be pretty fast. Can probably ingest millions in < 60s.
01:50
Kishore Nallan
01:50 PM
(we support partial document updates)
01:50
Kishore Nallan
01:50 PM
I recommend optimizing for search because those tend to be many many times more frequent than category updates.

1

Jacob
Photo of md5-c60456339dc3724ba99c3f7ca170d671
Jacob
01:51 PM
Correct. Thank you for your quick response!