#community-help

Optimizing Product Search by Category in Typsense Collection

TLDR Jacob is inquiring about the impacts of filtering based on category hierarchy in Typsense. Kishore Nallan suggests creating separate fields for various category levels and optimizing for search, despite possible frequent updates.

14mo

Join the chat

Oct 21, 2022 (14 months ago)

Photo of md5-c60456339dc3724ba99c3f7ca170d671

Jacob

01:40 PM

Hi guys! I have a performance question that maybe you can give me hint on.

I have this category hierarchy (in memory, not in typsense) which might consist of a maximum of 1k categories. Depth wise maybe max 10.

In my typsense collection of products, every document holds a single category_id value. What I want to do is to search for products that belong to a specific category_id AND all of its subcategories. I can easily traverse the category tree and identify all category_ids that I need to filter for.

My question is, how bad is it, performance wise, to potentially filter on hundreds of values?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:43 PM

The most performant way to address this is to have 10 fields, e.g. category.level0, category.level1 etc. so that when you want to filter you can do that at any level in the category hierarchy with a single filter by field.

Jacob

01:47 PM

That would be a nice solution. The only problem I see with that is when there are updates to the category tree, a lot of documents might need to be updated (there will be millions of products in the collection)

Kishore Nallan

01:49 PM

But those tend to be rare as well right? Updating those fields should be pretty fast. Can probably ingest millions in < 60s.

01:50

Kishore Nallan

01:50 PM

(we support partial document updates)

01:50

Kishore Nallan

01:50 PM

I recommend optimizing for search because those tend to be many many times more frequent than category updates.

Jacob

01:51 PM

Correct. Thank you for your quick response!

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Discussing Document Indexing Speeds and Typesense Features

Thomas asks about the speed of indexing and associated factors. The conversation reveals that larger batch sizes and NVMe disk usage can improve speed, but the index size is limited by RAM. Jason shares plans on supporting nested fields, and they explore a solution for products in multiple categories and catalogs.

23mo

Discussions on Typesense, Collections, and Dynamic Fields

Tugay shares plans to use Typesense for their SaaS platform and asks about collection sizes and sharding. Jason clarifies Typesense's capabilities and shares a beta feature. They discuss using unique collections per customer and new improvements. Kishore Nallan and Gabe comment on threading and data protection respectively.

35mo

Performance Characteristics of Filtering Search Results

Oskar queries the performance difference in filtering search results. Jason clarifies how filters work and provides performance improvement suggestions like increasing vCPUs and sharding the collection. Kishore Nallan explains filter IDs and document ID matching. The thread concludes with discussions on performance tradeoffs in filter implementation.

15mo