#community-help

Effective Data Storage in Typesense

TLDR Juri inquires about the best way to store warehouse data in Typesense. Jason recommends one collection with warehouse IDs, but suggests testing for performance.

26mo

Join the chat

Sep 16, 2021 (26 months ago)

Photo of md5-756d5da34cc5127c88730a39db749024

Juri

04:16 PM

Hi. e.g I have two warehouses. I have to search for items within both warehouses. Should I create for each warehouse it’s own collection? Or can I make for both one collection (so that I can search easily without having to merge & re-sort the results)? How strong is typesense actually, can I put 500GB of data in one collection? Or is it better to split it up to be safe of crashes and stuff?

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

06:03 PM

Juri In general I'd recommend storing data in one collection and store a warehouse id for eg in each record, if you want to filter data for only one warehouse.

You want to try that out first and then if you don't see the performance you expect, then split it across multiple collections

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Typesense Data Storage Issue with Cloud Run

Rizky inquired about data storage when Typesense is run on Cloud Run. Jason clarified that Typesense needs persistent data storage which Cloud Run doesn't offer.

10mo

Troubleshooting Typesense Document Import Error

Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.

8mo

Understanding Typesense's Functionality and Usage as a Search Engine

Tim asked about Typesense's function as a DB storage. Jason clarified Typesense functions primarily as a search engine and is not meant to be a primary data store. They also discussed data management strategies using Typesense. Joe provided further insight and recommended resources.

31mo