#community-help

Understanding Typesense Indexing and Memory Usage

TLDR Ed inquires about pros and cons of indexing in typesense. Kishore Nallan and Jason explain the purpose and benefits of Typesense as a secondary data store and how to optimize memory usage.

Powered by Struct AI

1

Sep 01, 2023 (3 months ago)
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
11:12 AM
I’m not a DB guy, what is the benefits/drawback of indexing in typesense, looks like i’m indexing everything
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:16 AM
Typesense is not designed to be a primary datastore. Our design decisions and trade-offs are tailored towards fast search and to act as a secondary data store. A primary data store will have to sacrifice some of that speed for harder consistency and durability guarantees.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
03:24 PM
To add to what Kishore said, the benefit of indexing data in Typesense is to speed up search. Primary databases were not designed with search performance as a consideration, so tend to perform slower on mid to large sized datasets. Typesense was designed specifically for this use-case, and stores the indices in memory which makes it super fast
Sep 04, 2023 (3 months ago)
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
07:53 PM
thanks, maybe I should remove some fields I don’t search against, to free up memory?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
09:13 PM
Yea, if you're not searching / filtering / faceting / sorting / grouping on a field, then you can just leave it out of the schema. These fields are unindexed and will just be stored on disk and won't count towards memory usage
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
09:36 PM
thanks

1