Using Typesense to Index Large Amounts of Data

TLDR Rafael wants to use Typesense to index 100M documents currently in MongoDBAtlas. Jason affirmed Typesense can handle it and asked for more details.

Powered by Struct AI
Jul 22, 2022 (17 months ago)
Photo of md5-f8d6fc9fcafe368f2f9b5ad7c7789e9e
03:09 PM
Hello folks, I have 100M of documentos, my ideia is use the Typesense to index this amount of data. Today I use MongodbAtlas and is impossible make a query in this amount of data. Exist a way todo thay with typesense , whats the size of server I need to do that. (I know this depende the size of data, usully my record have 3KB)
Photo of md5-8813087cccc512313602b6d9f9ece19f
03:26 PM
Rafael I've seen users index 100s of millions of documents, so this should be fine to handle in Typesense.

In the 3KB of data, will you be searching / filtering / sorting on all the fields or only a subset of them? If it's a subset, could you tell me the size of just the subset of fields (only these fields count towards ram usage, all other unindexed fields will be stored on disk)