Using Typesense to Index Large Amounts of Data
TLDR Rafael wants to use Typesense to index 100M documents currently in MongoDBAtlas. Jason affirmed Typesense can handle it and asked for more details.
Jul 22, 2022 (17 months ago)
In the 3KB of data, will you be searching / filtering / sorting on all the fields or only a subset of them? If it's a subset, could you tell me the size of just the subset of fields (only these fields count towards ram usage, all other unindexed fields will be stored on disk)
Indexed 3015 threads (79% resolved)
Understanding Data Storage in Typesense
Ethan wanted information on how to index large amounts of data. Jason guided that Typesense is for secondary data storage and all data for search results must be in Typesense.
Efficiently Indexing Large Number of Documents in Typesense
Henrique asked for an efficient way to index 30 million documents using typesense. Kishore Nallan suggested checking the import end-point response for success or error messages.
Indexing and Updating Large Data in Typesense from MongoDB
Vasudev asked about indexing large data from MongoDB to Typesense, and updating subsequent data. Kishore Nallan advised batched import and using the import API with upsert or update actions.