Handling High Volume of Updates with Typesense
TLDR Ahmad inquired about the ability of Typesense to handle 10-100k updates every 5 minutes. Jason confirmed Typesense can handle such volume with adequate CPU capacity and usage of the bulk import endpoint.
Nov 08, 2022 (11 months ago)
Ahmad
07:47 PM
Jason
07:50 PMCopy-pasting my response here:
Typesense should be able to handle that volume of updates, provided that you have sufficient CPU capacity and also that you use the bulk import endpoint to send all the 100K changes split over say 5-10 API calls, with a client-side timeout as high as 30 minutes for each API call.
Typesense
Indexed 2776 threads (79% resolved)
Similar Threads
Handling Frequently Updated Data on Typesense Server
Sangeetha expressed satisfaction about Typesense but sought advice on handling frequent data updates. Jason suggested setting up a webhook for changes or regularly polling the data store. Sangeetha found the advice helpful.
Handling High Volume Document Updates with Typesense
Stefan asked about Typesense's ability to handle regular document updates. Jason suggested using Typesense's bulk update feature. They also discussed Algolia's restriction on high volume requests.
Efficient Dataset Creation and Maintenance in Typesense with React
sonu asked about Typesense dataset management in a React project. Nick shared their solution using a postgres database, cron, and the import endpoint. Kishore Nallan suggested updating Typesense whenever MySQL data changes and explained primary databases' role. The dataset file would reside with Typesense, not affecting the front-end load time.