Optimal batchSize for Typesense Imports/Upserts
TLDR Nathan asked about increasing batchSize for imports/upserts. Jason recommended keeping server-side batchSize to default 40, and suggested parallel imports up to the number of CPU cores for optimal performance.
2
2
2
1
May 15, 2023 (4 months ago)
Nathan
04:52 PMJason
05:06 PMSo the larger the server-side batch size, the lesser priority your search requests get
Jason
05:07 PMNathan
05:08 PM1
Nathan
07:59 PMYour explanation above makes me think that parallel is better, but I want to double check.
Jason
08:03 PM2
2
2
Typesense
Indexed 2779 threads (79% resolved)
Similar Threads
Revisiting Typesense for Efficient DB Indexing and Querying
kopach experienced slow indexing and crashes with Typesense. The community suggested to use batch import and check the server's resources. Improvements were made but additional support was needed for special characters and multi-search queries.
Typesense Server Import Batching Issues
Alex had issues with import batching in production Typesense server, with failed syncs for batch sizes over 100. Kishore Nallan suggested an upgrade to version 0.22.1 and offered to check logs.
Resolving Typesense Documents Import Error
Aljosa experienced an error while using typesense `documents().import()`, related to handling of large document arrays. Jason clarified that batch_size controls server-side batching, not client-side. He advised splitting arrays to address the issue and committed to elaborating its functionality in the docs. Aljosa proposed amending the TypeScript types to accommodate batch_size in the import options.