Increasing Server Startup Speed Through Parallelization
TLDR Robert was experiencing long server startup times due to single-threaded data loading. Jason and Kishore Nallan suggested increasing parallelization by adjusting load settings and distributing data across multiple collections.
Oct 20, 2022 (14 months ago)
Robert
10:14 PMRobert
10:34 PMRobert
10:35 PMOct 21, 2022 (14 months ago)
Jason
12:32 AM--num-collections-parallel-load
and --num-documents-parallel-load
Documented here: https://typesense.org/docs/0.23.1/api/server-configuration.html#using-command-line-arguments
Jason
12:33 AMKishore Nallan
12:37 AMRobert
12:38 AM--num-documents-parallel-load=10000
Robert
12:38 AMKishore Nallan
12:40 AMTypesense
Indexed 3015 threads (79% resolved)
Similar Threads
Speeding Up Typesense Startup Time
Jessica experienced lengthy Typesense startup times. Kishore Nallan offered strategies around indexing times, advising to increase `thread-pool-size` and adjust the `num-collections-parallel-load`. They also offered to check schema for further optimization.
Loading Specific Collections and Deleting Part of Collection in Typesense
Chetan inquired about loading specific collections and deleting part of a collection in Typesense. Jason advised that this was not possible, but suggested trying the `--num-collections-parallel-load` and `--num-documents-parallel-load` options for future reference.
Optimizing Typesense Implementation for Large Collections
Oskar faced performance issues with his document collection in Typesense due to filter additions. Jason suggested trying a newer Typesense build and potentially partitioning the data into country-wise collections. They also discussed reducing network latency with CDN solutions.
Addressing Cluster Issue due to Excessive Data
Andrew had trouble with cluster operations due to excessive data and collections. Jason advised flushing the data and stated that the upcoming update will remedy such issues. Both agreed to stick to v0.19 and not to fill the cluster excessively.
Addressing Typesense Server Issues and Optimization Needs
Robert had an issue with a 'stuck' typesense server. Jason and Kishore Nallan gave advice on handling writes, configuration for high search volumes, and running multiple typesense instances. They also recommended monitoring CPU usage and updating the server version for bug fixes.