Loading Specific Collections and Deleting Part of Collection in Typesense
TLDR Chetan inquired about loading specific collections and deleting part of a collection in Typesense. Jason advised that this was not possible, but suggested trying the --num-collections-parallel-load and --num-documents-parallel-load options for future reference.
1
1Aug 21, 2023 (1 month ago)
Chetan
07:36 PM
Jason
07:37 PM1Chetan
07:39 PMJason
07:39 PMChetan
07:40 PM• demo call in 20 mins
• dataset i need to demo with loads fast (4 mins)
• other data takes much longer (up to hours)
• pushing all the data to a typesense server with no data will also take much longer
but ah sg.
Chetan
07:40 PMJason
07:41 PMJason
07:43 PM--num-collections-parallel-load and --num-documents-parallel-loadhttps://typesense.org/docs/0.25.0/api/server-configuration.html#using-command-line-arguments
Chetan
07:49 PMdecided to just try re-pushing and it should actually be done in time - thanks for the suggestion.
1
Typesense
Indexed 2764 threads (79% resolved)
Similar Threads
Speeding Up Typesense Startup Time
Jessica experienced lengthy Typesense startup times. Kishore Nallan offered strategies around indexing times, advising to increase `thread-pool-size` and adjust the `num-collections-parallel-load`. They also offered to check schema for further optimization.
Offloading Idle Keys to Disk for Efficient Memory Usage in Typesense
Thomas proposed offloading idle keys to disk in data-rich typesense systems. Although Kishore Nallan explains that this conflicts with their in-memory design philosophy, Thomas suggests deleting idle collections from RAM and re-indexing when requested, and shares that they have implemented a similar serverless solution at their company.
2Increasing Server Startup Speed Through Parallelization
Robert was experiencing long server startup times due to single-threaded data loading. Jason and Kishore Nallan suggested increasing parallelization by adjusting load settings and distributing data across multiple collections.
Deleting Documents from Typesense Cloud and Version Upgrade
Masahiro asked how to remove documents from Typesense. Kishore Nallan clarified it varies by version and suggested upgrading from 0.20 to 0.22. Masahiro agreed to test this.
Troubleshooting Typesense Document Import Error
Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.
3