#community-help

Loading Specific Collections and Deleting Part of Collection in Typesense

TLDR Chetan inquired about loading specific collections and deleting part of a collection in Typesense. Jason advised that this was not possible, but suggested trying the --num-collections-parallel-load and --num-documents-parallel-load options for future reference.

Powered by Struct AI
+11
white_check_mark1
9
1mo
Solved
Join the chat
Aug 21, 2023 (1 month ago)
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
07:36 PM
is there a way to only load specific collections on startup instead of all of them?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:37 PM
No, there’s no way to do this…
white_check_mark1
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
07:39 PM
np. i assume there's no way to delete part of the collection/db either?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:39 PM
Not when Typesense is not running… no
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
07:40 PM
context:
• demo call in 20 mins
• dataset i need to demo with loads fast (4 mins)
• other data takes much longer (up to hours)
• pushing all the data to a typesense server with no data will also take much longer
but ah sg.
07:40
Chetan
07:40 PM
i'll just cross my fingers.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:41 PM
Hmm if a particular collection only takes 4 mins to load at startup, are you able to index just that from scratch in a new Typesense instance?
07:43
Jason
07:43 PM
This might be risky trying this last minute, but for the future you might want to also experiment with the --num-collections-parallel-load and --num-documents-parallel-load

https://typesense.org/docs/0.25.0/api/server-configuration.html#using-command-line-arguments
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
07:49 PM
yea i have num-collections-parallel-load at 100000, but the pubmed dataset still takes a while.

decided to just try re-pushing and it should actually be done in time - thanks for the suggestion.
+11