Hello is there a way to export typesense documents...
# community-help
l
Hello is there a way to export typesense documents chunk by chunk? thanks in advance?
f
Hey @Luka , Typesense’s export endpoint already streams results in JSONL, and it does support a batch_size query parameter to control how many docs are returned per chunk in the stream. There isn’t a separate “chunk by chunk export” API beyond that. If you need to export only part of a collection at a time, the usual approach is to partition the export with filter_by and run multiple exports, or consume the streamed JSONL response incrementally on your side.
l
thanks is this written in documentation somewhere?
f
Matter of fact, no! Thanks for bringing this up.
l
thank you 😉 lmk when it's added
@Fanis Tharropoulos Hey just wondering can you please share an example in the meantime so I start using it now?
f
Yep, if you’re on v0.24+, you can use batch_size like this:
Copy code
curl -H "X-TYPESENSE-API-KEY: $TYPESENSE_API_KEY" \
 "http://localhost:8108/collections/companies/documents/export?batch_size=100"
If you want to export only part of a collection, combine it with filter_by
Copy code
curl -G -H "X-TYPESENSE-API-KEY: $TYPESENSE_API_KEY" \
    --data-urlencode 'filter_by=country:USA' \
    --data-urlencode 'batch_size=100' \
    "http://localhost:8108/collections/companies/documents/export"
batch_size defaults to 100 if you don’t pass it.
l
ok thanks