Hey all, I had a small question. When exporting a...
# community-help
t
Hey all, I had a small question. When exporting as a stream, is the order of documents deterministic? From the documentation, it looks like it’s in ascending order by id, which is perfect. I just want to double check, thank you! Oh, further, if there is or isn’t some level of determinism, is there any way to detect when an abort is occurring, in my case when a lambda times out, and to resume from there next time?
k
Yes it's ordered on internal ID, which is insertion order.
t
Awesome, thank you! Is there anyway for me to like capture that insertion order and reuse it after a process exit?
Or internal id, whatever the term is haha. I feel like I may not know the difference between internal id and regular id
I’d really like to basically have like a maxLastID variable, and if I am going to abort because my lambda times out, resume later with like filter_by: internal_id > maxLastID
k
Actually have you tried inserting records with the
id
in the following order:
Copy code
"id": "d",
"id": "c",
"id": "b",
"id": "a"
And check what order the export runs? I am not sure if we use user provided ID or internal ID for ordering.
t
i have not, and my ids are numeric strings. i will check, thank you!
k
With numeric string, it might follow lexographic ordeing (
11
will appear
1)
😮 1
t
ah, very sneaky. thanks!
Honestly, ordering doesn’t matter too much as long as it is consistent and I can do
filter_by: id > '111'
k
You can't really do
>
on strings.