#community-help

Estimating JSON Weight in .documents().export() Method

TLDR gab experiences crashes in a docker container during export and suspects a memory issue. Kishore Nallan suggests trying on 0.24.0 docker, but gab later discovers it was another task crashing the container.

Powered by Struct AI

1

10
8mo
Solved
Join the chat
Feb 07, 2023 (8 months ago)
gab
Photo of md5-8be2a24edf7a95c9c74abce4b1130c3e
gab
11:35 AM
Hi,
is there a way to estimate the weight of the JSON returned by the .documents().export() method ?
I'm running it in a docker container that is crashing doing the export. I cannot catch and log anything so I suspect a memory issue. I'm running a container with 8Go of allocated memory and wonder if the export would not exceed it.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:41 AM
We recently made an optimization for export that uses too much memory. There is already a fix that will be going out in a 0.24.1 patch release this week.
11:42
Kishore Nallan
11:42 AM
I can share a fixed temporary build if you can tell me how you are running Typesense.
gab
Photo of md5-8be2a24edf7a95c9c74abce4b1130c3e
gab
11:46 AM
I have an instance in typesense cloud running v0.23.0.rc27
I'm using Typesense package v1.4.5 for executing the export
I haven' checked yet what would be breaking going to 0.24 :thinking_face:
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:47 AM
Oh then this issue is not related to that change.
11:48
Kishore Nallan
11:48 AM
Can you try on 0.24.0 docker first? Since you already use it locally.
11:48
Kishore Nallan
11:48 AM
Or are you saying that you are running the export script inside Docker?
gab
Photo of md5-8be2a24edf7a95c9c74abce4b1130c3e
gab
11:49 AM
yes I'm running the export script inside Docker running in Google cloud run
11:51
gab
11:51 AM
But you might be right, I should try somehow to do the export locally targeting my remote instance
Feb 08, 2023 (8 months ago)
gab
Photo of md5-8be2a24edf7a95c9c74abce4b1130c3e
gab
08:38 AM
I found that was not related at all from the weight of this request but it was because of another asynchronous task that was silently crashing the container during the export.
thanks for the help

1