Hi Guys,
I have 3 million posts, and i’m bulk inde...
# community-helpd
Digamber Pradhan
03/09/2023, 11:49 AMHi Guys,
I have 3 million posts, and i’m bulk indexing and during the indexing process i run tino cURL error 28,
k
Kishore Nallan
03/09/2023, 11:50 AMThe post body might be too large
d
Digamber Pradhan
03/09/2023, 11:51 AMThe thing is it bails after different times
k
Kishore Nallan
03/09/2023, 11:52 AMWhat's the size of the file being posted?
d
Digamber Pradhan
03/09/2023, 11:52 AMAdditionally - the current code is doing a request to /health to check condition
Digamber Pradhan
03/09/2023, 11:52 AMthat’s when it’s being bailed
k
Kishore Nallan
03/09/2023, 11:53 AMAre you using the php client?
d
Digamber Pradhan
03/09/2023, 11:53 AMNo we’re making cURL requests
k
Kishore Nallan
03/09/2023, 11:53 AMSo the thing is if you write too fast, Typesense can return 503 status code as a backpressure mechanism to let writes catch up
Kishore Nallan
03/09/2023, 11:54 AMIf you get a 503, backoff and retry after a short interval.
d
Digamber Pradhan
03/09/2023, 11:57 AMok let me see if i can reduce the payload and test it out again.
Also - not sure if i’m getting a 503 error - will verify and check
Digamber Pradhan
03/09/2023, 12:06 PMAlso can you let me know how to measure payload size here ?
k
Kishore Nallan
03/09/2023, 12:07 PMThe raw jsonl post you are passing. Max payload size is 10 G so I don't think you are running into that.
d
Digamber Pradhan
03/09/2023, 12:07 PMyeah - ok, that’s probably not it - no way i’m going over 10G
Digamber Pradhan
03/09/2023, 12:07 PMwill do some more test