Importing data to Typesense Cloud with JS client
TLDR Zaiste attempted to import data to Typesense Cloud but encountered an error. Kishore Nallan explained the error was due to server side backpressure and suggested a curl approach for the import.
3
1
Dec 06, 2021 (23 months ago)
Zaiste
12:39 PMServerError: Request failed with HTTP code 503 | Server said: Not Ready or Lagging
. Iโm using the JS client. The connectionTimeoutSeconds
is set to 10
. I tried different values but it didnโt work. Is there something I could change to make the import work?Kishore Nallan
12:41 PMKishore Nallan
12:42 PM1
Kishore Nallan
12:43 PMKishore Nallan
12:44 PMcurl
. We can handle several GBs of uploads via curl. The backpressure exists to protect searches from getting affected adversely.Zaiste
01:07 PM100
elements (the batchSize
option to import
).Zaiste
01:08 PMKishore Nallan
01:08 PM1
Zaiste
01:10 PMcurl
- if I understand correctly the JS library is just a wrapper around your HTTP API right?Kishore Nallan
01:10 PMZaiste
01:11 PMKishore Nallan
01:11 PM1
Zaiste
01:12 PM1
Typesense
Indexed 2786 threads (79% resolved)
Similar Threads
Bulk Import 50MB JSON Files Error - Timeout and Solutions
madhweep encounters an error while bulk importing JSON files. Kishore Nallan provided help, but the issue persists. Jason intervenes and after troubleshooting, they concluded the cluster had run out of memory causing the issue. The problem was resolved by using a cluster with sufficient memory. Daniel also experienced a similar issue, resolved by increasing the timeout.
Troubleshooting Write Timeouts in Typesense with Large CSVs
Agustin had issues with Typesense getting write timeouts while loading large CSV files. Kishore Nallan suggested chunking data or converting to JSONL before loading. Through troubleshooting, they identified a possible network problem at AWS and found a workaround.
Troubleshooting Typesense Document Import Error
Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.
Typesense Server Bulk Import/Upsert Issue Resolved
Adam was confused about the discrepancy between the successful responses and the actual indexed data while working with a custom WP plugin integrating with Typesense. The issue was a bug related to fetching documents in the wrong order, not a Typesense problem.
Troubleshooting Indexing Duration in Typesense Import
Alan asked about lengthy indexing times for importing documents to Typesense. Jason suggested various potential causes, including network connectivity and system resources. They later identified the problem to be an error in Alan's code.