Using Typesense in Docker Container – Importing JSONL File
TLDR Hakim faced 'Empty reply from server' error when importing a JSONL file into a Typesense-Docker container. Kishore Nallan and Jason suggested checking the docker logs and increasing default RAM and CPU allocation. Finally, Hakim successfully indexed the documents using a PHP script.
Aug 26, 2022 (14 months ago)
Hakim
12:11 PMI'm using Typesense in a docker container. and I wanted to import a jsonl file (5M lines)
i used that curl call
curl -H "X-TYPESENSE-API-KEY: xyz" -X POST --data-binary @insee_current_insee.jsonl "<http://localhost:8108/collections/companies/document>s/import?action=create"but after around 20 minutes I got this error :
curl: (52) Empty reply from serverany idea what the problem might be ?
Kishore Nallan
12:16 PMHakim
12:23 PMthose are the last lines of
docker logsI20220826 10:31:38.936184 101 raft_server.cpp:534] Term: 5, last_index index: 6277, committed_index: 6277, known_applied_index: 6277, applying_index: 0, queued_writes: 1493, pending_queue_size: 0, local_seq
uence: 24922
E20220826 10:31:38.970467 101 raft_server.cpp:612] 1493 queued writes > healthy read lag of 1000
I20220826 10:31:38.970467 125 raft_server.h:60] Peer refresh succeeded!
E20220826 10:31:38.986366 101 raft_server.cpp:624] 1493 queued writes > healthy write lag of 500Hakim
12:40 PM{ "message": "Not Ready or Lagging"}
Jason
03:55 PMJason
03:55 PMAug 29, 2022 (14 months ago)
Hakim
08:04 AMJason
06:49 PMAug 30, 2022 (14 months ago)
Hakim
06:49 AM
Typesense
Indexed 2786 threads (79% resolved)
Similar Threads
Trouble with Typesense Memory Usage when Restarting Docker Container
Blend reports increased memory usage when restarting Docker with Typesense, information shared with Jason and Kishore Nallan. Potential data edge case identified as potential cause, although resolution undetermined.
Troubleshooting Typesense Document Import Error
Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.
Troubleshooting Typesense Server Error on Docker
vikram was facing an error with Typesense Server Docker container and loss of data on restart. Kishore Nallan guided to avoid mounting tmp directory from localhost and explained stopping the Docker container.
Large JSONL Documents Import Issue & Resolution
Suraj was having trouble loading large JSONL documents into Typesense server. After several discussions and attempts, it was discovered that the issue was due to data quality. Once the team extracted the data again, the upload process worked smoothly.
4
Issues with Importing Typesense Collection to Different Server
Kevin had problems migrating a Typesense collection between Docusaurus sites on different machines. Jason advised them on JSONL format, handling server hosting, and creating a collection schema before importing documents, leading to successful import.