#community-help

Troubleshooting Document Import Issue with Typesense on Docker

TLDR Harrison experienced an issue with Typesense getting stuck when importing JSONL documents. Harrison offered to provide a copy of the docker volume to aid troubleshooting, but failed to retrieve it. Kishore Nallan recommended observing the local sequence number in the logs.

Powered by Struct AI
May 29, 2022 (19 months ago)
Harrison
Photo of md5-43a35158b04c9c49110114370dbeae06
Harrison
04:01 PM
The logs arent hugely helpful and I realise i've accidentally cleared the ones showing what looks like the issue 😢
But essentially uploading a set of JSONL documents, where on is invalid using: /collections/stackoverflow/documents/import?dirty_values=coerce_or_drop and then doing an abrupt stop in docker, causes typesense to just get stuck trying to recover and it cant?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:04 PM
Have you been able to reproduce the issue?
04:05
Kishore Nallan
04:05 PM
Are you saying that Typesense server is just stuck and not able to finish loading docs on startup?
Harrison
Photo of md5-43a35158b04c9c49110114370dbeae06
Harrison
04:05 PM
Looks like it
04:05
Harrison
04:05 PM
well, Looked*
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:06 PM
Then how did it resolve?
Harrison
Photo of md5-43a35158b04c9c49110114370dbeae06
Harrison
04:06 PM
it didnt. I just wiped the container
04:06
Harrison
04:06 PM
unless it did start up correctly, and something else blocked it from being 'ready'
04:07
Harrison
04:07 PM
Would It be helpful If I send the docker volume that was causing it to get stuck in the given state?
04:07
Harrison
04:07 PM
Thought I had removed it but turns out I haven't :thumbsup: So should be useful to debug
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:07 PM
Yes any form of reproduceable artifact is good
04:08
Kishore Nallan
04:08 PM
The key thing to notice when you get that lag message is to see if the local sequence number in the logs -- to see whether it's increasing.
Harrison
Photo of md5-43a35158b04c9c49110114370dbeae06
Harrison
04:15 PM
Just bundling the volume up, it's reasonably chunky at about 6GB so might take a little bit
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:16 PM
No worries, it's pretty late here so I'll probably be able to download and look only in the morning.
Harrison
Photo of md5-43a35158b04c9c49110114370dbeae06
Harrison
04:17 PM
Fair enough, shall send once it's done (or more likely might go into a blob store)
Jun 15, 2022 (19 months ago)
Harrison
Photo of md5-43a35158b04c9c49110114370dbeae06
Harrison
07:04 PM
Sorry, I realise I completely forgot about this 😅 Unfortunately I wasn't able to retrieve the volume data, something wasn't happy about me trying to copy it across and cleared everything 😕

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Using Typesense in Docker Container – Importing JSONL File

Hakim faced 'Empty reply from server' error when importing a JSONL file into a Typesense-Docker container. Kishore Nallan and Jason suggested checking the docker logs and increasing default RAM and CPU allocation. Finally, Hakim successfully indexed the documents using a PHP script.

9
16mo

Fixing Errors on Typesense Cluster

Tugay is having issues with their Typesense cluster, and Jason is trying to diagnose the problem. They have ruled out issues with the data dir and are currently considering whether the errors could be due to high concurrent writes or running bulk migrations. They plan to test with 0.24.0.rcn56 Typesense version.

1

26
11mo

Troubleshooting Typesense Document Import Error

Christopher had trouble importing 2.1M documents into Typesense due to memory errors. Jason clarified the system requirements, explaining the correlation between RAM and dataset size, and ways to tackle the issue. They both also discussed database-like query options.

3

30
11mo

Improving Query Results and Docker Issue with Typesense

Draisy struggled with retrieving query results and a docker command not working with a newer version of typesense. Jason suggested getting Typesense logs and upgrading cluster, but problems persist.

2

12
5mo

Troubleshooting Write Timeouts in Typesense with Large CSVs

Agustin had issues with Typesense getting write timeouts while loading large CSV files. Kishore Nallan suggested chunking data or converting to JSONL before loading. Through troubleshooting, they identified a possible network problem at AWS and found a workaround.

2

59
32mo