#community-help

Resolving Typesense Docker Container "Stopping" issue

TLDR em1nos has an issue with a Typesense Docker container stopping regularly. Jason suggested potential disk or permission issues and offered several commands to diagnose the problem. Resolution is ongoing.

15mo

Join the chat

Jun 29, 2022 (15 months ago)

Photo of md5-f33ae8b829901656b655c985c894be3a

em1nos

07:33 PM

My local development Typesense Docker container keeps "stopping" after a while. This started today all of a sudden.

07:33

em1nos

07:33 PM

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

07:34 PM

It looks like its failing as it's trying to save a snapshot to disk. So it's likely that the disk space you've assigned to the Docker volume has run out of space.

em1nos

07:36 PM

But it's storing that on the host, right? It's linked.

07:36

em1nos

07:36 PM

And the host is not out of disk space.

07:41

em1nos

07:41 PM

This is inside the docker container.

Jason

07:43 PM

Could you share the full docker command you're using?

07:44

Jason

07:44 PM

Also, from inside the container, could you type journalctl and see if there's any interesting information around the timestamp when the typesense logs say "failed to save snapshot"

em1nos

07:44 PM

I'm using Lando, and the config looks like this.

07:45

em1nos

07:45 PM

# journalctl
No journal files were found.

Jason

07:46 PM

how about inside /var/log/syslog?

em1nos

07:46 PM

also note that it's actually failing on "renaming the snapshot"

07:47

em1nos

07:47 PM

no such log.

07:48

em1nos

07:48 PM

This has happened a few times tonight already. I can restart the container after which it works for another 10-20 minutes.

Jason

07:49 PM

Hmm, could you do du -h /data inside the container

em1nos

07:50 PM

Jason

07:50 PM

Ok that seems benign

07:50

Jason

07:50 PM

I wonder if there's some permission issue then, that prevents the docker container from writing to the host disk?

07:51

Jason

07:51 PM

Could you try creating say a 10MB file inside /data using say dd, from inside the container?

em1nos

07:53 PM

let me try that ... I'm not a Docker or linux pro, so hang on 🙂

07:59

em1nos

07:59 PM

yea copying a 14MB file to /data worked fine.

Jason

08:05 PM

I'm running out of ideas...

One last shot. Could you run ls -haRl /data when it's working fine, and then when you see an error, run the same command again and see if any of the permissions / file owners look different from before?

08:06

Jason

08:06 PM

Another thing to try is to may be move the typesense-data dir on your host to a different location, outside of your project dir... I'm wondering if there's some process on the host that changes file permissions / ownership inside your project dir that's causing this issue inside the container

em1nos

08:08 PM

Ok. I will try those things. Thank you for your help. It's very appreciated.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2764 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Trouble with Typesense Memory Usage when Restarting Docker Container

Blend reports increased memory usage when restarting Docker with Typesense, information shared with Jason and Kishore Nallan. Potential data edge case identified as potential cause, although resolution undetermined.

9mo

Troubleshooting Typesense Server Error on Docker

vikram was facing an error with Typesense Server Docker container and loss of data on restart. Kishore Nallan guided to avoid mounting tmp directory from localhost and explained stopping the Docker container.

11mo

Using Typesense in Docker Container – Importing JSONL File

Hakim faced 'Empty reply from server' error when importing a JSONL file into a Typesense-Docker container. Kishore Nallan and Jason suggested checking the docker logs and increasing default RAM and CPU allocation. Finally, Hakim successfully indexed the documents using a PHP script.

12mo

Trouble Running Typesense Docker Image on AMD Phenom II X4 955 Processor

Ian struggles to run Typesense on an AMD Phenom II X4 955 Processor, experiencing the docker image to shut down immediately upon startup. Jason and Kishore Nallan suggest multiple troubleshooting steps, identifying the potential issue as the processor's lack of support for MSSE4 flags.

11mo

Discussion About Typesense Nodes Not Synchronizing Correctly

Erick experienced an issue where documents weren't updated properly in a Typesense instance running on 3 nodes. Upon requesting debug logs and configs, Jason identified that these nodes weren't part of the same cluster. They couldn't resolve the nodes' failure to connect issue and recommended a fresh installation.

19mo