#community-help

Cluster State Unhealthy on Restart with Large Collection

TLDR Julian experienced an unhealthy cluster state on restarts with a large collection. Jason explained that it's expected, and suggested running the cluster persistently like Redis.

Powered by Struct AI

1

5
5mo
Solved
Join the chat
Apr 21, 2023 (5 months ago)
Julian
Photo of md5-309edd752adeb3e0ea515a8c8165ce45
Julian
04:53 PM
Strange situation: I have a collection with ~70K documents. Whenever I restart my cluster (or in my case my docker container), the cluster state is unhealthy for a couple of minutes. The logs are showing a lot entries 155892 queued writes > healthy read lag of 1000 (counting down). The health endpoint only return ok: true once the queued writes are finished. Happens over again on every restart. Am I doing something wrong here? Any hints are appreciated 🙏
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:42 PM
This is expected, since Typesense rebuilds the in-memory index on each process restart, and is unavailable to service requests until this is complete
Julian
Photo of md5-309edd752adeb3e0ea515a8c8165ce45
Julian
05:44 PM
Understood. Do you have a take on this when doing local development where you might have to reboot your cluster multiple times a day?
Apr 22, 2023 (5 months ago)
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
11:59 AM
I usually just let the cluster run persistently, very similar to Redis
Julian
Photo of md5-309edd752adeb3e0ea515a8c8165ce45
Julian
12:23 PM
OK, got it. Thanks Jason

1