Kyriakos Akriotis
06/16/2025, 8:12 AMCrashLoopBackoff throwing constantly and error:
typesense F20250613 09:10:49.139238 228 log_manager.cpp:310] Can't truncate logs before _applied_id=6937, last_log_kept=6935
Following some discussions in threads, I tried lowering the TYPESENSE_SNAPSHOT_INTERVAL_SECONDS but this is not solving the problem. The Pod (FOLLOWER) could not recover by itself, until I terminated it manually. I have two questions:
1. How can we solve this without manual intervention?
2. What was different in Typesense initialization when I terminated the Pod that didn't take place when it was restarting by itself during the CrashLoopBackoff events?
Thanks in advance!Kishore Nallan
06/16/2025, 8:21 AMCan't truncate logs before _applied_id=6937, last_log_kept=6935This typically happens if the follower's raft log is irreconcilable with the leader. Usually clearing the data directory of the follower and letting it sync fresh from the leader will help solve the issue. Restarting without termination will try to go off the existing logs and resume from that point which did not work.
Kyriakos Akriotis
06/16/2025, 8:23 AMKishore Nallan
06/16/2025, 8:27 AMKyriakos Akriotis
06/16/2025, 8:28 AMKishore Nallan
06/16/2025, 8:28 AMstate/log or state/meta directories.Kyriakos Akriotis
06/16/2025, 8:29 AM