Server Startup Troubleshooting
TLDR Robert was unsure why their server was taking a long time to start up. Jason explained that the server was still indexing data into memory.
1
Oct 19, 2022 (14 months ago)
Robert
06:47 PMW20221019 14:44:49.868903 120213 node.cpp:811] [default_group:127.0.0.1:21807:21808 ] Refusing concurrent configuration changing
E20221019 14:44:49.868935 120213 raft_server.cpp:589] Node not ready yet (known_applied_index is 0).
E20221019 14:44:49.868968 120291 raft_server.h:62] Peer refresh failed, error: Doing another configuration change
E20221019 14:44:58.869524 120213 raft_server.cpp:589] Node not ready yet (known_applied_index is 0).
I20221019 14:44:59.869647 120213 raft_server.cpp:534] Term: 6, last_index index: 976479, committed_index: 976479, known_applied_index: 0, applying_index: 271928, queued_writes: 243525, pending_queue_size: 0, local_sequence: 8653509
W20221019 14:44:59.869679 120213 node.cpp:811] [default_group:127.0.0.1:21807:21808 ] Refusing concurrent configuration changing
E20221019 14:44:59.869736 120388 raft_server.h:62] Peer refresh failed, error: Doing another configuration change
E20221019 14:45:07.870239 120213 raft_server.cpp:589] Node not ready yet (known_applied_index is 0).
I20221019 14:45:09.870419 120213 raft_server.cpp:534] Term: 6, last_index index: 976479, committed_index: 976479, known_applied_index: 0, applying_index: 275075, queued_writes: 246346, pending_queue_size: 0, local_sequence: 8754883
W20221019 14:45:09.870451 120213 node.cpp:811] [default_group:127.0.0.1:21807:21808 ] Refusing concurrent configuration changing
E20221019 14:45:09.870555 120325 raft_server.h:62] Peer refresh failed, error: Doing another configuration change
E20221019 14:45:16.870968 120213 raft_server.cpp:589] Node not ready yet (known_applied_index is 0).
I20221019 14:45:19.871181 120213 raft_server.cpp:534] Term: 6, last_index index: 976479, committed_index: 976479, known_applied_index: 0, applying_index: 282883, queued_writes: 253661, pending_queue_size: 0, local_sequence: 8910756
W20221019 14:45:19.871206 120213 node.cpp:811] [default_group:127.0.0.1:21807:21808 ] Refusing concurrent configuration changing
E20221019 14:45:19.871248 120316 raft_server.h:62] Peer refresh failed, error: Doing another configuration change
E20221019 14:45:25.871608 120213 raft_server.cpp:589] Node not ready yet (known_applied_index is 0).
I20221019 14:45:29.871889 120213 raft_server.cpp:534] Term: 6, last_index index: 976479, committed_index: 976479, known_applied_index: 0, applying_index: 286219, queued_writes: 256907, pending_queue_size: 0, local_sequence: 8941348
W20221019 14:45:29.871923 120213 node.cpp:811] [default_group:127.0.0.1:21807:21808 ] Refusing concurrent configuration changing
E20221019 14:45:29.871968 120343 raft_server.h:62] Peer refresh failed, error: Doing another configuration change
E20221019 14:45:34.872177 120213 raft_server.cpp:589] Node not ready yet (known_applied_index is 0).
I20221019 14:45:35.256573 120214 batched_indexer.cpp:250] Running GC for aborted requests, req map size: 250049
I20221019 14:45:39.872522 120213 raft_server.cpp:534] Term: 6, last_index index: 976479, committed_index: 976479, known_applied_index: 0, applying_index: 286940, queued_writes: 257606, pending_queue_size: 0, local_sequence: 8948464
Jason
06:48 PMapplying_index: 286219, queued_writes: 256907
applying_index: 286940, queued_writes: 257606
So it’s still indexing the on-disk data to memory on start up
Robert
06:49 PMJason
06:50 PMRobert
06:50 PM1
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Troubleshooting Typesense 503 Errors and Usage Queries
Kevin encountered 503s using typesense. Jason asked for logs and explained why 503s occur. They made recommendations to remedy the issue and resolved Kevin's import parameter confusion. User was asked to open a github issue for accepting booleans.
Addressing High CPU Usage in Typesense
Robert reported high CPU usage on Typesense, even after halting all incoming searches. Kishore Nallan suggested logging heavy queries and increasing thread count. The issue was resolved after Robert found and truncated unusually large documents in the database.
Resolving Server Stoppage Issues in Typesense Multi VM Cluster
gaurav faced issues with the Typesense server in a multi VM cluster, including automatic stoppage and errors. Kishore Nallan identified the lack of a quorum and suggested using three nodes. When the issue persisted, they advised running Typesense via `nohup` or `systemd` to prevent session closure from stopping the process.