#community-help

Understanding Typesense Log Files and Snapshot Settings

TLDR Bartek inquires about log files in Typesense and if they can be disabled. Kishore Nallan explains they are part of the raft logs for persistence and can be truncated by adjusting the snapshot interval.

Powered by Struct AI

1

8
7mo
Solved
Join the chat
Apr 27, 2023 (7 months ago)
Bartek
Photo of md5-b2bee2a5cdeadec2a545df719fb80a2d
Bartek
11:20 AM
Hi guys, I was woking on improving stability of our typesense instance. And I was wondering what do those logs mean:
 close a full segment. Current first_index: 8357 last_index: 8479 raft_sync_segments: 0 will_sync: 1 path: /usr/share/typesense/data/state/log/log_00000000000000008357_00000000000000008479

Renamed `/usr/share/typesense/data/state/log/log_inprogress_00000000000000008357' to `/usr/share/typesense/data/state/log/log_00000000000000008357_00000000000000008479'

Created new segment `/usr/share/typesense/data/state/log/log_inprogress_00000000000000008480' with fd=93

When I print those files I get the data I inserted to typesense, so is this log as in logging and I'm unwittingly logging all the data into files?
Or is it something else?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:25 AM
That's the data being written to the raft logs for persistence and for distribution to other nodes in the cluster.
Bartek
Photo of md5-b2bee2a5cdeadec2a545df719fb80a2d
Bartek
11:27 AM
Thanks, assuming a single instance can I disable this? Or expect those files to be at some point deleted to free up space?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:28 AM
Every time the snapshot runs (default 60 mins) those logs will be truncated.
Bartek
Photo of md5-b2bee2a5cdeadec2a545df719fb80a2d
Bartek
11:30 AM
I see, thanks 😄
11:35
Bartek
11:35 AM
Is there a setting to do that more often?
(I don't see that option on https://typesense.org/docs/0.24.1/api/server-configuration.html)
11:40
Bartek
11:40 AM
never mind I think you mean --snapshot-interval-seconds
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:42 AM
Yes

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Discussion About Typesense Nodes Not Synchronizing Correctly

Erick experienced an issue where documents weren't updated properly in a Typesense instance running on 3 nodes. Upon requesting debug logs and configs, Jason identified that these nodes weren't part of the same cluster. They couldn't resolve the nodes' failure to connect issue and recommended a fresh installation.

2

91
23mo

Typesense Log Issue Resolved

Jayesh requested help resolving large log files in Typesense. Kishore Nallan suggested adjusting the snapshot-interval-seconds flag. The suggestion resolved the issue.

1

4
4mo
Solved

"Resolving Startup Issue with Typesense Single Replica Set in Local `K8s` Setup"

Vishal encountered issues starting a Typesense single replica set due to directory permission issues. Kishore Nallan suggested checking if the volume was accessible for writes. Vishal resolved the problem by adjusting permissions on the host.

2

10
22mo
Solved

Troubleshooting Typesense Snapshot Errors in AWS Docker Container

Arthur experienced recurring Typesense snapshotting errors in an AWS Docker container. Kishore Nallan diagnosed two different issues: a known Google log issue to be fixed in the next release and an unknown file opening error, and guided Arthur on creating GitHub issues for both.

20
27mo
Solved

Addressing High CPU Usage in Typesense

Robert reported high CPU usage on Typesense, even after halting all incoming searches. Kishore Nallan suggested logging heavy queries and increasing thread count. The issue was resolved after Robert found and truncated unusually large documents in the database.

35
14mo
Solved