#community-help

Understanding Typesense Log Files and Snapshot Settings

TLDR Bartek inquires about log files in Typesense and if they can be disabled. Kishore Nallan explains they are part of the raft logs for persistence and can be truncated by adjusting the snapshot interval.

7mo

Join the chat

Apr 27, 2023 (7 months ago)

Photo of md5-b2bee2a5cdeadec2a545df719fb80a2d

Bartek

11:20 AM

Hi guys, I was woking on improving stability of our typesense instance. And I was wondering what do those logs mean:

 close a full segment. Current first_index: 8357 last_index: 8479 raft_sync_segments: 0 will_sync: 1 path: /usr/share/typesense/data/state/log/log_00000000000000008357_00000000000000008479

Renamed `/usr/share/typesense/data/state/log/log_inprogress_00000000000000008357' to `/usr/share/typesense/data/state/log/log_00000000000000008357_00000000000000008479'

Created new segment `/usr/share/typesense/data/state/log/log_inprogress_00000000000000008480' with fd=93

When I print those files I get the data I inserted to typesense, so is this log as in logging and I'm unwittingly logging all the data into files?
Or is it something else?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

11:25 AM

That's the data being written to the raft logs for persistence and for distribution to other nodes in the cluster.

Bartek

11:27 AM

Thanks, assuming a single instance can I disable this? Or expect those files to be at some point deleted to free up space?

Kishore Nallan

11:28 AM

Every time the snapshot runs (default 60 mins) those logs will be truncated.

Bartek

11:30 AM

I see, thanks 😄

11:35

Bartek

11:35 AM

Is there a setting to do that more often?
(I don't see that option on https://typesense.org/docs/0.24.1/api/server-configuration.html)

11:40

Bartek

11:40 AM

never mind I think you mean --snapshot-interval-seconds

Kishore Nallan

11:42 AM

Yes

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Discussion About Typesense Nodes Not Synchronizing Correctly

Erick experienced an issue where documents weren't updated properly in a Typesense instance running on 3 nodes. Upon requesting debug logs and configs, Jason identified that these nodes weren't part of the same cluster. They couldn't resolve the nodes' failure to connect issue and recommended a fresh installation.

23mo

Typesense Log Issue Resolved

Jayesh requested help resolving large log files in Typesense. Kishore Nallan suggested adjusting the snapshot-interval-seconds flag. The suggestion resolved the issue.

4mo

"Resolving Startup Issue with Typesense Single Replica Set in Local `K8s` Setup"

Vishal encountered issues starting a Typesense single replica set due to directory permission issues. Kishore Nallan suggested checking if the volume was accessible for writes. Vishal resolved the problem by adjusting permissions on the host.

22mo

Troubleshooting Typesense Snapshot Errors in AWS Docker Container

Arthur experienced recurring Typesense snapshotting errors in an AWS Docker container. Kishore Nallan diagnosed two different issues: a known Google log issue to be fixed in the next release and an unknown file opening error, and guided Arthur on creating GitHub issues for both.

27mo

Addressing High CPU Usage in Typesense

Robert reported high CPU usage on Typesense, even after halting all incoming searches. Kishore Nallan suggested logging heavy queries and increasing thread count. The issue was resolved after Robert found and truncated unusually large documents in the database.

14mo