#community-help

Understanding Cluster Metrics in Typesense

TLDR Sergio requested for more information on Cluster Metrics such as typesense_memory_mapped_bytes and typesense_memory_metadata_bytes. Jason and Kishore Nallan provided details on the intended use and meaning of these metrics.

15mo

Join the chat

Oct 05, 2022 (15 months ago)

Photo of md5-19856b8e92142bdd0747d7a3706736c8

Sergio

04:07 PM

Do you have documented details regarding Cluster Metrics? Gathered from /metrics.json?

04:07

Sergio

04:07 PM

For example typesense_memory_mapped_bytes or typesense_memory_metadata_bytes

04:28

Sergio

04:28 PM

And also, understand which is the optimal values for such, or warning ones

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

06:59 PM

Those are mainly for debugging purposes, so we haven’t yet documented it

07:00

Jason

07:00 PM

You generally want to be looking at typesense_memory_active_bytes (IIRC) for the actual RAM + Swap that typesense is using, and make sure it never exceeds the total system memory

Sergio

07:01 PM

Thought metadata_bytes where related to collection usage

07:02

Sergio

07:02 PM

And called my attention that fragmentation_ratio is not like documented in Redis. Even if apparently you use same logic.
> A fragmentation ratio less than 1.0 means that Redis requires more memory than is available on the system and so it has resorted to using swap memory resources.
https://www.dynatrace.com/news/blog/introducing-redis-server-monitoring/

Oct 06, 2022 (15 months ago)

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:34 PM

Some of these metrics are jemalloc metrics, see stats.mapped etc. on this page: https://manpages.ubuntu.com/manpages/jammy/man3/jemalloc.3.html

Fragmentation ratio is defined as memory allocated divided by memory active. It reflects the additional memory wasted during allocations by the allocator (jemalloc).

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Memory Leak Issue in Typesense Server v0.21.0

Jim was experiencing a potential memory leak while load testing an API. Jason and Kishore Nallan suggest it may be a known issue in v0.22.0 RC builds which is resolved in later versions. Jim identifies a specific search filter escalating the issue. Kishore Nallan confirms a fix in the 0.22 RC builds.

27mo

Optimizing Typesense Implementation for Large Collections

Oskar faced performance issues with his document collection in Typesense due to filter additions. Jason suggested trying a newer Typesense build and potentially partitioning the data into country-wise collections. They also discussed reducing network latency with CDN solutions.

11mo

Addressing Typesense Server Issues and Optimization Needs

Robert had an issue with a 'stuck' typesense server. Jason and Kishore Nallan gave advice on handling writes, configuration for high search volumes, and running multiple typesense instances. They also recommended monitoring CPU usage and updating the server version for bug fixes.

14mo

Issue Resolution and Upgrade Problems in Typesense Version 0.26rc16

Ankit reported an issue with Typesense, which was addressed by Kishore Nallan and Jason. However, Ankit experienced difficulties while trying to upgrade, with the server status showing as "Not ready or lagging" 503. The resolution remains incomplete.

2mo

Typesense Server Crash and Counting Issue

Dan reported a server crash and counting issues with Typesense server version 0.26.0.rc1. Jason suggested an upgrade to `0.26.0.rc13` while Kishore Nallan committed to investigate the issue based on a sample dataset from Dan.

3mo