#community-help

Understanding Cluster Metrics in Typesense

TLDR Sergio requested for more information on Cluster Metrics such as typesense_memory_mapped_bytes and typesense_memory_metadata_bytes. Jason and Kishore Nallan provided details on the intended use and meaning of these metrics.

Powered by Struct AI
Oct 05, 2022 (15 months ago)
Sergio
Photo of md5-19856b8e92142bdd0747d7a3706736c8
Sergio
04:07 PM
Do you have documented details regarding Cluster Metrics? Gathered from /metrics.json?
04:07
Sergio
04:07 PM
For example typesense_memory_mapped_bytes or typesense_memory_metadata_bytes
04:28
Sergio
04:28 PM
And also, understand which is the optimal values for such, or warning ones
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:59 PM
Those are mainly for debugging purposes, so we haven’t yet documented it
07:00
Jason
07:00 PM
You generally want to be looking at typesense_memory_active_bytes (IIRC) for the actual RAM + Swap that typesense is using, and make sure it never exceeds the total system memory
Sergio
Photo of md5-19856b8e92142bdd0747d7a3706736c8
Sergio
07:01 PM
Thought metadata_bytes where related to collection usage
07:02
Sergio
07:02 PM
And called my attention that fragmentation_ratio is not like documented in Redis. Even if apparently you use same logic.
> A fragmentation ratio less than 1.0 means that Redis requires more memory than is available on the system and so it has resorted to using swap memory resources.
https://www.dynatrace.com/news/blog/introducing-redis-server-monitoring/
Oct 06, 2022 (15 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:34 PM
Some of these metrics are jemalloc metrics, see stats.mapped etc. on this page: https://manpages.ubuntu.com/manpages/jammy/man3/jemalloc.3.html

Fragmentation ratio is defined as memory allocated divided by memory active. It reflects the additional memory wasted during allocations by the allocator (jemalloc).

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Memory Leak Issue in Typesense Server v0.21.0

Jim was experiencing a potential memory leak while load testing an API. Jason and Kishore Nallan suggest it may be a known issue in v0.22.0 RC builds which is resolved in later versions. Jim identifies a specific search filter escalating the issue. Kishore Nallan confirms a fix in the 0.22 RC builds.

4

30
27mo

Optimizing Typesense Implementation for Large Collections

Oskar faced performance issues with his document collection in Typesense due to filter additions. Jason suggested trying a newer Typesense build and potentially partitioning the data into country-wise collections. They also discussed reducing network latency with CDN solutions.

5

67
11mo

Addressing Typesense Server Issues and Optimization Needs

Robert had an issue with a 'stuck' typesense server. Jason and Kishore Nallan gave advice on handling writes, configuration for high search volumes, and running multiple typesense instances. They also recommended monitoring CPU usage and updating the server version for bug fixes.

1

30
14mo

Issue Resolution and Upgrade Problems in Typesense Version 0.26rc16

Ankit reported an issue with Typesense, which was addressed by Kishore Nallan and Jason. However, Ankit experienced difficulties while trying to upgrade, with the server status showing as "Not ready or lagging" 503. The resolution remains incomplete.

1

19
2mo

Typesense Server Crash and Counting Issue

Dan reported a server crash and counting issues with Typesense server version 0.26.0.rc1. Jason suggested an upgrade to `0.26.0.rc13` while Kishore Nallan committed to investigate the issue based on a sample dataset from Dan.

11
3mo