Increased Memory Usage After Implementing `pinned_hits` in Typesense
TLDR John experienced a significant increase in memory usage after using pinned_hits
at query time in Typesense. Kishore Nallan suggested that the increase was due to the filtering of the dataset but the issue remains unresolved.
Nov 19, 2022 (11 months ago)
John
04:40 PMpinned_hits
at query time and see rapidly increasing memory usage. First an almost immediate jump from 900 MB to 1.6 GB and then a slow but consistent increase from 1.6 GB to now 3.3 GB since Friday. Is this expected behaviour? Looks a bit concerningJohn
04:41 PM{
"system_cpu1_active_percentage": "0.00",
"system_cpu2_active_percentage": "0.00",
"system_cpu_active_percentage": "0.00",
"system_disk_total_bytes": "105089261568",
"system_disk_used_bytes": "1497489408",
"system_memory_total_bytes": "8344117248",
"system_memory_used_bytes": "4312408064",
"system_network_received_bytes": "71278807601",
"system_network_sent_bytes": "11252234532",
"typesense_memory_active_bytes": "3264450560",
"typesense_memory_allocated_bytes": "3173333928",
"typesense_memory_fragmentation_ratio": "0.03",
"typesense_memory_mapped_bytes": "3366674432",
"typesense_memory_metadata_bytes": "65228800",
"typesense_memory_resident_bytes": "3264450560",
"typesense_memory_retained_bytes": "1692180480"
}
Nov 20, 2022 (11 months ago)
Kishore Nallan
02:27 AMJohn
08:33 AMNov 21, 2022 (11 months ago)
John
10:26 AMIt started happening after we added
pinned_hits
at query time (quite many), together with filter_curated_hits
and some fairly large filters (only get documents where their group ID is not in list)Kishore Nallan
11:16 AMJohn
11:49 AMKishore Nallan
11:53 AMTypesense
Indexed 2786 threads (79% resolved)
Similar Threads
Troubleshooting Stalled Writes in TypeSense Instance
Robert was experiencing typesense instances getting stuck after trying to import documents. Kishore Nallan provided suggestions and added specific logs to diagnose the issue. The two identified queries causing troubles but the issues had not been fully resolved yet.
Slow, High CPU Write Operations After Collection Drop in Typesense
Himank discussed an issue in Typesense where deleting and recreating a collection led to slow write operations and high CPU usage. Kishore Nallan suggested using an alias to avoid this issue. Numerous tests and debugging was conducted as pboros contributed with local testing. Kishore Nallan aimed to start implementing a range delete and full db compaction after deletion to potentially solve the issue.
Addressing High CPU Usage in Typesense
Robert reported high CPU usage on Typesense, even after halting all incoming searches. Kishore Nallan suggested logging heavy queries and increasing thread count. The issue was resolved after Robert found and truncated unusually large documents in the database.
Resolving Typesense Result Issue in Document Collection Queries
Mike was encountering errors when searching for specific query in their Typesense document collection. Jason suggested it may be due to the `drop_tokens_threshold` setting. There was a misunderstanding but after further explanation from Jason, Mike understood and decided to continue the conversation via email.
Memory Leak Issue in Typesense Server v0.21.0
Jim was experiencing a potential memory leak while load testing an API. Jason and Kishore Nallan suggest it may be a known issue in v0.22.0 RC builds which is resolved in later versions. Jim identifies a specific search filter escalating the issue. Kishore Nallan confirms a fix in the 0.22 RC builds.