Hi, we are trying to find the right `analytics-flu...
# community-help
n
Hi, we are trying to find the right
analytics-flush-interval
for 100+ analytics rules. but how can i judge how close we are to filling in mem buffers given current flush rate?
f
Hey @Nicholas Tjørnelund , There currently isn't a direct “buffer fill %” or memory-bytes metric for analytics, so the best proxy is the pending counts from GET /analytics/status. That endpoint shows how much analytics data is still sitting in memory waiting to be compacted or flushed. The simplest way to judge whether your analytics-flush-interval is reasonable is to watch those numbers over time. If they rise between flushes and then fall back down cleanly after each flush, you’re generally in a healthy range. If they keep growing across flush cycles, or only partially drop after a flush, then the interval is likely too long for your current traffic and number of rules. For example, if you check /analytics/status and see query_log_events: 8,000 and doc_counter_events: 3,000, then a few minutes later those become 16,000 and 6,200, and after a flush they only fall to 12,000 and 4,900, that usually means the system is accumulating analytics faster than it can clear them with the current interval. On the other hand, if those numbers grow during the interval and then drop close to zero after each flush, that’s a good sign that you still have headroom. The main thing to watch most closely is log-style analytics and high-cardinality document counters, since those can keep growing in memory between flushes more than the popular/no-hits query counters do.
n
@Fanis Tharropoulos Also how do i know when a flush has happened or will happen. i only adjust frequency in cluster settings, any way this is possible too?
f
There isn’t currently a dedicated API that tells you exactly when the next analytics flush will happen, or returns the timestamp of the last one. In practice, the two ways to tell are by polling GET
/analytics/status
or by checking the server logs (which isn't available to you on Cloud). If the pending analytics counts rise for a while and then suddenly drop, that’s a good indication that a flush has just happened. One thing to keep in mind is that
analytics-flush-interval
controls how often the background worker is allowed to flush, but the flush itself is handled by a periodic worker loop, so it may happen a little after the exact interval you configured rather than at an exact wall-clock boundary.
👍 1