Resolving "Not Ready or Lagging" Error and Improving Upsert Performance
TLDR Anton experienced a "Not Ready or Lagging" error when deleting collections and upserting new data. Kishore Nallan identified it was caused by the server lagging behind in writes, and suggested increasing the write lag threshold configuration. Kishore Nallan also mentioned a future build that could improve upsert performance.
Jun 16, 2021 (32 months ago)
Anton
12:39 PM{ \"message\": \"Not Ready or Lagging\"}
when i try to delete my collections. Do you know what may cause this issue?
Kishore Nallan
12:41 PMKishore Nallan
12:41 PM--healthy-read-lag and --healthy-write-lag parameters here: https://typesense.org/docs/0.20.0/guide/configure-typesense.html#using-command-line-argumentsAnton
12:41 PMAnton
12:41 PMAnton
12:43 PMKishore Nallan
12:49 PMforcibly closed error. It indicates that the server has closed the connection with a RST packet, but it can also mean that some form of client / tcp timeout causing a connection drop. Finally (and least likely), check if the server itself crashed.Anton
12:51 PM"message": "Not Ready or Lagging"
}
for several minutes already
Anton
12:52 PMKishore Nallan
12:52 PMAnton
12:52 PMKishore Nallan
12:52 PMAnton
12:52 PMKishore Nallan
12:52 PMAnton
12:56 PMJun 21, 2021 (31 months ago)
Anton
03:47 PMAnton
03:47 PMAnton
03:48 PMAnton
03:48 PMKishore Nallan
03:49 PMI presume this is happening during updates, correct?
Anton
03:52 PMAnton
03:53 PMAnton
03:53 PMAnton
03:53 PMKishore Nallan
04:19 PMSee this snippet in the log line:
last_index index: 29278 ... applying_index: 27909Kishore Nallan
04:19 PMAnton
04:21 PMKishore Nallan
04:21 PM--healthy-read-lag and --healthy-write-lag values to higher values. This way the server won't reject writes.Anton
04:21 PMKishore Nallan
04:21 PMKishore Nallan
04:21 PMKishore Nallan
04:22 PMKishore Nallan
04:23 PMJun 22, 2021 (31 months ago)
Anton
08:07 AM
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Slow, High CPU Write Operations After Collection Drop in Typesense
Himank discussed an issue in Typesense where deleting and recreating a collection led to slow write operations and high CPU usage. Kishore Nallan suggested using an alias to avoid this issue. Numerous tests and debugging was conducted as pboros contributed with local testing. Kishore Nallan aimed to start implementing a range delete and full db compaction after deletion to potentially solve the issue.
Handling Kinesis Stream Event Batching with Typesense
Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.
Large JSONL Documents Import Issue & Resolution
Suraj was having trouble loading large JSONL documents into Typesense server. After several discussions and attempts, it was discovered that the issue was due to data quality. Once the team extracted the data again, the upload process worked smoothly.
4
Resolve Connection Error on Records Upsert
Jainil faced consistent connection errors while upserting records. Jason identified an OOM issue, suggesting a capacity upgrade. Auto-scaling was discussed and the upgrade implementation process, which was in progress, was clarified.
Typesense Server Bulk Import/Upsert Issue Resolved
Adam was confused about the discrepancy between the successful responses and the actual indexed data while working with a custom WP plugin integrating with Typesense. The issue was a bug related to fetching documents in the wrong order, not a Typesense problem.