Systemic Deletion of Collection in Typesense on Amazon EC2 Instance
TLDR Tatu's Typesense collections were mysteriously disappearing. After investigating with Kishore Nallan, they discovered it was due to misuse of the PHP library which deleted the whole collection instead of a single record.
May 03, 2021 (28 months ago)
Tatu
11:04 AMAre there any reasons why Typesense would drop a collection by itself? Any idea where I could start diagnosing something like this?
Kishore Nallan
11:06 AMHave you checked that there is enough RAM and Typesense isn't restarting by checking the logs?
Tatu
11:11 AMI have one smaller collection that is also doing this but a bit less often, and four collections that are stable.
I don't see anything relevant in the log, no mentions of a restart. But a cold start from disk should anyways be possible?
Kishore Nallan
11:12 AMKishore Nallan
11:13 AMYes, certainly. What are the contents of the Typesense data directory? Check the date time of the files.
Tatu
11:14 AMTatu
11:15 AMKishore Nallan
11:17 AMTatu
11:19 AMTatu
11:20 AMKishore Nallan
11:22 AMKishore Nallan
11:22 AMTatu
11:22 AMTatu
11:23 AMKishore Nallan
11:24 AMStarting Typesense
occur in the logs? I presume the logs themselves are preserved for all the 4 days of uptime.Kishore Nallan
11:25 AMStarting Typesense
)Tatu
11:26 AMKishore Nallan
11:27 AMTatu
11:28 AMKishore Nallan
11:28 AMTatu
11:29 AMTatu
11:29 AMKishore Nallan
11:31 AMTatu
11:32 AMBut I wouldn't put it past me to find out something on the application side is actually doing the damage.
Tatu
11:33 AMKishore Nallan
11:33 AMTatu
11:35 AMKishore Nallan
11:35 AMTatu
11:36 AMKishore Nallan
11:37 AMKishore Nallan
11:37 AMTatu
11:38 AMTatu
11:39 AMKishore Nallan
11:39 AMMay 14, 2021 (28 months ago)
Kishore Nallan
01:15 PMJun 28, 2021 (27 months ago)
Kishore Nallan
01:20 PMSep 24, 2021 (24 months ago)
Tatu
12:04 PM$client->collections['products']->documents($id)->delete()
, not $client->collections['products']->delete($id)
which I was doing and which coincidentally deletes the whole collection. The syntax "looks" valid, which is why it took me a long time to figure out what's wrong.Maybe a point of improvement to the PHP library would be to throw a warning if a parameter was used with collection deletion, but I can raise an issue about this in the typesense-php repo.
Kishore Nallan
12:05 PMTypesense
Indexed 2764 threads (79% resolved)
Similar Threads
Slow, High CPU Write Operations After Collection Drop in Typesense
Himank discussed an issue in Typesense where deleting and recreating a collection led to slow write operations and high CPU usage. Kishore Nallan suggested using an alias to avoid this issue. Numerous tests and debugging was conducted as pboros contributed with local testing. Kishore Nallan aimed to start implementing a range delete and full db compaction after deletion to potentially solve the issue.



Issues Deleting Collection Using Typesense with Enabled CORS
Roman encountered issues deleting collections using typesense with enabled CORS which isn't listed as a common issue in the documentation. Jason tried to provide possible solutions by asking Roman to upgrade to `0.23.0.rc47` and later suggested to open a Github issue after the problem wasn't resolved.



Trouble with Typesense Memory Usage when Restarting Docker Container
Blend reports increased memory usage when restarting Docker with Typesense, information shared with Jason and Kishore Nallan. Potential data edge case identified as potential cause, although resolution undetermined.



Discussion About Typesense Nodes Not Synchronizing Correctly
Erick experienced an issue where documents weren't updated properly in a Typesense instance running on 3 nodes. Upon requesting debug logs and configs, Jason identified that these nodes weren't part of the same cluster. They couldn't resolve the nodes' failure to connect issue and recommended a fresh installation.


Handling Kinesis Stream Event Batching with Typesense
Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.

