https://typesense.org logo
Join Slack
Powered by
If we can’t recover a permanently removed collecti...
# community-help
d
If we can’t recover a permanently removed collection, as it seems from the doc. What is the best practice for keeping constant snapshots in typesense cloud? Should we periodically request snapshots and prune older ones? Or are there security measures to avoid accidentally deleting a whole collection like I just did? xD It’s also super weird because all I did was:
Copy code
await TypesenseAdminClient.collections(IndexNames.onde_sites)
    .documents()
    .upsert(indexedSpace, { action: "upsert" });
but I needed to use an API key for some reason.
👀 1
j
Since Typesense is meant to be used as a secondary datastore and not a primary datastore, you'd want to setup backups on your primary datastore. And then if you need to resuscitate a collection in Typesense, you'd just resync from your primary datastore. Restoring the native backups in Typesense takes as long as just resyncing the data via the import API (since we're still rebuilding the in-memory indices in either case). Since it's easier to do the latter, we haven't exposed the backup feature available in Typesense in the Typesense Cloud UI. We do do backups as part of config changes though.
That code snippet shouldn't have deleted the collection. Are you able to replicate it consistently?
d
I see, so for any change in typesense we should be prepared to just reload from the database?
That sounds fine enough 🙂 I tried again my setup after I did just that. but I think it’s difficult from a reliability perspective unless I’m actively doing it xD
as in, we haven’t used our script to load all data into typesense since we first turned it up and we had to fix a few issues with schemas before. If this had been an actual emergency it would have been weird. But i guess it makes sense for us the clients to figure out the best reliable setup for reloading stuff into typesense.
Is there an event log we can have access to to determine at what time this collection was deleted? I really want to root cause whether it was my action or something else. I really don’t have any other setup that touches typesense.
j
Side note: One UX confusion I've seen before - some operation ends up saturating CPU, so the UI is unable to fetch the set of collections and shows a state as though there are no collections in the cluster, when in reality, it's just that the whole cluster is out of CPU and so the list of collections can't be fetched.
Is there an event log we can have access to to determine at what time this collection was deleted?
If it was deleted via the API, then we have a log of it on our side. We can comb through it if you email support with your cluster id and collection name
but I think it’s difficult from a reliability perspective unless I’m actively doing it
Even one time is one too many to cause a collection to be deleted when doing an upsert. Can you share a set of curl commands that replicates the issue at least once in a set of repeated runs?
d
I’ll try to reproduce with the new collection. But I’ll also send a support ticket.
Side note: One UX confusion I’ve seen before - some operation ends up saturating CPU, so the UI is unable to fetch the set of collections and shows a state as though there are no collections in the cluster, when in reality, it’s just that the whole cluster is out of CPU and so the list of collections can’t be fetched.
I’ll include this in the email, but we aren’t yet live and we were able to look at another collection we had in the same cluster
Open in Slack
PreviousNext
Powered by