#community-help

Disaster Recovery and Backups with Typesense Cloud

TLDR Viji asked about performing backups in Typesense Cloud for disaster recovery purposes. Jason provided guidance on how to backup and restore data, while explaining why Typesense does not provide snapshot functionality like AWS Aurora RDS.

10mo

Join the chat

Dec 12, 2022 (10 months ago)

Photo of md5-d2def4ce72082649c7191218a9e73146

Viji

07:26 PM

We are using Typesense Cloud without High Availability (for cost reasons) and to plan for Disaster Recovery, we would like to take a backup every x hours and store them on our S3 bucket. Has anyone done this? What are the steps involved?

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

07:34 PM

You want to backup the output of the following endpoints:

GET /collections
GET /collections/<collection_name>/documents/export
GET /collections/<collection_name>/synonyms
GET /collections/<collection_name>/overrides
GET /aliases
GET /keys

Viji

07:52 PM

Thank you Jason! So, how would we restore from these backups since you manage the cluster?

Jason

07:53 PM

You would then use the create collections/synonyms/documents/aliases/keys endpoints to add this data back into the cluster if/when needed

Dec 13, 2022 (10 months ago)

Photo of md5-19856b8e92142bdd0747d7a3706736c8

Sergio

10:24 AM

This is because the Snapshot functionality is not for Cloud edition right?

Jason

05:56 PM

We do use the snapshots functionality internally in Typesense Cloud, but we don’t expose those snapshots to you in Typesense Cloud

Dec 22, 2022 (10 months ago)

Viji

07:09 PM

Can you expose these snapshots like AWS does with Aurora RDS?

Dec 23, 2022 (10 months ago)

Jason

04:31 AM

Yeah, we were thinking about doing this… May I know what your use case is for having access to these snapshots?

Viji

12:51 PM

So that we could restore from them for Disaster Recovery purposes.

Jason

04:06 PM

I see, one reason we initially didn’t expose these snapshots is because Typesense is not intended to be a primary datastore - meaning, you have your primary copy of the data in your primary data store and you’re only syncing a copy of it into Typesense. So for disaster recovery, we figured users would just resync their data into Typesense from their primary datastore…

04:06

Jason

04:06 PM

This is unlike RDS, where they are the primary datastore

Viji

04:24 PM

Fair enough. We just have to account for the time to process the data

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Snapshotting and Exporting Data from Typesense Cloud

User Masahiro asked about creating snapshots of Typesense Cloud node. Jason suggested to use the documents/export endpoint. The conversation revealed Masahiro's intent to switch to a self-hosted GCE due to cost and lack of TS Cloud features.

26mo

Issues with Typesense and k8s Snapshot Restoration

Arnob experienced data loss and errors with Typesense in k8s. Kishore Nallan explained corruption could be from premature pod termination. To resolve, Kishore Nallan suggested deleting the data directory on the malfunctioning pod for automatic restoration from the leader.

1mo

Cloning and Backups in Typesense Cloud

Muhamed asked about cloning and backups in Typesense Cloud. Jason clarified cloning methods and confirmed that backups were handled by Typesense Cloud.

10mo

Questions on Backup/Disaster Recovery for Self-Hosted Typesense Cluster

Lane asked questions regarding the backup/disaster recovery for a Typesense cluster. Jason provided detailed answers for each query, guiding the user through the necessary steps and locations for backup and recovery.

11mo

Discussing Features for Typesense Cloud Dashboard

Stephano expressed the need for a Typesense Cloud dashboard to manage collections. Jason confirmed its development, also including features like editing and deleting items. James and Mica added comments about API key management and self-hosted Typesense compatibility.

30mo