#community-help

Recommendations for Running Typesense on GCP

TLDR Alexandru asks about best practices for running Typesense on Google Cloud Platform (GCP). Kishore Nallan recommends using the Compute Engine and setting up a cluster if the data isn't fairly static.

22mo

Join the chat

Nov 11, 2021 (22 months ago)

Photo of md5-828e11a7dff2fb061b968a75c8ea1a00

Alexandru

01:12 AM

Are there any recommendations or best practises on how to run Typesense on GCP? Would you recommend to create 3 VM instances running on Compute Engine as a Cluster to achieve HA? I understand all indexing happens in memory so if a node dies, the db/indexing goes along with it.

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

02:06 AM

👋 Alexandru How often you expect your data to change? If it's fairly static, you can even get away with hosting 2 independent instances and rely on background indexing and using alias to swap to the new collection.

Otherwise, yes, clustering is the best approach to have HA. Even though Typesense is in-memory, it can survive restarts because the raw data is kept on disk and is re-indexed on restart.

Alexandru

03:08 AM

I expect the data to grow to ~50M over the next few years and trying to estimate very roughly what would be required on our end.

What service would be best suited in GCP? Some of the data might fall under PII over time and we’d love to self-host it.

Kishore Nallan

05:16 AM

The memory usage will depend on shape of data. You can index what you have and check how memory grows (there is some fixed overhead initially which should not be extrapolated).

Alexandru

01:47 PM

I guess Compute Engine is the only option in GCP? Can you establish a typesense cluster with google cloud run?

Kishore Nallan

01:48 PM

Yes, you have to use Compute Engine. set up 3 different instances and cluster them.

01:48

Kishore Nallan

01:48 PM

https://typesense.org/docs/guide/high-availability.html

Alexandru

01:53 PM

Thanks so much for the help!

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2764 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Deploying Typesense on Google Cloud Platform (GCP)

Stephane asked for advice on deploying Typesense on GCP and encountered an issue with persistent storage. Jason suggested using Linux VMs on GCP instead, which Stephane thanked them for and found a solution in the documentation.

12mo

Typesense Performance in Cloud Run

Bill asked about the performance of Typesense in Cloud Run. CaptainCodeman and Kishore Nallan expressed doubts about this setup, while Thomas proposed using flexsearch and AWS Lambdas as a suitable alternative.

19mo

Discussing Multi-tenant Indexing and Typesense Clusters

Semyon enquired about search and indexing performance for multi-tenant indexing. Virginio suggested using Typesense cloud, but Semyon noted the cost difference against bare metal instances. Jason proposed vertical scaling before multi-cluster setup.

2mo

Deploying Typesense on Google Cloud Run

Nicolas attempted to deploy Typesense on Google Cloud Run but encountered problems. CaptainCodeman advised that Cloud Run is unsuitable for Typesense and suggested using a container optimized VM.

11mo

Configuring Typesense HA Cluster on AWS

Agustin was looking for best practices to run a Typesense HA cluster on AWS, particularly regarding horizontal scaling. Kishore Nallan recommended using standalone instances or an ASG for static data, rather than clustering.

27mo