Hey <!channel>, I'm excited to announce the availability of our new Firebase Extension for Typesense: https://github.com/typesense/firestore-typesense-search It syncs data from your Firestore database to Typesense automatically on changes. If you're a Firebase user, I'd love to get your feedback.

Nice! I probably won't use this because we don't sync our exact firestore data to typesense, but do some processing on it beforehand, but cool to see nonetheless. A couple of questions: • Because Firestore is an object store that allows having nested data, how do you handle users wanting to search nested data? IIRC typsense does not support this (yet) • How does this scale...because firestore/firebase functions are "infinitely" scalable & typesense is not, I forsee issues arising from too much attempted indexing activity at once.

Great questions!

Because Firestore is an object store that allows having nested data, how do you handle users wanting to search nested data? IIRC typsense does not support this (yet)

The current MVP version does not handle nested data in Firestore, because Typesense server doesn't support it yet. We've been discussing adding support, especially for Firebase users, based on demand for this...

How does this scale...because firestore/firebase functions are "infinitely" scalable & typesense is not, I forsee issues arising from too much attempted indexing activity at once.

Scalability would be a function of the Typesense cluster configuration. High-throughput reads & writes would require larger number of CPU cores to handle the load. Another thing I've been thinking about is may be adding a buffering mechanism on the Firestore side that can accumulate changes and bulk insert changes into Typesense, which should help accommodate higher throughput writes on lower number of CPU cores as well. Question for you: is the pre-processing you're doing only to handle nested fields, or do you do additional transformations?

nested fields + attaching userIds for filtering that aren't present in the firestore documents.

Ooh interesting, thanks for that

essentially. We actually have an intermediate firestore collection that contains this data so we can make use of atomicity in firestore & then we propagate the data in it to typesense using something similar to your extension with a filter that only does so if at least one user is associated with the document.

So if Typesense server supported nested fields, you'd still need a way to specify a filter on the Firestore collection, to indicate what subset of those records are synced to Typesense via the extension?

This is huge!!! Congratulations!!!

@Jason Bosco yup, but we may have a unique set up. I wouldn't go solving for our use case unless you see others with the same needs.

This is amazing.