#random

Transforming Firebase Collection Data before Indexing to Typesense

TLDR David wants to transform data before indexing to Typesense. Jason suggests forking the extension and using a Google Cloud function. David successfully implements the solution.

Powered by Struct AI

3

1

May 12, 2023 (7 months ago)
David
Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd
David
12:52 AM
Hello again,

If I'm monitoring a firebase collection using the Typesense extension, is it possible to transform that data at all before it is indexed to Typesense? I want to add some more data to each record so I can filter on those values.

The values I want to add to each record are also the two fields in my Firestore Collection Path - pathOne/{thisValue}/pathTwo/{andThisValue}/pathThree - if I can't add them to a record using a firebase cloud function, can I filter on these values any other way?

Thanks 🙏
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:54 AM
This is not possible unfortunately. So you would have to fork the extension and run your own version of the cloud function provided by the extension

1

May 16, 2023 (7 months ago)
David
Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd
David
03:20 PM
Okay, unless there is some other configuration for filtering our data that we should explore, we may have to pursue this option..

Would it require publishing the forked version of the extension to the firebase extension marketplace? And has it been done before?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:27 PM
You don’t have to publish it. You’d essentially take the code in this file and paste it inside a Google Cloud function inside your own project.

1

04:28
Jason
04:28 PM
> And has it been done before?
Yup, this is not an uncommon scenario.

1

Jun 05, 2023 (6 months ago)
David
Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd
David
06:08 PM
We've done this and everything has gone smoothly so far.

But does this also require a custom backfill function?

When backfilling data, we don't get any errors in the console, but the function runs and never enters the batching loops to write to typesense
06:16
David
06:16 PM
Nvm, think I answered my own question.. We need to modify typesenseDocumentFromSnapshot so the changes apply to both writing and backfilling..

1