#community-help

Syncing Firebase 'ref' Field with Typesense

TLDR Guillermo wanted to index a 'ref' field from firebase with typesense. Jason provided a solution which involved syncing ref.path as an addition to the existing fields. Guillermo confirmed the solution worked.

14mo

Join the chat

Jun 29, 2022 (15 months ago)

Photo of md5-6f48d9114b22b98994ccbbd9192d9718

Guillermo

01:48 PM

I’ am working with typesense and firebase and I want to index a field type ‘ref’, Is there a way to transform the ref field to string when indexing? I was trying defining it with “string*” but no luck

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

01:51 PM

Guillermo ref is an internal firebase field right?

01:52

Jason

01:52 PM

The extension just goes off of whatever fields Firestore returns as part of this call: https://github.com/typesense/firestore-typesense-search/blob/1b9d22bc1e3fbb55fd548003abbadcbf2be8a3fc/functions/src/utils.js#L32

01:53

Jason

01:53 PM

So if ref isn't returned by Firestore there, then the extension won't be able to sync it

Guillermo

02:02 PM

I see,
else if (value instanceof admin.firestore.DocumentReference) {
return null;
}

Jason

02:03 PM

Ahh, hmmm... May be we should just remove that else if block...

02:03

Jason

02:03 PM

So it returns value from the else block

02:08

Jason

02:08 PM

Could you open a Github issue for this? Happy to accept a PR for this, or I can take a look in a few days.

Guillermo

02:11 PM

Great! I’m on it. We will have to see if then we can index that field to typesense in some way since its basically an object

02:11

Guillermo

02:11 PM

Thanks Jason

Jason

02:13 PM

Oh I see, if it is an object, then may be we should add that type to this else if block which handles objects: https://github.com/typesense/firestore-typesense-search/blob/1b9d22bc1e3fbb55fd548003abbadcbf2be8a3fc/functions/src/utils.js#L16

02:13

Jason

02:13 PM

something like else if ((typeof value === "object" && value !== null) || value instanceof admin.firestore.DocumentReference)

Guillermo

02:17 PM

the documentReference has a property path that returns a string with the path inside the DB, maybe we could work with that

Jason

02:18 PM

So may be we just sync ref.path as a field into Typesense?

Guillermo

02:19 PM

That would work for me at least, I’m not sure if there’s another functionality of the ref field that somebody will need later

02:49

Guillermo

02:49 PM

I created the issue, thanks Jason

Jul 05, 2022 (14 months ago)

Jason

06:45 PM

Following up from this:

I think I misunderstood that you were going to do a PR for it... I'll take take a look later today and get back to you.

Guillermo

06:52 PM

Ahh! I see, sorry I guess I could give it a try!

Jason

06:53 PM

That would be great! Especially if you can test it as well with your project

06:53

Jason

06:53 PM

Here's the developer workflow for the extension: https://github.com/typesense/firestore-typesense-search#development-workflow

Jul 06, 2022 (14 months ago)

Guillermo

01:55 PM

Hi Jason, I can’t seem to be able to create a branch for the issue I created, maybe you have to assign it to me?

Jason

01:56 PM

You would have to fork the repo, and then create a branch in your forked repo

Guillermo

03:48 PM

what params should I be giving the npm run emulator command?

npm run emulator
gives me
error: unknown option '--test-params'

03:52

Guillermo

03:52 PM

With the typesense server I had no problem

Jason

04:06 PM

Hmm, could you run firebase --open-sesame extdev first, and then try running the emulator?

Guillermo

04:25 PM

npm run emulator:

ext:dev:emulators:start is no longer supported. Instead, navigate to a Firebase project directory and add this extension to the extensions manifest by running:

firebase ext:install

04:26

Guillermo

04:26 PM

and when I try firebase ext:install:

i  extensions: ensuring required API <http://firebaseextensions.googleapis.com|firebaseextensions.googleapis.com> is enabled...

Error: HTTP Error: 403, Permission denied to get service [<http://firebaseextensions.googleapis.com|firebaseextensions.googleapis.com>]

Jason

04:37 PM

Hmmm, not sure why that error shows up. Could you just do a PR for whatever you have? I can then take it test it and take it from there

Guillermo

05:27 PM

done, I literally just changed a single line in the code but I cannot test it 😅

06:13

Guillermo

06:13 PM

https://github.com/typesense/firestore-typesense-search/pulls

Jul 07, 2022 (14 months ago)

Jason

02:38 AM

Thank you for the PR Guillermo. I've posted an update on Github

Guillermo

01:35 PM

It works great! You have to name the field ‘fieldname.path’ on the schema.

01:35

Guillermo

01:35 PM

Jason

02:15 PM

Hmmm... In my tests it shows up as ref.path

02:15

Jason

02:15 PM

Is "consumer" the name of your firestore collection?

Guillermo

02:19 PM

yes

02:22

Guillermo

02:22 PM

This is the error on firestore when the typesense field was the same as the firestore collection

02:22

Guillermo

02:22 PM

but naming the schema field consumer.path worked

02:24

Guillermo

02:24 PM

That’s maybe something to add to the documentation.

Jul 12, 2022 (14 months ago)

Jason

03:38 AM

Guillermo May I know what your firestore collection structure looks like that generates a ref key in the Firestore doc? I'm trying to replicate this on my Firestore collection

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2764 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Setup and Issues in Connecting Typesense Instance with Firebase Emulator

Michael.M seeks help on connecting a local Typesense instance to the Firebase local emulator. Jason suggests looking at emulator commands and creating a specific file. After various attempts and troubleshooting steps, Michael.M is able to establish the connection.

16mo

Troubleshooting 400 Error When Upgrading Typesense Firestore Extension

Orion experienced a `400` error after updating the Typesense Firestore extension, causing issues with cloud functions. They traced the issue back to a data type conflict in their Typesense collection schema after updating. With help from Jason and Kishore Nallan, they resolved the issue by recreating the collection.

11mo

Troubleshooting 409 Errors with Firestore to Typesense Cloud Function

Orion encounters 409 errors with `ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite` cloud function. Jason suggests possible solutions like querying Firestore on each change or tracking sync state in a collection. Both agreed on adding a config option. Orion proposed contributing a PR for the change.

7mo

Firestore to Typesense Backfill Issue with Dynamic Paths

Greg experienced issues with Firestore to Typesense backfill not working, and Jason determined it might be related to dynamic paths in Firestore collections not being supported. An RC version of the extension with dynamic path support was considered but needs further review before being shared with Greg.

3mo

Resolving Issues with Firebase Extension and Typesense Configuration

Ernie sought clarification on how to handle the Firebase Extension and typesense collection configuration issues. Jason explained how to correctly specify the path and refresh the page to see updated data. After applying the advice given, Ernie successfully saw the changes.

12mo