#random

Configuring Typesense Extension for Nested Fields in Firebase Collection

TLDR David had issues indexing nested fields using the Typesense extension. Jason offered advice on specifying schemas and prioritizing specific fields, but there remained unresolved challenges with the schema.

7mo

Join the chat

May 08, 2023 (7 months ago)

Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd

David

05:28 PM

Hey all, I'm trying to install the typesense extension on a firebase collection where each document has some nested fields, but I want to index only a few of these nested fields with each typesense record.

• Data doesn't sync with typesense when I add field1.nestedField1 in the Firestore Collection Fields input when configuring the extension.
• Data does sync with typesense if I only put field1 in the same input, but then my typesense schema appears to auto-update to include every nested field within field1
Should I be able to target nested fields in Firestore Collection Fields with dot notation? If so, should each value that I provide to Firestore Collection Fields correlate with a field item in my typesense schema? And is the schema supposed to auto-update after I connect typesense to my firebase collection?

I'm not finding much information online about how to properly configure the firebase extension to work with nested fields, but I know that's a fairly new feature. Thanks in advance.

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

05:57 PM

David The extension doesn’t allow you to send a subset of fields from a nested object into Typesense. It only allows syncing top-level keys/values fully.

05:58

Jason

05:58 PM

In the Typesense Schema though, you could choose to only index a subset of fields within the nested object. Only those fields will be stored in memory and the other fields will be stored on disk and returned when the document is a hit

David

06:56 PM

Okay, makes sense. So in Firestore Collection Fields I should specify field1, and then in my schema I need to provide info for field1, as well as all relevant nested fields I want to index, like:

    "fields":[
    {
      "name": "field1",
      "type": "object"
    },
    {
      "name": "field1.nested1",
      "type": "object[]"
    },
    {
      "name": "field1.nested1.nested2",
      "type": "string[]"
    },
    {
      "name": "field1.nested1.nested3",
      "type": "object[]"
    },
    {
      "name": "field1.nested1.nested3.nested4",
      "type": "string[]"
    },
    {
      "name": "field1.nested1.nested3.nested5",
      "type": "string[]"
    }
]

06:58

David

06:58 PM

But then when I do connect to firestore and begin syncing data to typesense, my schema in collection settings increases in length by abouce 4x, to include every other nested field in field1 ... You're saying this is also expected?

Jason

07:02 PM

You’d only need to mention the “leaf” level node

07:02

Jason

07:02 PM

So if you only need to index field1.nested1.nested3.nested5, you can skip the other items in your example, from the schema

07:03

Jason

07:03 PM

Adding this is what causes the more specific field definitions to be ignored.

{
      "name": "field1",
      "type": "object"
    },

07:03

Jason

07:03 PM

Essentially the more generic field definitions take precedence if they’re specified, so you want to take those out and leave the most specific one(s)

David

07:04 PM

Oh okay, that's good to know. Thanks for the additional info, much appreciated 🙏

08:01

David

08:01 PM

I'm still not getting any data when I try to index field1.nested1.nested2 which looks like

field1: {
  nested1: [ ... { nested2: <value> } ]
}

and I see

Error: Request failed with HTTP code 404 | Server said: Field field1.nested1.nested2 not found.

in the firebase logs

08:11

David

08:11 PM

may not be supported

Jason

08:23 PM

Oh hmm, I thought we addressed that, looks like it got lost in the mix in that thread. Mind creating a separate issue for it, so we can track it?

David

08:49 PM

https://github.com/typesense/typesense/issues/1006

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3005 threads (79% resolved)

Join Our Community

Similar Threads

Nested Objects Issue in Firebase & Typesense Integration

Shaun encountered issues with nested objects being flattened in Typesense. Jason found the root issue and provided a solution involving updating the Firebase extension to 1.0.3.

6mo

Specific Collection Backfill and Schema Config in Firebase Extension

Loic asked Jason about backfilling specific collections with the Firebase Extension and configuring collection schemas. The latter explained current limitations and gave advices about handling attributes for search/filtering, storage, and return.

20mo

Discussion on Firebase Extension for Typesense

Jason announced the release of a Firebase Extension for Typesense, leading to a discussion with Gabe about capabilities and scalability concerns. Other users, John and A, simply expressed their admiration.

31mo

Firestore to Typesense Backfill Issue with Dynamic Paths

Greg experienced issues with Firestore to Typesense backfill not working, and Jason determined it might be related to dynamic paths in Firestore collections not being supported. An RC version of the extension with dynamic path support was considered but needs further review before being shared with Greg.

6mo

Issues with Schema Creation and Nested Fields

Sean encountered a problem with schema creation involving auto nested fields. Kishore Nallan suggested checking the API response for errors and adding problematic fields to the schema as optional. Sean confirmed the advice.

2mo