#random

Configuring Typesense Extension for Nested Fields in Firebase Collection

TLDR David had issues indexing nested fields using the Typesense extension. Jason offered advice on specifying schemas and prioritizing specific fields, but there remained unresolved challenges with the schema.

Powered by Struct AI

4

May 08, 2023 (7 months ago)
David
Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd
David
05:28 PM
Hey all, I'm trying to install the typesense extension on a firebase collection where each document has some nested fields, but I want to index only a few of these nested fields with each typesense record.

• Data doesn't sync with typesense when I add field1.nestedField1 in the Firestore Collection Fields input when configuring the extension.
• Data does sync with typesense if I only put field1 in the same input, but then my typesense schema appears to auto-update to include every nested field within field1
Should I be able to target nested fields in Firestore Collection Fields with dot notation? If so, should each value that I provide to Firestore Collection Fields correlate with a field item in my typesense schema? And is the schema supposed to auto-update after I connect typesense to my firebase collection?

I'm not finding much information online about how to properly configure the firebase extension to work with nested fields, but I know that's a fairly new feature. Thanks in advance.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:57 PM
David The extension doesn’t allow you to send a subset of fields from a nested object into Typesense. It only allows syncing top-level keys/values fully.
05:58
Jason
05:58 PM
In the Typesense Schema though, you could choose to only index a subset of fields within the nested object. Only those fields will be stored in memory and the other fields will be stored on disk and returned when the document is a hit
David
Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd
David
06:56 PM
Okay, makes sense. So in Firestore Collection Fields I should specify field1, and then in my schema I need to provide info for field1, as well as all relevant nested fields I want to index, like:
    "fields":[
    {
      "name": "field1",
      "type": "object"
    },
    {
      "name": "field1.nested1",
      "type": "object[]"
    },
    {
      "name": "field1.nested1.nested2",
      "type": "string[]"
    },
    {
      "name": "field1.nested1.nested3",
      "type": "object[]"
    },
    {
      "name": "field1.nested1.nested3.nested4",
      "type": "string[]"
    },
    {
      "name": "field1.nested1.nested3.nested5",
      "type": "string[]"
    }
]
06:58
David
06:58 PM
But then when I do connect to firestore and begin syncing data to typesense, my schema in collection settings increases in length by abouce 4x, to include every other nested field in field1 ... You're saying this is also expected?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:02 PM
You’d only need to mention the “leaf” level node

1

07:02
Jason
07:02 PM
So if you only need to index field1.nested1.nested3.nested5, you can skip the other items in your example, from the schema

1

07:03
Jason
07:03 PM
Adding this is what causes the more specific field definitions to be ignored.

{
      "name": "field1",
      "type": "object"
    },

07:03
Jason
07:03 PM
Essentially the more generic field definitions take precedence if they’re specified, so you want to take those out and leave the most specific one(s)
David
Photo of md5-7621a536ac5e9f5ac6d4713602bef5fd
David
07:04 PM
Oh okay, that's good to know. Thanks for the additional info, much appreciated 🙏

1

08:01
David
08:01 PM
I'm still not getting any data when I try to index field1.nested1.nested2 which looks like
field1: {
  nested1: [ ... { nested2: <value> } ]
}

and I see
Error: Request failed with HTTP code 404 | Server said: Field field1.nested1.nested2 not found.

in the firebase logs
08:11
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
08:23 PM
Oh hmm, I thought we addressed that, looks like it got lost in the mix in that thread. Mind creating a separate issue for it, so we can track it?

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3005 threads (79% resolved)

Join Our Community

Similar Threads