Defining Nested JSON Schema for Querying

I asked you about this last week but am still confused. You upgraded my instance to .24 release to support nested objects. My JSON object looks like this:

{
...
    "contributors": {
        "contributor1": {
            "bio": "Something about this author",
            "firstName": "Lewis",
            "lastName": "Carroll",
            "type": "author"
        }
        "contributor2": ...
        "contributorX": ...
    }
...
}

It can have an unknown number of contributors. How do I define the schema for that? I am going to have to query by a contributors first & last name, type, and be able to group by the author.

Greg Are you able to structure that data as an array of objects like this, this will make it much easier to query:

{
...
    "contributors": [
        {
            "bio": "Something about this author",
            "firstName": "Lewis",
            "lastName": "Carroll",
            "type": "author"
        },
        { ... },
        { ... }
    ]
...
}

Not without significant refactor and data migration

Haven’t tested this, but could you try something like this:

{
  "name": "collection_name",
  "fields": [
    {"name": "contributors\..*.firstName", "type": "auto" },
    {"name": "contributors\..*.lastName", "type": "auto" },
    {"name": "contributors\..*.type", "type": "auto" },
  ]
}

Ok. Trying it now. Thank you

You would have to index at least one document before those additional fields show up

I did.

Hmm

Let me check my record. I hard coded some records for testing.

Shouldn’t be needed…

This is what the schema looks like:

{
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "contributor..*.firstName",
      "nested": false,
      "optional": true,
      "sort": false,
      "type": "auto"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "contributor..*.lastName",
      "nested": false,
      "optional": true,
      "sort": false,
      "type": "auto"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "contributor..*.type",
      "nested": false,
      "optional": true,
      "sort": false,
      "type": "auto"
    },

This is the record in the Typesense :

contributors
{ "contributor1": { "firstName": "Lewis", "lastName": "Carroll", "type": "Author" }, "contributor2": { "firstName": "Greg", "lastName": "Mascherino", "type": "Illustrator" } }

Btw, forgot to mention this:

{
  "name": "collection_name",
  "enable_nested_fields": true, // <=== this is required
  "fields": [
    {"name": "contributors\..*.firstName", "type": "auto" },
    {"name": "contributors\..*.lastName", "type": "auto" },
    {"name": "contributors\..*.type", "type": "auto" },
  ]
}

That property is not getting saved.

exports.bookContentsSchema = {
  name: 'book-contents',
  enabled_nested_fields: true,
  fields: [
    {
      facet: false,
      name: 'ageDemographicFrom',
      optional: true,
      type: 'int32',
    },
    {
      facet: false,
      name: 'ageDemographicTo',

Result in Typesense clould:

{
  "created_at": 1664488909,
  "default_sorting_field": "",
  "enable_nested_fields": false,
  "fields": [
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "ageDemographicFrom",
      "nested": false,
      "optional": true,
      "sort": true,

There was a type in my earlier snippet. It should be enable_nested_fields not enabled_nested_fields

Sorry. I should have looked closer. My bad. That’s what I get for just copying and pasting.

my bad too!

No luck.

{
    "code": 404,
    "error": "Could not find a field named `contributors..*.firstName` in the schema."
}

That 404 is during a search right?

Got it. That makes sense.

Could you share the full search query you’re using with all the search params, and also the full schema?

Schema from Node:

exports.bookContentsSchema = {
  name: 'book-contents',
  enable_nested_fields: true,
  fields: [
    {
      facet: false,
      name: 'ageDemographicFrom',
      optional: true,
      type: 'int32',
    },
    {
      facet: false,
      name: 'ageDemographicTo',
      optional: true,
      type: 'int32',
    },
    {
      facet: false,
      name: 'bisac1',
      optional: true,
      type: 'string',
    },
    {
      facet: false,
      name: 'bisac2',
      optional: true,
      type: 'string',
    },
    {
      facet: false,
      name: 'bisac3',
      optional: true,
      type: 'string',
    },
    {
      facet: false,
      name: 'chunk',
      optional: false,
      type: 'string',
    },
    {
      facet: false,
      name: 'cfi',
      optional: false,
      type: 'string',
    },
    {
      name: 'contributor..*.firstName',
      type: 'auto',
    },
    {
      name: 'contributor..*.lastName',
      type: 'auto',
    },
    {
      name: 'contributor..*.type',
      type: 'auto',
    },
    {
      facet: false,
      name: 'genres',
      optional: true,
      type: 'string[]',
    },
    {
      facet: true,
      name: 'isbn',
      optional: false,
      type: 'string',
    },
    {
      facet: false,
      name: 'keywords',
      optional: true,
      type: 'string[]',
    },
    {
      facet: false,
      name: 'pageCount',
      optional: true,
      type: 'int32',
    },
    {
      facet: false,
      name: 'physicalCopyLink',
      optional: true,
      type: 'string',
    },
    {
      facet: false,
      name: 'publicationDate',
      optional: true,
      type: 'int64',
    },
    {
      facet: false,
      name: 'published',
      optional: true,
      type: 'bool',
    },
    {
      facet: false,
      name: 'subtitle',
      optional: true,
      type: 'string',
    },
    {
      facet: false,
      name: 'thumbnail',
      optional: true,
      type: 'string',
    },
    {
      facet: false,
      name: 'title',
      optional: false,
      type: 'string',
    },
  ],
};

Actually it might be easier if you can give me permission to access your data on your Typesense Cloud cluster (we require explicit permission before we can access customer data). Would that be ok?

Yes, you can access it. Cluster ID: ujgk3a7dsmvz1l04p

There’s a typo in the schema:

"name": "contributor..*.firstName"

Should be: "name": "contributors..*.firstName"

It is contributor

Actually that shouldn’t make a difference given the regex…

I think that worked

Cool, looks like regex and nested fields still needs work

Thank you for the help!