#community-help

Resolving Auto-Schema and Configuration Issues in Typesense

TLDR Narayan was struggling with auto-schema and configuration issues in Typesense. With the help of Kishore Nallan, they understood and solved the problems by adding 'optional' to all nested fields. They will find a way to handle 'None'.

2mo

Join the chat

Sep 22, 2023 (2 months ago)

Photo of md5-6441fd8d08a3a44e184f7cba8ddb1d72

Narayan

03:05 AM

Is there a way to disable auto-schema? It introduces additional fields in my base schema for nested objects fields and then expects them to be or array-type. This prevents me to add those objects to typesense. Also, the object type field is also set to not be indexed. If I specify a schema I would expect that auto schema does not kick in😅

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:08 AM

In Typesense a field can have only one type. So the types of children of object fields are stored in the schema and validated. If you define a person field as a type of object and then index an object with a person.name string field, this information is added to the schema so that we can ensure that future person.name values are also string.

Narayan

05:03 AM

Thanks. I re-read the docs and I think I understand now. I ran into some other issues with the data I was importing and the schema expectations.

05:05

Narayan

05:05 AM

Can you help me with how I should configure the schema for this case:

Schema

{
    "name": "Company",
    "enable_nested_fields": True,
    "fields": [
        {
            "name": "id",
            "type": "string"
        },
        {
            "name": "persons",
            "type": "object[]",
            "index": False,
            "optional": True,
        },
        {
            "name": "persons.id",
            "type": "int64[]"
        },
        {
            "name": "persons.addresses.zip",
            "type": "string[]"
        }
    ]
}

When I try to import the following docs typesense complain:

[
    {
        "id": "131415",
        "persons": []
    },
    {
        "id": "789",
        "persons": [
            {
                "id": 1,
                "addresses": [{"zip": None}]
            },
            {
                "id": 2,
                "addresses": [{"zip": "11112"}]
            },
            {
                "id": 3,
                "addresses": [{"zip": "11113"}]
            }
        ]
    },{
        "id": "1011112",
        "persons": [
            {
                "id": 4,
                "addresses": [{"zip": "11111"}]
            },
            {
                "id": 5,
                "addresses": [{"zip": "11112"}]
            },
            {
                "id": 6,
                "addresses": [{"zip": "11115"}]
            }
        ]
    }
]

The errors:

[{'code': 400, 'document': '{"id": "131415", "persons": []}', 'error': 'Field `persons.addresses.zip` has been declared in the schema, but is not found in the document.', 'success': False}]

05:06

Narayan

05:06 AM

I’ve also tried to mark the fields persons.id and persons.addresses.zip as optional but it still complains for the first two docs. Only when I provide all the necessary fields and none of them are None , like in the 3rd case, it works.

Kishore Nallan

05:08 AM

Let me try running that, but is there a reason for defining those child fields?

Narayan

05:10 AM

this is just a toy example. but I need to index these fields for faster search/filtering based on these properties.

Kishore Nallan

05:10 AM

The issue is this empty doc:

 {
        "id": "131415",
        "persons": []
    },

Narayan

05:12 AM

but I’ve marked persons as optional. Should I instead just use {"id":131415} when there are no persons?

05:12

Narayan

05:12 AM

btw, {"id":131415} this too does not work and still raises the same error

Kishore Nallan

05:13 AM

There is also:

"addresses": [{"zip": None}]

Narayan

05:13 AM

yep, how should I configure the schema so that these cases can be still be added to typesense?

Kishore Nallan

05:13 AM

You have defined persons as optional but have explicit child reference which is not marked as optional, so the explicit child field paths get prirority.

05:14

Kishore Nallan

05:14 AM

Add "optional": True, to the child fields

05:15

Kishore Nallan

05:15 AM

And avoid having None

Narayan

05:16 AM

let me give that a try.

05:21

Narayan

05:21 AM

adding optional to all nested fields helped. thanks! I’ll figure out how to deal with None

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Issues with Schema Creation and Nested Fields

Sean encountered a problem with schema creation involving auto nested fields. Kishore Nallan suggested checking the API response for errors and adding problematic fields to the schema as optional. Sean confirmed the advice.

2mo

Trouble in Implementing Deeply Nested Search

Anirudh is struggling to implement a two-level nested search. Jason asked for some specific examples to study the issue. Anirudh provided some material, realizing that adding top fields helped but might over-index. Jason then suggested reporting this issue on GitHub.

Defining Nested JSON Schema for Querying

Greg struggled in defining a schema for querying a JSON object with nested fields. Jason offered solutions, with the final resolution involving the use of "object" as a field type for the contributors in the schema.

15mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

6mo

Troubleshooting Query Error with Optional Nested Objects

Jordan encountered an error while querying optional nested objects. Jason suggested ensuring at least one document has the field or defining fields explicitly in the schema.

9mo