#community-help

Issue with Dynamic Nested Field Type in TS Collection Schema

TLDR Justin encountered an issue with dynamic nested fields not showing as float. Jason suggested reproducing the error with a snippet, while Kishore Nallan was tagged for assistance.

5mo

Join the chat

Apr 11, 2023 (6 months ago)

Photo of md5-cd1242c8fc450a418cdc8d8dd3f044ff

Justin

08:50 PM

hi TS team! i'm hoping for help with a nested field question. essentially, i'm struggling to get a (dynamic) nested field to show up as a float instead of an int64. more details in the thread...

08:50

Justin

08:50 PM

i have documents that have the following numericalSpecifications field, which can contain different keys for different documents (e.g., one document might have an Inside Height key, while another might have Load Capacity):

    'numericalSpecifications': 
      {'Inside Height': {'units': 'inch', 'value': 2.25},
       'Inside Length': {'units': 'inch', 'value': 11.75},
       'Inside Width': {'units': 'inch', 'value': 10.75}
      },

i've defined my TS collection schema as follows:

    "enable_nested_fields": True,
    "fields": [
        ...
        {"name": "numericalSpecifications", "type": "object", "index": True, "facet": True, "optional": True},
        {"name": "numericalSpecifications\..*\.units", "type": "string", "optional": True},
        {"name": "numericalSpecifications\..*\.value", "type": "float", "facet": True, "optional": True},

i then ingested a collection of documents, and when i retrieve the schema, i see things like this:

  {'facet': True,
   'index': True,
   'infix': False,
   'locale': '',
   'name': 'numericalSpecifications.Inside Width.value',
   'optional': True,
   'sort': True,
   'type': 'int64'},

08:51

Justin

08:51 PM

i'm wondering what i'm doing wrong. despite declaring "numericalSpecifications\..*\.value", "type": "float", i see dynamic fields like numericalSpecifications.Inside Width.value assuming type int64

08:52

Justin

08:52 PM

i assume it's a mistake in how i've defined the dynamic nested field schema, but not sure what the correct syntax should be

08:52

Justin

08:52 PM

thanks for reading!

Apr 12, 2023 (6 months ago)

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

05:36 AM

Hmm, that seems like it should work… Could you clone this snippet, change it to your schema and a few sample documents, reproduce the issue and share that snippet here?

Justin

04:56 PM

thanks Jason i appreciate the reply. attached is a repro case

04:57

Justin

04:57 PM

here's what i noticed: if a particular numericalSpecifications.foo.value field looks like an integer, that field will be indexed as an int64, ignoring the {"name": "numericalSpecifications\\..*\\.value", "type": "float", "facet": true, "optional": true} schema directive

04:58

Justin

04:58 PM

in my sample document, i've made a couple of the numerical specs look like floats, and Overall Depth look like an integer. when i retrieve the collection, you can see that they've been indexed with different types

Apr 13, 2023 (6 months ago)

Jason

05:30 AM

Justin Thank you, that looks like a bug. Could you copy-paste that snippet into a GitHub issue? We’ll look into it.

CC: Kishore Nallan

Justin

06:33 AM

sure, i'd be happy to. will follow up with a link to the issue later

04:22

Justin

04:22 PM

issue created: https://github.com/typesense/typesense/issues/973
let me know if i can elaborate with any further details. thanks!

Apr 18, 2023 (5 months ago)

Justin

05:51 PM

thanks Kishore Nallan -- https://github.com/typesense/typesense/issues/973#issuecomment-1512806426

Apr 19, 2023 (5 months ago)

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:18 AM

👍 works?

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Firestore to Typesense Backfill Issue with Dynamic Paths

Greg experienced issues with Firestore to Typesense backfill not working, and Jason determined it might be related to dynamic paths in Firestore collections not being supported. An RC version of the extension with dynamic path support was considered but needs further review before being shared with Greg.

4mo

Defining Nested JSON Schema for Querying

Greg struggled in defining a schema for querying a JSON object with nested fields. Jason offered solutions, with the final resolution involving the use of "object" as a field type for the contributors in the schema.

13mo

Troubleshooting Invalid Field Error in Firestore Document Indexing

Darren receives an error when indexing Firestore documents with empty array in "_grades" field. Jason suggests submitting a bug report and manually setting the schema. The user still experiences issues. Kishore Nallan reproduces the bug, but suggests a solution might exist with an explicit 'string[]' type definition. Further investigation is needed.

22mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

4mo

Troubleshooting 400 Error When Upgrading Typesense Firestore Extension

Orion experienced a `400` error after updating the Typesense Firestore extension, causing issues with cloud functions. They traced the issue back to a data type conflict in their Typesense collection schema after updating. With help from Jason and Kishore Nallan, they resolved the issue by recreating the collection.

12mo