#community-help

Auto Detection Issue with Wildcard Field for Single Array Value

TLDR Aljosa consulted Jason and Kishore Nallan about an issue with auto-detection related to wildcard fields. It was clarified that a string can be coerced into a single-element array. Kishore Nallan confirmed it's a known issue and soon to be addressed on GitHub.

28mo

Join the chat

Oct 01, 2021 (28 months ago)

Photo of md5-6bb7313b20c5179141d6908d6c09b2d5

Aljosa

07:52 PM

👋 I'm using auto detection with a wildcard field for a certain type of data which can't be known beforehand. It works very well except for one specific use case.

I have documents for which 99% of a particular wildcard field will be a string array, but sometimes there is only a single value in the array. It seems like it's not perhaps not possible to have an array field with a single value right now, is that right ?

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

07:54 PM

Hi Aljosa, that's correct. If it is a single value, it still has to be inside a single element array.

07:55

Jason

07:55 PM

But I can see how it would be useful to automatically coerce a single value to a single element array. Could you open a Github issue with this request?

Aljosa

07:58 PM

Hey Jason, ty for the quick answer!

I am actually sending it inside a single value array, so normally you're saying that single value arrays would still be automatically detected as arrays and not be auto detected as just string?

Jason

07:58 PM

Oh yes, that's correct

07:59

Jason

07:59 PM

Or at least that's the desired behavior! If it's not working like that, it's a bug 🙂

Aljosa

08:02 PM

I think the bug is on my side 😄 . Just realized I don't do a pre-check on whether a field may be an array across all documents, so when I get to a document which does not have multiple values, I end up sending it as string.. oops! (and whichever type gets sent first remains the valid type for all documents in the collection)

I'll fix and if it still doesn't work, I'll open a bug 🙂

08:16

Aljosa

08:16 PM

It works perfectly now with my change.

Really love your product ❤️

08:24

Aljosa

08:24 PM

Reading a bit more now about dealing with "dirty data" and the coerce_or_reject behavior.

Does it make sense that it should be possible to coerce a string into a single element array ?

The opposite not being possible makes sense, coercing an array into a string is nonsensical

Jason

08:33 PM

> Does it make sense that it should be possible to coerce a string into a single element array ?
Yup! This is what I thought you meant originally. We don't do this currently, and I think it makes sense to attempt this coercion

Oct 02, 2021 (28 months ago)

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:07 AM

We had a recent issue opened on GitHub with the same suggestion. Going to address it.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Troubleshooting Invalid Field Error in Firestore Document Indexing

Darren receives an error when indexing Firestore documents with empty array in "_grades" field. Jason suggests submitting a bug report and manually setting the schema. The user still experiences issues. Kishore Nallan reproduces the bug, but suggests a solution might exist with an explicit 'string[]' type definition. Further investigation is needed.

25mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

6mo

Discussions on Typesense, Collections, and Dynamic Fields

Tugay shares plans to use Typesense for their SaaS platform and asks about collection sizes and sharding. Jason clarifies Typesense's capabilities and shares a beta feature. They discuss using unique collections per customer and new improvements. Kishore Nallan and Gabe comment on threading and data protection respectively.

35mo

Querying with Typesense-Js and Handling Null Values

michtio was querying using typesense-js and receiving fewer results than expected. Kishore Nallan suggested using different query parameters. Further discussion led to the handling of 'null' values and filtering syntax in the search queries. The thread ended with Jason offering migration support from Algolia to Typesense.

17mo

Resolving Typesense Result Issue in Document Collection Queries

Mike was encountering errors when searching for specific query in their Typesense document collection. Jason suggested it may be due to the `drop_tokens_threshold` setting. There was a misunderstanding but after further explanation from Jason, Mike understood and decided to continue the conversation via email.

22mo