#community-help

Discussing Support for Nested Objects in Typesense

TLDR Martin expressed the need for support for nested objects in Typesense. Kishore Nallan mentioned the possibility of flattening objects before indexing and restoring them later. An issue has been created on GitHub to track this.

28mo

Join the chat

Sep 20, 2021 (28 months ago)

Photo of md5-4c44d17869045701f2ca4035972ad4c5

Martin

06:49 AM

Thanks for the update. This functionality is high on my wish list, as many of my records would benefit from support for objects, and I am used to this from Elasticsearch. But of course I can wait, and work around this by flattening the data.

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

06:49 AM

Do you have arrays inside nested objects that you index into ES?

06:50

Kishore Nallan

06:50 AM

Also nested array of objects. Majority of the implementation complexity was dealing with nested array of objects.

Martin

07:33 AM

I work in scholarly publishing, but indexing a blog is similar and maybe more familiar. A typical use case is indexing blog posts with their authors. There can be one or more authors, and each other has givenName, familyName, affiliation, and potentially more.

07:36

Martin

07:36 AM

There could be arrays within nested objects if I want to support e.g. multiple affiliations, and these could be an array of nested objects (e.g. affiliation name, affiliation country, etc.), but the basic use case that covers 80-90% is support for objects and arrays of objects, but no arrays within the objects.

Kishore Nallan

07:36 AM

👍

07:38

Kishore Nallan

07:38 AM

We might have to just flatten the document on Typesense side before indexing and then unflatten it back during display. We might have to store some kinda metadata that allows us to reverse the flattening.

Martin

07:43 AM

A pragmatic approach not supporting all edge cases would work for me.

Kishore Nallan

07:56 AM

Would it possible for you to put a sample object here for reference? For e.g. the one involving multiple authors involving affiliations which itself could be nested.

Photo of md5-88d88db4789daa0e3abef8c3ca27772b

Andrew

08:41 AM

Hi Kishore Nallan Martin we would also really non flat objects. Not having this is causing major problems

Martin

08:49 AM

Kishore Nallan This is an example from an Elasticsearch index, with a nested array of objects (nameIdentifiers), and an array of objects at the top level (subjects, dates, rightsList, descriptions): https://api.datacite.org/application/vnd.datacite.datacite+json/10.6084/m9.figshare.90828

Kishore Nallan

09:45 AM

When you query would you want to query all the keys of a nested structure or would you want to zero in on a set of specific keys? For e.g. would you just want to query creators which is a nested object directly or query as creators.name etc. which refer to primitive types.

Martin

10:53 AM

The latter, so creators.name . This particular query is very common. So queries would be for primitive types. An edge case would be queries for creators.givenName & creators.familyName which Elasticsearch addresses with either nested fields (as you want givenName and familyName to belong to the same creator) or a combined field combining both values. I usually use the latter (this is for an index with more than 20 million records).

Kishore Nallan

11:01 AM

> as you want givenName and familyName to belong to the same creator
Yes, this is one tricky thing to handle when you flatten fields: you don't want to end up matching the first name of one person with last name of another.

Sep 29, 2021 (28 months ago)

Andrew

03:04 PM

Kishore Nallan Martin did a Github issue end up being created for this?

Kishore Nallan

03:05 PM

Yes there is an issue tracking this on GitHub.

Sep 30, 2021 (28 months ago)

Andrew

04:39 PM

can you let me have the ID Kishore Nallan?

Kishore Nallan

04:42 PM

https://github.com/typesense/typesense/issues/227

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Handling Nested Objects in Typesense

Darren queried about storing arrays of objects in Typesense. Kishore Nallan advised that Typesense doesn't support nesting but shared alternative reindexing methods. They added improved nesting support is planned for the future.

24mo

Nested Objects and Arrays in Typesense

Robert seeks advice on managing nested objects in Typesense. Kishore Nallan informs of upcoming support for nested objects and provides a current workaround. Robert indicates interest in tracking this feature. Kishore Nallan provides a link to follow the issue.

18mo

Configuring Typesense Extension for Nested Fields in Firebase Collection

David had issues indexing nested fields using the Typesense extension. Jason offered advice on specifying schemas and prioritizing specific fields, but there remained unresolved challenges with the schema.

7mo

Nested Objects Issue in Firebase & Typesense Integration

Shaun encountered issues with nested objects being flattened in Typesense. Jason found the root issue and provided a solution involving updating the Firebase extension to 1.0.3.

6mo

Nested Field Search with Typesense

Justin asks if searching within object arrays is supported, Jason says it isn't yet. Users discuss Elasticsearch's nested object type as a potential solution, and Justin confirms that field names with spaces work in the current Typesense version.

9mo