#community-help

Struggles with Parsing Highlight Data in Go Library

TLDR Jack expressed difficulty parsing new highlight data in Go library, considering creating own API. Kishore Nallan and Jason discussed related problems and potential solutions including using Typescript generics. No solution decided upon.

2mo

Join the chat

Sep 21, 2023 (2 months ago)

Photo of md5-763597f84b704376f42cabfa33614ae8

Jack

01:26 PM

The type for the new highlight data is missing in the official Go library and it's somewhat difficult to parse (it ends up being a map[string]interface{} which can contain more map[string]interface{}s or []interface{}s), has anyone worked around this? I'm currently considering just calling the API directly and building my own thin wrapper around it.

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:32 PM

Yeah the new highlight structure overlays on the original field structure which means that nested fields can have an arbitrary number of levels.

Jack

01:32 PM

Yeah the new highlight type seems like it'd be hard to represent in Go as-is due to the fact you can have object fields.

Kishore Nallan

01:33 PM

Wondering if there is a way to maybe make the client accept some type for desererialization

Jack

01:35 PM

It seems like you can end up with any of the following for a highlight:

{ "matched_tokens": [], ... }```

[
{ "matched_tokens": [], ... },
{ "matched_tokens": [], ... },
...
]

{
"key1": { "matched_tokens": [], ... },
"key2": { "matched_tokens": [], ... },
...
}```

01:35

Jack

01:35 PM

Oh you can have []object too :thinking_face: certainly makes it hard to represent in OpenAPI or in most languages I'd imagine.

Kishore Nallan

01:38 PM

Unfortunately what's easy for dynamically typed languages like JS is doubly difficult in typed. This was very convenient for JS.

Jack

01:38 PM

I guess when deserialising I could extensively list all of the fields and their types, not ideal though as it'd be nice if there was some generic representation for highlight data.

Kishore Nallan

01:38 PM

People wanted to be able to access the nested structure directly, which is why we ended up with this approach.

Jack

01:38 PM

Yeah it makes sense for JS, much more convenient

Sep 22, 2023 (2 months ago)

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

03:47 PM

Btw, in Typescript, there's a concept of generics where we can allow a user to define the types for their docs and pass that into the Typesense Typescript library and that gets used internally to also represent the highlight structure, by just replacing the leaf matched field to the highlight structure (not sure if we've implemented this yet, but it should be theoretically possible. But we've implemented this pattern to represent the hits[].document field for eg)

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

JavaScript Client's Return of Highlights Inquiry

Daniel questioned why the JavaScript client returned highlights in an array instead of an object. Kishore Nallan explained it was due to specific issues with statically typed languages needing defined JSON structures for parsing.

32mo

Integration Issues with Highlights in Typesense

Michael was struggling with integrating highlights into their work. Dima and Jason offered clarifications and suggestions. Michael decided to try Jason's solution.

4mo

Transitioning from Meilisearch to Typesense - Questions and Suggestions

Al is moving from Meilisearch to Typesense and asked for similar matching information features. They also proposed adding daily backups. Kishore Nallan helped them find a workaround, while noting expected complexities, and agreed to include their suggestions in their backlog.

33mo

Discussing Typesense Search Highlighting Capabilities

Jack enquiries about getting highlight data to include all fields in an object on Typesense. Jason clarifies that only specific fields in 'query_by' will be returned, which resolves the issue for Jack.

2mo

Understanding and Implementing Typesense Dart Library with Flutter

Alexandro sought help with the Typesense Dart library. Jason explained that the library is in progress, discussed utilizing other HTTP libraries, and provided detailed instructions on utilizing Typesense with Flutter. Alexandro provided feedback on the Typesense UI and expressed interest in creating a tutorial video.

32mo