#community-help

Discussing Typesense Search Highlighting Capabilities

TLDR Jack enquiries about getting highlight data to include all fields in an object on Typesense. Jason clarifies that only specific fields in 'query_by' will be returned, which resolves the issue for Jack.

2mo

Join the chat

Sep 26, 2023 (2 months ago)

Photo of md5-763597f84b704376f42cabfa33614ae8

Jack

03:25 PM

Is it possible to get highlight data to include all fields in an object? I thought I saw Typesense do it at some point, but it doesn't seem to be doing it now. For example, with this field:

"tags": [
  { "key": "TagKey1", "value": "TagValue1" },
  { "key": "TagKey2", "value": "TagValue2" }
]

Searching for TagValue1 gives me this highlight data:

"highlight": {
    "tags": [
        {
            "value": {
                "matched_tokens": [ "TagValue1" ],
                "snippet": "<mark>TagValue1</mark>"
            }
        },
        {
            "value": {
                "matched_tokens": [],
                "snippet": "TagValue2"
            }
        }
    ]
}

Is there a way to get the key for this matching field too? I can extract it from the document by using the index into the tags array, but I thought I had key exposed at some point in the highlight data.

03:26

Jack

03:26 PM

Here's the schema I am using for this:

{
    "name": "things",
    "enable_nested_fields": true,
    "fields": [
        {
            "name": "tags.value",
            "type": "string[]",
            "optional": true
        }
    ]
}

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

03:33 PM

We didn't include unmatched fields/values in arrays of objects to avoid redundant data and increase response sizes.

So the access pattern typically is, you'd first check highlight.field for a display value and fall back to document.field if it's not present

Jack

03:34 PM

I must have been seeing things then 😄 cheers. I'll get the associated key field (non-indexed) from the document then (using the positional index of the tag in the highlight data).

Jason

03:34 PM

(Side note: we had to include all objects, even if they don't have a match, inside an array of objects though, to preserve the array index)

Jack

03:34 PM

But just the indexed fields, right?

Jason

03:34 PM

Correct

Jack

03:35 PM

Perfect, thanks for the help 😄

Jason

03:35 PM

Oh wait, it's technically the fields in query_by which could be a subset of the indexed fields

Jack

03:35 PM

Oh :thinking_face: changing tags.value in my query_by to just tags makes it return the key! Is this expected (even though key isn't indexed) or should I not depend on this behaviour?

Jason

03:36 PM

That is indeed expected behavior. Basically highlights will return fields in query_by

03:37

Jason

03:37 PM

Although, need to check if we have a test case for this!

Jack

03:37 PM

Ah fantastic - I'm glad I'm not going crazy 😁 I must have changed tags to tags.value at some point in my query_by. The only indexed field in tags is value anyway, so I will switch it back to tags. Appreciated!

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Phrase Search Relevancy and Weights Fix

Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.

111

8mo

Query on "weighted_score" & Issue with Synonym Highlighting

Stefan asked about "weighted_score" field and reported a possible synonym highlighting issue. Kishore Nallan clarified the use of "weighted_score". The possible synonym issue is still being investigated.

19mo

Querying and Indexing Multiple Elements Issues

Krish queried fields with multiple elements, which Kishore Nallan suggested checking `drop_tokens_threshold`. Krish wished to force OR mode for token, but Kishore Nallan admitted the feature was missing. Krish was able to resolve the issue with url encoding.

12mo

Troubleshooting Typo Highlighting in Search Queries

Stefan queried "chews" and "Roche", despite having a typo distance of 4, was highlighted. Kishore Nallan requested to test on v0.20 RC. Random results like "Sachets" and "Lachesca" were also highlighted. On single record query, highlighting works. Kishore Nallan assured to address this issue in the pending release.

33mo

Issues with Typesense Search and JSON Formatting

Babin had issues with not getting all fields in a search. The problem was due to a JSON formatting extension in VS Code. Jason also clarified the requirement of `query_by` in multi-search POST method.

9mo