#community-help

Discussing Typesense Search Highlighting Capabilities

TLDR Jack enquiries about getting highlight data to include all fields in an object on Typesense. Jason clarifies that only specific fields in 'query_by' will be returned, which resolves the issue for Jack.

Powered by Struct AI

3

13
2mo
Solved
Join the chat
Sep 26, 2023 (2 months ago)
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
03:25 PM
Is it possible to get highlight data to include all fields in an object? I thought I saw Typesense do it at some point, but it doesn't seem to be doing it now. For example, with this field:
"tags": [
  { "key": "TagKey1", "value": "TagValue1" },
  { "key": "TagKey2", "value": "TagValue2" }
]

Searching for TagValue1 gives me this highlight data:
"highlight": {
    "tags": [
        {
            "value": {
                "matched_tokens": [ "TagValue1" ],
                "snippet": "<mark>TagValue1</mark>"
            }
        },
        {
            "value": {
                "matched_tokens": [],
                "snippet": "TagValue2"
            }
        }
    ]
}

Is there a way to get the key for this matching field too? I can extract it from the document by using the index into the tags array, but I thought I had key exposed at some point in the highlight data.
03:26
Jack
03:26 PM
Here's the schema I am using for this:
{
    "name": "things",
    "enable_nested_fields": true,
    "fields": [
        {
            "name": "tags.value",
            "type": "string[]",
            "optional": true
        }
    ]
}
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
03:33 PM
We didn't include unmatched fields/values in arrays of objects to avoid redundant data and increase response sizes.

So the access pattern typically is, you'd first check highlight.field for a display value and fall back to document.field if it's not present
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
03:34 PM
I must have been seeing things then 😄 cheers. I'll get the associated key field (non-indexed) from the document then (using the positional index of the tag in the highlight data).

1

Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
03:34 PM
(Side note: we had to include all objects, even if they don't have a match, inside an array of objects though, to preserve the array index)
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
03:34 PM
But just the indexed fields, right?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
03:34 PM
Correct

1

Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
03:35 PM
Perfect, thanks for the help 😄
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
03:35 PM
Oh wait, it's technically the fields in query_by which could be a subset of the indexed fields
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
03:35 PM
Oh :thinking_face: changing tags.value in my query_by to just tags makes it return the key! Is this expected (even though key isn't indexed) or should I not depend on this behaviour?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
03:36 PM
That is indeed expected behavior. Basically highlights will return fields in query_by
03:37
Jason
03:37 PM
Although, need to check if we have a test case for this!
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
03:37 PM
Ah fantastic - I'm glad I'm not going crazy 😁 I must have changed tags to tags.value at some point in my query_by. The only indexed field in tags is value anyway, so I will switch it back to tags. Appreciated!

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Phrase Search Relevancy and Weights Fix

Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.

6

111
8mo
Solved

Query on "weighted_score" & Issue with Synonym Highlighting

Stefan asked about "weighted_score" field and reported a possible synonym highlighting issue. Kishore Nallan clarified the use of "weighted_score". The possible synonym issue is still being investigated.

17
19mo

Querying and Indexing Multiple Elements Issues

Krish queried fields with multiple elements, which Kishore Nallan suggested checking `drop_tokens_threshold`. Krish wished to force OR mode for token, but Kishore Nallan admitted the feature was missing. Krish was able to resolve the issue with url encoding.

34
12mo
Solved

Troubleshooting Typo Highlighting in Search Queries

Stefan queried "chews" and "Roche", despite having a typo distance of 4, was highlighted. Kishore Nallan requested to test on v0.20 RC. Random results like "Sachets" and "Lachesca" were also highlighted. On single record query, highlighting works. Kishore Nallan assured to address this issue in the pending release.

1

25
33mo

Issues with Typesense Search and JSON Formatting

Babin had issues with not getting all fields in a search. The problem was due to a JSON formatting extension in VS Code. Jason also clarified the requirement of `query_by` in multi-search POST method.

4

17
9mo
Solved