Hey all! Been working with TS Cloud for an internal knowledge management project and it's been great, however, it's important for our use-case to show context around multiple (sometimes all) highlights. Currently this means sending the entire highlighted field over the wire and handling snippets clientside, which for large docs is an immense amount of data and is slowing everything down. There are a couple issues on GH mentioning this, such as this one. I was wondering if there's been any further discussion, or any robust workarounds found. As the current approach is not sustainable for us.

Typesense is optimized to find the best matched text segment that contains most/all keywords in the query to show as highlight. It will require significant effort to rewire that to handle multiple snippets within a large document without compromising on performance. Splitting a long piece of text into smaller documents and then doing

group_by

on the

document_id

is the best work around at the moment. If there are no paragraphs to split on, maybe just using a ballpark of 200 words per document will be sufficient.