Hi guys, sorry for asking before doing my diligence, but I'm moving from Meilisearch to Typesense and would like to know this: On MS, you can ask for info about the matches in the retrieved results (position in the indexed string, length, etc...), is there a similar feature in TS ?

Typesense response JSON will contain the matched tokens. So far we have not had any use case where the position of a match is useful but happy to learn more.

Sure, check out this:

I see. So you use those indices to wrap the text with your own highlighting?

In my particular case, on the UI, they are separate span elements with some events attached (i.e. they are clickable, etc..).

Can you give me a quick example snippet of how the HTML will look?

I could prob. do the same with parsing, but it's easier that way.

Sorry I meant the HTML used for those highlights.

HTML looks like:

Okay, got it. I'm trying to see if there is an easy work around for now. One thing you can do is set the

highlight_start_tag

as

<mark class="foo">

and then attach the event handlers to the

.foo

element?

Yes, I thought of a similar thing as well.

The actual reason for not exposing offsets is because it can get confusing when unicode is involved, especially for non-latin languages. Whether the encoding is UTF-8 or UTF-16 etc.

I actually ran into that issue and asked them (MS) about it but got ignored. So, what you can do is just provide the offsets on the "buffer" of the string.

So took a middle ground of exposing just the matched tokens, as find and replace seemed easier since you then don't have to worry about byte offset vs unicode points etc.

Yeah, that's true.

Typesense uses UTF-8 so we can expose those offsets, but depending on the language and client used, response parsed encoding is not certain. Seemed like a can of worms 🙂

If you provide the offset using the raw buffer as reference, is super easy to split the text (in a browser, using javascript)

Got it. Thanks. I will add to our backlog. I think we should be able to expose it.

Thanks to you Kishore, you've been very kind.

No problem. Sometimes, we think hard about whether a new feature can sufficiently be replaced with another (highlight tags in this case), since each feature adds a bit of bloat and complexity.

Do you plan to add backups to hosted typesearch?

We already have a snapshot end-point that can back up the whole Typesense DB. So it should not be too difficult to add that. How would your backup related workflow look like? Do you for e.g. want to take a snapshot and then shut down the instance and then restart it? Or, do you want to be able to export the data?

Ok, but if the instance dies, for whatever reason, will the snapshot go down with it?

No, it will be stored externally.

What I would like to have is some sort of daily backup/snapshot, and being able to rollback/restore to it.

From a compliance or HA perspective?

You nailed it, compliance.

The wounds from implementing SOC-2 at a previous job is still raw 🙂

Haha

I will be happy to priortize this feature. We just launched a UI for browsing collections and searching on data. Next on the roadmap is self-serve for version upgrades, which is currently not available on the self-serve interface. We will be happy to tackle backups as part of that. Maybe begin with offering snapshots, and then add a way to restore it as well.

That would be super nice!

You're welcome. I've bookmarked this thread and I will ping you when we have snapshots ready! 👋