Hi, just out of curiosity, why is there no cursor pagination in Typesense?

Much harder to pull off because cursor requires having a field whose value is static. In search we have to compute a text match score based on overlap of query tokens with the record. So there is no static score within a record to base the cursor on. Also, it requires having some kind of snapshot view of the data to retain consistency across pagination.

Thanks. That makes perfect sense but why is the usage of snapshot or cache not desirable in this situation?

We could technically cache the result ids on which pagination happens, but the on-disk data which actually stores the content could change due to updates between pagination. So the results might look odd if a keyword on a record that's fetched was actually removed in an interim update.

If we cache complete records instead of just IDs, would it be a waste of resources as people typically don't go beyond the first page?

Yes correct and actual data will be too large to store in memory anyway. May I know what problem you are facing?

I am using Firestore as the primary database but it’s not possible to do a full-text search or expensive querying in Firestore. Typesense seems like a perfect solution for the full text search but expensive querying would require some kind of caching solution.

What do you mean by expensive querying?

A geoquery combined with multiple range filters on different fields

We've work planned to improve range filter performance for next release.

The offset pagination is more expensive to run especially after the user goes past multiple pages

Typesense as a search engine is optimised for providing top results from a large set. Deep pagination is a problem for search. Even Google limits to top 10-20 pages for the same reason.

Referring to your answer that the actual data will be too large to store in memory anyway - what if we cache records without the indexed text?

So that we can discuss specifics can you please DM an actual heavy query and what response time you see?

I haven’t implemented the query yet. To be honest I am so used to Firestore that I am just unnecessary pre-optimizing and wondering how would this query behave at scale