#community-help

Discussing ES Percolation Feature's Applicability to Typesense

TLDR Nick discussed potential benefits of having ES percolation-like functionality in Typesense. Jason acknowledged the importance but confirmed such a feature doesn't exist in Typesense and suggested Nick to open a GitHub issue.

Powered by Struct AI
Jan 19, 2023 (11 months ago)
Nick
Photo of md5-1ffe9f5b775348258bf398610d6a5ac4
Nick
06:54 PM
Hi, we have a custom hosted elasticsearch instance, but are going through a rearchitecture. We have been targeting algolia, but was still looking around due to requiring us to manually implement our notification functionality. We had been working on migrating from a brute force approach to ES percolators. The idea being we save each “search” as a document, then we can ask elasticsearch “here is a new document, which searches match it?“. When we have the searches that match it, our mapping of users with that search tells us who to send a notification to. Our original approach requires us to essentially loop through all relevant searches each time a new piece of content is published to determine which searches match. I didn’t see percolator-like functionality mentioned in the Typesense docs, nor was it called out in the comparison. If this isn’t something supported by typesense, then it might be worth highlighting there
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:03 PM
TIL about the percolation feature in ES! We don’t have this in Typesense.

ES has such a breadth of features that it’s going to be hard to list every single one of them in a comparison matrix, especially because ES doesn’t cover just site/app search but a whole variety of adjacent use cases (like the percolation feature).

Algolia and Typesense on the other hand, we focus just on site/app search. So the features listed in that matrix are from the point of view of the common features you’d need to implement site/app search.
Nick
Photo of md5-1ffe9f5b775348258bf398610d6a5ac4
Nick
07:08 PM
Thanks for taking a look. Yeah, content alerting seems to be a pretty often-available feature for websites (ebay, rvtrader, etc), but there doesn’t seem to be a lot out there right now besides just rolling your own fairly custom functionality. The way we look at it is a site search experience has the natural extension to “save” a search. Then there is the natural extension to get notifications when something matches the search. So, I understand the distinction, but i think the functionality provided by percolators are relevant to site search use cases. I imagine people just handle it through brute force means typically
07:11
Nick
07:11 PM
Just wanted to say that to underline that there is a bit of opportunity there. Even if it was just storing the searches and brute forcing it internally. I’d love to be able to subscribe to a webhook that sends us the search/content when a new piece of content matches an existing search
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:17 PM
Yeah, it’s definitely an interesting feature. I’ve heard about this ask from one other user so far… (They didn’t call it “percolation” though).

Could you open a GitHub issue describing how this feature would work ideally, along with the use-case you described?

We can then gauge demand and see how to prioritize
Nick
Photo of md5-1ffe9f5b775348258bf398610d6a5ac4
Nick
07:22 PM
Yeah, i think more broadly it is referred to as saved searches, content alerts, or on the technical side reverse searches from a functionality standpoint. Percolators is just the elasticsearch name for it. Never seen that used anywhere else