Using Typesense for In-house Log Search

TLDR Gokay asked if using Typesense for a log search project is feasible. Jason replied it may become expensive due to being in-memory and missing some features but it's possible in some contexts.

Powered by Struct AI
Oct 14, 2022 (14 months ago)
Photo of md5-e8129121b07d8278f8a19f89046a29d4
06:13 PM
Great product!
I have a question for you. I'm trying to setup a log search for my side project. ELK stack is too heavy and overkill for my project and pain in the ass to setup. I'm considering an inhouse solution and I've found typesense. Do you think it'll work for my case? I'll store access logs.
Photo of md5-8813087cccc512313602b6d9f9ece19f
06:15 PM
Typesense is an in-memory search engine, so depending on the volume of your logs, it might become cost prohibitive to store all logs in RAM, since RAM is still more expensive than disk.

Also, Typesense was designed more searching through structured data for sites / app search, so you might find some aggregation features that you’d typically find in ES missing in Typesense.

That said, I do know a few users who use Typesense for log search - but in a specialized context. Their users generate logs in their application, and they allow a portion of those logs (recent ones) to be searched for / filtered by their end users. So it’s still site/app search, but the underlying data their searching is user-generated logs (think audit events, app events, etc)