I think that having built-in RAG feature in Typesense is really great! BUT. It might be... improved a bit. I am reading documentation: > Typesense currently supports the following LLM platforms: > OpenAI > Cloudflare Workers AI > vLLM Well, in my experience, the platforms supporting /chat/completions API are QUITE compatible. Almost 100% compatible. I have some code working with the following /chat/completions API providers:

api.fireworks.ai
api.omnistack.sh
api.scaleway.ai
api.cerebras.ai
api.deepinfra.com
glhf.chat
api.groq.com
openrouter.ai
api.sambanova.ai
api-inference.huggingface.co
api.together.xyz

(there are much more providers, and there are new providers popping up all the time) And my client code is the same! Well, I have configuration files, per provider, and I found out that one provider didn't support one feature (specifying "seed"), so I changed the corresponding config file, thats it. My point is: Typesense can rely on /chat/completions API And you can just write in Typesense documentation: 1. RAG uses /chat/completions API ... 2. ... the following parameters of /chat/completions should be supported by provider: ... 3. ... we tested Typesense RAG with the following /chat/completions API providers: ... P.S. I really want to test RAG feature, but have no OpenAI / CloudFlare account, and can not run big models locally with vLLM... Would like to use something like this https://deepinfra.com/meta-llama/Llama-3.3-70B-Instruct-Turbo - the most cheap 70B model I found...