About the conversation API again, we are having some difficulty getting it to work for our use case, like it kind of works but it is not great, I was wondering if you had more documentation about how it actually works? Like how do you manage the interaction between typesense's collection and the LLM? What is the flow of data when the user makes a request?

We take the question, along with all the search parameters you specified (and you want to make sure you're using

query_by: embedding

to do a semantic search so retrieval works properly), fetch the top results for that search query, and then send it to the LLM along with the system prompt

Yes thanks that's super helpful. I had seen that other thread but wanted to make sure I understood the limitations before going that way.

This is a demo we just wrapped up last week: https://conversational-search-pg-essays.typesense.org

@Jason Bosco Is it also possible to add azure-openai. Furthermore, before sending the searchresult to the LLM, is it possible to somehow send it to a reranker (open-source or e.g. cohere) Another major thing is the search itself. Since you mentioned the use query_by "embedding", is it generally possible to also add additional factors, other than just vector search

Super helpful thanks for the demo, it helps anchor the core use case. I think I gravitate more towards the other approach you highlighted where a user actually wants a search but does not necessarily know how to set the correct filters. Or when a follow up search depends on previous results

We actually have a demo that may match your usecase. Instead of human readable responses, the LLM instead formats the correct query to Typesense. Here's a link to the source code and here's the live versions

Really cool thanks!

This demo is limited to just generating queries, and does not keep track of the current convo context, so all subsequent queries won't keep the previous ones in mind

Ok, I'll see what we can do with the previous responses context