https://typesense.org logo
Join Slack
Powered by
Hey folks, I'm trying to get to grips with the con...
# community-help
e
Hey folks, I'm trying to get to grips with the conversational search (RAG) functionality. I've got my own collection set up, got the conversation history thing set up, the model does appear to work (i.e. I can get a generated response that makes sense based on what I ask), however if I don't have any errors in my query / collection, I always get an empty answer. That is to say, I can only seem to get an actual text, human-readable response from the LLM if I have some kind of error in my request, like in my example screenshot. I've also shown what happens if I don't have any errors in the request. Can anybody help me understand why please? My understanding was that it should use the details in my data to build the text response, is that wrong?
j
It looks like the LLM is using its past training data to respond, if no results are found from your collection
You will have to explicitly instruct it to not use past data and only use the provided context.
For eg, see the system prompt in the example code snippet here: https://typesense.org/docs/27.0/api/conversational-search-rag.html#create-a-conversation-model
Could you make sure you're using v27?
Could you also share your conversation model configuration JSON, collection schema and also a curl request of your search query with all the parameters you're using?
That will help debug why a 404 is returned
e
Hey Jason, just gathering some of this info now. Sorry this has taken ages, I sent this and then promptly went on holiday 😅
Here’s the way I’m creating the model:
Copy code
### Create conversation model
POST <http://localhost:8108/conversations/models>
Content-Type: application/json
X-TYPESENSE-API-KEY: xyz

{
    "id": "conv-model-1",
    "model_name": "cloudflare/@cf/meta/llama-2-7b-chat-fp16",
    "history_collection": "conversation_store",
    "api_key": "<snip>",
    "account_id": "<snip>",
    "system_prompt": "You are an assistant for question-answering. You can only make conversations based on the provided context. If a response cannot be formed strictly using the provided context, politely say you do not have knowledge about that topic.",
    "max_bytes": 16384
}
Version is 27:
Copy code
$ typesense-server --help
Typesense 27.0
Invalid configuration: Data directory is not specified.
Command line usage: ./typesense-server --data-dir=string --api-key=string [options] ...
Running it locally on my Mac as a homebrew service currently
Collection for the data:
Copy code
### Create the deals collection in local Typesense
POST <http://localhost:8108/collections>
Content-Type: application/json
X-TYPESENSE-API-KEY: xyz

{
    "name": "deals",
    "fields": [
        {"name": "brandId", "type": "int64"},
        {"name": "advertiserName", "type": "string"},
        {"name": "brandLogoUrl", "type": "string" },
        {"name": "searchDestinationId", "type":  "int64" },
        {"name": "searchDestinationName", "type":  "string" },
        {"name": "ttiCode", "type":  "int64" },
        {"name": "hotelName", "type":  "string" },
        {"name": "departureAirportIata", "type":  "string" },
        {"name": "departureAirportName", "type":  "string" },
        {"name": "searchIatas", "type":  "string[]" },
        {"name": "departureDate", "type":  "string" },
        {"name": "duration", "type":  "int64" },
        {"name": "boardBasisCode", "type":  "string" },
        {"name": "boardBasisName", "type":  "string" },
        {"name": "pricePence", "type":  "int64" },
        {"name": "totalPricePence", "type":  "int64" },
        {"name": "usps", "type":  "string[]" },
        {"name": "adults", "type":  "int64" },
        {"name": "children", "type":  "int64" },
        {"name": "infants", "type":  "int64" },
        {"name": "childAges", "type":  "int64[]" },
        {"name": "deepLink", "type":  "string" },
        {"name": "deepLinkUrl", "type":  "string" },
        {"name": "offerScore", "type":  "int64" },
        {
            "name": "embedding",
            "type": "float[]",
            "embed": {
                "from": [
                    "advertiserName",
                    "searchDestinationName",
                    "hotelName",
                    "departureAirportName",
                    "departureDate",
                    "boardBasisName",
                    "usps"
                ],
                "model_config": {
                    "model_name": "ts/all-MiniLM-L12-v2"
                }
            }
        }
    ]
}
A request for a search:
Copy code
### Converse...
POST <http://localhost:8108/multi_search?q=where+could+I+go+on+holiday+for+less+then+>£600?&conversation=true&
    conversation_model_id=conv-model-1
Content-Type: application/json
X-TYPESENSE-API-KEY: xyz

{
    "searches": [
        {
            "collection": "deals",
            "query_by": "embedding",
            "exclude_fields": "embedding"
        }
    ]
}
Which does yield results, but no response from the AI agent:
Copy code
{
    "conversation": {
        "answer": "",
        "conversation_history": [
            {
                "user": "where could I go on holiday for less then £600?"
            },
            {
                "assistant": ""
            }
        ],
        "conversation_id": "c972456d-e581-442c-9ef6-48bb905368d1",
        "query": "where could I go on holiday for less then £600?"
    },
    "results": [
        {
            "facet_counts": [],
            "found": 10,
            "hits": [
                {
Part of me was wondering if there’s too much data in the original documents or something, but I’m not really sure
It’s basically as much information as we need for it to be able to be useful really
I think I’ve got this one working now. I’ve changed to use a different Cloudflare model and I’m now getting a response!
👍 1
Open in Slack
PreviousNext
Powered by