I have a question, regarding conversational search, if I index 1000 documents, which are rates with fluctuating

total

values, if I ask a question like

Give me the rate with highest total value

, it never returns the right value, there are many higher than the result The reason I'm asking this question here, is there seems to be a limitation across many different RAG solutions which prevents the total dataset from being included in the results, at least without severely impacting performance sample data

{"id":"000xme7XknFWEjneQrLy","city":"Arlington","date":"1726729200","delivery":"76014","description":"daily chassis $50 \nif required\npre-pul...","fsc":"34%","ingate":"APM Terminals - Port Elizabeth","metro":"NYC","oneWayMiles":"1591","outgate":"NYC","perMile":"$2.54","rate":"6038","state":"TX","total":"8091"} <------- much higher rate
{"id":"02W4G73phW4YSbrchImT","city":"Mobile","date":"1714633200","delivery":"36619","description":"Accessorial charges \n  \nstandard chass...","fsc":"37%","ingate":"40' Wando Welch Terminal - WWT N59","metro":"CHS","oneWayMiles":"581","outgate":"CHS","perMile":"$3.41","rate":"2893","state":"AL","total":"3963"} <------- rate return in answer
... many others

question

{
  "question": "Give me the rate with highest total value"
}

response

{
  "conversation": {
    "answer": "The rate with the highest total value is $3963 for the city of Mobile, Alabama.",
    "conversation_history": {
      "conversation": [
        {
          "user": "Give me the rate with highest total value"
        },
        {
          "assistant": "The rate with the highest total value is $3963 for the city of Mobile, Alabama."
        }
      ],
      "id": "d3368b47-73c0-408a-868e-c7207a291f5d",
      "last_updated": 1728516115,
      "ttl": 86400
    },
    "conversation_id": "d3368b47-73c0-408a-868e-c7207a291f5d",
    "query": "Give me the rate with highest total value"
  },