is this normal? it takes more than 20 seconds to p...
# community-help
a
is this normal? it takes more than 20 seconds to perform vector search over 4M docs if filters are used. Machine specs: CPU - E7-4830V4 * 4 (112 CPU) RAM - 1TB Typesense version: v28 Query
Copy code
{
  "searches": [
    {
      "q": "SEARCH_QUERY",
      "query_by": "embedding",
      "collection": "COLLECTION_NAME",
      "filter_by": "date:>=-63158400 && date:<=-61516800",
      "exclude_fields": "embedding",
      "vector_query": "embedding:([], distance_threshold:0.40)",
      "per_page": 20
    }
  ]
}
Response
Copy code
{
    "results": [
        {
            "facet_counts": [],
            "found": 20,
            "hits": [SOME DOCS],
            "out_of": 4848421,
            "page": 1,
            "request_params": {
                "collection_name": "COLLECTION NAME",
                "first_q": "SOME QUERY",
                "per_page": 20,
                "q": "SOME QUERY"
            },
            "search_cutoff": false,
            "search_time_ms": 21753
        }
    ]
}
Search time is really high
Copy code
"search_time_ms": 21753
k
You can set it as high as 100. Basically when the filter clause is super sparse, the hnsw graph traversal becomes slow because so many nodes have to be checked to fetch enough results.
a
yes, tried that too. but still facing same delay
k
With flat search cutoff we will completely skip hnsw traversal so the slowdown could be because of filtering itself.
a
the delay happens only when i include a filter with the vector query. otherwise, its super snappy.
k
Try doing only filter without vector query?
a
yes that too works snappy
only when combining both, delays a lot
k
Did you set flat search cutoff value above the number of "found" value in the response?
a
yeah. tried both min and max. and also equivalent to found value.
k
Post the full query without vector query?
a
This is only filter query. this takes atmost 600ms.
Copy code
{
  "searches": [
    {
      "q": "*",
      "collection": "<COLLECTION_NAME>",
      "filter_by": "date:>=-63158400 && date:<=-61516800",
      "exclude_fields": "embedding",
      "per_page": 20
    }
  ]
}
Only vector query. this gives almost instant results, less than 200 ms
Copy code
{
  "searches": [
    {
      "q": "<SEARCH_QUERY>",
      "query_by": "embedding",
      "collection": "<COLLECTION_NAME>",
      "exclude_fields": "embedding",
      "vector_query": "embedding:([], distance_threshold:0.40)",
      "per_page": 20
    }
  ]
}
Same query with combination of both parameters. it takes around 20000 ms.
Copy code
{
  "searches": [
    {
      "q": "<SEARCH_QUERY>",
      "query_by": "embedding",
      "collection": "<COLLECTION_NAME>",
      "filter_by": "date:>=-63158400 && date:<=-61516800",
      "exclude_fields": "embedding",
      "vector_query": "embedding:([], distance_threshold:0.40, flat_search_cutoff:30)",
      "per_page": 20
    }
  ]
}
k
Can you try upgrading to v29 -- we have improved range filtering performance
a
okay. let me try to schedule the upgrade with infra team, test it out and let you know! thanks a lot for your time!! i was really impressed and inspired by the project btw ๐Ÿ‘
๐Ÿ™Œ 1
Hi @Kishore Nallan. Now the filtering works faster for non-numerical fields (example: topic, category and other string[] fields). But for fields like date, its really slow. takes more than 20 seconds.
r
is date an indexed field?
k
Can you also try enabling range index?
a
Obviously the date field is indexed. but havenโ€™t enabled range index. Will try that now.
k
The other thing I will try is to use the
..
operator for the range instead of doing
date:>=-63158400 && date:<=-61516800"
a
does the request remains open until the field alteration is completed? thought it would run in background. yeah also tried that
..
operator too. still facing the delay. the alteration is in process for range index.
k
The request will remain open just to indicate progress.
a
can i exit it? no issues?
k
Yes you can quit it
๐Ÿ‘ 1
a
yeah, that solved the issue!!
๐Ÿ‘ 1
range_index: true
thanks a lot for the timely response!
k
Glad to hear!