#community-help

Optimizing Database Search Using k Value and flat_search_cutoff

TLDR Koushik wanted to optimize a brute force search in a database using k value and flat_search_cutoff and also needed help with "filter_by" parameter. Kishore Nallan provided guidance and directed Koushik to relevant documentation.

Powered by Struct AI
Jul 28, 2023 (4 months ago)
Koushik
Photo of md5-237fc2ec43c199f75556faf6aa5d6a0b
Koushik
11:19 AM
hi, while doing brute force search on our created type sense database, found some k value and flat_search_cutoff could help us to optimise our search. Could anyone help me to get a detail idea how we can use both the params ?
ref ::
{"searches":[{"q":"*", "filter_by": "category:shoes", "vector_query": "vec:([0.96826, 0.94, 0.39557, 0.306488], k:100, flat_search_cutoff: 20)" }]}'

Another point is how I could supply multiple combination in "filter_by",
like "filter_by": "category:shoes, meterial:leather,make:india"
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:21 AM
That query looks fine to me. What error are you getting?
Koushik
Photo of md5-237fc2ec43c199f75556faf6aa5d6a0b
Koushik
11:25 AM
need to understand how I could use different values for k and flat_search_cutoff.
I understood flat_search_cutoff would be the number of expected items. but how k value associates with the query ?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:26 AM
K is max number of results you want to get.
Koushik
Photo of md5-237fc2ec43c199f75556faf6aa5d6a0b
Koushik
11:27 AM
ok , then flat_search_cutoff. is not it aslo pointing to max number ?
11:30
Koushik
11:30 AM
Another point is how I could supply multiple combination in "filter_by",
like "filter_by": "category:shoes, meterial:leather,make:india"
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:30 AM
We have explained how flat search cutoff works in our docs here: https://typesense.org/docs/0.24.1/api/vector-search.html#brute-force-searching
11:30
Kishore Nallan
11:30 AM
Check filter by documentation in the docs. There are examples.
Koushik
Photo of md5-237fc2ec43c199f75556faf6aa5d6a0b
Koushik
11:36 AM
ok but I only can see this, where no multiple "filter_by" example provided. Can you help me on this ?
Image 1 for ok but I only can see this, where no multiple "filter_by" example provided. Can you help me on this ?
Koushik
Photo of md5-237fc2ec43c199f75556faf6aa5d6a0b
Koushik
12:35 PM
ok thanks ... its working
12:36
Koushik
12:36 PM
But looks k value and flat search cutoff not working.
even if I am giving 5 for both query returning 10 documents.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
12:37 PM
Try setting per_page param lower. If that's higher that takes precedence.
12:37
Kishore Nallan
12:37 PM
Default is 10
12:37
Kishore Nallan
12:37 PM
We've fixed this in recent builds.
Koushik
Photo of md5-237fc2ec43c199f75556faf6aa5d6a0b
Koushik
12:38 PM
ok ... shall try both ... thank you very much/

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community