Hi! Question, why is the total number of hits limited by

per_page

when

k

is not defined, instead of limited by the

distance_threshold

? Using

k

or

per_page

forces me to limit the results of a semantic search manually, unlike keyword search

Because the nearest neighbor search can get unbounded pretty fast without being grounded in number of results. We use HNSW under the hood and that does not provide a way to search only on threshold. K is also needed for the approximate algorithm to prune results at each hop of the graph traversal. So this is inherent to how HNSW is designed