Hi guys! I got a couple of questions: 1. When usin...
# community-help
a
Hi guys! I got a couple of questions: 1. When using search I would like to get results in pages bigger than 250 items. At the moment queries like this /collections/companies/documents/search?q=*&filter_by=additionaldata_company_competences:=["Steuerberatung","test"]&include_fields=rootfield__document_uri,additionaldata_company_competences&page=1&per_page=1000 end up with error "Only upto 250 hits can be fetched per page.". Is there a way to get more? 2. I cant find a way to filter the collection by term containing special symbols ("_","-"). The following query collections/companies/documents/search?q=*&filter_by=rootfield__document_uri:="di-r_3"&include_fields=rootfield__document_uri returns items with rootfield__document_uri "dir3","dir_3","di-r_3". How to solve that issue?
r
any reason why you can't fetch the next page?
you could even issue 4 requests at the same time to fetch page 1, 2, 3 and 4.
j
@Anton Khatunzev @Andrew Sittermann Summarizing part of our conversation here: 1. We have a limit of 250 records per page for performance and anti-abuse reasons. I think what you'd find useful is to just fetch all records (
q=*
) matching a filter. We've been thinking about adding the ability to add
filter_by
and
include_fields
to the documents export endpoint, which won't have this pagination limit, since there's no sorting performance overhead involved. In the meantime, fetching multiple pages across multiple requests is the workaround. Like indexer mentioned above, you could actually issue requests for multiple pages in one http requests using the multi_search endpoint. 2. This is a result of us removing all special characters before indexing. So
dir_3
actually gets indexed as
dir3
. We're planning to address this in an upcoming release (not v0.21.0, but the one after).
v
@Jason Bosco documents export with filter_by and include_fields will be very useful.
👍 1
j
Looks like we have 3 asks for this now! Mind creating a Github issue for this, so we can track?
a