#community-help

Multiple Filters and JSON Requests in Typesense

TLDR Manish asked about multiple filter_by arguments, JSON input, and using multisearch. Jason offered typesense documentation links, examples, and how to use JSON formatted requests with multisearch. Ed shared a similar use case.

Powered by Struct AI

5

1

Jun 16, 2023 (6 months ago)
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
05:43 PM
is there a way to supply multiple filter_by arguments? I'm looking at https://typesense.org/docs/0.24.1/api/search.html#filter-parameters , but there isn't any example of that.
05:43
Manish
05:43 PM
Also, if all of the filter_bys are done via url params, then how do you run "&&", etc., mentioned elsewhere?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:45 PM
You can use any complex expressions in filter_by. Eg:

filter_by: field1:=a && field2:=b
05:46
Jason
05:46 PM
You want urlencode the value of the filter_by parametr
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
05:46 PM
I see. let me try that. It would be good to have an example of this in the docs.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:47 PM
It’s documented under this table: https://typesense.org/docs/0.24.1/api/search.html#filter-parameters

Under “Multiple Conditions”
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
05:51 PM
I saw that. But, I think having an example with the full query params, is what I was looking for.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:52 PM
Ok will update this snippet right on top to include more parameters as examples: https://typesense.org/docs/0.24.1/api/search.html#search

1

05:52
Jason
05:52 PM
There’s also a way to automatically have curl urlencode the parameters
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
05:58 PM
is there a way to pass these arguments as JSON? My list of arguments is getting long and cumbersome
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:58 PM
Yeah the multi-search endpoint supports JSON: https://typesense.org/docs/0.24.1/api/federated-multi-search.html
05:59
Jason
05:59 PM
You can send a single search into it, even though it’s called multi-search
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
06:00 PM
and it would take these as well? exclude_fields, highlight_fields, per_page, etc.?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:00 PM
Yup, any search parameters
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
06:00 PM
cool, let me try that. Should simplify.
06:14
Manish
06:14 PM
Ahh... So much better with JSON! No weird URL encoding stuff.

1

Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
09:15 PM
I have an interesting use case and looking at multisearch [although just working on the idea now]
I have a search query [free text], example Operations Manager
Filters: employment-type = ‘full-time’, job-type=‘remote’, country = ‘Germany’
these are all facets with counts on them.
Naturally when you pick a filter - example full time, the count for all other employment-types is set to 0. I need to be able to show the count for part-time aswell even though you clicked on ‘full-time’, also other filters now would reflect the total count, example, job-type now has only 10 full-time jobs that are of job-type = remote.

The idea is to send another query to get facet counts using existing query + filters, except for the ones you’re active on, example on the employment-type.

This query would look like this:
search: Operations Manager,
Filters: job-type = ‘remote’, country = ‘Germany’
You see I purposely didn’t include the ‘employment-type’ filter, this way it would me facet counts for full-time and part time.
09:15
Ed
09:15 PM
am I headed on the right direction here?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
09:16 PM
Yup, that’s the exact approach you want to take
09:16
Jason
09:16 PM
Here’s an example of this in action: https://ecommerce-store.typesense.org/

If you open the browser dev console, and then the network tab, then click on some of the filters, you’ll see the type of queries generated inside the multi-search request
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
09:17 PM
but to improve the response time these can be done in one http request?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
09:17 PM
Correct, that’s the multi_search endpoint
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
09:17 PM
awesome
09:20
Ed
09:20 PM
what size cluster does this run on?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
09:22 PM
That example runs on a cluster that has a few other indices as well - 32GB RAM, 4vCPUs, SDN (in Oregon, Frankfurt, Mumbai)
09:22
Jason
09:22 PM
Image 1 for
09:23
Jason
09:23 PM
Probably has around 34M records across all collections
09:23
Jason
09:23 PM
The products collection itself is tiny - probably 50MB
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
09:24 PM
got it 😄
09:24
Ed
09:24 PM
thanks for sharing

1

09:30
Ed
09:30 PM
https://showcase-nextjs-typesense-ecommerce-store.vercel.app/
just this could run on 2vCPU and 1GB memory?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
09:31 PM
Yup, even 0.5GB RAM should be sufficient
Ed
Photo of md5-120c789e9edae8b90bf59cf0e2612b66
Ed
09:31 PM
awesome
09:31
Ed
09:31 PM
thanks again

1

Jun 20, 2023 (5 months ago)
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
02:40 PM
This stumped me -- the query_by parameter is outside the JSON, while all the rest of them are within the JSON. Why?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:24 PM
That’s just an example of how you can set parameters common to all searches as query parameters. And you can override individual search parameters inside the JSON
04:24
Jason
04:24 PM
You can just leave out query_by in the query parameter if you don’t need this mechanism
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
04:25 PM
So, query_by can be inside the JSON as well?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:25 PM
Correct
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
04:25 PM
Hmm... maybe I missed it. Would be good to have a full example of all the possible parameters / keys possible in a JSON request to TS.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:26 PM
You can use any of the search parameters listed here, inside a multi_search request: https://typesense.org/docs/0.24.1/api/search.html#search-parameters

1

04:26
Jason
04:26 PM
Essentially, you want to think of a multi_search request as a batch of individual search requests
Manish
Photo of md5-f0a83cd20895941fd74c026f9f15b61f
Manish
07:14 PM
Yeah, there's reference. And there are full examples. I think it would be good to expand the documentation with lots of examples.

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Discussing Typesense Search Request Performance

Al experienced longer-than-reported times for Typesense search requests, sparking a detailed examination of json parsing, response times and data transfer. Jason and Kishore Nallan helped solve the issue.

2

37
33mo

Moving from Algolia to Typesense: Questions and Answers

Juan sought advice from Kishore Nallan about moving from Algolia to Typesense, handling MultiSearch, setting parameters, checking imported documents, and a specific syntax query.

1

22
22mo

Enhancing Vector Search Performance and Response Time using Multi-Search Feature

Bill faced performance issues with vector search using multi_search feature. Jason and Kishore Nallan suggested running models on a GPU and excluding large fields from the search. Through discussion, it was established that adding more CPUs and enabling server-side caching could enhance performance. The thread concluded with the user reaching a resolution.

3

140
1mo

Improving Record Retrieval Speed from Typesense

Yoshi sought ways to accelerate Typesense record retrieval. Jason advised upgrading to high availability and using the documents/export endpoint. They also noted a high volume of writes consuming significant CPU capacity as a possible performance factor.

1

12
3mo

Integrating Semantic Search with Typesense

Krish wants to integrate a semantic search functionality with typesense but struggles with the limitations. Kishore Nallan provides resources, clarifications and workarounds to the raised issues.

6

75
11mo