<@U01PL2YSG8L> was there any progress on filtering...
# community-help
l
@Kishore Nallan was there any progress on filtering multiple values within objects in an array? We'd like to use this functionality to enable the empty value search. https://github.com/typesense/typesense/issues/227#issuecomment-1211828348
k
No progress yet, but this is high on our roadmap for v28 and will be picking it up in the next few weeks.
🙌 1
l
any recommendations to handle this for now? I need to find a solution until end of september 🫠
k
Can you describe your use case and schema?
l
Of course. Here is a basic example: We have a collection of countries along with some animalData, which holds data about individual animals. To filter on missing data, we also have a boolean for each animal in the country record. Now we want to filter e.g: • countries that have more than 300 lions or 100 Tigers • countries that have more than 300 lions or no lions at all (doesn't work) so something like •
(animalData.count:>300 && animalData.name:=lion) || (animalData.count:>100 && animalData.name:=tiger)
•
(animalData.count:>300 && animalData.name:=lion) || isLionDataNull:true
Schema
Copy code
{
  name: "animals_collection",
  fields: [
    { name: "country", type: "string" },
    { name: "animalData", type: "object[]" },
    { name: "/^is.*DataNull$/", type: "bool" }
  ]
}
Data
Copy code
[
  {
    country: "USA",
    id: "1",
    animalData: [
      { data: { count: 221 }, name: "Lion" },
      { data: { count: 915 }, name: "Tiger" }
    ],
    isLionDataNull: false,
    isTigerDataNull: false,
    isElephantDataNull: true
  },
  {
    country: "Canada",
    id: "2",
    animalData: [
      { data: { count: 915 }, name: "Tiger" },
      { data: { count: 821 }, name: "Elephant" }
    ],
    isLionDataNull: true,
    isTigerDataNull: false,
    isElephantDataNull: false
  }
];
I think what i will do is flatten the "animalData" object, as we only have ~10 entries there. I think that should work, although its not great
j
If you need to filter on animal data, I would recommend creating that as the top-level object. So create, one record per animal, and add country as a property of each animal record. Then use the group_by parameter to fetch unique country names
l
Could you explain more why you prefer restructuring the records to flattening the data structure? We thought about that before, but figured it would be best to keep the structure as "countries" because there is a lot of "country" specific data that we'd have to duplicate over many records.
with the flattening it works in my experiments btw
j
If you have a limited number of animals, then it should be fine to flatten
🙌 1
l
@Jason Bosco i've tried to implement the flattening +
exists
approach, but encountered an issue with the algolia instantsearch client: • how do i define such filters described here across multiple facets - e.g.
lionsExist OR lion.data.count
or (
lionsExist OR lion.data.count AND lion.data.avgAge) OR (...)
? The documentation says that in Algolia it is not permitted to do queries across formats like this. Is this fixed in the typesense adapter? • Otherwise how we should best modify the instantsearch client to format the query into this format?
j
That limitation does not exist in Typesense's
filter_by
parameter. But it needs to be in Typesense's filter format, and not Algolia's. With the configure widget for eg, you can pass a
filters
prop and that will get passed verbatim to Typesense. You'd need to find an equivalent property in the widget you're using to set the
filters
string, and use Typesense's
filter_by
format in there
l
My understanding is that each instantsearch widget only updates the internal state of that filter-type, e.g. range will only modify the internal range filter state, not set the actual search query parameters. But with the proposed logic, i would need to modify the filter_by query based on two widgets. So i'm wondering where the (best) place is, where instantsearch combines the internal state parameters into the final
query_by
string
j
I'm out of my depth with Instantsearch knowledge here, so I don't know the exact solution. But from the perspective of the typesense adapter, the internal instantsearch state gets passed into the adapter and then the adapter transforms it... except for the
filters
parameter, that gets passed through to Typesense as is. So you want to find a way in Instantsearch, to set the
filters
state value somehow