https://typesense.org logo
Join Slack
Powered by
Hello <@U076GF3T0HY>, Nice meeting u again. So i c...
# community-help
a
Hello @Fanis Tharropoulos, Nice meeting u again. So i came up below error recently, can u help me solve it?
Copy code
"error": "`filter_by` has too many operations. Maximum allowed: 100. Use `--filter-by-max-ops` command line argument to customize this value.
f
It seems you have more than 100 different filter_by operators running at one query. Are you using Typesense Cloud or self hosted?
a
its cloud version, and currently in free tier
f
Can you share the query? 100 different filters seem overkill
a
Copy code
{
  "searches": [
    {
      "query_by": "partNumber",
      "per_page": 250,
      "highlight_full_fields": "partNumber",
      "collection": "supplierinventories",
      "q": "*",
      "facet_by": "supplierInfo",
      "filter_by": "((partNumber:68001M79G01 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:68002M79G01 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:68003M79G00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:68004M79G00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:57300M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:84511M79G01 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:71711M66L00-5PK && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:71741M66L00-5PK && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:71732M66L10-5PK && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:71731M66L10-5PK && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:35501M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:35502M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:35121M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:35321M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:84611M79G00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:35650M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:35750M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:83811M79G01 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:83821M79G11 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:83831M79G01 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:83841M79G01 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:72321M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:72322M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:72331M76G00-5PK && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:43250M67K00-27N && adjustedQuantity:<4 && adjustedQuantity:>0)) || ((partNumber:990J0M66L00-010 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:71781M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:71782M66L00 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:990J0M66L01-010 && adjustedQuantity:<1 && adjustedQuantity:>0)) || ((partNumber:990J0M66LQ0-010 && adjustedQuantity:<1 && adjustedQuantity:>0))",
      "max_facet_values": 100,
      "page": 1
    }
  ]
}
in this query every partnumber quantity condition can be different every time, thats why we are not using 'in' condition
f
Why are you consistently saying that adjustedQuantity <1 and <0? It's basically =1
a
Actually its dynamic, but in this case its luckily 1, it can be 10 also.
f
This is being generated by code at runtime?
a
yeah
f
Could you please explain the whole use-case? Acceptance criteria, edge cases etc
a
So basically u can say a customer want lots of parts and every part can have different asked quantity. So i want to fetch the documents which has the part number customer asked and quantity should be more than 0 and less than it asked quantity. So basically i want to find parts which are partial in quantity.
f
It would be 
partNumber && adjustedQuantity:[lower...upper]
. Is there ever a document where quantity is less than 0?
a
No there is no document having quantity less than 0
f
Then you can just say 
partNumber && adjustedQuantity:<upper_bound
a
but there can be case of 0
which we want to exclude
Also isnt there any way to use (
--filter-by-max-ops
) and increase limit ?
f
Okay got it. Since you're evaluating things at runtime, create a map of the key being the upper bound, and the value being the list of part numbers that match it, and then create a filter that would look like 
partNumber:[sku1,sku2,sku3] && adjustedQuantity:[1...upper1] || partNumber[sku4,sku5,sku6] && adjustedQuantity:[1...upper2]
Your current approach is going to be really inefficient, that's why we're trying to keep a max of 100 different filter operations per query
🙌 1
a
let me try
I will keep u updated
but still dont we have access to tweak the limit?
f
We do, but let's try to avoid it, since there's a better solution for this
a
yeahhh , but right now this bug came in production😅
So i was thinking a quick fix for now
f
You're not using HA, though, right? Even if we were to update the max filters allowed, there would be downtime
a
Ohh
HA?
f
High Availability. You mentioned the Trial configuration, which doesn't include it
a
Ohh yeahh yeahh, we are running on trial config, so basically its available in HA config only.
k
Please DM me your cluster ID.
f
so basically its available in HA config only.
No, I was talking about downtime. If you don't have HA, restarting the Typesense server will bring production down altogether until it's back up
k
I've bumped that limit and restarted. Since it's a small cluster, blip was small.
🙌 1
a
Thanks a lot 🙌
cc. @Sahil Rally @Atishay Jain
s
Couple of Questions 1. What is the bumped Limit now ? 2. What is the Limit on Production Paid cluster ? 3. Why there is a Limit 4. We have the use case where there can be 500 Parts in thequery, how would we handle that ? 5. partNumber:[sku1,sku2,sku3] && adjustedQuantity:[1...upper1] , we can do this but in worst case all the adjustedQuntities could be different leading to same issue again 6. Can we increase this limit for us on Paid Servers , otherwise this could be blocker for us to move ahead with TS.
@Kishore Nallan
k
I've bumped it to 1000 now. It's counter productive to have such large filters usually. It exists as a soft limit which you can increase. We can change this at the moment from our end: just set a sufficiently large number for your use case and it should be fine.
👍 1
a
@Fanis Tharropoulos @Kishore Nallan i have one more such limit problem, it's regarding 250 hits per page in a query . Is there any way we can increase this limit to some number as per our use case?
s
“It exists as a soft limit which you can increase.” -> Pls note it. Do you know how to increase it ? @Kishore Nallan Are you saying 1000 is the soft limit and we can increase it further ? Error message says Max Limit is 150 which you bumped to 1000.
f
The limit was 150 which we increased to 1000. Do we need more than 1000 filters?
j
Are you saying 1000 is the soft limit and we can increase it further
Correct.
i have one more such limit problem, it's regarding 250 hits per page in a query. Is there any way we can increase this limit to some number as per our use case?
This is also a soft limit for performance reasons. But we can increase it for you from our side. We've bumped it up to 1000 hits per page for your cluster.
👍 1
r
@Bernhard Pertl same issue I think?
2 Views
Open in Slack
PreviousNext
Powered by