#community-help

Typesense Bug Fix with `canceled_at` Field and Upgrade Concerns

TLDR Mateo reported an issue regarding the treatment of an optional field by Typesense which was confirmed a bug by Jason. After trying an upgrade, an error arose. Jason explained the bug was due to a recent change and proceeded to downgrade their version. Future upgrade protocols were discussed.

8mo

Join the chat

Dec 26, 2022 (10 months ago)

Photo of md5-da3527dcf5d076c2193d90ba6a2964bb

Mateo

02:14 PM

Hey guys! good morning!
I've added a field canceled_at to an appointments collection:

    {
      "facet": false,
      "index": false,
      "name": "canceled_at",
      "optional": true,
      "type": "int64"
    }

But I've noticed that even though I marked it as optional, when I'm trying to index my appointments only those who have canceled_at as not null are being uploaded to typesense, the desired behaviour would to upload all appointments with or without a canceled_at field.

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

02:19 PM

This is a bug in 0.23.1. We’ve already fixed it in the latest RC build. Could you upgrade to 0.24.0.rcn46

Mateo

02:23 PM

Got it, I'm kinda new to the project and Ruby/Rails overall, sorry if these are silly questions, I see we have typesense (0.8.0) in the Gemfile.lock, would it be a different version in Ruby?

02:25

Mateo

02:25 PM

Typesense Server     typesense-ruby
>= v0.23.0     >= v0.14.0
>= v0.21.0     >= v0.13.0
>= v0.20.0     >= v0.12.0
>= v0.19.0     >= v0.11.0
>= v0.18.0     >= v0.10.0
>= v0.17.0     >= v0.9.0
>= v0.16.0     >= v0.8.0
>= v0.15.0     >= v0.7.0
>= v0.12.1     >= v0.5.0
>= v0.12.0     >= v0.4.0
<= v0.11     <= v0.3.0

02:26

Mateo

02:26 PM

Maybe v0.14.1 could work?

Jason

03:09 PM

I was referring to the version of Typesense server that you’re using, not the client library.

03:09

Jason

03:09 PM

How are you running Typesense server btw?

Mateo

03:15 PM

Oh okay, I think we upload from our Rails app to Typesense cloud directly

Jason

03:17 PM

Ah ok, if you can send an email to [email protected] with you cluster ID, we can upgrade you to the latest RC build from our side

Dec 27, 2022 (10 months ago)

Mateo

01:56 PM

Good morning Jason!
Thank you for upgrading the RC build, it was effective yesterday, I reindexed the appointments today again, even tried deleting the collection and recreating it, but unfortunately I can still see only the appointments with the canceled_at field not null

Jason

02:51 PM

Hmm, could you take a batch of say just 2 documents with canceled_at set to null, import them and share the response of the import API call from Typesense?

Mateo

03:40 PM

It worked! the client was sending "" instead of 0 or nil and typensense was failing with a 400 saying canceled_at must be a int64

03:42

Mateo

03:42 PM

One last question would be I cannot filter canceled_at value like canceled_at != 0 to get all appointments not canceled?

Jason

03:55 PM

Yeah, we don’t support doing != on integer fields at the moment unfortunately. But we’re very close to wrapping it up in the next few weeks.

Until then, you would have to create a new boolean field called say is_canceled: true|false and use that to filter.

Mateo

04:04 PM

I have a production issue after the update, all the requests to typesense in a page are returning:

400
message    "Could not parse the filter query: unbalanced `&&` operands."

We send a request like this:

04:04

Mateo

04:04 PM

Do you know what could be the issue Jason?

Jason

04:08 PM

Hmm, wonder if there’s some issue with escaping. Could you run this curl command:

curl '' -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"

substituting ${TYPESENSE_API_KEY} with your actual API key?

and then post the output?

Mateo

04:09 PM

Sure, it returned this:

{"facet_counts":[{"counts":[{"count":28140,"highlighted":"06dbed29-f19c-4631-9a93-b9a4b777724e","value":"06dbed29-f19c-4631-9a93-b9a4b777724e"},{"count":14037,"highlighted":"53af34f9-813b-4ab0-b5ef-e3787d493ace","value":"53af34f9-813b-4ab0-b5ef-e3787d493ace"},{"count":2829,"highlighted":"6462dec6-b278-4280-b32b-6d94d7d26531","value":"6462dec6-b278-4280-b32b-6d94d7d26531"},{"count":1302,"highlighted":"fb72e6e4-0e05-4c8c-bca5-cc0b67b7dfec","value":"fb72e6e4-0e05-4c8c-bca5-cc0b67b7dfec"},{"count":819,"highlighted":"5d0835ef-8d6c-4d3b-b164-8f55a1b5d0ff","value":"5d0835ef-8d6c-4d3b-b164-8f55a1b5d0ff"},{"count":337,"highlighted":"cc7247bc-a75a-4ebb-9746-2e016e7f89f6","value":"cc7247bc-a75a-4ebb-9746-2e016e7f89f6"},{"count":328,"highlighted":"10e2645b-9b28-4479-b7e9-11f8b4fc3c2e","value":"10e2645b-9b28-4479-b7e9-11f8b4fc3c2e"},{"count":224,"highlighted":"7ef796cc-670d-43e2-94c2-80fe9e6c4566","value":"7ef796cc-670d-43e2-94c2-80fe9e6c4566"},{"count":151,"highlighted":"4a7f505f-91df-4f30-8e70-6b73430ce6f2","value":"4a7f505f-91df-4f30-8e70-6b73430ce6f2"},{"count":126,"highlighted":"8715f70d-d60d-43cf-ba1e-840fa8a8293e","value":"8715f70d-d60d-43cf-ba1e-840fa8a8293e"}],"field_name":"appointment_type_id","stats":{"total_values":21}}],"found":48558,"hits":[{"document":{"answer_identifiers":[],"appointment_type_id":"53af34f9-813b-4ab0-b5ef-e3787d493ace","appointment_type_name":"Inbound","arrival_time":1675717200,"checkin_id":"","checkin_status":"","color":"#276EF1","commodity_type":"MISC","confirmation_id":"","created_at":1671555271,"created_by_id":"7905e4dc-92d7-4e6d-8699-8a3e2c9f0697","created_by_name":"Angelo Amore","dock_id":"b8f7a401-7815-41b7-9cc6-53a2f35f0138","dock_name":"Columbia Dock #04","equipment_type_id":"9383706a-0f3c-4865-bc93-74e9c9dfea3b","facility_id":"0259b832-33ec-4fdf-b385-a866a66a619a","facility_name":"Columbia","has_attachments":false,"id":"61be0afe-0b8e-4602-af4c-b470d69bdfb8","less_than_truckload":false,"purchase_order_identifiers":["P31915"],"quantity":"","recurring_appointment_blueprint_id":"","repeating":false,"scheduler_id":"d77219aa-6b46-4a55-a5b7-444f6cd4d1c0","scheduler_name":"GPI","shipper_id":"6779f5e3-0406-4338-8964-5ff70a18b4a0","status":"scheduled","time_zone":"America/New_York","updated_at":1671555271},"highlight":{},"highlights":[]}],"out_of":48558,"page":1,"request_params":{"collection_name":"appointment_production","per_page":1,"q":"*"},"search_cutoff":false,"search_time_ms":6}

04:09

Mateo

04:09 PM

seems like it worked

Jason

04:09 PM

Yeah…

04:10

Jason

04:10 PM

So I suspect there might be some issue with escaping & when you send the query to Typesense?

04:11

Jason

04:11 PM

If you have & in the field you’re trying to filter, you want to surround that full string with backticks

Mateo

04:13 PM

The problem is that I think we do not control the url formatting, we use react-instantsearch-dom and a refinement list to do it

Jason

04:15 PM

Hmm, the adapter should do this for you. Could you open the network tab of the browser dev console, then click on the UI elements that generate this failing search query, right click on the network request to Typesense, click on “Copy as CURL” and paste that here?

Mateo

04:16 PM

curl ''

Jason

04:19 PM

Hmm, the typesense-instantsearch-adapter that you’d use with react-instantsearch-dom will only generate queries using the multi_search endpoint

04:19

Jason

04:19 PM

So this query seems to be coming from some other custom component

Mateo

04:21 PM

curl ''

04:21

Mateo

04:21 PM

These are the two requests I see to typesense

04:21

Mateo

04:21 PM

in the page with the problems

Jason

04:22 PM

This second query also seems to work fine…

curl '' -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"

04:22

Jason

04:22 PM

Do you see see the error from Typesense in the browser’s dev console, in the network response for these two API calls?

Mateo

04:23 PM

Yes, I see we get {"message": "Could not parse the filter query: unbalanced && operands."} in network tab

04:24

Mateo

04:24 PM

Error: Request failed with HTTP code 400 | Server said: Could not parse the filter query: unbalanced && operands.

Jason

04:24 PM

And you’re clicking on copy as curl on the network request and pasting it here right?

Mateo

04:25 PM

Yes, the only thing I do is delete the headers to not expose the api key

04:25

Mateo

04:25 PM

but that's exactly how I do it

Jason

04:25 PM

Oh wait, are you using a scoped API key?

04:25

Jason

04:25 PM

I wonder if the filters embedded inside the scoped API key have an issue with the unbalanced &&

Mateo

04:29 PM

Can I DM you the cURL copy?

04:29

Mateo

04:29 PM

with full headers

Jason

04:29 PM

Sure