Typesense Bug Fix with `canceled_at` Field and Upgrade Concerns
TLDR Mateo reported an issue regarding the treatment of an optional field by Typesense which was confirmed a bug by Jason. After trying an upgrade, an error arose. Jason explained the bug was due to a recent change and proceeded to downgrade their version. Future upgrade protocols were discussed.
2
1
Dec 26, 2022 (10 months ago)
Mateo
02:14 PMI've added a field
canceled_at
to an appointments collection: {
"facet": false,
"index": false,
"name": "canceled_at",
"optional": true,
"type": "int64"
}
But I've noticed that even though I marked it as optional, when I'm trying to index my appointments only those who have canceled_at as not null are being uploaded to typesense, the desired behaviour would to upload all appointments with or without a canceled_at field.
1
Jason
02:19 PMMateo
02:23 PMMateo
02:25 PMTypesense Server typesense-ruby
>= v0.23.0 >= v0.14.0
>= v0.21.0 >= v0.13.0
>= v0.20.0 >= v0.12.0
>= v0.19.0 >= v0.11.0
>= v0.18.0 >= v0.10.0
>= v0.17.0 >= v0.9.0
>= v0.16.0 >= v0.8.0
>= v0.15.0 >= v0.7.0
>= v0.12.1 >= v0.5.0
>= v0.12.0 >= v0.4.0
<= v0.11 <= v0.3.0
Mateo
02:26 PMJason
03:09 PMJason
03:09 PMMateo
03:15 PMJason
03:17 PMDec 27, 2022 (10 months ago)
Mateo
01:56 PMThank you for upgrading the RC build, it was effective yesterday, I reindexed the appointments today again, even tried deleting the collection and recreating it, but unfortunately I can still see only the appointments with the
canceled_at
field not nullJason
02:51 PMMateo
03:40 PM""
instead of 0
or nil
and typensense was failing with a 400 saying canceled_at must be a int64
Mateo
03:42 PMcanceled_at != 0
to get all appointments not canceled?Jason
03:55 PMUntil then, you would have to create a new boolean field called say
is_canceled: true|false
and use that to filter.Mateo
04:04 PM400
message "Could not parse the filter query: unbalanced `&&` operands."
We send a request like this:
Mateo
04:04 PMJason
04:08 PMcurl '' -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"
substituting
${TYPESENSE_API_KEY}
with your actual API key?and then post the output?
Mateo
04:09 PM{"facet_counts":[{"counts":[{"count":28140,"highlighted":"06dbed29-f19c-4631-9a93-b9a4b777724e","value":"06dbed29-f19c-4631-9a93-b9a4b777724e"},{"count":14037,"highlighted":"53af34f9-813b-4ab0-b5ef-e3787d493ace","value":"53af34f9-813b-4ab0-b5ef-e3787d493ace"},{"count":2829,"highlighted":"6462dec6-b278-4280-b32b-6d94d7d26531","value":"6462dec6-b278-4280-b32b-6d94d7d26531"},{"count":1302,"highlighted":"fb72e6e4-0e05-4c8c-bca5-cc0b67b7dfec","value":"fb72e6e4-0e05-4c8c-bca5-cc0b67b7dfec"},{"count":819,"highlighted":"5d0835ef-8d6c-4d3b-b164-8f55a1b5d0ff","value":"5d0835ef-8d6c-4d3b-b164-8f55a1b5d0ff"},{"count":337,"highlighted":"cc7247bc-a75a-4ebb-9746-2e016e7f89f6","value":"cc7247bc-a75a-4ebb-9746-2e016e7f89f6"},{"count":328,"highlighted":"10e2645b-9b28-4479-b7e9-11f8b4fc3c2e","value":"10e2645b-9b28-4479-b7e9-11f8b4fc3c2e"},{"count":224,"highlighted":"7ef796cc-670d-43e2-94c2-80fe9e6c4566","value":"7ef796cc-670d-43e2-94c2-80fe9e6c4566"},{"count":151,"highlighted":"4a7f505f-91df-4f30-8e70-6b73430ce6f2","value":"4a7f505f-91df-4f30-8e70-6b73430ce6f2"},{"count":126,"highlighted":"8715f70d-d60d-43cf-ba1e-840fa8a8293e","value":"8715f70d-d60d-43cf-ba1e-840fa8a8293e"}],"field_name":"appointment_type_id","stats":{"total_values":21}}],"found":48558,"hits":[{"document":{"answer_identifiers":[],"appointment_type_id":"53af34f9-813b-4ab0-b5ef-e3787d493ace","appointment_type_name":"Inbound","arrival_time":1675717200,"checkin_id":"","checkin_status":"","color":"#276EF1","commodity_type":"MISC","confirmation_id":"","created_at":1671555271,"created_by_id":"7905e4dc-92d7-4e6d-8699-8a3e2c9f0697","created_by_name":"Angelo Amore","dock_id":"b8f7a401-7815-41b7-9cc6-53a2f35f0138","dock_name":"Columbia Dock #04","equipment_type_id":"9383706a-0f3c-4865-bc93-74e9c9dfea3b","facility_id":"0259b832-33ec-4fdf-b385-a866a66a619a","facility_name":"Columbia","has_attachments":false,"id":"61be0afe-0b8e-4602-af4c-b470d69bdfb8","less_than_truckload":false,"purchase_order_identifiers":["P31915"],"quantity":"","recurring_appointment_blueprint_id":"","repeating":false,"scheduler_id":"d77219aa-6b46-4a55-a5b7-444f6cd4d1c0","scheduler_name":"GPI","shipper_id":"6779f5e3-0406-4338-8964-5ff70a18b4a0","status":"scheduled","time_zone":"America/New_York","updated_at":1671555271},"highlight":{},"highlights":[]}],"out_of":48558,"page":1,"request_params":{"collection_name":"appointment_production","per_page":1,"q":"*"},"search_cutoff":false,"search_time_ms":6}
Mateo
04:09 PMJason
04:09 PMJason
04:10 PM&
when you send the query to Typesense?Jason
04:11 PM&
in the field you’re trying to filter, you want to surround that full string with backticksMateo
04:13 PMreact-instantsearch-dom
and a refinement list to do itJason
04:15 PMMateo
04:16 PMcurl ''
Jason
04:19 PMmulti_search
endpointJason
04:19 PMMateo
04:21 PMcurl ''
Mateo
04:21 PMMateo
04:21 PMJason
04:22 PMcurl '' -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"
Jason
04:22 PMMateo
04:23 PM{"message": "Could not parse the filter query: unbalanced
&& operands."}
in network tabMateo
04:24 PMError: Request failed with HTTP code 400 | Server said: Could not parse the filter query: unbalanced
&& operands.
Jason
04:24 PMMateo
04:25 PMMateo
04:25 PMJason
04:25 PMJason
04:25 PMMateo
04:29 PMMateo
04:29 PMJason
04:29 PMJason
04:30 PMJason
04:32 PM{"filter_by":"shipper_id:004bed58-2588-430c-be7b-a7bc6a842905"}
Jason
04:34 PM-
inside the string without escaping.{“filter_by”:“shipper_id:=[`004bed58-2588-430c-be7b-a7bc6a842905`]“}
Jason
04:34 PM:
to :=
and then surround the id with backticks and square bracketsJason
04:42 PMJason
04:42 PMJason
04:44 PM&&
anything mentioned in the filter_by
search param with the filter_by
embedded inside the scoped search API key. Looks like we don’t account for cases when filter_by
is empty, and still end up ANDing an empty string, with the filter inside the scoped API KeyJason
04:45 PMfilter_by
in the search query and then the query works with the scoped search api keyJason
04:45 PMMateo
04:47 PMMateo
04:48 PMJason
04:49 PMAlternatively, you could provision a new cluster on 0.21 and then reindex your data in it.
Jason
04:49 PMMateo
04:52 PMJason
04:53 PMMateo
04:55 PMMateo
04:55 PMJason
04:56 PMJason
04:56 PMMateo
04:58 PM1
Jason
04:59 PMMateo
06:01 PMI wanted to ask, is there any way we can schedule this upgrades at specific times in the future? we want to create a flow to maintain our clusters up-to-date with the new versions but also avoid clients to experiment any issues, so we were thinking in having 2 clusters, one for test/development and another for staging/prod, and schedule upgrades at specific day/time so the QA team can test first with the Test Cluster and then do that for the staging/prod one
Jason
06:02 PMMateo
06:09 PMJason
06:10 PM1
Mateo
06:14 PMJason
06:14 PMDec 28, 2022 (9 months ago)
Jason
05:09 PMJan 03, 2023 (9 months ago)
Jason
03:41 PMJan 30, 2023 (8 months ago)
Mateo
02:01 PMJason
08:25 PMJason
08:25 PMTypesense
Indexed 2786 threads (79% resolved)
Similar Threads
Issues with Indexing Data using Typesense in Go
Konrad was having trouble with a "Forbidden" error using Typesense in Go when indexing data remotely. David offered multiple troubleshooting steps and suggestions. Ultimately, Konrad discovered the issue lied within the application code.
Issue with Numerical Faceting in Combination with Group_by
Vamshi encounters issues with numerical faceting when used in conjunction with group_by in Typesense. After a detailed query script, Kishore Nallan identifies this as a bug, encouraging the user to raise a Github issue.
Issue with Query Expectations on Typesense Search
Sean was having an issue with their search query on Typesense. Kishore Nallan suggested adjusting the 'drop_tokens_threshold' parameter. After making the adjustment, Sean found an issue with the order of the results, which was resolved by updating Typesense version.
Issue with Embedding Error in Version 0.25.0.rc63
Bill reported a bug in version 0.25.0.rc63 regarding a problem with updating or emplacing a document and receiving an embedding error. This was resolved in version 0.25.0.rc65, but further discussion ensued regarding the function of 'index' in the update feature.
Troubleshooting Typesense Data Load Issues on Ecommerce Platform
Dương requested help with Typesense data load issues on non-localhost systems. Kishore Nallan and David helped him isolate the problem. It was finally resolved when Dương identified and corrected a router configuration issue.