we upgraded our cluster this morning to a newer version and typesense #community-help

we upgraded our cluster this morning (to a newer v...

Rob Bast

03/05/2025, 7:02 AM

we upgraded our cluster this morning (to a newer version) and we lost one of the fields in one of our collections. any idea why this could have happened?

Rob Bast

03/05/2025, 7:03 AM

it was a boolean facet field named

vat_validation_required

. I just added it back and now our production data set is working again. cluster id is

2v0kybqdzjp1ts6rp

if you plan to investigate.

Rob Bast

03/05/2025, 7:04 AM

issue was in the

orders_

collection.

Kishore Nallan

03/05/2025, 7:52 AM

The only time I've seen something like this happen is when a previous bug allowed a bad field to be persisted which is not allowed in the next version owing to the bug being fixed. I'll check.

Rob Bast

03/05/2025, 9:22 AM

well I was able to add it manually after the migration of the cluster, so I would assume the field was not bad in that sense?

Rob Bast

03/11/2025, 12:55 PM

Hmm, also getting related errors now when searching by this boolean facet

Rob Bast

03/11/2025, 12:55 PM

"Error with filter field `vat_validation_required`: Unidentified field data type, see docs for supported data types."

Rob Bast

03/11/2025, 12:55 PM

message has been deleted

Rob Bast

03/11/2025, 12:56 PM

i guess something changed between 28.0 rc candidate and 28 final release

Rob Bast

03/11/2025, 1:05 PM

the facet_by for this field works, but the filter_by does not

Rob Bast

03/11/2025, 1:06 PM

the facet_by does indicate there are no documents with value 'true' currently, could that be why the filter_by would fail?

Fanis Tharropoulos

03/11/2025, 1:13 PM

Try using

<boolean_filter_name>:=true

Fanis Tharropoulos

03/11/2025, 1:18 PM

Also, could you post what the orders preset looks like and what the collection of orders looks like as well?

Kishore Nallan

03/11/2025, 1:19 PM

I checked the server, and there are no errors or anything strange in the logs.

Kishore Nallan

03/11/2025, 1:19 PM

If you want to be sure, I recommend reindexing the documents onto a new collection and switching your app over to that.

Rob Bast

03/11/2025, 2:48 PM

@Fanis Tharropoulos that is a bit difficult cause it is a toggleRefinement widget from algolia via the TypesenseInstantsearchAdapter.

Rob Bast

03/11/2025, 2:49 PM

Collection

Copy code

{
  "created_at": 1739795064,
  "default_sorting_field": "updated_at",
  "enable_nested_fields": true,
  "fields": [
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "identifier",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "customer",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "object"
    },
    {
      "async_reference": false,
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "customer.id",
      "optional": false,
      "reference": "customers_20250217.id",
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "customer.name",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "customer.company",
      "optional": true,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": true,
      "locale": "",
      "name": "customer.email",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": true,
      "locale": "",
      "name": "reference",
      "optional": true,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "billing.address",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "billing.postalcode",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "shipping.address",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "shipping.postalcode",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "created_at",
      "optional": false,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "int32"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "updated_at",
      "optional": false,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "int32"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "billing.country",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "shipping.country",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "status",
      "optional": true,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "webshop",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "paymentmethod",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "shippingmethod",
      "optional": true,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "string"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "intracommunautair",
      "optional": false,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "bool"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "deadline",
      "optional": true,
      "range_index": true,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "int32"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "is_picked",
      "optional": false,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "bool"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "is_printed",
      "optional": false,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "bool"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "amount_inc",
      "optional": false,
      "range_index": true,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "float"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "amount_exc",
      "optional": false,
      "range_index": true,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "float"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "margin",
      "optional": false,
      "range_index": true,
      "sort": true,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "float"
    },
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "vat_validation_required",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "boolean"
    }
  ],
  "name": "orders_20250217",
  "num_documents": 753623,
  "symbols_to_index": [],
  "token_separators": [
    "+",
    "-",
    "@",
    "."
  ]
}

For preset see image:

Fanis Tharropoulos

03/11/2025, 3:16 PM

You somehow created a field of type

boolean

. It should have been

bool

. Don't know how that wasn't caught when creating the collection

Copy code

{
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "vat_validation_required",
      "optional": false,
      "sort": false,
      "stem": false,
      "stem_dictionary": "",
      "store": true,
      "type": "boolean" <- Here
    }

Rob Bast

03/13/2025, 8:03 AM

huh, weird

Rob Bast

03/13/2025, 8:03 AM

I'll go and change that

Rob Bast

03/13/2025, 8:34 AM

fixed, everything working as expected now, thanks for pointing that out 😅

🙌 1

Fanis Tharropoulos

03/14/2025, 7:44 AM

It caught me by surprise as well! Don't know how I noticed it

Open in Slack

Previous Next