#community-help

Resolving Error with Incorrect Field Type during Schema Update

TLDR Samuel encountered an error updating the schema for an existing collection. Kishore Nallan suggested specifying a concrete type, and later found inconsistent data within the collection that caused the error.

Powered by Struct AI
+15
raised_hands2
43
6mo
Solved
Join the chat
Mar 01, 2023 (7 months ago)
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
10:28 AM
Hello, It is possible to find source of Request failed with HTTP code 400 | Server said: Field ** has an incorrect type. error when updating schema on already existing collection? I have tried to export data and check for any corrupted fields but wasn't able to find anything wrong
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:34 AM
Might be some edge case because field name ** sounds weird. Are you able to share the data directory with me via DM so I can take a look? You can just zip it and share if it's something you are comfortable sharing.
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
10:35 AM
I'm running on cloud typesense and field name is not ** I just recorded it for purpose of sending this message original error is Error: e: Request failed with HTTP code 400 | Server said: Field properties.WORK.assignee.value has an incorrect type.
10:36
Samuel
10:36 AM
where properties.WORK.assignee.value can be null | [] | string[]
10:36
Samuel
10:36 AM
{
  "fields": [
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "properties.WORK.assignee.value",
      "optional": true,
      "sort": false,
      "type": "auto"
    }
  ]
}
10:36
Samuel
10:36 AM
This is schema update I'm trying to apply
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:40 AM
There was an issue that I fixed a couple of weeks back that involved a nested value being null. This might be related to it. Are any of the existing records null on-disk (for this field)?
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
10:41 AM
Yes there are some
10:41
Samuel
10:41 AM
Cluster has Typesense v0.24.0 version might be related ?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
10:42 AM
Hold on let check if that could be an issue.
10:42
Kishore Nallan
10:42 AM
*that == null values
+11
raised_hands1
10:52
Kishore Nallan
10:52 AM
Hmm I am not able to reproduce that on a simple collection. Mind if I take a look at your collection? Can you DM me the cluster ID and collection name.
+11
11:19
Kishore Nallan
11:19 AM
Ah, I just realized that the type cannot be auto --> it must be a concrete type.
11:19
Kishore Nallan
11:19 AM
While the type can be auto during schema declaration, for schema changes, it needs to be a concrete type.
11:22
Kishore Nallan
11:22 AM
Try using string[] --> it should work!
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
12:08 PM
Thank you, works perfectly :))
raised_hands1
Mar 17, 2023 (6 months ago)
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
08:20 AM
I have this problem again on same cluster 😞
08:21
Samuel
08:21 AM
Image 1 for
08:21
Samuel
08:21 AM
Image 1 for
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:22 AM
Do you already have records with that value but with a different type?
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
08:24 AM
It worked perfectly until yesterday but something has happened and our schema got "reseted" without any action from us. After that I cannot update schema back to support this field. I believe that if this field configuration was present every insert of document with invalid data would get rejected. Or is there different behaviour?
08:24
Samuel
08:24 AM
I can download data again a check the types there if needed
09:04
Samuel
09:04 AM
Didn't find any inconsistency in our data
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:30 AM
Ok let me check. Give me an hour or so.
+11
11:25
Kishore Nallan
11:25 AM
Can you please post the update schema you tried (in the screenshot) as text here?
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
11:26 AM
sure
11:27
Samuel
11:27 AM
{
  "fields": [
    {
      "facet": true,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "properties.WORK.assignee.value",
      "optional": true,
      "sort": false,
      "type": "string[]"
    }
  ]
}
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:34 AM
Thanks, debugging now. Will keep you posted shortly.
+11
11:40
Kishore Nallan
11:40 AM
properties.WORK.assignee.value seems to be an array of objects?
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
11:40 AM
No it's always array of string
11:41
Samuel
11:41 AM
[{
  "lastUpdateBy": "KOBQAiHm8eTYmdYDSJtHfnxOBIv1",
  "updatedAt": {
    "_nanoseconds": 583000000,
    "_seconds": 1676508480
  },
  "value": [
    "KOBQAiHm8eTYmdYDSJtHfnxOBIv1"
  ]
}
,{
  "lastUpdateBy": "AJSUE9y9RtZw5BY7AIqafyPHhgT2",
  "updatedAt": {
    "_nanoseconds": 415000000,
    "_seconds": 1675938146
  },
  "value": [
    "AJSUE9y9RtZw5BY7AIqafyPHhgT2"
  ]
}]
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:41 AM
See here I found one with an array of objects:

"properties":{
      "WORK":{
         "assignee":{
            "lastUpdateBy":"CWzdjyOShMg4bHeGO4NkJzigLnE2",
            "updatedAt":{
               "_nanoseconds":660000000,
               "_seconds":1678966234
            },
            "value":[
               {
                  "displayName":"Social media manager",
                  "id":"Social media manager"
               }
            ]
         },
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
11:41 AM
oh wow
11:42
Samuel
11:42 AM
I'm sorry, I did all I was able via jq '.properties.WORK.assignee | select(.properties...' but didn't found that object
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:42 AM
Sorry for the trouble here -- we really should be surfacing the id of the bad document along with the error message
11:43
Kishore Nallan
11:43 AM
Check doc id NR4FCzUQgqjQGtP6Tzzy
11:49
Kishore Nallan
11:49 AM
The other thing I just realized we are unable to support right now is to add a new object[] field. This is a limitation that we have to address.
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
11:49 AM
Thank you for your help! I found the docs with invalid data and sorry I took your time
11:50
Samuel
11:50 AM
just one question, If I will fix those bad data, and I will update the schema, then I will try to insert document with incos. data will it get rejected ?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:50 AM
No worry. I truly consider every question an area for us to improve on.
Samuel
Photo of md5-7f4ba9a2fc0e9dd5200c8ff347f1ea6e
Samuel
11:52 AM
for example I will have set a correct schema type now to string[] and I will do the document.insert() which will contain invalid data. Will it get rejected ?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:53 AM
I don't get your question. If you fix that document and retry the schema update you have to target primitive types like string [] yes
11:54
Kishore Nallan
11:54 AM
Once you have a schema change done, subsequent docs inserted have to conform
+11