I m hoping someone might have some insight into an issue I m typesense #community-help

I'm hoping someone might have some insight into an...

Tim Stoute

03/26/2025, 9:05 PM

I'm hoping someone might have some insight into an issue I'm facing and unable to get past. I'm using the TypeSense Firestore Extension in Firebase (typesense/firestore-typesense-search@2.0.1). I've installed and configured the extension in Firebase. I create a TypeSense collection in TypeSense Cloud. I've included the schema below. Each time I try to sync my Firebase collection with the TypeSense collection by adding a "typesense_sync" collection in Firestore and adding record to to the collection with id = backfill and trigger = true I get errors in the Firebase Functions logs like this: "Error importing document with error: Field

uid

has been declared in the schema, but is not found in the document." But - I am certain that a uid field is in every one of the 15 documents in the Firestore collection. Other fields do not cause this error. My TypeSense Schema:

Copy code

{
  "created_at": 1743004503,
  "default_sorting_field": "createdAt",
  "enable_nested_fields": false,
  "fields": [
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "administrative_area_level_1",
      "optional": true,
      "sort": false,
      "stem": false,
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "city",
      "optional": true,
      "sort": false,
      "stem": false,
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "copy",
      "optional": true,
      "sort": false,
      "stem": false,
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "countryName",
      "optional": true,
      "sort": false,
      "stem": false,
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "createdAt",
      "optional": false,
      "sort": true,
      "stem": false,
      "store": true,
      "type": "int64"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "provinceOrState",
      "optional": true,
      "sort": false,
      "stem": false,
      "store": true,
      "type": "string"
    },
    {
      "facet": false,
      "index": true,
      "infix": false,
      "locale": "",
      "name": "uid",
      "optional": false,
      "sort": false,
      "stem": false,
      "store": true,
      "type": "string"
    }
  ],
  "name": "questions",
  "num_documents": 0,
  "symbols_to_index": [],
  "token_separators": []
}

Jason Bosco

03/26/2025, 9:06 PM

Could you share a screenshot of the logs from the Firebase console?

Jason Bosco

03/26/2025, 9:06 PM

with this particular log line fully expanded

Tim Stoute

03/26/2025, 9:08 PM

message has been deleted

Jason Bosco

03/26/2025, 9:29 PM

Do the log lines above or before show you the actual document ID at all?

Jason Bosco

03/26/2025, 9:29 PM

Also, may I know if you're using the recently release 2.0.1 version of the extension?

Tim Stoute

03/26/2025, 9:44 PM

Yes, the version of the Extension is 2.0.1 and I've attached another screen cap of the log errors. If there are other log lines you would like to see details on please let me know. Thanks

Jason Bosco

03/26/2025, 9:55 PM

Could you expand the line that says "Import error in a batch of documents"? That should have the exact document ID that threw the error

Jason Bosco

03/26/2025, 9:55 PM

@Fanis Tharropoulos We should improve the logging of those individual errors, so we include at least the document ID in the logs. Otherwise currently it's only in the batch import results

Tim Stoute

03/26/2025, 9:58 PM

Ok, I see, the id of the first failed document is 1Hzs4Ih9JvqAHisW4OBu, and here is the top portion of that log line expanded: "Import error in a batch of documents from 1Hzs4Ih9JvqAHisW4OBu to zwMyBjMXSv5XnkMvEh26 ImportError: 0 documents imported successfully, 15 documents failed during import. Use

error.importResults

from the raised exception to get a detailed error reason for each document. at ImportError.TypesenseError [as constructor] (/workspace/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js2328) at new ImportError (/workspace/node_modules/typesense/lib/Typesense/Errors/ImportError.js2528) at Documents.<anonymous> (/workspace/node_modules/typesense/lib/Typesense/Documents.js15439) at step (/workspace/node_modules/typesense/lib/Typesense/Documents.js4823) at Object.next (/workspace/node_modules/typesense/lib/Typesense/Documents.js2953) at fulfilled (/workspace/node_modules/typesense/lib/Typesense/Documents.js2058) at process.processTicksAndRejections (nodeinternal/process/task queues105:5) { importResults: [ { code: 400, document: '{"administrative_area_level_1":"Ontario","city":"Toronto","countryName":"CA","createdAt":1741982413,"copy":"This is the question copy","provinceOrState":"ON","id":"1Hzs4Ih9JvqAHisW4OBu"}', error: 'Field

uid

has been declared in the schema, but is not found in the document.', id: '1Hzs4Ih9JvqAHisW4OBu', success: false },"

Jason Bosco

03/26/2025, 9:59 PM

Could you now check in Firestore how that document with that ID looks?

Tim Stoute

03/26/2025, 10:01 PM

it's looks as I would expect in Firestore, with a bunch of fields, including a string field uid "9EbaPVXQv0PuJ4ET0Lynv9F06b52"

Jason Bosco

03/26/2025, 10:44 PM

Hmmm, I vaguely remember an issue like in an older version of Typesense where the validation error message returned was incorrect. Could you try upgrading to v28.0 and try this again? If the same error is still thrown, could you share that full document's JSON here?

Tim Stoute

03/26/2025, 10:49 PM

ok, I have changed my Cluster config to v28.0 and waiting for that configuration change to complete. Note that I had reverted from v28.0 to v27.1 to try to work around this issue prior to requesting help. Can I email you the entire json doc? If so, what email?

Jason Bosco

03/26/2025, 11:46 PM

You can email it to support at typesense dot org. Could you attach it as a .json file

Fanis Tharropoulos

03/27/2025, 8:15 AM

The id of the document that failed to import is always returned. Is there something else to be done to provide a better DX?

Tim Stoute

03/27/2025, 1:01 PM

@Jason Bosco I have emailed the json file. @Fanis Tharropoulos the document id is not in the log line that is generated for each document failure, for example: "Error: Error importing document with error: Field

uid

has been declared in the schema, but is not found in the document."

Fanis Tharropoulos

03/27/2025, 1:21 PM

"Import error in a batch of documents from 1Hzs4Ih9JvqAHisW4OBu to zwMyBjMXSv5XnkMvEh26 ImportError: 0 documents imported successfully, 15 documents failed during import. Use

error.importResults

uid

has been declared in the schema, but is not found in the document.', id: '1Hzs4Ih9JvqAHisW4OBu', success: false },"

Copy code

importResults: [
    {
      code: 400,
      document: '{"administrative_area_level_1":"Ontario","city":"Toronto","countryName":"CA","createdAt":1741982413,"copy":"This is the question copy","provinceOrState":"ON","id":"1Hzs4Ih9JvqAHisW4OBu"}', <-- Also here
      error: 'Field `uid` has been declared in the schema, but is not found in the document.',
      **id: '1Hzs4Ih9JvqAHisW4OBu',** <-- Is this not it?
      success: false
    },"

Tim Stoute

03/27/2025, 1:35 PM

ah ok, yes, I see it there - thanks

🙌 1

Tim Stoute

03/28/2025, 12:11 PM

was the json file helpful in determining why the typesense_sync is failing with errors on the uid field when syncing to the Firestore collection? ?

Fanis Tharropoulos

03/28/2025, 1:57 PM

I replied via email, there was a mis-configuration in terms of the document and the collection schema. Could you try again?

9 Views

Open in Slack

Previous Next