Troubleshooting 409 Errors with Firestore to Typesense Cloud Function
TLDR Orion encounters 409 errors with ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite
cloud function. Jason suggests possible solutions like querying Firestore on each change or tracking sync state in a collection. Both agreed on adding a config option. Orion proposed contributing a PR for the change.
1
1
1
Oct 21, 2022 (12 months ago)
Orion
01:54 PMext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite
cloud function. I don’t understand why this would throw an error as it shouldn’t matter if a document exists to update it… any idea what I’m doing wrong here?Jason
05:37 PMFeb 25, 2023 (7 months ago)
Orion
10:15 PMLog says "creating document" and not the usual "upserting" which I'm sure is part of this problem but its unclear how. This is the log from document creation:
{
"textPayload": "Creating document {\"measure_text_length\":343,\"rank\":0.5,\"text\":\"Indexers certainly have a very strong want for data determinism, they are constantly monitoring and worried about their data veracity. it's very time consuming\\nConsumers are also feeling this too since we've seen cases with \\\"wrong\\\" data returned by queries\\nBut no we have not had an attack like that yet that we know of, but what's your point?\",\"title\":\"slack\",\"type\":\"message\",\"url\":\"\",\"id\":\"aHR0cHM6Ly9ibG9ja3NjaWVuY2V0ZWFtLnNsYWNrLmNvbS9hcmNoaXZlcy9DMDM2TkFXUDBDVC9wMTY3NzI2MTc1MjM5NTI0OT9jaWQ9QzAzNk5BV1AwQ1QmdGhyZWFkX3RzPTE2NzYwNTk2NjEuMjQzMDg5\"}",
"insertId": "63f8fbbb0005cfd5c189c11f",
"resource": {
"type": "cloud_function",
"labels": {
"project_id": "knowledge-management-333914",
"region": "us-central1",
"function_name": "ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite"
}
},
"timestamp": "2023-02-24T18:02:35.380885Z",
"severity": "DEBUG",
"labels": {
"instance_id": "00c61b117c5a189878a95fae39681518ed338d4fa826a35986e68469d2ec5b3070fede155da6333715bb7dfc26d0c4c0f650d7a4a775945d9115",
"execution_id": "qu4q7bkf0lq7"
},
"logName": "projects/knowledge-management-333914/logs/cloudfunctions.googleapis.com%2Fcloud-functions",
"trace": "projects/knowledge-management-333914/traces/a9c2419e85dd694ae3fe11d14d31bebf",
"receiveTimestamp": "2023-02-24T18:02:35.518729400Z"
}
And this is the error:
{
"textPayload": "Error: Request failed with HTTP code 409 | Server said: A document with id aHR0cHM6Ly9ibG9ja3NjaWVuY2V0ZWFtLnNsYWNrLmNvbS9hcmNoaXZlcy9DMDM2TkFXUDBDVC9wMTY3NzI2MTc1MjM5NTI0OT9jaWQ9QzAzNk5BV1AwQ1QmdGhyZWFkX3RzPTE2NzYwNTk2NjEuMjQzMDg5 already exists.\n at ApiCall.customErrorForResponse (/workspace/node_modules/typesense/lib/Typesense/ApiCall.js:229:21)\n at ApiCall.performRequest (/workspace/node_modules/typesense/lib/Typesense/ApiCall.js:118:48)\n at processTicksAndRejections (internal/process/task_queues.js:95:5)",
"insertId": "63f8fbbb000bdb5feae60206",
"resource": {
"type": "cloud_function",
"labels": {
"project_id": "knowledge-management-333914",
"region": "us-central1",
"function_name": "ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite"
}
},
"timestamp": "2023-02-24T18:02:35.777055Z",
"severity": "ERROR",
"labels": {
"instance_id": "00c61b117c5a189878a95fae39681518ed338d4fa826a35986e68469d2ec5b3070fede155da6333715bb7dfc26d0c4c0f650d7a4a775945d9115",
"execution_id": "qu4q7bkf0lq7"
},
"logName": "projects/knowledge-management-333914/logs/cloudfunctions.googleapis.com%2Fcloud-functions",
"trace": "projects/knowledge-management-333914/traces/a9c2419e85dd694ae3fe11d14d31bebf",
"receiveTimestamp": "2023-02-24T18:02:35.852702927Z"
}
Feb 26, 2023 (7 months ago)
Jason
12:43 AMJason
12:43 AMOrion
12:48 AMOrion
12:49 AM{
"textPayload": "Upserting document {\"measure_text_length\":343,\"platform\":\"\",\"rank\":0.5,\"text\":\"Indexers certainly have a very strong want for data determinism, they are constantly monitoring and worried about their data veracity. it's very time consuming\\nConsumers are also feeling this too since we've seen cases with \\\"wrong\\\" data returned by queries\\nBut no we have not had an attack like that yet that we know of, but what's your point?\",\"title\":\"slack\",\"type\":\"message\",\"url\":\"\",\"id\":\"aHR0cHM6Ly9ibG9ja3NjaWVuY2V0ZWFtLnNsYWNrLmNvbS9hcmNoaXZlcy9DMDM2TkFXUDBDVC9wMTY3NzI2MTc1MjM5NTI0OT9jaWQ9QzAzNk5BV1AwQ1QmdGhyZWFkX3RzPTE2NzYwNTk2NjEuMjQzMDg5\"}",
"insertId": "63f8fbba00080662c369637e",
"resource": {
"type": "cloud_function",
"labels": {
"region": "us-central1",
"function_name": "ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite",
"project_id": "knowledge-management-333914"
}
},
"timestamp": "2023-02-24T18:02:34.525922Z",
"severity": "DEBUG",
"labels": {
"execution_id": "qu4q00yzm4k2",
"instance_id": "00c61b117c5a189878a95fae39681518ed338d4fa826a35986e68469d2ec5b3070fede155da6333715bb7dfc26d0c4c0f650d7a4a775945d9115"
},
"logName": "projects/knowledge-management-333914/logs/cloudfunctions.googleapis.com%2Fcloud-functions",
"trace": "projects/knowledge-management-333914/traces/88dc0c0598a37d19e7d0901e25dfeb9c",
"receiveTimestamp": "2023-02-24T18:02:34.855731622Z"
}
Orion
12:49 AMJason
12:50 AMJason
12:51 AMOrion
12:51 AMJason
12:53 AM> One potential solution to this could be to query Firestore on each change trigger and push the latest version of the Firestore document to Typesense, instead of using the snapshot document from the event
But from a cost perspective, do you think doing these reads for each Firestore write is reasonable? I guess it will essentially increase write costs by 33%?
Orion
12:56 AMOrion
12:57 AMJason
12:58 AMOrion
01:03 AM1. document added to firestore
2. TS update_or_create triggered -> TS document doesn't exist, creates it
3. document updated with "platform" field
4. TS update_or_create triggered -> TS document exists, updates it
Ahhh... I see what you mean now...
Orion
01:03 AMJason
01:03 AMOrion
01:08 AMJason
01:10 AMOrion
01:11 AMJason
01:11 AMOrion
01:12 AMOrion
01:13 AMJason
01:14 AMJason
01:15 AMOrion
01:22 AM• pull from firestore directly instead of using the snapshot -> increase write costs
• track sync state in a collection -> increased storage usage (and still need more than just the snapshot anyway? I guess this requires the first option too)
• periodically sync -> increased cost from queries and storage and unneeded periodic checks when no activity is happening?
Orion
01:23 AMOrion
01:23 AMJason
01:27 AM#2 also increases storage costs, so #1 might be better
#3 requires a completely different extension, since it doesn’t rely on triggers
Orion
01:28 AMOrion
01:29 AMOrion
01:31 AMJason
01:31 AMOrion
01:33 AMJason
01:34 AMJason
01:35 AMOrion
01:35 AMJason
01:36 AMOrion
01:37 AMJason
01:37 AM1
Jason
01:38 AM1
Orion
01:45 AMOrion
01:45 AMOrion
03:14 AMOrion
03:15 AMJason
09:08 PM1
Typesense
Indexed 2786 threads (79% resolved)
Similar Threads
Troubleshooting 400 Error When Upgrading Typesense Firestore Extension
Orion experienced a `400` error after updating the Typesense Firestore extension, causing issues with cloud functions. They traced the issue back to a data type conflict in their Typesense collection schema after updating. With help from Jason and Kishore Nallan, they resolved the issue by recreating the collection.
Handling Order of Firestore Events for Synchronization with Typesense.
Ross ran into an issue with Firestore events triggering out of order, causing synchronization inconsistency between Firestore and Typesense. With advice and input from Jason and Kishore Nallan, they implemented a debouncing solution using redis, ensuring that the latest Firestore data is synced to Typesense accurately.
Handling Kinesis Stream Event Batching with Typesense
Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.
High Cost Concerns with firestore-typesense-search Extension
Minyong is confused by the high costs of using the firestore-typesense-search extension and asks for ways to reduce them. Jason confirms that the extension does not batch requests and advises Minyong to consult Firebase support.
Syncing Firebase 'ref' Field with Typesense
Guillermo wanted to index a 'ref' field from firebase with typesense. Jason provided a solution which involved syncing `ref.path` as an addition to the existing fields. Guillermo confirmed the solution worked.