#community-help

Issue with "Search Firestore with Typesense" Extension

TLDR Rajbek had an issue synchronizing data between Firestore and Typesense collection. Jason helped diagnose the issue and the problem was resolved by Rajbek increasing the memory allocation of the specific cloud function.

Join the chat

Sep 17, 2023 (1 week ago)

Photo of md5-fa2b564839c22b335c78e8bf9f3f7c67

Rajbek

03:19 AM

Hello everyone! I have an issue with "Search Firestore with Typesense" extension. I have succesfully installed and synchronized Typesense cloud cluster collection with my Firestore collection. However, the number of documents after backfill hits into 1000 ceil. Now after adding new docs into my Firestore collection the number stuck at 1016 docs. I see the logs of "ext-firestore-typesense-search-indexToTypesenseOnFirestoreWrite" cloud function after manually updating a random doc in my Firestore collection and it seems fine "Function execution took 412 ms, finished with status: 'ok'". But the updated document is still abscent in the Typesense cloud collection with the total ceil of 1016.

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

03:23 AM

Could you post the complete logs from Firestore? Even if the function execution has completed, it might have logged errors before that?

Rajbek

03:25 AM

Thank you, Sir! That's the last logs

03:27

Rajbek

03:27 AM

Is csv ok? I can send the same info in JSON.

03:39

Rajbek

03:39 AM

Initially, I tested the extension only with one field. First, I didn't have such an issue with the number of docs in the Typsesense collection. After, I reinvented my Firestore collection scheme, deleted the extension in the Firebase and the collection in the Typesense cloud.

03:39

Rajbek

03:39 AM

Then, I installed the both agagin and now I have the issue.

03:41

Rajbek

03:41 AM

In the first try I had a field "id" in my Firestore collection, after I deceided to rename it into "registerID" to not mess with the typical "id" field of the Typsense.

Jason

03:45 AM

There should be logs for another backfill function, could you share those? Since the 1k doc issue is with the backfill right?

Rajbek

03:50 AM

Sure, one moment please🙏 But what about not upserting/synchronising doc after updating it in the firestore?

03:57

Rajbek

03:57 AM

ext-firestore-typesense-search-backfillToTypesenseFromFirestore last logs

10:29

Rajbek

10:29 AM

Jason Thank you. The issue is resolved. As you noticed the problem was indeed with the backfill. I increased the memory of the cloud function "ext-firestore-typesense-search-backfillToTypesenseFromFirestore" from 2Gb to 4Gb.🙂👍

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2776 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Discussing Firestore Extension and Collection Schema in Typesense

Loic asks about settings in Firestore Extension and experiences issues with the collection schema in Typesense. Discussion with Jason is ongoing.

Issue with Query Expectations on Typesense Search

Sean was having an issue with their search query on Typesense. Kishore Nallan suggested adjusting the 'drop_tokens_threshold' parameter. After making the adjustment, Sean found an issue with the order of the results, which was resolved by updating Typesense version.

Issues with Semantic Search in OpenAI Embeddings

Semyon was having issues while creating a schema for semantic search with OpenAI embeddings. Jason gave multiple suggestions for troubleshooting but none of them worked. The error was narrowed down to possibly being related to Semyon's specific OpenAI account or the OpenAI API key. The thread ended with Jason suggesting Semyon to check billing and make a direct API call to OpenAI.

Issues with Typesense Repo and Version Upgrade

Ankit reports a problem with the typesense repo which Jason and Kishore Nallan promptly address and fix. However, the same user encounters an issue when upgrading, making the server status "Not ready or lagging". The process is ongoing.

Resolving Issues While Upgrading Typesense Server

Ayush had queries about upgrading their Typesense server and updating collections using auto-embeddings. Kishore Nallan clarified all issues, but later discovered that reindexing of non auto-embedded vector fields only is currently supported during altering.