#community-help

Handling Collection Errors in HA Clusters

TLDR Mischa was encountering random collection existence errors when updating schemas in a HA cluster. Jason suggested waiting between deletions and recreations or using an alias for zero downtime switches.

Join the chat

Sep 18, 2023 (2 weeks ago)

Photo of md5-5faa26225a23ba2f1f83f2258d39478c

Mischa

05:21 PM

Why is it that when I run this, I randomly get an error telling me that one of my collections already exists?

export const recreateCollection = async (schema: CollectionCreateSchema) => {
  const client = await getTypesenseAdminClient()
  if (await collectionExists(client, schema)) {
    (`Deleting collection ${schema.name}`)
    await client.collections(schema.name).delete()
  }

  (`Creating collection ${schema.name}`)
  await client.collections().create(schema)
}

"httpStatus":409,"stack":["ObjectAlreadyExists: Request failed with HTTP code 409 | Server said: A collection with name xxxx already exists.

05:21

Mischa

05:21 PM

const collectionExists = async (client: Client, schema: CollectionCreateSchema) => {
  try {
    await client.collections(schema.name).retrieve()
    return true
  } catch (e) {
    if ((e as TypesenseError).httpStatus === 404) {
      return false
    } else {
      throw e
    }
  }
}

05:22

Mischa

05:22 PM

how else can I update my schema when there are breaking changes?

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

07:38 PM

If you're running a HA cluster, this might be due to eventual consistency. For eg, if the delete request is executed by one node and hasn't yet been executed by all nodes in the cluster, and in the meantime a create request makes it way to a different node, that will throw an error.

I'd recommend giving a few second between deleting and trying to recreate a collection with the same name.

07:40

Jason

07:40 PM

To answer your second question, you can either do an in-place change (only recommended in non-production envs since it is a blocking operation): https://typesense.org/docs/0.25.1/api/collections.html#modifying-an-existing-field

or use an alias: https://typesense.org/docs/0.25.1/api/collections.html#using-an-alias

Sep 20, 2023 (1 week ago)

Mischa

04:21 AM

i tried automating modifying existing fields and it was a nightmare

04:22

Mischa

04:22 AM

i'm okay with deleting and recreating the schema but i don't really see how to do that on a HA cluster in an automated fashion. waiting a few seconds wasn't giving good results

Jason

04:25 AM

Using an alias will let you do a zero downtime switch from an old collection to a new collection: https://typesense.org/docs/0.25.1/api/collections.html#using-an-alias

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2786 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Issues with Duplicate Entries in New Collections

Ivan reported an issue wherein duplicate entries appear in his database collection. This occurs sporadically when he deletes a collection, creates a new one with the same name, and adds the same entries. Kishore Nallan identified this as a race condition, suggesting a larger timeout value to avoid premature disconnection. Unable to increase the timeout, Ivan instead reduced his import chunks to 500 entries, which successfully eliminated the error.

Issue with Embedding Error in Version 0.25.0.rc63

Bill reported a bug in version 0.25.0.rc63 regarding a problem with updating or emplacing a document and receiving an embedding error. This was resolved in version 0.25.0.rc65, but further discussion ensued regarding the function of 'index' in the update feature.

1mo

Issue with "Search Firestore with Typesense" Extension

Rajbek had an issue synchronizing data between Firestore and Typesense collection. Jason helped diagnose the issue and the problem was resolved by Rajbek increasing the memory allocation of the specific cloud function.

Discussing Firestore Extension and Collection Schema in Typesense

Loic asks about settings in Firestore Extension and experiences issues with the collection schema in Typesense. Discussion with Jason is ongoing.

Issue with Query Expectations on Typesense Search

Sean was having an issue with their search query on Typesense. Kishore Nallan suggested adjusting the 'drop_tokens_threshold' parameter. After making the adjustment, Sean found an issue with the order of the results, which was resolved by updating Typesense version.