#community-help

Handling Collection Errors in HA Clusters

TLDR Mischa was encountering random collection existence errors when updating schemas in a HA cluster. Jason suggested waiting between deletions and recreations or using an alias for zero downtime switches.

Powered by Struct AI
8
1w
Solved
Join the chat
Sep 18, 2023 (2 weeks ago)
Mischa
Photo of md5-5faa26225a23ba2f1f83f2258d39478c
Mischa
05:21 PM
Why is it that when I run this, I randomly get an error telling me that one of my collections already exists?

export const recreateCollection = async (schema: CollectionCreateSchema) => {
  const client = await getTypesenseAdminClient()
  if (await collectionExists(client, schema)) {
    (`Deleting collection ${schema.name}`)
    await client.collections(schema.name).delete()
  }

  (`Creating collection ${schema.name}`)
  await client.collections().create(schema)
}

"httpStatus":409,"stack":["ObjectAlreadyExists: Request failed with HTTP code 409 | Server said: A collection with name xxxx already exists.
05:21
Mischa
05:21 PM
const collectionExists = async (client: Client, schema: CollectionCreateSchema) => {
  try {
    await client.collections(schema.name).retrieve()
    return true
  } catch (e) {
    if ((e as TypesenseError).httpStatus === 404) {
      return false
    } else {
      throw e
    }
  }
}
05:22
Mischa
05:22 PM
how else can I update my schema when there are breaking changes?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:38 PM
If you're running a HA cluster, this might be due to eventual consistency. For eg, if the delete request is executed by one node and hasn't yet been executed by all nodes in the cluster, and in the meantime a create request makes it way to a different node, that will throw an error.

I'd recommend giving a few second between deleting and trying to recreate a collection with the same name.
07:40
Jason
07:40 PM
To answer your second question, you can either do an in-place change (only recommended in non-production envs since it is a blocking operation): https://typesense.org/docs/0.25.1/api/collections.html#modifying-an-existing-field

or use an alias: https://typesense.org/docs/0.25.1/api/collections.html#using-an-alias
Sep 20, 2023 (1 week ago)
Mischa
Photo of md5-5faa26225a23ba2f1f83f2258d39478c
Mischa
04:21 AM
i tried automating modifying existing fields and it was a nightmare
04:22
Mischa
04:22 AM
i'm okay with deleting and recreating the schema but i don't really see how to do that on a HA cluster in an automated fashion. waiting a few seconds wasn't giving good results
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:25 AM
Using an alias will let you do a zero downtime switch from an old collection to a new collection: https://typesense.org/docs/0.25.1/api/collections.html#using-an-alias