#community-help

Ensuring Required Collections at Application Startup

TLDR KS needed a way to ensure required collections are created at application startup. Dima suggested using a script that checks current collections and compares them to a yaml file, and then sending requests to the API, as well as using Typesense PATCH API with "drop": true to delete fields.

Powered by Struct AI

2

1

11
5mo
Solved
Join the chat
Jun 19, 2023 (5 months ago)
KS
Photo of md5-9cc02c1d61e365404a2a5a7cf385b98b
KS
02:59 PM
Since creating collections is not idempotent, is there a recommended way to ensure required collections are created at application startup, sort of like a schema migration? I can check all the collections that currently exist and create new ones but for updating collection this is a little problematic. Wish there was an upsert for collections.
Dima
Photo of md5-1b62114a658b760944aa7d2b4c274460
Dima
03:03 PM
We wrote a small script, which checks current collections, schemas, synonyms, overrides and api-keys, compares them to yaml file and then sends requests to API
KS
Photo of md5-9cc02c1d61e365404a2a5a7cf385b98b
KS
03:04 PM
For comparing schema changes, do you persist the old schema somewhere or is this done during build/deploy time?
03:05
KS
03:05 PM
Or versioned in code, maybe?
Dima
Photo of md5-1b62114a658b760944aa7d2b4c274460
Dima
03:05 PM
You can retrieve current schema from Typesense API. Main pain is that fields are immutable, so if you’re adding some parameter to field you have to delete and create it in one API call. Also collections have some immutable fields too
03:05
Dima
03:05 PM
We have VCS for this yaml file, yes. We run this script after running typesense (through kubernetes job in one helm chart)

1

KS
Photo of md5-9cc02c1d61e365404a2a5a7cf385b98b
KS
03:07 PM
Delete the collection or the field? Is it possible to just delete a field? Can’t find such a thing in the API.
03:09
Dima
03:09 PM
You can call PATCH API with "drop": true and it will delete a field

1

KS
Photo of md5-9cc02c1d61e365404a2a5a7cf385b98b
KS
03:10 PM
Got it, thanks for the doc, I think this should help me tackle this!
Dima
Photo of md5-1b62114a658b760944aa7d2b4c274460
Dima
03:11 PM
Also this terraform provider can be useful: https://github.com/Kekenika/terraform-provider-typesense

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3005 threads (79% resolved)

Join Our Community

Similar Threads

Discussing Features for Typesense Cloud Dashboard

Stephano expressed the need for a Typesense Cloud dashboard to manage collections. Jason confirmed its development, also including features like editing and deleting items. James and Mica added comments about API key management and self-hosted Typesense compatibility.

10

50
32mo
Solved

Issue with Typesense Schema and Ruby Client

Mateo faces issues while creating a schema and using Ruby client for Typesense. Jason suggests using a new field instead of 'id' and provides assistance for Ruby client errors.

33
10mo

Guide on Updating Collections in Application Development

Ajeet requested guidance on updating collections for an application, citing issues like lack of schema validation. Jason suggested using regex for field restriction and confirmed collection alias functionality. CaptainCodeman shared a plan to use version numbers for collections avoiding delays.

3
23mo
Solved

Updating Collection Schema and Querying Unspecified Fields in Typesense

Stefan asked about updating a collection schema. Kishore Nallan stated it's not currently possible but suggested automatic schema detection in version 0.20. Rishabh sought a workaround and Jason suggested creating a new collection with a new schema and re-indexing the data. Rishabh also asked about defining custom field IDs which Jason confirmed is not possible with Typesense.

9
34mo
Solved

Methods for Fetching, Querying, and Modifying Collections in Typesense

Bill inquired about performing OR queries, querying empty arrays and modifying collections in Typesense. Kishore Nallan explained the current limitations and provided workarounds and recommendations for each case. The conversation also touched upon the usage of cache in Typesense and the workings of the _eval function.

5

52
10mo
Solved