#community-help

Adding New Fields to Items in Typesense Without Wiping Data

TLDR Alex wanted to add more fields to their Typesense items without wiping data. Kishore Nallan explained they had to create a new collection instead, and suggested using automatic schema detection for future changes. They also discussed potential downsides of indexing every field.

Powered by Struct AI
white_check_mark1
7
27mo
Solved
Join the chat
Jun 02, 2021 (27 months ago)
Alex
Photo of md5-faf0fdba0b6739a6706f05c15b6738c6
Alex
08:50 AM
Hey. We would like to add some more fields to our items in typesense. We also want these fields indexed. Our typesense server is running in production, so we don’t wanna wipe all data to make this change. Is it possible to add new fields (indexed ) to the schema and our items on the fly?
08:56
Alex
08:56 AM
From what i can read from similar asked question, we have to make a new collection, is that the only way? This means, we must upgrade our Typesense Cloud memory to do this. We would like to have no downtime for this change.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
08:56 AM
👋 Alex We don't have a way to modify the schema at the moment but that's on our backlog. You can just create a new collection and swap to it. The Collection Alias features makes this really easy to do without updating your app configuration: https://typesense.org/docs/0.20.0/api/collection-alias.html
08:59
Kishore Nallan
08:59 AM
For now, if your cluster is also not having enough memory, then you have to index on another cluster and swap over.

For future use: you can use automatic schema detection to help with schema changes. For e.g you can have a collection with a field defined as: ```{"name": ".*_int32", "type": "int32"}``` This way, you can easily add a new field called age_int32` and this will be automatically be inferred and indexed by Typesense.
09:00
Kishore Nallan
09:00 AM
Automatic schema detection is available on v0.20.0
Alex
Photo of md5-faf0fdba0b6739a6706f05c15b6738c6
Alex
09:08 AM
Thanks Kishore!
Do you see any downsides to adding:
{"name": ".*", "type": "auto" },
{"name": ".*_facet", "type": "auto", "facet": true }

to our schema? To make sure we never have to swap clusters again.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:12 AM
The only downside is that every field present in the document will be indexed. Using something like .*_int32 gives you flexibility. If you have two integer fields only one of which must be indexed, then suffix the indexable field name with _int32 but for the other field name don't do that, so it is only stored and not indexed.
white_check_mark1