#community-help

Updating Collections Strategy and Faceting New Field

TLDR Nithin asked about strategies for updating collections and faceting new fields. Kishore Nallan suggested creating another collection, indexing in the background and using aliases to switch live traffic over, and shared details about the upcoming release.

34mo

Join the chat

Apr 13, 2021 (34 months ago)

Photo of md5-843a1b2e5e652fcf41ea99f9e07f0d6c

Nithin

01:04 PM

Hi guys, I want to check on what is the recommended strategy for updating an existing collection ? Let's say we want to start faceting a new field in the documents? Deleting and recreating the collection with downtime seems too expensive.

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:06 PM

👋 As of now there is no way to modify the schema live, though that’s a very important todo item on our list. You can create another collection, index it in the background and use aliases to swap the live traffic over to that: https://typesense.org/docs/0.19.0/api/collection-alias.html#collection-alias

01:12

Kishore Nallan

01:12 PM

In the upcoming 0.20 release we are introducing automatic schema detection and support for regular expression in field names that will already help you work around this problem. For example you can define a field named .*_facet and define that to be facetable. So any field matching that regexp will be automatically added to facet index Nithin

01:13

Kishore Nallan

01:13 PM

We already have a RC build for 0.20 that I can share if you are interested.

Nithin

01:15 PM

Thanks Kishore, I am interested in trying it out.

Kishore Nallan

01:19 PM

Here’s the docker image: typesense/typesense:0.20.0.rc39

Just define a field like this:

{"name": ".*_str_facet", "type": "string", "facet": true}

So any field name ending with _str_facet will then be treated as a facetable field automatically. This lays the foundation for us to allow schema updates in future for non-dynamic fields from older versions of Typesense.

01:20

Kishore Nallan

01:20 PM

There is also full automatic type detection in the new version. WIP docs here for that: https://github.com/typesense/typesense-website/blob/775c22547727dee8bdae226a8dad90bb0e02df16/docs-site/content/0.20.0/api/collections.md#with[…]tion

Nithin

01:28 PM

Thanks, I am already using the rc, will give this a try.

02:01

Nithin

02:01 PM

Is it possible to search or facet using nested fields when using auto ?

Kishore Nallan

02:02 PM

We don’t support indexing and searching on nested fields. You need to flatten all nested fields to top-level fields using a customer.name kinda convention and then search on that. Again, this is something that’s on our roadmap to address.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3005 threads (79% resolved)

Join Our Community

Similar Threads

Discussions on Typesense, Collections, and Dynamic Fields

Tugay shares plans to use Typesense for their SaaS platform and asks about collection sizes and sharding. Jason clarifies Typesense's capabilities and shares a beta feature. They discuss using unique collections per customer and new improvements. Kishore Nallan and Gabe comment on threading and data protection respectively.

35mo

Faceting Objects in Typesense

John wants to facet objects in Typesense, but faces issues. Kishore Nallan suggests creating a feature request on GitHub.

9mo

Changing Facet Value of Indexed Collections

Roshan wanted to alter the facet value of fields after indexing. Kishore Nallan explained the current limitation, suggested pre-creation of collection before syncing. Faceted search is also encountered an unknown issue, Kishore Nallan advised to raise the issue in Github.

23mo

Cold Start Problem with Dynamic Collections

Adrian reported cold start issues with dynamic collections. Jason suggested using wildcard `*` for query_by parameters, upgrading to `0.25.0.rc34`, and clarified conventions. Adrian's issues were resolved but they reported a limitation that will potentially be addressed.

6mo

Release and Features of Typesense v0.20

Jason announced the Typesense v0.20 release and its new features. Mica inquired about the schema and was informed that new fields can be added without reindexing. Alex congratulated the team.

33mo