#community-help

Handling HTTP Connections during Schema Changes in Typesense

TLDR Michael asked about HTTP request behavior during schema changes and was reassured by Jason that interruption won't stop execution. They finally decided to reset the node to a clean slate.

Powered by Struct AI

3

1

Sep 14, 2023 (2 months ago)
Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
05:11 PM
Does the http request for updating a collection leave the connection open over the entire reindexing process? If so, what happens in case of a timeout/connection issue?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:15 PM
> Does the http request for updating a collection leave the connection open over the entire reindexing process
That's correct. Indexing is a synchronous process in Typesense.

> If so, what happens in case of a timeout/connection issue?
Typesense will still process all the data it has received up until the connection was terminated.
Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
05:40 PM
So if I'm making a schema change, say making an indexed field no longer indexed. If the connection is interrupted, will the collection's configuration settings reflect the old schema or the new one?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:31 PM
It will reflect the new one, after it completes executing it fully
06:32
Jason
06:32 PM
On a side note, for the size of your dataset, doing an in-place schema migration might be time-consuming. I would recommend following this approach instead: https://typesense.org/docs/0.25.1/api/collections.html#using-an-alias

1

Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
06:32 PM
So interrupting the http connection will not stop the execution?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:33 PM
Correct
Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
06:34 PM
Interesting, I might have accidentally kicked off the process twice
06:51
Michael
06:51 PM
There's no way to cancel these processes, right?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
06:53 PM
No unfortunately
06:54
Jason
06:54 PM
If you're still in dev mode, one thing we could do is clear all data in the node (including API keys), so it's a clean slate and you can create a new collection with the updated schema and reindex your data in it
Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
06:55 PM
Yea I think that makes sense
06:55
Michael
06:55 PM
Same cluster id right?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:44 PM
Yeah the cluster ID will still remain the same

1

07:45
Jason
07:45 PM
Could you confirm that you want to do this? And also confirm that the cluster ID starts with vfy...
Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
07:45 PM
Yes let's do it

1

Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:59 PM
You're all set
07:59
Jason
07:59 PM
Your cluster should be in a clean slate state now
Michael
Photo of md5-ca6495d5be926db80e09aabf066f4b8b
Michael
08:02 PM
Thanks a lot!

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Discussing Migration Handling for TypeSense

Jack asks how to handle "migrations" for TypeSense. Kishore Nallan recommends creating a new collection, reindexing data, then using the alias feature to swap traffic. Further clarification on blocking during schema alteration is provided.

1

11
2mo

Discussion About Typesense Nodes Not Synchronizing Correctly

Erick experienced an issue where documents weren't updated properly in a Typesense instance running on 3 nodes. Upon requesting debug logs and configs, Jason identified that these nodes weren't part of the same cluster. They couldn't resolve the nodes' failure to connect issue and recommended a fresh installation.

2

91
23mo

Handling Collection Errors in HA Clusters

Mischa was encountering random collection existence errors when updating schemas in a HA cluster. Jason suggested waiting between deletions and recreations or using an alias for zero downtime switches.

8
2mo

Flushing Queue for Schema Altering in Typesense

Dima experienced repeated schema alterations due to a misconfiguration, causing server timeouts. Kishore Nallan recommended increasing the timeout and adjusting max_retries to resolve the issue.

3

28
8mo

Typesense Bug Fix with `canceled_at` Field and Upgrade Concerns

Mateo reported an issue regarding the treatment of an optional field by Typesense which was confirmed a bug by Jason. After trying an upgrade, an error arose. Jason explained the bug was due to a recent change and proceeded to downgrade their version. Future upgrade protocols were discussed.

3

74
10mo