#community-help

Handling 503 Errors and Reducing Cluster Pressure

TLDR Gustavo faced frequent 503 errors despite having ample memory. Jason suggested handling 503s per Typesense documentation and adjusting settings to reduce unnecessary upserts and cluster pressure.

Powered by Struct AI

1

8
4mo
Solved
Join the chat
Jun 14, 2023 (4 months ago)
Gustavo
Photo of md5-da3527dcf5d076c2193d90ba6a2964bb
Gustavo
10:01 PM
I'm getting error 503 Not Ready or Lagging for almost all my requests since 2 hours ago. I have almost 2x the memory being used. Isn't that enough?
Image 1 for I'm getting error `503 Not Ready or Lagging` for almost all my requests since 2 hours ago. I have almost 2x the memory being used. Isn't that enough?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
10:06 PM
May I know your cluster id?
Gustavo
Photo of md5-da3527dcf5d076c2193d90ba6a2964bb
Gustavo
10:09 PM
v601y2x3upjea4tip
10:10
Gustavo
10:10 PM
This is from Google Cloud Logging. You can see the moment when it started to have errors.
Image 1 for This is from Google Cloud Logging. You can see the moment when it started to have errors.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
10:11 PM
I see writes coming in regularly into the cluster. So the 503 you see is Typesense’s backpressure mechanism to make sure CPU is reserved for searches.

Here’s how to handle 503s: https://typesense.org/docs/guide/syncing-data-into-typesense.html#handling-http-503s
Gustavo
Photo of md5-da3527dcf5d076c2193d90ba6a2964bb
Gustavo
10:12 PM
Hum, I'm probably performing some useless upserts that don't change the data but put pressure into the cluster.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
10:12 PM
If you’re using the python client, you want to increase connection_timeout_seconds and retry_interval_seconds to as high as say 10 minutes and 5 minutes each

https://github.com/typesense/typesense-python/blob/765c99668bd9832a9dd523405bc54444b980c475/typesense/configuration.py#L39-L41
Gustavo
Photo of md5-da3527dcf5d076c2193d90ba6a2964bb
Gustavo
10:13 PM
Got it. I'll try to prevent some upserts that don't change data and then increase those settings.

1