#community-help

Airbyte Timeout Issue with Typesense Sync

TLDR Jamshid experienced timeouts when syncing MySQL records with Airbyte. Jason suggested it's related to an Airbyte issue, leading Jamshid to create a custom Typesense connector to resolve the problem.

Powered by Struct AI
+11
6
3mo
Solved
Join the chat
Jun 16, 2023 (3 months ago)
Jamshid
Photo of md5-317ed6510eb7587c9e9243fb6ebc4e87
Jamshid
06:53 PM
We have a table on MySQL that has around 18K records. We are syncing that to typesense through Airbyte. In that table, there are multiple json fields and when all are selected to sync, Airbyte constantly fails. In the logs it says timeout occurred on the destination side.

Our typesense cluster is 0.5 GB RAM, 2 vCPUs, Regular SSD, Single Node, Single Region.

I was wondering if these timeouts are because of our typesense resources.

Any thoughts?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
07:50 PM
It’s most likely related to this: https://github.com/airbytehq/airbyte/issues/24097
07:50
Jason
07:50 PM
The default timeout in the python client (which is what AirByte uses) is low, and has to be increased
07:51
Jason
07:51 PM
I would recommend following up with AirByte about that issue
Jamshid
Photo of md5-317ed6510eb7587c9e9243fb6ebc4e87
Jamshid
10:08 PM
Thanks. That makes sense. Since you shared the link, there is some comments there.
Jun 20, 2023 (3 months ago)
Jamshid
Photo of md5-317ed6510eb7587c9e9243fb6ebc4e87
Jamshid
10:14 PM
Okay I made my custom typesense connector and started using it. Now I don’t get timeouts https://hub.docker.com/r/jamshidhashimi/typesense
+11