#community-help

AWS Typesense Cloud Performance Troubles

TLDR Todd experienced latency issues with Typesense Cloud performance on AWS. Jason suggested testing the same request via ab and checking the average response times to help diagnose the problem.

14mo

Join the chat

Aug 26, 2022 (14 months ago)

Photo of md5-cccf0b87668408fef09dd77e1948fced

Todd

07:16 PM

Hey all! So, we’re using Typesense Cloud with AWS, and we’ve just gotten opentelemetry rolling, and we’re using it to fine tune our performance. However, we’re seeing some performance hiccups that we’d really like to resolve.

So I’ve included a segment of our trace that basically is just a typesense query (1.2 seconds). However, when I look in Typesense cloud at the max search latency, I see a much, much lower value (255 ms). I should mention we run a second search immediately afterwards, and it doesn’t have this discrepency at all (259 ms).

We’ve made sure that both our AWS region and our Typesense regions are equivalent (Oregon), but we’re still seeing roughly 1000ms of delay on our first search.

Basically, is there anything we can change about our configuration to reduce this. Can we give the Typesense cloud access to our VPC and use internal URLs, or anything like that? Is there more information I can provide? Thank you

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

07:21 PM

The search latency graph in Typesense Cloud shows the search processing time for a single query and excludes network latency. Also, if you’re sending a multi_search request (which based on the screenshots you’re not), the dashboard will still show the latency of each search within the multi_search request.

07:22

Jason

07:22 PM

I’m surprised the network latency is this high in the same region.

07:22

Jason

07:22 PM

Could you try making the same request via say ab from a server running in Oregon to Typesense Cloud, and post the output here?

07:22

Jason

07:22 PM

Trying to see what the avg response times look like, or if this was a blip

Todd

07:27 PM

Jason - That’s a great question. I’ll post data here when I get it. First time using ab, but it doesn’t look too bad.

09:46

Todd

09:46 PM

Hm, I’m at a bit of a loss I guess. When I run these requests via ab in our Jumphost instance, requests are running consistently at < 50 ms time. I checked on curl and had issues, so I switched to curl. Requests are clocking in around 100 to 150 ms total time. I’m at a bit of a loss for why these times would be so much longer in ECS or NodeJS at random times.

Aug 28, 2022 (14 months ago)

Jason

06:40 PM

I wonder if there’s some network warmup time in ECS?

06:41

Jason

06:41 PM

Also, may be try allocating more CPU to ECS?

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2776 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Optimizing Typesense Implementation for Large Collections

Oskar faced performance issues with his document collection in Typesense due to filter additions. Jason suggested trying a newer Typesense build and potentially partitioning the data into country-wise collections. They also discussed reducing network latency with CDN solutions.

8mo

Troubleshooting Write Timeouts in Typesense with Large CSVs

Agustin had issues with Typesense getting write timeouts while loading large CSV files. Kishore Nallan suggested chunking data or converting to JSONL before loading. Through troubleshooting, they identified a possible network problem at AWS and found a workaround.

29mo

Network Errors and Duplicate Requests in Typesense

Ethan and Rushil were experiencing more than 300 network errors and duplicate requests from Typesense. After rigorous troubleshooting, Jason identified the issue as an Instantsearch-induced multiple re-rendering and implemented a fix. Multiple requests reduced, but networking errors persisted, suggested to be due to client-side internet connectivity.

2mo

Stability Issues with Self-Hosted Typesense Server on AWS ECS

Ryan reports stability issues with a self-hosted typesense server, whilst Jason suggests various potential factors to consider, including configuration, CPU allocation, modifying the healthcheck, and testing on EC2 instances. However, no definitive solution is reached.

1mo

Discussing Typesense Search Request Performance

Al experienced longer-than-reported times for Typesense search requests, sparking a detailed examination of json parsing, response times and data transfer. Jason and Kishore Nallan helped solve the issue.

30mo