#community-help

Discussion on OpenAI Embeddings Response Time and Alternatives

TLDR Walter discusses the varying response time when using OpenAI embeddings. Jason confirms this issue and Manish suggests using the E5 model in Typesense RC as a better alternative.

3mo

Join the chat

Jun 28, 2023 (3 months ago)

Photo of md5-b0a343a23053bb091cc198f636ad4103

Walter

04:42 PM

thanks kishore for fixing that bug and creating a new release.

I've noticed that the response time of search requests can be anywhere from 1-4s when using the open ai embeddings. Has that been your experience as well?

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

04:50 PM

Unfortunately that’s what we’ve seen with a few other users as well. Sometimes even more - OpenAI API response times seem to vary widely

Walter

04:53 PM

Ah ok thanks. Do you know how vertex compares?

Jason

06:06 PM

Haven’t heard any feedback about Vertex

Photo of md5-f0a83cd20895941fd74c026f9f15b61f

Manish

07:05 PM

I switched from OpenAI embeddings to use the E5 model, that's in Typesense RC. Works much better, latency is really good. See https://threads.typesense.org/kb

Walter

07:22 PM

Thanks for your comment Manish that's really helpful.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2779 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Finding Similar Documents Using JSON and Embeddings

Manish wants to find similar JSON documents and asks for advice. Jason suggests using Sentence-BERT with vector query and provides guidance on working with OpenAI embeddings and Typesense. They discuss upcoming Typesense features and alternative models.

5mo

Optimum Cluster for 1M Documents with OpenAI Embedding

Denny inquired about the ideal cluster configuration for handling 1M documents with openAI embedding. Jason recommended a specific configuration, explained record size calculation, and clarified embedding generation speed factors and the conditions that trigger openAI.

1mo

Integrating OpenAI Embeddings with DocSearch Scraper

Marcos was looking for how to use OpenAI embeddings with DocSearch. Jason guided with an update to the scraper config, and suggested the GTE built-in model for generic use.