#community-help

Switching from External to Auto Embedding: Pros and Cons

TLDR Somesh inquires about switching to auto embedding from external ones. Kishore Nallan explains that auto embedding quickens query time but slows down documents’ encoding during indexing.

1mo

Join the chat

Oct 21, 2023 (1 month ago)

Photo of md5-a9a351e11d64f05b41fec183816a0cda

Somesh

12:03 PM

Hi,Just want to know that should I switch from external embeddings to auto-embedding,Is it faster or beneficial in anyway.The product that I am working on already uses external embeddings and I have been given the task to convert it to auto Embedding

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:46 PM

The major benefit of auto embedding is during query time. You avoid making an external api call to encode the query so it's faster.

03:47

Kishore Nallan

03:47 PM

The downside is that it's slower to encode documents during indexing compared with an external embedding end-point unless you use a GPU instance for Typesense -- that will speed up auto embedding generation significantly.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Speeding up Document Indexing with Auto-Embedding

Somesh queries the long document indexing time using auto-embedding. Kishore Nallan confirms it's normal and suggests using an external service or running Typesense on GPU.

Discussing Indexing and Embedding Performance in Typesense

Dima had queries about indexing with embedding in Typesense. Kishore Nallan and Jason provided solutions, including reducing documents sent in an API call and running embeddings on a GPU. They facilitated Dima with the latest RC.

4mo

Optimum Cluster for 1M Documents with OpenAI Embedding

Denny inquired about the ideal cluster configuration for handling 1M documents with openAI embedding. Jason recommended a specific configuration, explained record size calculation, and clarified embedding generation speed factors and the conditions that trigger openAI.

3mo