Switching from External to Auto Embedding: Pros and Cons

TLDR Somesh inquires about switching to auto embedding from external ones. Kishore Nallan explains that auto embedding quickens query time but slows down documents’ encoding during indexing.

Powered by Struct AI
Join the chat
Oct 21, 2023 (1 month ago)
Photo of md5-a9a351e11d64f05b41fec183816a0cda
12:03 PM
Hi,Just want to know that should I switch from external embeddings to auto-embedding,Is it faster or beneficial in anyway.The product that I am working on already uses external embeddings and I have been given the task to convert it to auto Embedding
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:46 PM
The major benefit of auto embedding is during query time. You avoid making an external api call to encode the query so it's faster.
Kishore Nallan
03:47 PM
The downside is that it's slower to encode documents during indexing compared with an external embedding end-point unless you use a GPU instance for Typesense -- that will speed up auto embedding generation significantly.