#community-help

Speeding up Document Indexing with Auto-Embedding

TLDR Somesh queries the long document indexing time using auto-embedding. Kishore Nallan confirms it's normal and suggests using an external service or running Typesense on GPU.

Join the chat

Nov 14, 2023 (2 weeks ago)

Photo of md5-a9a351e11d64f05b41fec183816a0cda

Somesh

11:32 AM

Hi,it's taking me 40-50 min to index documents using auto-embedding,is this normal?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

11:34 AM

Yes, unfortunately embedding generation is a slow process given the models used. To speed things up, you have to either use an external embedding service (which probably uses a GPU under the hood) or run Typesense itself on GPU.

Somesh

11:34 AM

My batch size is 320k and my import batch size is 20k

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Switching from External to Auto Embedding: Pros and Cons

Somesh inquires about switching to auto embedding from external ones. Kishore Nallan explains that auto embedding quickens query time but slows down documents’ encoding during indexing.

1mo

Optimum Cluster for 1M Documents with OpenAI Embedding

Denny inquired about the ideal cluster configuration for handling 1M documents with openAI embedding. Jason recommended a specific configuration, explained record size calculation, and clarified embedding generation speed factors and the conditions that trigger openAI.

3mo

Discussing Indexing and Embedding Performance in Typesense

Dima had queries about indexing with embedding in Typesense. Kishore Nallan and Jason provided solutions, including reducing documents sent in an API call and running embeddings on a GPU. They facilitated Dima with the latest RC.

4mo