#community-help

Optimum Cluster for 1M Documents with OpenAI Embedding

TLDR Denny inquired about the ideal cluster configuration for handling 1M documents with openAI embedding. Jason recommended a specific configuration, explained record size calculation, and clarified embedding generation speed factors and the conditions that trigger openAI.

Powered by Struct AI

2

12
3mo
Solved
Join the chat
Sep 01, 2023 (3 months ago)
Denny
Photo of md5-9ad0e03e3fe5fac2f75e521afcf2a7d7
Denny
12:37 AM
Hi - If I have about 1M documents (using embedding with openAI), what cluster should I be using? Approx 2 request per second or ~160k per day.
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
12:38 AM
The vector search section under here gives you a formula for calculating RAM usage: https://typesense.org/docs/guide/system-requirements.html#choosing-ram
Denny
Photo of md5-9ad0e03e3fe5fac2f75e521afcf2a7d7
Denny
12:43 AM
thanks for checking!

1

01:17
Denny
01:17 AM
how do i check the size of each record?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
01:18 AM
For keyword search: https://typesense.helpscoutdocs.com/article/161-how-do-i-calculate-average-record-size

For vector search, it's just number of dimensions * 7 bytes
Denny
Photo of md5-9ad0e03e3fe5fac2f75e521afcf2a7d7
Denny
01:33 AM
Will increasing the size of my cluster make embedding generation faster?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:49 AM
If you’re using OpenAI, embedding generation speed is completely dependent on OpenAI api response times, which I’ve seen take a second or two for each api call.

If you’re using a built-in model, then enabling GPU Acceleration will speed up the embedding generation process: https://typesense.helpscoutdocs.com/article/174-gpu-acceleration
Denny
Photo of md5-9ad0e03e3fe5fac2f75e521afcf2a7d7
Denny
02:57 AM
Ah I see… I’ve noticed the syncing is a lot slower due to this.
02:57
Denny
02:57 AM
When I update a document, using upsert, will it trigger OpenAI?
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:58 AM
Only if the value for the embed.from fields specified in the schema change
02:59
Jason
02:59 AM
Btw, you want to make sure you use the import endpoint with as many as 1K documents per API call, so the call to openai is also a bulk embedding call

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Issues with Embeddings on Collection with 80K Documents

Samuel experienced issues when enabling embeddings on a large collection, leading to an unhealthy cluster. Kishore Nallan suggested rolling back to a previous snapshot, advised on memory calculations for OpenAI embeddings, and confirmed that creating a new cluster should solve the problem.

1

39
1w
Solved

Integrating OpenAI Embeddings with DocSearch Scraper

Marcos was looking for how to use OpenAI embeddings with DocSearch. Jason guided with an update to the scraper config, and suggested the GTE built-in model for generic use.

10
3mo
Solved

Utilizing Vector Search and Word Embeddings for Comprehensive Search in Typesense

Bill sought clarification on using vector search with multiple word embeddings in Typesense and using them instead of OpenAI's embedding. Kishore Nallan and Jason informed him that their development version 0.25 supports open source embedding models. They also resolved Bill's concerns regarding search performance, language support, and limitations in the search parameters.

11

225
4mo
Solved

Finding Similar Documents Using JSON and Embeddings

Manish wants to find similar JSON documents and asks for advice. Jason suggests using Sentence-BERT with vector query and provides guidance on working with OpenAI embeddings and Typesense. They discuss upcoming Typesense features and alternative models.

8

64
7mo
Solved

Discussion on Performance and Scalability for Multiple Term Search

Bill asks the best way for multi-term searches in a recommendation system they developed. Kishore Nallan suggested using embeddings and remote embedder or storing and averaging vectors. Despite testing several suggested solutions, Bill continued to face performance issues, leading to unresolved discussions about scalability and recommendation system performance.

3

105
1w