Issue with Inbuilt Model for Creating Embeddings
TLDR Shikhar experiences issues generating embeddings for all documents using an in-built model. Kishore Nallan suggests a re-run or trying to debug the issue, but the problem remains unresolved.
Aug 16, 2023 (3 months ago)
Shikhar
04:15 AMKishore Nallan
04:56 AMShikhar
05:14 AMShikhar
05:14 AMKishore Nallan
05:18 AMShikhar
08:38 AMShikhar
08:40 AMand some do not?
Kishore Nallan
08:57 AMShikhar
09:02 AMKishore Nallan
09:09 AMTypesense
Indexed 3015 threads (79% resolved)
Similar Threads
Issues with Embeddings on Collection with 80K Documents
Samuel experienced issues when enabling embeddings on a large collection, leading to an unhealthy cluster. Kishore Nallan suggested rolling back to a previous snapshot, advised on memory calculations for OpenAI embeddings, and confirmed that creating a new cluster should solve the problem.
Issues with Cluster Upgrade and Embedding Field
Gustavo had issues upgrading their cluster and their embedding field wasn't being filled. Jason helped to solve the upgrade issue and advised re-indexing the documents to solve the embedding field issue. Both problems were successfully resolved.
Discussing Indexing and Embedding Performance in Typesense
Dima had queries about indexing with embedding in Typesense. Kishore Nallan and Jason provided solutions, including reducing documents sent in an API call and running embeddings on a GPU. They facilitated Dima with the latest RC.
Utilizing Vector Search and Word Embeddings for Comprehensive Search in Typesense
Bill sought clarification on using vector search with multiple word embeddings in Typesense and using them instead of OpenAI's embedding. Kishore Nallan and Jason informed him that their development version 0.25 supports open source embedding models. They also resolved Bill's concerns regarding search performance, language support, and limitations in the search parameters.
Optimum Cluster for 1M Documents with OpenAI Embedding
Denny inquired about the ideal cluster configuration for handling 1M documents with openAI embedding. Jason recommended a specific configuration, explained record size calculation, and clarified embedding generation speed factors and the conditions that trigger openAI.