Issues with Cluster Upgrade and Embedding Field
TLDR Gustavo had issues upgrading their cluster and their embedding field wasn't being filled. Jason helped to solve the upgrade issue and advised re-indexing the documents to solve the embedding field issue. Both problems were successfully resolved.


Aug 02, 2023 (1 month ago)
Gustavo
07:14 PMJason
07:23 PMGustavo
07:23 PMGustavo
07:24 PMJason
07:24 PMJason
07:24 PMGustavo
07:25 PMJason
07:26 PMGustavo
07:26 PMGustavo
07:28 PMembedding
field is computed from the topics
field which is an array of strings. After looking for a bug in my search functionality, I noted that a document in Typesense didn't have the embedding
field when I retrived it with the SDK.Gustavo
07:29 PMGustavo
07:33 PMJason
07:34 PM
Gustavo
07:48 PMposts-v1
collection? There's a document there with ID z3xMqOyt8d0xdTeEaPyb
that contains the topics
field, but not the embedding
.Gustavo
07:48 PMv601y2x3upjea4tip
Jason
08:01 PMGustavo
08:02 PMJason
08:13 PM
Aug 03, 2023 (1 month ago)
Jason
04:10 AMJason
04:10 AMJason
04:11 AMGustavo
10:22 AMJason
02:32 PMHowever builds after that have a different bug that’s causing your particular dataset to not load in them. So I’d recommend not upgrading up from rc53 for now. We’re working on the fix.
Gustavo
04:20 PMJason
04:21 PMJason
04:21 PMJason
04:24 PM
Gustavo
04:26 PMJason
04:31 PMGustavo
05:01 PMJason
05:48 PMGustavo
06:11 PM
Gustavo
08:00 PM
Aug 14, 2023 (1 month ago)
Gustavo
10:16 PMJason
10:17 PMGustavo
10:19 PMgZKGF5rItp8UWLdmKzIg
: had the problem, but I deleted and recreated, so now it's OKAbHg5oQacJ1Ufw9JjI9v
: still missing the embedding fieldJason
10:24 PMcreatedAt
timestamp), but the cluster was upgraded to rc59 only on Aug 3. So this is an old document that was created on a prior RC build which didn’t have the error handling for this particular case.See this: https://typesense-community.slack.com/archives/C01P749MET0/p1691079700751459?thread_ts=1691003672.362599&cid=C01P749MET0
Jason
10:26 PMGustavo
10:28 PMGustavo
10:28 PMJason
10:29 PMBest to just delete the whole collection and recreate it
Gustavo
10:30 PMGustavo
11:01 PMGustavo
11:01 PMGustavo
11:01 PMJason
11:01 PMGustavo
11:01 PMGustavo
11:02 PMJason
11:02 PMGustavo
11:02 PMGustavo
11:04 PM
Aug 16, 2023 (1 month ago)
Gustavo
04:46 PMtopics
array. I have a document with: id: 'WtSy5f9KgdvxGB0i6HGz',
topics: [
'Direito Trabalhista',
'Direito Previdenciário',
'Benefícios trabalhistas',
'Auxílio maternidade',
'Licença maternidade'
]
Weirdly, when I make a semantic search with "Auxílio maternidade", it doesn't appear.
Gustavo
04:47 PMtopics
doesn't include it? I wonder if there's something wrong with the embedding itself.Gustavo
04:48 PM "embed": {
"from": [
"topics"
],
"model_config": {
"api_key": "...",
"model_name": "openai/text-embedding-ada-002"
}
},
"facet": false,
"index": true,
"infix": false,
"locale": "",
"name": "embedding",
"num_dim": 1536,
"optional": false,
"sort": false,
"type": "float[]"
},
Gustavo
04:48 PMlocale
because the topics
include diacritics?Gustavo
04:55 PMtopics: ['Software development']
just to make it have a topic that's different from all other documents in the collection. I search for "Software development" and it still doesn't appear in the first 50 results. There's no other document with anything even close to that topic.Gustavo
04:56 PMJason
04:57 PMJason
04:57 PMquery_by: topics,embedding
Gustavo
04:59 PMGustavo
05:00 PMGustavo
05:01 PMJason
05:02 PMGustavo
05:02 PMGustavo
05:02 PMJason
05:02 PMGustavo
05:03 PMtopics
field is different.
Gustavo
05:05 PMtopics
is the same as how it was before I manually changed it to test.Gustavo
05:06 PMSame topics, same search.
Before: didn't appear in the results.
Now: appears normally.
Gustavo
05:07 PMJason
05:11 PM
Typesense
Indexed 2764 threads (79% resolved)
Similar Threads
Utilizing Vector Search and Word Embeddings for Comprehensive Search in Typesense
Bill sought clarification on using vector search with multiple word embeddings in Typesense and using them instead of OpenAI's embedding. Kishore Nallan and Jason informed him that their development version 0.25 supports open source embedding models. They also resolved Bill's concerns regarding search performance, language support, and limitations in the search parameters.



Troubleshooting Typesense Node Embedding Field Error after Upgrade
Denny was having an error with the embedding field after upgrading a typesense node. Jason revealed that this was a bug in the v0.25.0 version. They suggested upgrading to 0.25.1.rc4 to fix the issue, which Denny did successfully.

Handling Kinesis Stream Event Batching with Typesense
Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.


Troubleshooting 400 Error When Upgrading Typesense Firestore Extension
Orion experienced a `400` error after updating the Typesense Firestore extension, causing issues with cloud functions. They traced the issue back to a data type conflict in their Typesense collection schema after updating. With help from Jason and Kishore Nallan, they resolved the issue by recreating the collection.



Discussing Indexing and Embedding Performance in Typesense
Dima had queries about indexing with embedding in Typesense. Kishore Nallan and Jason provided solutions, including reducing documents sent in an API call and running embeddings on a GPU. They facilitated Dima with the latest RC.
