Hi team! I’m currently testing built-in embedding ...
# community-helpd
Dima
07/18/2023, 8:41 AMHi team! I’m currently testing built-in embedding feature from the alpha doc. Do you support indexing with embedding for updating imports, for instance with
emplacek
Kishore Nallan
07/18/2023, 8:42 AMYes updates should be handled.
d
Dima
07/18/2023, 2:04 PMGreat, it’s working now, I did something wrong on the first attempt 🙃
But I faced with another problem: I ran re-indexing and found that while in-build embedding works on indexing it affects search performance. It is totally fine and expected, so I increased available CPU on the node 2 -> 4 -> 8 -> 16, but it’s still not enough. Can I decrease concurrency of embedding / indexing somehow? For now I’m sending 500 rows in one batch, maybe I should decrease this amount?
j
Jason Bosco
07/18/2023, 3:30 PMYeah reducing the number of documents you send in an import API call, and pacing them out would be the way to reduce indexing concurrency
Jason Bosco
07/18/2023, 3:32 PMOn a side note, if you have over say 100K documents, I would recommend running the embedding on a GPU, which speeds up embedding generation for local models significantly
Jason Bosco
07/18/2023, 3:33 PMGPU support is already in the latest build, but we need to put together docs on how to set it up, since it unfortunately involves setting external dependencies like CUDA (too large to bundle within Typesense)
d
Dima
07/18/2023, 3:39 PMGot it, thank you!
Dima
07/18/2023, 5:04 PMAnd one last question — does built-in embedding save into snapshot? Or all content have to be reindexed and embedded after every restart?
j
Jason Bosco
07/18/2023, 5:05 PMOnce the embeddings are generated, they are indeed saved into snapshot during the next hourly snapshot in the most recent RC builds (earlier builds had a bug that caused embeddings to be regenerated on each restart)
d
Dima
07/18/2023, 5:28 PMAh, maybe it’s the reason. Do you have tar.gz with latest RC at hand or it’s better just wait for a release?
j
Jason Bosco
07/18/2023, 6:16 PMd
Dima
07/18/2023, 6:17 PMExtracted binary, thank you
👍 1