Hey everyone :wave:. Thanks for this amazing produ...
# community-help
d
Hey everyone šŸ‘‹. Thanks for this amazing product, we're very glad to be customers so far šŸ™‚. Also, to mention that the pace of innovation / feature additions is quite impressive. I have two questions that could alleviate the need for quite some custom code in our current data pipelines to create embeddings. Abstracted issue: I have a series of fields that are important for a model to use as input and some fields that are "leading the model astray" which I want to exclude. 1. Triggering "remote embeddings" for a given document, is there a way to specify which fields of the document to be included in the embedding input? 2. I am assuming that with remote embeddings, every update to a document leads to a re-creation of an embedding (and thereby model API call and resulting cost). If including/excluding model fields is supported, is a recalculation of the embedding only triggered if one if the included fields change? Cheers, Daniel
f
You can specify which fields your embedding field will embed from using the
embed
property's
from
prop. Since it's an array of strings, you can pass in any
string
field: https://typesense.org/docs/29.0/api/vector-search.html#creating-an-auto-embedding-field The embeddings will only be re-calculated when an embedding field changes, not when the document is changed, so there's o extra API call to your remote embedder there
d
Thanks @Fanis Tharropoulos, that was too obvious šŸ™ˆ. Another question that comes to mind: For the usage of remote embeddings, is there a way to add response caching to control cost and/or latency? I see there is a new caching feature introduced in v29 (which is not yet well documented btw). But could I e.g. also instruct Typesense to use my LiteLLM instance instead of the model provider directly?
LiteLLM or other caching solutions…
f
Is LiteLLM OpenAI compatible? From a quick read, I saw that it standardizes the API format, but is it hosted on a remote URL like we describe in the docs? Apart from that, if you want to use your own model, you can pass in the embeddings generated from the model yourself as well. Lastly, could you describe what part of the documentation based on caching is not up to par? We strive for a fully documented API and feedback is always welcome
d
Lastly, could you describe what part of the documentation based on caching is not up to par? We strive for a fully documented API and feedback is always welcome
At this point I feel the documentation on caching is quite hidden (could only find it in https://typesense.org/docs/29.0/api/server-configuration.html#resource-usage). I’d expect it’s own section somewhere in the vicinity of https://typesense.org/docs/29.0/api/vector-search.html#index-embeddings. Furthermore, I think it should explain in more detail how the caching behaves exactly. Technically speaking, e.g. the question on my mind is: how is the caching key constructed / which input goes in? Does it hash only the document contents or does it also include e.g. the chosen model and any parameters going into the embedding generation? Under which conditions will the embedding be recalculated, when will it be taken from the cache? Is the cache persistent and thereby survives server restarts? Is the cache distributed and therefore works in a clustered scenario? Some one of these questions are going beyond what I need, but these are examples of questions that come to my mind when I read about a caching feature. I could look at the implementation to get answers, but the implementation might just be a snapshot and the behaviour might change without me noticing. Documentation typically serves more as a ā€œspecificationā€, describing how things should work.
Is LiteLLM OpenAI compatible? From a quick read, I saw that it standardizes the API format, but is it hosted on a remote URL like we describe in the docs?
Yes, it is. Thanks for the pointer!
šŸ™Œ 1
f
At this point I feel the documentation on caching is quite hidden (could only find it in https://typesense.org/docs/29.0/api/server-configuration.html#resource-usage). I’d expect it’s own section somewhere in the vicinity of https://typesense.org/docs/29.0/api/vector-search.html#index-embeddings.
Furthermore, I think it should explain in more detail how the caching behaves exactly.
Technically speaking, e.g. the question on my mind is: how is the caching key constructed / which input goes in? Does it hash only the document contents or does it also include e.g. the chosen model and any parameters going into the embedding generation? Under which conditions will the embedding be recalculated, when will it be taken from the cache? Is the cache persistent and thereby survives server restarts? Is the cache distributed and therefore works in a clustered scenario?
Some one of these questions are going beyond what I need, but these are examples of questions that come to my mind when I read about a caching feature. I could look at the implementation to get answers, but the implementation might just be a snapshot and the behaviour might change without me noticing. Documentation typically serves more as a ā€œspecificationā€, describing how things should work.
We'll definitely add mention of it under the vector search docs, and we'll try to cover some technical aspects of the implementation, but we're trying to keep documentation readable for new-comers as well, so we may not go into full detail. Thanks for the feedback!
šŸ‘ 1