Hi Everyone, I have a question regarding best prac...
# community-help
a
Hi Everyone, I have a question regarding best practices when using Typesense as a vector database for RAG use cases. In most implementations I’ve seen, each chunk is stored together with: • the embedding vector • the chunk content itself • and some metadata (document id, hierarchy, source, etc.) However, I’ve also seen architectures where only the vector and metadata are stored in the vector DB, and the actual content is kept in a separate database (e.g., Postgres, object storage, etc.), retrieved later via ID. From your experience, what is the most common and recommended approach when using Typesense for RAG?
a
For Typesense specifically, the recommended approach is to store everything together: embeddings, chunk content, and metadata in a single collection. Since Typesense is a full search engine with integrated vector capabilities, when you perform a vector search it returns complete documents (all fields + similarity score) in a single query, eliminating the need for a secondary content lookup from Postgres or S3. Also, it has a built-in auto-embedding feature (embed property on vector fields) that generates vectors from your text at index time using local or remote models (OpenAI, Google, etc.), simplifying your ingestion pipeline. Co-locating everything also lets you combine hybrid search (text + vector) and metadata filtering in one query for better retrieval quality. You'll also be able to use the built-in conversational RAG pipeline (conversation=true) that automatically sends search results as context to an LLM and manages conversation history.
a
Thank you @Alan Martini for your quick response. Is there any limit (hard or recommended) on the size of the
content
field when storing chunks in Typesense? We’re trying to define an appropriate chunk size and want to make sure we stay within optimal limits.
a
There is no hard limit on the size of string field values in Typesense. Your content field can hold large blocks of text without being truncated or rejected. The practical constraint for your RAG use case is the token limit of the embedding model you're using. Typesense sends the full text to the model without truncation on its side, so: - Built-in local models (ts/e5-small, ts/all-MiniLM-L12-v2): 512-token context window (~1,000-2,000 characters of English). Keep chunks within this limit for optimal embedding quality. - Remote models (OpenAI text-embedding-3-small): up to 8,191 tokens, but retrieval quality is often best with chunks in the 256-1,024 token range. But the content field itself has no size cap for storage and retrieval