Guys, I need your advice about quality of semantic...
# community-help
n
Guys, I need your advice about quality of semantic search. My documents are between half-screen and screen long, I am building embeddings this way:
Copy code
"name": "embedding",
      "type": "float[]",
      "num_dim" : 4096,
      "embed": {
        "from": [
          "title",
          "content",
          "tags1",
          "tags2",
          "organization",
        ],
        "model_config": {
          "query_prefix": "query:Given a search query, retrieve relevant passages that answer the query",
          "model_name": "openai/Qwen/Qwen3-Embedding-8B",
          "api_key": ...
          "url": "https://api.studio.nebius.ai/v1/embeddings"
        }
      }
The search results are... well, "it could be better". Not "Google like" (i mean google like 10 years ago). Before I use heavy artillery like Conversational Search (RAG) or something else... what are my options? • To pre-process my documents, spilt them into chunks (1 chunk = 1 paragraph) and compute embeddings for these chunks instead of full documents... is it good idea? • To use better embedding model ? (I think Qwen3-Embedding-8B is good enough, correct me if i wrong) Something else?
k
I'm afraid that there is no golden answer here. You have to try a few different models and identify the gaps.
n
Ok, will try some embedding models, and... I think this should work in my case: https://typesense.org/docs/guide/tips-for-searching-common-types-of-data.html#long-pieces-of-text Imagine, I split my "screen long" documents into chunks about... 120–220 tokens (what is the right chunk size by the way?). OK I store each chunk in as a separate document in Typesense. Now: 1. There are absolutely great examples of semantic search, the Hacker News comments example is great! But, all these examples deal with relatively SHORT pieces of text, I found NO chunking example. Am I missing something? 2. OK, I store each chunk in as a separate document in Typesense, but I want to display in the UI the complete, original document, with some additional metadata fields (my metadata) like title, tags, organization. As far as I understand this is something like: do search in "chunks collection", display to user "original docs collection". Something like One-to-many relation https://typesense.org/docs/30.1/api/joins.html#one-to-many-relation right? So my parent collection named "doc"
Copy code
{
  "name":  "doc",
  "fields": [
    {"name": "title" ... },
    {"name": "content" ... },
    {"name": "tags1" ... },
    {"name": "tags2" ... },
    {"name": "organization" ... },
    ...
  ]
}
And my child collection named "chunk"
Copy code
{
  "name":  "chunk",
  "fields": [
    {"name": "chunk_str", "type": "string" ... },
    {"name": "chunk_embed", "type": "float[]",  "embed": { "from": [chunk_str] ...} ... },
    {"name": "doc_id", "type": "string", "reference": "doc.id"}
  ] 
}
And I search this way (semantic search):
Copy code
{
    "q": "device to type things on",
    "collection": "doc",
    "query_by": "$chunk(chunk_embed)",
    "vector_query":  $chunk( chunk_embed:([], k: 200) ),
    "prefix": "false",
    "exclude_fields": "$chunk(chunk_embed)"
    "per_page": 20,
    "page": 1
}
Is it the right syntax? ( I am trying to combine multiple Typesense features, which is a bit difficult... pls take a look if possible) thanks in advance @Kishore Nallan!
k
Would you be able to provide some examples where you are not getting expected results? I can then use that to suggest whether it's a chunking problem or something else.
n
Anyway: Thank you for your replies on Saturday! examples... uff... this means... schema ... doc collection is some format... query... can you guide me how to provide reproducible Typesense example? ( and, I still think chunking will make sense in my case...)
a
Are you in cloud? If yes, you can send a search query where it doesnt get good results and we can verify internally the collection schema, model and so on
n
Hi Alan, no I am self-hosting... I think I solve it increasing
distance_threshold
a bit (it was 0.51 before) - I understand, with increasing distance_threshold user will see more garbage results after good results... that's life... My client code looks like this:
Copy code
const threshold = 0.56;
const fetchK = 40;

const vectorQuery = `embedding:([], ef:10, distance_threshold:${threshold}, k:${fetchK})`;

const searchParameters = {
  q: QUERY,
  query_by: 'embedding',
  exclude_fields: 'embedding,content,content_indexed',
  per_page: fetchK,
  prefix: 'false',
  vector_query: vectorQuery,

  drop_tokens_threshold: 0,
  num_typos: 0,
  prioritize_num_matching_fields: false,
  page: 1
};

client.collections('docs').documents().search(searchParameters).then( ... );
How bad it is? and about document size vs embedding model capabilities: I am using this https://huggingface.co/Qwen/Qwen3-Embedding-8B embedder, and they advertise it with "Context Length: 32k"... if my "screen-long" documents fit into 32k tokens - am I fine and need no chunking?