Hello, Do collection curations apply to semantic/...
# community-help
t
Hello, Do collection curations apply to semantic/vector-only searches? I've tested between two presets, one that only queries a vector field and another that performs a hybrid search against a vector field and several string fields. When using the hybrid preset, all curations apply normally, but switching over to the vector-only search preset, the curations don't seem to affect the search results anymore.
a
They do. Curation is applied before any search even begins. This seems like a bug. Could you kindly create a reproducible example using the gist below as a reference? https://gist.github.com/auth/github?return_to=https%3A%2F%2Fgist.github.com%2Fjasonbosco%2F7c3432713216c378472f13e72246f46b
👍 1
t
@Alan Martini After some additional validation, I'm pretty sure this is a bug. Here is a reproduceable example:
Copy code
### Run Typesense via Docker ########################################
set -x

export TYPESENSE_API_KEY=xyz
export TYPESENSE_HOST=<http://localhost:8108>

docker stop typesense-repro 2>/dev/null
docker rm typesense-repro 2>/dev/null
rm -rf "$(pwd)"/typesense-data-dir-repro
mkdir "$(pwd)"/typesense-data-dir-repro

# Wait for Typesense to be ready
docker run -d -p 8108:8108 --name typesense-repro \
            -v"$(pwd)"/typesense-data-dir-repro:/data \
            typesense/typesense:29.0.rc31 \
            --data-dir /data \
            --api-key=$TYPESENSE_API_KEY \
            --enable-cors

# Wait till typesense is ready.
until curl -s -o /dev/null -w "%{http_code}" "$TYPESENSE_HOST/health" -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" | grep -q "200"; do
  sleep 2
done

curl -s "$TYPESENSE_HOST/debug" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" | jq


curl -s "$TYPESENSE_HOST/collections" \
       -X POST \
       -H "Content-Type: application/json" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
       -d '
          {
  "name": "articles",
  "fields": [
    {
      "name": "title",
      "type": "string"
    },
    {
      "name": "articleType",
      "type": "string"
    },
    {
      "name": "titleVector",
      "type": "float[]",
      "embed": {
        "from": [
          "title"
        ],
        "model_config": {
          "model_name": "ts/all-MiniLM-L12-v2"
        }
      }
    }
  ]
}
' | jq

curl -s "$TYPESENSE_HOST/collections/articles/documents/import?action=create" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -H "Content-Type: text/plain" \
        -X POST \
        -d '{"id": "1","title": "How to curate typesense searches", "articleType": "doc" }
            {"id": "2","title": "Understanding Typesense Curations", "articleType": "doc" }
            {"id": "3","title": "My Personal Life Update", "articleType": "blog" }'  | jq

curl "$TYPESENSE_HOST/collections/articles/overrides/blog-filter" -X PUT \
-H "Content-Type: application/json" \
-H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" -d '{
  "rule": {
    "query": "personal",
    "match": "contains"
  },
  "filter_by": "articleType:=Blog"
}' | jq

# Curation isn't applied to this vector only query
curl -s "$TYPESENSE_HOST/multi_search" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '
          {
            "searches": [
              {
                "collection": "articles",
                "q": "What are the latest personal articles?",
                "query_by": "titleVector",
                "exclude_fields": "titleVector"
              }
            ]
          }'  | jq

# Curation is applied to this hybrid query
curl -s "$TYPESENSE_HOST/multi_search" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '
          {
            "searches": [
              {
                "collection": "articles",
                "q": "What are the latest personal articles?",
                "query_by": "titleVector,title",
                "exclude_fields": "titleVector"
              }
            ]
          }'  | jq

docker stop typesense-repro
docker rm typesense-repro
a
Thanks @Thomas Reither With the reproducible steps is pretty clear that is a bug. We will then open a issue on our github. Would you like to do it? (to receive notifications and etc) https://github.com/typesense/typesense/issues
t
a
Perfect, thanks Thomas, really appreciate your help and bringing this bug to us.