Hi Team, When I search for "iphone 16", I get results, but not for "iphone16". My query looks like this:

{
  "q": "iphone16",
  "page": "1",
  "per_page": "125",
  "query_by": "title,category,brand,color",
  "exclude_fields": "embedding",
  "enable_synonyms": true,
  "split_join_tokens": "always"
}

You can check out the

split_join_tokens

parameter:

i am already using

split_join_tokens

in my query!

Oh, didn't see that, sorry. I can't seem to replicate it on a local cluster. What version of Typesense are you using?

i'm using v28.0

Can you provide some reproduction steps on a fresh cluster using curl?

Sure, could you please clarify what exact behavior you'd like me to reproduce? An example or specific use case would be helpful.

The iPhone 16 example. Creating a collection with a sample dataset and providing the query that produces unexpected results

I created the schema using the auto schema detection feature. Below are the two

curl

queries I used:

curl -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
"<http://localhost:8108/collections/my_store/documents/search?q=iphone%2016&query_by=title,category,brand,color&exclude_fields=embedding&enable_synonyms=true&split_join_tokens=always>"

curl -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
"<http://localhost:8108/collections/my_store/documents/search?q=iphone16&query_by=title,category,brand,color&exclude_fields=embedding&enable_synonyms=true&split_join_tokens=always>"

Please create a reproduction script in this manner: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b

Hi @Kishore Nallan, Thank you for sharing the script. Here is my reproduction script.

export TYPESENSE_API_KEY=xyz

curl "<http://localhost:8108/debug>" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}"

curl "<http://localhost:8108/collections>" \
       -X POST \
       -H "Content-Type: application/json" \
       -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
       -d '{
         "name": "ecommerce_store",
         "fields": [
           {"facet": false,"index": true,"infix": false,"locale": "","name": "title","optional": true,"sort": false,"stem": true,"stem_dictionary": "","store": true,"type": "string"},
           {"facet": true,"index": true,"infix": false,"locale": "","name": "category","optional": true,"sort": false,"stem": true,"stem_dictionary": "","store": true,"type": "string[]"},
           {"facet": true,"index": true,"infix": false,"locale": "","name": "brand","optional": true,"sort": false,"stem": true,"stem_dictionary": "","store": true,"type": "string"},
           {"facet": false,"index": true,"infix": false,"locale": "","name": "color","optional": true,"sort": false,"stem": true,"stem_dictionary": "","store": true,"type": "string"},
           {"facet": false,"index": true,"infix": false,"locale": "","name": "price","optional": true,"sort": true,"stem": false,"stem_dictionary": "","store": true,"type": "float"}
         ]
       }'

curl "<http://localhost:8108/collections/ecommerce_store/documents/import?action=create>" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -H "Content-Type: text/plain" \
        -X POST \
        -d '{"id": "124","title": "Apple iPhone 16 128GB - Ultramarine","category": ["phones","apple"],"brand": "apple","color": "ultramarine","price": 999}
            {"id": "125","title": "Apple iPhone 16 Plus 256GB - Black","category": ["phones","apple"],"brand": "apple","color": "Black","price": 444}'

Working example:

curl "<http://localhost:8108/multi_search>" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "ecommerce_store",
              "q": "iphone 16",
              "query_by": "title,category,brand,color"
            }
          ]
        }'

This will give empty result :

curl "<http://localhost:8108/multi_search>" \
        -X POST \
        -H "Content-Type: application/json" \
        -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" \
        -d '{
          "searches": [
            {
              "collection": "ecommerce_store",
              "q": "iphone16",
              "query_by": "title,category,brand,color"
            }
          ]
        }'

You've

"stem": true

Which will change the query to a form which will NOT trigger the split joins behavior.