We are using `symbols_to_index` to to add characte...
# community-help
ó
We are using
symbols_to_index
to to add characters like - or /. We would like to filter product reference like
143/2024
, but the text and symbols are treated as different words instead of a single piece of text:
Copy code
{
                    "document": {
                        "externalReference": "143/2024"
                    },
                    "highlight": {
                        "externalReference": {
                            "matched_tokens": [
                                "143",
                                "/",
                                "2024"
                            ],
                            "snippet": "<mark>143</mark><mark>/</mark><mark>2024</mark>"
                        }
                    },
                    "highlights": [
                        {
                            "field": "externalReference",
                            "matched_tokens": [
                                "143",
                                "/",
                                "2024"
                            ],
                            "snippet": "<mark>143</mark><mark>/</mark><mark>2024</mark>"
                        }
                    ],
                    "text_match": 1736172819518056569,
                    "text_match_info": {
                        "best_field_score": "3315704398333",
                        "best_field_weight": 15,
                        "fields_matched": 1,
                        "num_tokens_dropped": 0,
                        "score": "1736172819518056569",
                        "tokens_matched": 3,
                        "typo_prefix_score": 0
                    }
                },
                {
                    "document": {
                        "externalReference": "2024/143"
                    },
                    "highlight": {
                        "externalReference": {
                            "matched_tokens": [
                                "2024",
                                "/",
                                "143"
                            ],
                            "snippet": "<mark>2024</mark><mark>/</mark><mark>143</mark>"
                        }
                    },
                    "highlights": [
                        {
                            "field": "externalReference",
                            "matched_tokens": [
                                "2024",
                                "/",
                                "143"
                            ],
                            "snippet": "<mark>2024</mark><mark>/</mark><mark>143</mark>"
                        }
                    ],
                    "text_match": 1736172819518056569,
                    "text_match_info": {
                        "best_field_score": "3315704398333",
                        "best_field_weight": 15,
                        "fields_matched": 1,
                        "num_tokens_dropped": 0,
                        "score": "1736172819518056569",
                        "tokens_matched": 3,
                        "typo_prefix_score": 0
                    }
                },
                {
                    "document": {
                        "externalReference": "143/2024"
                    },
                    "highlight": {
                        "externalReference": {
                            "matched_tokens": [
                                "143",
                                "/",
                                "2024"
                            ],
                            "snippet": "<mark>143</mark><mark>/</mark><mark>2024</mark>"
                        }
                    },
                    "highlights": [
                        {
                            "field": "externalReference",
                            "matched_tokens": [
                                "143",
                                "/",
                                "2024"
                            ],
                            "snippet": "<mark>143</mark><mark>/</mark><mark>2024</mark>"
                        }
                    ],
                    "text_match": 1736172819518056569,
                    "text_match_info": {
                        "best_field_score": "3315704398333",
                        "best_field_weight": 15,
                        "fields_matched": 1,
                        "num_tokens_dropped": 0,
                        "score": "1736172819518056569",
                        "tokens_matched": 3,
                        "typo_prefix_score": 0
                    }
                }
As you can notice, the first and third one contains the exact match, but the one in the middle no. As it treats each part as separated, the three of them are "perfect" matches, but they are not. How can I achieve it??? Honestly, I spent too much time already on this issue when it should just work. How the heck can I put the exact matches to the top, and treat this text "143/2024" as a whole word???? Collection settings:
Copy code
"symbols_to_index": [
    "_",
    "-",
    "/",
    "\\",
    ".",
    "-"
  ],
  "token_separators": []
k
Seems like a bug. Can you please create a github issue with a small reproducible example in this format: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b Happy to look into why this is happening.
And, sorry for the hassle!
f
Are the
token_separators
in the field level or the collection's root?
ó
That's collection level, as field level still doesn't work as per: https://github.com/typesense/typesense/issues/2365#issuecomment-2955379801