How does the `symbols_to_index` work at a per fie...
# community-help
ó
How does the
symbols_to_index
work at a per field basis? If the collection was created in a previous version, can I use alter and modify one filed to use it even if at collection level it's empty? Because I'm trying it, and it's not working (at least with a hybrid search)
j
There's a bug with per field symbols to index that we're working on fixing at the moment. CC: @Fanis Tharropoulos
ó
Perfect! Knowing that I'm not crazy is enough 😅
😅 1
@Jason Bosco Do you have an ETA? Like a week, month or more? It's to be able to schedule the upgrade
j
The fix is actually already done, just needs to go through code review and into an RC build. So probably like a week tops
🙌 2
j
When I tried to add symbols_to_index: [‘-’, ‘.’] at the field level, I did this hoping to solve for an exact search scenario so when searching for ‘“stocktoncollege.com”’, only stocktoncollege.com appears, not both stocktoncollege.com and stockton-college.com
ó
As Jason said, the fix is still not real eased. But it's a similar use case
j
Yes, hoping the fix will resolve @Óscar Vicente
j
The fix is now in
29.0.rc22
👍 1
Could you upgrade to this and see if that fixes the issue you're seeing?
ó
Isn't it backported? I thought this was a 28 feature
Yup, I can try right now, it will take a while
j
We usually don't backport - we follow a roll-forward approach to versioning
1
ó
I'm still getting the same weird behavior when using a hybrid search. We have a code reference and a name field. The Code reference is something like
CTTI-2024-15
while the name is just a name.
Copy code
q": "CTTI-2024-15",
"query_by": "externalReference,title",
"prefix": true,
"filter_by": "(id:*||$favorites(userId:=7537b880-912f-44a7-aaf0-b6079cc07865))",
"query_by_weights": "100,5",
"text_match_type": "sum_score"
The document with that exact externalReference won't be the first ever, more like the tenth. And the text_match info will show 0, and no highlight even with the rerank. Something is going on with this.
Copy code
{
  "document": {
    "externalReference": "CTTI-2025-15"
  },
  "highlight": {},
  "highlights": [],
  "hybrid_search_info": {
    "rank_fusion_score": 0.16521739959716797
  },
  "text_match": 0,
  "text_match_info": {
    "best_field_score": "0",
    "best_field_weight": 0,
    "fields_matched": 0,
    "num_tokens_dropped": 1,
    "score": "0",
    "tokens_matched": 0,
    "typo_prefix_score": 255
  },
  "vector_distance": 0.3783947229385376
}
The indexed field:
Copy code
{
  "facet": false,
  "index": true,
  "infix": false,
  "name": "externalReference",
  "optional": true,
  "sort": false,
  "stem": false,
  "store": true,
  "type": "string",
  "symbols_to_index": [
    "_",
    "-",
    "/",
    "\\",
    ".",
    "-"
  ]
}
j
Hmm ok this might be a different issue then. Mind opening a GitHub issue with a small reproducible set of curl commands like this: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b
ó
@Jason Bosco Issue created here! https://github.com/typesense/typesense/issues/2365#issue-3065599605 Let me know if further clarification is needed. Also, if there's a workaround I could use for now until it works.