#community-help

Handling Special Characters in Typesense Queries

TLDR Jack had an issue with Typesense returning all results when querying with special characters. Jason indicated that Typesense removes special characters, suggesting a modification in the symbols_to_index setting. Jack found a solution by implementing a client-side filter for special characters only queries.

Powered by Struct AI
Oct 17, 2023 (1 month ago)
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
04:37 PM
When querying my Typesense collection with a special character (e.g. @), I seem to get all results back when I'd actually expect no results (none of my documents contain an @). Is there an option to disable this behaviour? We depend on there being some highlight information in the results, but when searching for special characters we don't get any highlight data back (as the result doesn't actually match the query).
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
04:38 PM
By default, Typesense removes special characters from the search query. So q=@ will actually translate to q=* by default.

You want to add @ to the symbols_to_index setting when creating the collection, to change this behavior
Jack
Photo of md5-763597f84b704376f42cabfa33614ae8
Jack
04:40 PM
Ah right. I don't think I want to add @ to symbols_to_index as none of my results have @ in them (and I'd have to do it for every other special character too!) - ideally if a query consists entirely of non-indexed symbols then it should return nothing, but I understand treating it as *. I'll implement something on our client-side to not send a query if it only consists of special characters.
04:42
Jack
04:42 PM
You wouldn't happen to know where in the Typesense code it strips symbols from the query would you? Ideally I'd like to implement the same logic on our client.