#community-help

Sanitizing Queries with Special Characters in Typesense

TLDR Daniel asked about sanitizing queries containing special characters in Typesense. Kishore Nallan recommended using backticks for literal string values and fixing the schema for field names with special characters, sticking to alphanumeric and underscores.

Powered by Struct AI

1

13
7mo
Solved
Join the chat
Apr 25, 2023 (7 months ago)
Daniel
Photo of md5-a7605b093448e06efb0a1c150153cad6
Daniel
03:19 PM
We're running into issues where our filter_by properties sometimes contain open parenthesis, but no closing parenthesis (so something like "Cup Size (mm" rather than "Cup Size (mm)"), and this is leading to issues.
Does Typesense have a good way of sanitizing the queries so that characters like this don't become a factor? Or do we have some way of boxing filters so that this isn't an issue?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:23 PM
👋 Wrap string values with backticks to treat as literal. E.g.

title: `Cup size (mm`
Daniel
Photo of md5-a7605b093448e06efb0a1c150153cad6
Daniel
03:24 PM
Ah, thanks! We'll give it a try
03:55
Daniel
03:55 PM
Does it work for the fields, too? We're doing something like
'filter_by': '`descriptors.Cup Size (mm`:=`4 x 5`'

(The field itself is typo'd, but we're finding that we're having issues even if there is both open and close parenthesis)
Image 1 for Does it work for the fields, too? We're doing something like
```'filter_by': '`descriptors.Cup Size (mm`:=`4 x 5`'```
(The field itself is typo'd, but we're finding that we're having issues even if there is both open and close parenthesis)
03:56
Daniel
03:56 PM
This is giving us the following error, if any parentheses are in the filter_by field, from the looks of it
Image 1 for This is giving us the following error, if any parentheses are in the filter_by field, from the looks of it
03:57
Daniel
03:57 PM
If it matters, this is on v0.25, so maybe a quirk of the new version? (and using a wildcard for the facets?)
04:01
Daniel
04:01 PM
If there's limitations on what we can use as field names in documents, we could use some kind of placeholder value (like underscores, etc.) to get around it
04:02
Daniel
04:02 PM
(though obviously it'll be easiest for us to just use what we already have in the collection, but it's not too late to redo things like this)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:24 PM
Sorry, this won't help with field names. You have fix your schema.
04:25
Kishore Nallan
04:25 PM
We have traditionally not validated against field names. We cannot enforce it now because it will break backward compatibility.
Daniel
Photo of md5-a7605b093448e06efb0a1c150153cad6
Daniel
05:20 PM
Ahh ok
Apr 26, 2023 (7 months ago)
Daniel
Photo of md5-a7605b093448e06efb0a1c150153cad6
Daniel
01:29 AM
Kishore Nallan In that case, if there are letters we should avoid, is there a list so that we can work on some kind of converter? (And then something to convert it back to normal text)

I take it that parenthesis are bad, but are things like = and : also an issue?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:09 AM
Best to stick to alphanumeric and underscores.

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Phrase Search Relevancy and Weights Fix

Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.

6

111
8mo
Solved

Merging Results of Multi-Search Queries: Workarounds and Future Features

Julian wanted to merge results from two similar collections. Kishore Nallan confirmed it's a feature that might be implemented but hasn't been prioritised. Both Julian and robert plan to use client-side solutions for now.

12
16mo

Issue with Search Duration on Typesense Database

Robert reported an issue about query time delay when adding a `filter_by` constraint in a large Typesense database. Kishore Nallan explained that this happens due to the order of operation and also promised to look into this issue further. Robert withdrew his interest in sponsoring the improvement due to moving from the project.

13
10mo

Moving from Algolia to Typesense: Questions and Answers

Juan sought advice from Kishore Nallan about moving from Algolia to Typesense, handling MultiSearch, setting parameters, checking imported documents, and a specific syntax query.

1

22
22mo
Solved

JavaScript Client's Return of Highlights Inquiry

Daniel questioned why the JavaScript client returned highlights in an array instead of an object. Kishore Nallan explained it was due to specific issues with statically typed languages needing defined JSON structures for parsing.

14
32mo
Solved