Typesense

Typesense

Command MenuMenu

K

#community-help

Sanitizing Queries with Special Characters in Typesense

TLDR Daniel asked about sanitizing queries containing special characters in Typesense. Kishore Nallan recommended using backticks for literal string values and fixing the schema for field names with special characters, sticking to alphanumeric and underscores.

Powered by Struct AI

1

13

5mo

Apr 25, 2023 (5 months ago)

Photo of md5-a7605b093448e06efb0a1c150153cad6

Daniel

03:19 PM

We're running into issues where our filter_by properties sometimes contain open parenthesis, but no closing parenthesis (so something like "Cup Size (mm" rather than "Cup Size (mm)"), and this is leading to issues.
Does Typesense have a good way of sanitizing the queries so that characters like this don't become a factor? Or do we have some way of boxing filters so that this isn't an issue?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:23 PM

👋 Wrap string values with backticks to treat as literal. E.g.

title: `Cup size (mm`

Photo of md5-a7605b093448e06efb0a1c150153cad6

Daniel

03:24 PM

Ah, thanks! We'll give it a try

03:55

Daniel

03:55 PM

Does it work for the fields, too? We're doing something like

'filter_by': '`descriptors.Cup Size (mm`:=`4 x 5`'

(The field itself is typo'd, but we're finding that we're having issues even if there is both open and close parenthesis)

Image 1 for Does it work for the fields, too? We're doing something like<br><pre><code>'filter_by': '`descriptors.Cup Size (mm`:=`4 x 5`'</code></pre><br>(The field itself is typo'd, but we're finding that we're having issues even if there is both open and close parenthesis)

03:56

Daniel

03:56 PM

This is giving us the following error, if any parentheses are in the filter_by field, from the looks of it

Image 1 for This is giving us the following error, if any parentheses are in the filter_by field, from the looks of it

03:57

Daniel

03:57 PM

If it matters, this is on v0.25, so maybe a quirk of the new version? (and using a wildcard for the facets?)

04:01

Daniel

04:01 PM

If there's limitations on what we can use as field names in documents, we could use some kind of placeholder value (like underscores, etc.) to get around it

04:02

Daniel

04:02 PM

(though obviously it'll be easiest for us to just use what we already have in the collection, but it's not too late to redo things like this)

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

04:24 PM

Sorry, this won't help with field names. You have fix your schema.

04:25

Kishore Nallan

04:25 PM

We have traditionally not validated against field names. We cannot enforce it now because it will break backward compatibility.

Photo of md5-a7605b093448e06efb0a1c150153cad6

Daniel

05:20 PM

Ahh ok

Apr 26, 2023 (5 months ago)

Photo of md5-a7605b093448e06efb0a1c150153cad6

Daniel

01:29 AM

Kishore Nallan In that case, if there are letters we should avoid, is there a list so that we can work on some kind of converter? (And then something to convert it back to normal text)

I take it that parenthesis are bad, but are things like = and : also an issue?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:09 AM

Best to stick to alphanumeric and underscores.

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2764 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Phrase Search Relevancy and Weights Fix

Jan reported an issue with phrase search relevancy using Typesense Instantsearch Adapter. The problem occurred when searching phrases with double quotes. The team identified the issue to be related to weights and implemented a fix, improving the search results.

Merging Results of Multi-Search Queries: Workarounds and Future Features

Julian wanted to merge results from two similar collections. Kishore Nallan confirmed it's a feature that might be implemented but hasn't been prioritised. Both Julian and robert plan to use client-side solutions for now.

Issue with Search Duration on Typesense Database

Robert reported an issue about query time delay when adding a `filter_by` constraint in a large Typesense database. Kishore Nallan explained that this happens due to the order of operation and also promised to look into this issue further. Robert withdrew his interest in sponsoring the improvement due to moving from the project.

Resolving Special Character Search Errors

suraj had issues searching for data containing special characters. Kishore Nallan resolved the issue by advising suraj to remove the parameters for 'preSegmentedQuery' and 'tokenSeparators'.

Moving from Algolia to Typesense: Questions and Answers

Juan sought advice from Kishore Nallan about moving from Algolia to Typesense, handling MultiSearch, setting parameters, checking imported documents, and a specific syntax query.

Powered By