#community-help

Adding Colon as Token Separator in Java Client

TLDR Ivy encountered issues adding ":" as a token separator using the Java client for Typesense. Jason asked them to open a Github issue for further investigation.

6mo

Join the chat

Jun 08, 2023 (6 months ago)

Photo of md5-2f39998c05c5409f8c3521187ec34d4e

Ivy

02:25 PM

Hey all! I'm trying to add : as a token separator using the Java client, but in Typesense, I don't see it. So not sure if it worked and it's not showing up or it didn't work 😅

How does Typesense use "" as a token_separator also?

"token_separators": [
    "",
    "-",
    ",",
    "\""
  ] ```

val typesenseSchema = CollectionSchema()
.name(schema.name.asString)
.fields(typesenseFields)
.enableNestedFields(true)
.tokenSeparators(listOf(":", "-", ",", "\""))```

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

04:45 PM

CC: Kishore Nallan ^

Ivy

04:48 PM

This might be an issue on Typesense vs the Java client Jason

Image 1 for This might be an issue on Typesense vs the Java client <@4L6c7>

04:48

Ivy

04:48 PM

[","] works, but not [":"] or [":", ","]

Jason

04:55 PM

Hmmm, mind copy-pasting that schema into a Github issue? Need to take a closer look

Ivy

06:47 PM

Done! https://github.com/typesense/typesense/issues/1053 Thanks 🙂

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Restricting `token_separators` to a Specific Field in Typesense

Loic asked Jason about applying `token_separators` to a specific field in Typesense. Jason suggested opening a github issue to add this feature.

18mo

Updating token_separators in Typesense Cloud Issue

Loic encountered an error while trying to update token_separators in Typesense Cloud. Jason suggested creating a new schema and reindexing data.

16mo

Typesense Search Issue with Special Characters

Saurabh faced an issue in Typesense when searching for "security". Kishore Nallan explained special characters are removed and suggested adding "-" as a token separator in the configuration.

6mo