Chandar Venkata Rama
05/14/2025, 4:42 PMAlan Martini
05/14/2025, 5:07 PMsymbols_to_index
causes Typesense to preserve and index additional characters as part of the in-memory search index. This might increase the number of distinct tokens, depending on how many distinct special characters you have between the same set of tokens (word).
Since Typesense is an in-memory database, the more distinct tokens you have, the more memory is needed.
I would recommend you to benchmark it with your dataset to assess the exact memory impact