#community-help

Achieving Stemming Support with Typesense

TLDR Sabyasachi asked how to implement stemming in Typesense, which Kishore Nallan explained must be handled externally. Sabyasachi later shared they created an extra field for storing stemmed content.

22mo

Join the chat

Jan 08, 2022 (22 months ago)

Photo of md5-3badcffdc9bad0939ba26ebfebc3bd43

Sabyasachi

06:08 AM

How are you achieving stemming support? I think Typesense does not directly support stemming, but what are you folks using as work around? (without irrelevant results showing up).

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

01:52 PM

You will have to handle stemming outside by stemming both text and query before sending them to Typesense.

Jan 10, 2022 (22 months ago)

Sabyasachi

03:58 AM

Thanks.

Jan 11, 2022 (22 months ago)

Sabyasachi

04:05 AM

For anyone’s future reference:

Here is what I did: added a separate field in the schema: text_stemmed. I used nltk.PorterStemmer for stemming the content and store the resultant string in text_stemmed.

While querying I use the same method to generate stemmed query string. I concatenate the original query and the stemmed query. In the search params, I added the text_stemmed at the last of the query_by param. So the exact matches are still prioritized higher.

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 2786 threads (79% resolved)

Join Our Community Manage subscription

Similar Threads

Implementing Stemming, Lemmatization, Stopwords with Typesense

Carlo asked about implementing stemming, lemmatization, stopwords with Typesense. Kishore Nallan suggested the Porter stemmer and mentioned stopwords is under development. Gustavo suggested using GPT-3.5-Turbo.

2mo

Typesense Support for Non-English Languages

omega enquired about Typesense's support for non-English languages. Kishore Nallan suggested using separate fields for different languages.

13mo

Understanding Query Parsing & Search Algorithm in Typesense

robert inquired about typesense's query parsing and search algorithm. Kishore Nallan explained that typesense supports prefix matching and typo tolerance, but does not stem queries.

14mo