#community-help

Memory Leak Issue with Typesense in Cyrillic

TLDR Alexander found a memory leak when using Typesense's docker image that was rectified by Kishore Nallan with a new build (0.23.0.rc53), which also supports Cyrillic search.

Powered by Struct AI
6
17mo
Solved
Join the chat
Mar 31, 2022 (18 months ago)
Alexander
Photo of md5-7bdd13dd55fe010daa6dededaa70ef8e
Alexander
12:59 PM
Hey community, I am struggling with strange issue with RAM usage.
I’m using typesense/typesense:0.23.0.rc8 docker image (rc version for locale aware indexing for queries in Cyrillic) but I found out lately that typesense container apparently has memory leak and it takes ~24h to reach 8gb (server limit) regular RAM usage with total 30mb index and ~20000 documents in multiple indexes.
I tried to upgrade to latest rc version (rc49) but with this version cyrillic not supported at all (neither query nor faceting).

Am I missing some configuration introduced in latest rc’s?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
01:01 PM
👋 do you have a sample snippet that shows a sample document indexed and queried which shows the problem you face with latest RC?
Alexander
Photo of md5-7bdd13dd55fe010daa6dededaa70ef8e
Alexander
04:41 PM
Search query:
{
    "searches": [
    {
        "per_page": 24,
        "query_by": "Код,Артикул,Наименование,Бренд",
        "sort_by": "",
        "highlight_full_fields": "Код,Артикул,Наименование,Бренд",
        "collection": "all",
        "q": "*",
        "facet_by": "Категория,ВНаличии,ДилерскаяЦенаРуб,Бренд,Страна",
        "filter_by": "",
        "max_facet_values": 300,
        "page": 1
    }]
}

Response:
{
    "results": [
    {
        "facet_counts": [],
        "found": 0,
        "hits": [],
        "out_of": 11653,
        "page": 1,
        "request_params":
        {
            "collection_name": "all",
            "per_page": 5,
            "q": "стол"
        },
        "search_cutoff": false,
        "search_time_ms": 1
    }]
}

Indexed document:

{
  "id": "5692",
  "Артикул": "282701",
  "Бренд": brand",
  "Категория": "Столы холодильные",
  "КатегорияЛатиницей": "stoly-kholodilnye",
  "Код": "5692",
  "Модель": "GN 11/TN",
  "Наименование": "GN 11/TN стол охл. (-2+10), 2 двери, 1390х700х850мм, HICOLD RUS",
  "НаименованиеПолное": "GN 11/TN стол охл. (-2+10), 2 двери, 1390х700х850мм, HICOLD RUS",
  "Популярность": 1000000051,
  "Статус": "Основной ассортимент",
  "Страна": "Россия"
}

04:44
Alexander
04:44 PM
Just tested one more time changing typesense RC version from rc8 to rc49 and back and with rc8 everything works as expected (except leaking RAM)
rc49 - empty search results for cyrillic query (quering against fields indexed with ‘locale: “ru”’)
Apr 04, 2022 (17 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:00 PM
Ok let me look into this and get back to you.
Apr 14, 2022 (17 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
07:08 AM
Alexander I was able to track down and fix that memory leak: thank you so much for bringing this to my attention. Since it was confined to Cyrillic, our existing tests didn't catch the leak.

I've also published a new build 0.23.0.rc53 that contains this fix, and I also tested that ru search works fine on this build, see this gist: https://gist.github.com/kishorenc/b207c23ff59198dbaecca7007fdea0db