Hello everyone, I'm currently investigating the pe...
# community-help
d
Hello everyone, I'm currently investigating the performance impact of enabling locales on a field. Generally, I've observed that on loading (which also indicates indexing speed during document insertion), there's a performance hit of about 10-20% for most European languages I've tested (including en, fr, it, nl, pt, es). However, for Russian, the impact is significantly higher, with performance being about 800% slower. I have a small reproduction dataset available if needed. Interestingly, CPU usage drops during this process. Some performance profiling showed that most of the time is spent in the ICU library initializing a translator, which, by the way, uses a global mutex. I'll provide more details in this thread as I continue my investigation. A few initial thoughts: • The 10-20% performance hit compared to English is this what you expect • The Russian tokenizer seems particularly problematic. This seems coming from this code "loosing" the translator between iteration in src/tokenizer.cpp
Copy code
} else if(is_cyrillic(locale)) {
        // init transliterator but will only transliterate during tokenization
        UErrorCode translit_status = U_ZERO_ERROR;
        if(!transliterator) {
            transliterator = icu::Transliterator::createInstance("Any-Latin; Latin-ASCII",
                                                                 UTRANS_FORWARD, translit_status);
        }
Looking forward to any insights or suggestions you might have!
docker image:
typesense/typesense:29.0.rc28
conf
Copy code
TYPESENSE_NUM_DOCUMENTS_PARALLEL_LOAD=10000
TYPESENSE_NUM_COLLECTIONS_PARALLEL_LOAD=1
my intial dataset has country names in different locales for ~1M document
Copy code
serial_number: 35
  "country": {
    "code": "uy",
    "name_with_code_de": "Uruguay|uy",
    "name_with_code_en": "Uruguay|uy",
    "name_with_code_es": "Uruguay|uy",
    "name_with_code_fr": "Uruguay|uy",
    "name_with_code_it": "Uruguay|uy",
    "name_with_code_nl": "Uruguay|uy",
    "name_with_code_pt": "Uruguai|uy",
    "name_with_code_ru": "Уругвай|uy",
    "name_with_code_tr": "Uruguay|uy"
    }
with following schema
Copy code
{"facet": true,"index": true,"infix": false,"locale": "en","name": "country.name_with_code_en","optional": true,"sort": false,"stem": false,"stem_dictionary": "","store": true,"type": "string"},
    {"facet": true,"index": true,"infix": false,"locale": "ru","name": "country.name_with_code_ru","optional": true,"sort": false,"stem": false,"stem_dictionary": "","store": true,"type": "string"},
    ....
    {"facet": true,"index": true,"infix": false,"locale": "","name": "serial_number","optional": true,"range_index": true,"sort": false,"stem": false,"stem_dictionary": "","store": true,"type": "int64"}
what i did is to copy one locale at a time with either setting the locale to the local of the field or setting it to empty (but still coping it in that local) so i have with_local_es collection containing serial_number and country.name_with_code_es in spanish (with locale set to es) , and without_locales_es conatianing serial_number and country.name_with_code_es in spanish but with locles set to '' Perf report when loading ru
Copy code
-   96,56%     0,00%  typesense-serve  libc.so.6          [.] 0x000078f097186ee8                                                                                                                                                                                               ◆
     0x78f097186ee8,
   - std::__future_base::_State_baseV2::_M_do_set(std::function<std::unique_ptr<std::__future_base::_Result_base, std::__future_base::_Result_base::_Deleter> ()>*, bool*),
      - 95,14% Index::batch_memory_index(Index*, std::vector<index_record, std::allocator<index_record> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, tsl::htrie_map<char, field, tsl::ah::str_hash<char>, unsigned short> const&▒
         - 94,68% Index::compute_token_offsets_facets(index_record&, tsl::htrie_map<char, field, tsl::ah::str_hash<char>, unsigned short> const&, std::vector<char, std::allocator<char> > const&, std::vector<char, std::allocator<char> > const&),
            - 94,07% Index::tokenize_string(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, field const&, std::vector<char, std::allocator<char> > const&, std::vector<char, std::allocator<char> > const&, std::unordered_map<std::__c▒
               - 51,69% Tokenizer::Tokenizer(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char, std::allocator<char>▒
                  - 51,45% Tokenizer::init(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&),
                     - 49,50% icu_71::Transliterator::createInstance(icu_71::UnicodeString const&, UTransDirection, UErrorCode&),
                        - 49,02% icu_71::TransliteratorIDParser::instantiateList(icu_71::UVector&, UErrorCode&),
                           - 48,97% icu_71::TransliteratorIDParser::SingleID::createInstance(),
                              - 48,94% icu_71::Transliterator::createBasicInstance(icu_71::UnicodeString const&, icu_71::UnicodeString const*),
                                 - 34,00% icu_71::TransliteratorAlias::create(UParseError&, UErrorCode&),
                                    - 33,70% icu_71::Transliterator::createInstance(icu_71::UnicodeString const&, UTransDirection, UParseError&, UErrorCode&),
                                       - 33,24% icu_71::TransliteratorIDParser::instantiateList(icu_71::UVector&, UErrorCode&),
                                          - 33,16% icu_71::TransliteratorIDParser::SingleID::createInstance(),
                                             - 33,15% icu_71::Transliterator::createBasicInstance(icu_71::UnicodeString const&, icu_71::UnicodeString const*),
                                                - 32,17% icu_71::TransliteratorRegistry::get(icu_71::UnicodeString const&, icu_71::TransliteratorAlias*&, UErrorCode&),
                                                   - 32,10% icu_71::TransliteratorRegistry::find(icu_71::UnicodeString&, icu_71::UnicodeString&, icu_71::UnicodeString&)
k
Yeah we had to use the transliteration to allow typo tolerance for Cyrillic. It started out as low cost (effort wise) to do that because our typo tolerance system supports only edit distance of 2 bytes but locales that had several bytes per logical alphabet posed issues.
d
side note, this is not theoretic issue, my investigation started because i see when i ramping indexing threads (like 50-60 thread indexing 10 document) the speed increase hit a maximum and than cpu usage drop to stabilize around 200% and response time increases.
thank you for the quick answer . i understand that , and i may disable locale for cyrilic. what surpise me is the name of function that doesn't seems to be something to be called frequently (icu_71:Transliterator:createBasicInstance ), i havn't dig far but seems for me is a quick win to have that translator somehow persisted , may be i m totally off and than best solution is to disable the locales for cyrilic
k
We might have to use a pool of tokenizers so this init is cached.
d
especially it has a global mutex on creation which break the scaling
Upon further analysis of the code, I believe that in our specific case, we may not need locale-specific tokenization, and the non-localized version should suffice. My understanding is that localized text processing helps capture nuances that are typically lost in non-localized versions (for example, distinguishing between ê, è, and é, which are different from the ASCII character e). However, since we are not dealing with large descriptions or extensive text, but rather with country/state names, sports, and related terms (such as club names), I expect that this "compression" will not result in significant information loss. Do you confirm if my understanding is correct? Additionally, I still believe that caching the initialization process would be advantageous, as pinpointing its impact can be quite challenging. I spent several hours tracking down the performance issue, constructing a minimal reproduction case, and discovered that it only occurs with Russian.
k
> I expect that this "compression" will not result in significant information loss. Yes, for general names the distinction will not matter much. What you lose however for Cyrillic is typo tolerance. You might have to disable it entirely via
num_typos: 0
-- otherwise doing fuzzy matching on the unicode byte representation of Cyrillic characters can cause bad matches to surface. > Additionally, I still believe that caching the initialization process would be advantageous Yes, we will need to fix this. Appreciate the help in identifying the root cause. Will be looking into it next week.
🙏 1
✅ 1