https://typesense.org logo
Join Slack
Powered by
# community-help
  • a
    Hey guys - wanted to sanity-check an approach for merchandising we’re looking at on faceted browse pages. Setup: our category pages are a wildcard search (
    q=*
    ) with a 
    filter_by
    that defines the category. Merchandisers pin products and set a curated sort per category, which we’d been storing as curation overrides matched on 
    rule.filter_by
    . Problem: 
    rule.filter_by
    has to match the request’s 
    filter_by
    exactly. So the moment a shopper ticks a facet (e.g. adds 
    && size:=M
    ), the request’s 
    filter_by
    no longer matches the rule, the override stops applying, and the pins + curated sort disappear instantly. Our POC fix: stop relying on stored-rule matching for browse. At query time we read the campaign’s curation and inject it as native params - 
    pinned_hits
    + 
    hidden_hits
    + 
    sort_by
    , with 
    filter_curated_hits=true
    - so they work with the 
    filter_by
    instead of getting dropped. (search campigns use overrides since they match on 
    q
    , which filtering doesn’t change.) The reason we took this approach was to avoid a “mapping” process, it means PLP for us is now almost instant on save in our Navigstion maping module. Example - “Mens Shoes”, pin SKUs A, B, C, curated sort `trending:desc`: • Shopper selects Size 10 • Request: 
    filter_by = <category> && size:=10
    , 
    pinned_hits = A:1,B:2,C:3
    , 
    sort_by = trending:desc
    , 
    filter_curated_hits = true
    • Result: B (no size 10) slides out, A & C hold their pinned positions, trending sort persists ✅ Two things we’d like to confirm: 1. Is query-time`pinned_hits`/`sort_by`the right pattern here, or is there a cleaner native way to keep a stored curation applying when the request 
    filter_by
    is a superset of the rule’s`filter_by`? 2. Does`hidden_hits`also remove those docs from`facet_counts`/ numeric stats, or only from the returned hits? (We’re propagating`hidden_hits` into our facet + histogram sub-queries to keep counts/range bounds consistent, but want to know the intended behaviour.)
    f
    • 2
    • 1
  • f
    Wrong thread
  • y
    Hey guys, I work on React Native projects and one of the things that I do part of my job is porting Rust and C++ libraries to React Native. Now, the app that I am currently working with needs a powerful search solution, and I was wondering if there is a sensible way to pack TypeSense into a mobile app?
    🚀 1
    a
    • 2
    • 12
  • g
    I have become aware that our facet counts are off when our multi_search also contains the query param 
    q
    and multiple 
    query_by
    fields. Is this expected behavior or does it sound like a flawed request body?
    f
    a
    • 3
    • 12
  • o
    Hi, I've a question regarding token separators. With the default config, when I index "WordA-WordB" I get this behavior on search: 1. "WordA" -> match 2. "WordB" -> match 3. "WordA-" -> match 4. "WordAWordB" -> match (to my surprise) 5. "WordA-WordB" -> no match 6. "-WordB" -> no match What I don't understand is case 5...
    f
    • 2
    • 4
  • s
    Hey there! Just a quick question about the roadmap and progress on JOINs: Is there any noteworthy ETA for searching across JOINed collections? Background: We have a use case of semantically(*1) searching positions (own collection with title, description etc.). Positions are bundled in parent record, giving more context (through additional context like title, description etc.). For the moment, we ingest that parent record as nested field in each item. However, this is getting pretty large and searches etc. become also cumbersome if you are e.g. just interested in that parent details. So we are eagerly awaiting options to search across JOINed collections to get the most out of it. 😬 We are already using the filtering across JOINed collections. Implementing that was a blessing finally. ❤️ *1 - actually hybrid search would be favored but semantic would be sufficient initially.
    f
    • 2
    • 2
  • g
    Hi all, I'm following GitHub issue #1014 (geo search clustering with per-cluster counts): github.com/typesense/typesense/issues/1014 The core question is whether geo 
    precision
    clustering can return the number of elements in each cluster (a 
    _group_found
    style count), which is what's needed for map clustering UIs. Right now the 
    found
    key returns the total, not per-cluster counts. Any news on this? Is per-cluster counting planned or available in a recent build? Thanks.
    j
    • 2
    • 4
  • j
    We have ran into performance issues with our 3-node typesense cluster on k8’s. Importing a handful (a few hundred) of documents takes so long that it times out or in this case we end up with a node stuck with 5000 queued writes that it does not get rid of even after 20 minutes. Can you help us diagnose the issue?
    j
    • 2
    • 1
  • a
    @Kishore Nallan @Jason Bosco Please help with conversational RAG model API This cURL mentioned in docs does not work for google/openai models. It gives me errors like `{"message": "Model namespace `` is not supported."}` and `{"message": "Model namespace 
    google
    is not supported."}`. I am using typesense v30.2
    Copy code
    ✗ curl -H "X-TYPESENSE-API-KEY: ${TYPESENSE_API_KEY}" <http://localhost:8108/debug>

{"state":1,"version":"30.2"}

CODE:
model_config = {
            "api_key": GEMINI_API_KEY,
            "model_name": GEMINI_MODEL_NAME,
            "history_collection": HISTORY_COLLECTION_NAME,
            "max_bytes": 16384,
            "system_prompt": "You are an assistant for question-answering. You can only make conversations based on the provided context. If a response cannot be formed strictly using the provided context, politely say you do not have knowledge about that topic.",
        }
create_model = tse.client.conversations_models.create(model=model_config)
print(create_model)

OUTPUT:
typesense.exceptions.RequestMalformed: [Errno 400] Gemini API error: {"message":"Bad request."}
    f
    • 2
    • 64
  • m
    Hi folks, we raise an issue with Multisearch using the go library, it doesn't return any of the metadata, this is blocking our integration at the moment using the library github.com/typesense/typesense-go/issues/226
    h
    p
    • 3
    • 5
  • s
    Question out of curiosity: With RFC 10008 being published now, any thoughts of using it and overcome things like
    TIP
    Since vector search queries tend to be large because of the large dimension of the query vector, we are using the 
    multi_search
    end-point that sends the search parameters as a POST request body.
    as per the typesense documentation? I guess any query (whether vector or not) could simply use 
    QUERY
    over 
    GET
    or 
    POST
    now. In terms of backwards compatibility, just adding the 
    QUERY
    method might not hurt actually. Have you had any thoughts on that at all yet? The typesense API just feels like a 100% fit for that new method, 😅
    f
    • 2
    • 2
  • j
    Hi, quick question regarding 
    typesense-instantsearch-adapter
    , is there a timeline for this change to be versioned and published? github.com/typesense/typesense-instantsearch-adapter/pull/274 It doesn't look like it's part of the latest version (3.0.2) which has been published
    f
    • 2
    • 2
  • s
    We just ran into another situation on our self hosted Typesense cluster again. Background / setup: We are integrating a customer's LLM Proxy Provider as endpoint for Typesense to generate the embedding for a semantic search. We have ingesting pipelines that are fully automated. Additionally, we have the collection models created as part of the infrastructure setup. Issue observed: At some point in time the semantic search on the platform started to fail. It returned with:
    Copy code
    typesense.exceptions.RequestMalformed: [Errno 400]
Malformed vector query string: could not find a field named `embedding`.
    Investigation: There have been issues with the customer's LLM Proxy Provider, leading to Typesense not able to generate the embedding. Typesense seems to decide on its own to drop the embedding entirely from the collection. Consequences: Once the LLM Proxy Provider is available again, we need to redeploy our services, setting up Typesense collections. Once done, Typesense has to go through all documents, checking whether embeddings exist (they aren't likely removed from disk, but just not linked anymore). Questions: • Is that behavior indeed intended? • Are there options to enforce the embedding field to stay, even if the provider has (long-lasting) issues?
    j
    f
    • 3
    • 3
  • n
    Has anyone had experience self hosting typesense with >500M records? As well as all the infra supporting it? Would love to pick your brain on some best practices!
    j
    • 2
    • 3
  • r
    Hello, We're looking for some guidance regarding Typesense performance after a large indexing operation. Our primary question is: Is it expected for Typesense to continue consuming very high CPU and exhibit degraded search performance after a full indexing operation of this size? If so, how long should this typically last? Is there anything we can do on our side to speed up recovery or reduce the impact? Environment • Typesense v30.2 • Single node (Frankfurt) • 2 vCPUs • 16 GB RAM • Regular SSD • Automatic Capacity Upgrade: Disabled What we did • Performed a full site indexing. • Indexed approximately 600,000 documents over ~60 hours. • Documents are distributed across two collections: ◦ ~500,000 documents ◦ ~100,000 documents What we're seeing • CPU usage has remained consistently very high even after indexing completed. • Search performance has degraded. • Search requests intermittently fail to return results for queries that should have matches. • This is impacting our production environment. If this isn't expected behavior, or anything we can for optimization? Any advice would be greatly appreciated. Thanks! @Kishore Nallan @Jason Bosco
    f
    • 2
    • 5
  • t
    Hello, we have encountered some performance issues on particular collections and after some investigations, we come to conclusion that 
    include_fields
    plays some role in performance, but our observation is that even fields that are not requested, nor queried play a major role in the overall query performance In particular we have tested: we create multiple collections that are exactly same (schema, documents) except having different embedding field in following variants • no embedding field • embedding field with 3072 dimensions • embedding field with 768 dimensions On this collections we perform queries that do NOT query this embedding fields (ie. we set 
    include_fields
    to some other fields than the embedding one described above) Our performance observation is that even if we have not requested the field and do not search over it (not used in 
    include_fields
    , 
    query_by
    or other search params), it plays major role in performance:
    Copy code
    | config                      | est. `per_page=250` | vs today    |
|-----------------------------|---------------------|-------------|
| today (3072 dims)           | 613 ms              | —           |
| 768 dims                    | \~253 ms            | 2.4x faster |
| embedding out of collection | \~156 ms            | 3.9x faster |
    Questions: • Can you please confirm this hypothesis? • This is because some kind of deserialization / materialization going on on in Typesense under the hood? • If so any recommended solutions? ◦ split into multiple collections doing joins? ◦ having enough memory cache? ◦ any other possible workarround / maybe we are doing something wrong? Thank you very much for clarifications and shedding light into this
    f
    • 2
    • 4
  • s
    Hey guys, have you seen the email from Google about Firebase Extensions being shut down?
    You are receiving this communication because one or more of your *Firebase*projects has active installations of Firebase Extensions.
    We’re writing to inform you that the Firebase Extensions service is being deprecated and will be decommissioned. Already installed Extensions will continue executing. Please review the details below to understand the upcoming timelines. Migration options will be provided in September 2026.
    What you need to know
    Key changes:
    • Installation and updates cutoff: You can install new extensions or redeploy and update existing extension instances until March 31, 2027. After this date, the Firebase console and Firebase CLI will no longer support installation or configuration modifications for Firebase Extensions.
    • Uninterrupted execution of installed extensions: Any extension instances that are already installed before March 31, 2027 will continue to execute normally after the decommissioning date.
    Impact after March 31, 2027:
    • Inability to modify configurations or update installed instances: You won't be able to re-deploy or update your installed extensions to pick up new code versions, reconfigure parameters, or apply critical bug and security patches through the Firebase Extensions service.
    • Loss of management access: You won't be able to edit configurations, or fetch configurations of installed instances (which will be helpful for migration to a new solution later). Service failures may occur if third-party API changes require patches, as you cannot apply these through the Firebase Extensions service.
    • Manual deletion required: You won't be able to uninstall extensions via the Firebase console. Instead, to uninstall an extension, you will need to manually delete all its associated Firebase and Google Cloud resources (such as functions, secrets, and service accounts).
    What you need to do
    • Uninstall unused instances: If you no longer use your installed extensions, uninstall them before March 31, 2027.
    • Update active instances: If you actively use your installed extensions, update them to their latest versions now and keep them updated up until March 31, 2027.
    • Await migration path: In September 2026, the Firebase team will reach out with comprehensive migration documentation, tools, and detailed transition options. These resources will help you migrate to self-managed functions with similar behavior, which you can continue to update and reconfigure after the Firebase Extensions service decommissioning.
    Timelines:
    • September 2026: Firebase will release comprehensive migration documentation, tools, and detailed transition options.
    • March 31, 2027 (Decommission date): This is the final day to install new extensions or redeploy and update existing instances using current Firebase Extensions functionality.
    f
    • 2
    • 6
  • f
    Hi everyone! Is there an estimated release date for v31? We've been tracking #2993 (keeping remote embedding fields instead of dropping them on collection load, closing #2988) — it looks merged/near-merged but we can't find a stable release with it yet. Just wondering if there's a rough timeline, or if v31 is still in active development with no ETA. Thanks!
    k
    • 2
    • 1
  • a
    in-built nl_query is not accurate with multi search in multiple collections, it does not work with common params(some embedd error). Please help urgent. @Fanis Tharropoulos @Kishore Nallan
    f
    • 2
    • 29
  • e
    Hey, ran into some confusing curation behavior on v30.2 and wanted to check whether it's a known bug before I open an issue. When a curation rule's trigger query contains a stopword, the rule's filter_by and replace_query seem to be matched against two different versions of the query, so they never both fire. I have a stopword set containing 
    for
    , and a curation rule (match: exact) that has both a replace_query and a filter_by. Then I search for 
    shoes for men
     with that stopword set active. What I see: - If the rule's rule.query is 
    shoes for men
    (the raw thing the user types), only replace_query fires — the filter_by is silently dropped. - If the rule's rule.query is 
    shoes men
    (what's left after the stopword for is removed), only filter_by fires — the replace_query is silently dropped. So a single rule ends up evaluated against two different strings. I'd have expected a matched rule to apply all of its actions, and for rule matching to be consistent.
    a
    • 2
    • 1
  • m
    We've got a 3-node Typesense cluster (on baremetal Kubernetes, all NVMe drives with ample resources for Typesense) that has been experiencing random high CPU load, eventually causing the cluster to die as a result of read/write lag. Any ideas on what to do here? We've heavily improved our schema and document indexing system which has resulted in 3x faster indexing speeds, but the cluster still eventually ends up no longer working. The screenshot is from earlier today where one node was pinned to 100% CPU (24 cores), 0 pending writes (and not receiving any writes for hours to begin with). Initially we thought this would be related to our full reindexing with an inefficient schema but the issue seems to occur even with our schema improvements and incremental indexing implementation (which has both significantly reduced the size of the documents we import into TS).
    a
    j
    • 3
    • 8
  • a
    Hi! Self-hosted 30.2, multi-tenant, ~250M docs already sharded into 6-month collections (~70M in the largest). Every query is scoped to a single tenantId. Want to sanity-check a few assumptions before we commit to a workaround. Can you confirm: 1. Are 
    filter_by
    clauses evaluated independently and then intersected, with no reordering by selectivity? i.e. is total cost ~ the sum of the per-clause costs, rather than bounded by the most selective one? 2. Is 
    :!=
    always computed as the complement of the equality set over the whole collection? 3. For 
    date:>=a && date:<=b
    , is the cost proportional to the documents inside the range, independent of the other clauses in the same filter? 4. Does 
    range_index:true
    accelerate the scan across a matched range, or only the endpoint lookup? We A/B'd identical int64 values in a 
    range_index
    field vs a plain one, both AND-ed with the tenant filter (3M docs, 15 runs): 16/21ms vs 14/19ms for a 450-day range, no measurable difference. Our workaround is to fold the tenant into the indexed value: a separate int64 field 
    tenantDay = tenantId * 100000 + epochDay
    , so a date window becomes 
    tenantDay:[t*100000+d1 .. t*100000+d2]
    instead of a date range, and the same idea for the status field. Every clause then hits a tenant-local posting list, with identical 
    found
    . Is a composite key the right approach for wide ranges and low-cardinality fields in a multi-tenant collection, or is there something supported we're missing? Shard by tenant as well would mean thousands of collections, so we'd rather not
    j
    • 2
    • 8
  • m
    We think we found a security issue. Any mitigation? There does not seem to be any way to limit_hits when using Union. The docs typesense.org/docs/30.2/api/federated-multi-search.html#… say:
    "The pagination parameters (page, per_page, offset, limit, and limit_hits) of individual searches are ignored. Only global pagination parameters passed as query parameters are considered."
    So that means even if you have scoped search api keys, anyone can just call multi-search and get past any limit_hits. Thoughts? Thank you! Side note - does anyone have recommendations on how to BOTH (1) benefit from typesense cloud SDN, while at the same time (2) putting it in front of a proxy using anything in AWS? For example, WAF + Cloudfront + AWS Lambda Authorizer?
    k
    a
    • 3
    • 5
  • m
    This may be easy for someone familiar with Typesense joins and scoped search keys—any guidance would be greatly appreciated. I have two collections: • `books`: 
    id
    , 
    title
    , and 
    author_id
    , where 
    author_id
    references 
    authors.id
    • `authors`: 
    id
    and 
    full_name
    Questions: 1. Can Typesense use this reference in both directions? • Search 
    books
    and include the related author’s 
    full_name
    • Search 
    authors
    and include all related books • Is the reference on 
    books.author_id
    enough, or is a reverse reference also required on 
    authors
    ? 2. How do scoped search keys work with these joins? • If the key allows searching 
    books
    but not 
    authors
    , can a books query still return joined author fields? • Must the parent key allow both collections? • Do scoped filters apply only to the main collection, or also to joined collections?
    j
    a
    • 3
    • 7
  • g
    Prefix filtering in filter_by on a string[] field seems capped at 4 distinct values — expected? Typesense v31.0.rc12 I filter products by category path. Each product has a string[] of hierarchical paths: "categories[].categories_id.slugPath": ["alimentation/miel", "alimentation/miel/miel-de-foret"] Field schema: {"name":"categories[].categories_id.slugPath","type":"string[]","facet":true,"infix":true,"optional":true} To get "this category + all its descendants" I use a prefix filter: categories[].categories_id.slugPath:[
    alimentation
    , 
    alimentation/*
    ] → found: 60 The equivalent exact filter on our denormalized label field returns 167: categories.fr-CH.lvl0:=
    Alimentation
    → found: 167 Faceting the truncated result shows exactly what happened — only 4 paths actually drove the match: 24 alimentation/autres-produits/superaliments 16 alimentation/autres-produits/condiments/miso 13 alimentation/fruits-baies-et-noix/produits-de-fruits/fruits-secs 7 alimentation/miel --- 60 = found Things that do not help: • max_candidates (tried up to 10000) — seems to apply only to q, not filter_by • exhaustive_search=true Questions 1. Is this 4-value cap on prefix expansion in filter_by intended / documented? 2. Is there a way to raise it for filter_by (a max_candidates equivalent)? 3. If not, what's the recommended way to filter "category + descendants" over a string[] of paths? Enumerating every descendant value in :[...] works, but requires a separate facet query first. Thanks!
    j
    • 2
    • 2
  • a
    How can i sort search results by specific collections? Like i want to show matches 
    brands
    collection first and then others
    f
    • 2
    • 3
  • r
    Hello, Our Typesense node is running at nearly 100% CPU utilization almost continuously, and the node frequently becomes unhealthy. Background After performing a full site indexing, the CPU remained at 100% for an extended period, causing search requests to fail. After discussing the issue here previously, we upgraded the instance from 2 vCPUs to 4 vCPUs. Current environment • Typesense v30.2 • Single-node cluster (Frankfurt) • 4 vCPUs • 16 GB RAM • Regular SSD • Automatic Capacity Upgrade: Disabled Our data is distributed across two collections: • ~500,000 documents • ~100,000 documents We expected this dataset and traffic volume to be well within the capacity of the node. Search traffic is only around 5–7 requests per second, yet the CPU remains close to 100% most of the time and the node frequently becomes unhealthy. Has anyone experienced something similar, or are there any recommendations on what we should investigate next? Thanks! @Kishore Nallan @Jason Bosco
    a
    • 2
    • 1
  • m
    Can someone clarify these docs regarding un-indexed fields? > “You want to NOT mention these fields in the collection's schema or mark these fields as `index: false`…” I am confused on the wording. It can be understood as: Do not mention the fields in the schema, and also do not mark them as 
    index: false
    .     But I think the intended meaning is: Either do not mention the fields in the schema, or add them to the schema with 
    index: false
    .     In other words, are both of these valid ways to make a field unindexed and stored only on disk?
    f
    • 2
    • 1
  • r
    Hi everyone, I'm pretty new to the Typesense community and have been experimenting with building Typesense from source for both AMD64 and ARM64 targets. So far, I've had great success with AMD64. I'm building with Bazel, and the resulting image passes tests and appears to work as expected. Where I'm getting stuck is ARM64. When I build from source and run the ARM64 image, write requests hang indefinitely on the first write operation. I've been able to reproduce this on both Apple Silicon and Linux ARM environments. What's interesting is that the official 
    typesense/typesense
    The ARM64 image from Docker Hub works perfectly on the same machines, suggesting I may be missing something in my build process rather than a general ARM issue. Before I go too far down the rabbit hole, I was curious if anyone could point me toward how the official ARM64 images are produced. Are they built directly from the public repo with Bazel, or is there a different toolchain, configuration, compiler version, build flags, or CI pipeline involved? I'm mainly trying to learn the supported build path for both AMD64 and ARM64, and would appreciate any guidance from folks who have worked with the build system. Thanks in advance!
    j
    • 2
    • 2
  • p
    Has anyone migrated from private RTD-hosted doc site search to Typesense? I would like to hear from you.
    a
    • 2
    • 2