#community-help

Optimizing Uploads to Typesense and Dealing with Hits Limit

TLDR Masahiro asked how to upload documents to Typesense faster and about their search feature limit_hits. Jason suggested batching documents and debugged the search query. Issue was resolved.

33mo

Join the chat

Apr 25, 2021 (33 months ago)

Photo of md5-366dff6b5f9b1a7d0f404fdc3261e573

Masahiro

06:18 AM

Hi,
I have 3 questions.
1. Faster way to upload documents to Typesense. (I’m okay with uploading speed now, but if possible I wanna make it faster.)
2. ~~advantages over meilisearch (Typesense cloud and unlimited filters?)~~ https://typesense.org/docs/overview/comparison-with-alternatives.html#typesense-vs-meilisearch
3. limit_hits did not work. search request retrieves all documents.
I’d appreciate it if you could answer my questions😄

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

08:42 PM

Masahiro 1. The fastest way to import documents is through the documents/import endpoint and ingesting a large batch of documents at once (assuming the server has sufficient capacity to handle the volume). I've been able to import 2M records, in 3 mins, importing 10K docs in one import call

08:44

Jason

08:44 PM

2. Looks like you found the comparison table! Besides the feature set, the biggest highlight I'd say is that Typesense can run in a clustered mode, whereas Meilisearch can only run on a single node and does not support a clustered configuration, which makes it a potential single point of failure in a production setting. So it's not production-ready yet.

08:45

Jason

08:45 PM

3. Could you share the exact search query you're using with limit_hits?

Apr 26, 2021 (33 months ago)

Masahiro

12:18 AM

1. thanks! I will add this tips to documentation later. and besides, Do you have any plans to add FAQ section? It will help people to find answers more quickly!
2. wow-cool. Wny not mention in the table? 😆
3. This is the query I did

const searchRequests = {
    searches: [
      {
        collection: 'users',
        q: `${ids}`,
        query_by: 'userId',
        limit_hits: 10,
        page: 1,
        per_page: 10
      }
    ]
  };
  try {
    const result = await client.multiSearch.perform(searchRequests);
    console.log(result);
    // console.log(result['results'][0]['hits']);
  } catch (e) {
    cons

Jason

02:26 AM

1. No plans yet, but may be we should add one! May be to the overview section? Feel free to add it, and I can then also add a few questions that have come up in the past.
2. It's mentioned very lightly in the table as "Fault Tolerance" . I'll update it to make it clearer.
3. What happens when you change page to 2 in the same query?

Masahiro

04:56 AM

because limit_hits: was set to 10, query failed.
so, I tried with this query, but did not work.
{
collection: ‘users’,
q: ${ids},
query_by: ‘userId’,
page: 2,
per_page: 10
}

05:11

Masahiro

05:11 AM

query found 450 documents,but hits were 10.
thanks it solved!!

Jason

05:21 AM

Yup, it will still return the number of documents found, but it won't let you paginate through them based on limit_hits

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Moving from Algolia to Typesense: Questions and Answers

Juan sought advice from Kishore Nallan about moving from Algolia to Typesense, handling MultiSearch, setting parameters, checking imported documents, and a specific syntax query.

22mo

Querying with Not-in in Typesense

Masahiro inquired about using not-in queries in Typesense. Kishore Nallan explained how to conduct such queries by using the "-" operator in the query string, and assisted Masahiro with issues stemming from a high number of exclusion tokens. The problem was eventually resolved by switching to the `multi_search` endpoint.

33mo

Typesense Capabilities and Troubleshooting Queries

A had issues with refinement lists and analytics in Typesense. Jason provided a possible solution and recommended the analytics widget. They clarified import size limits and helped identify a filter issue in A's query. Upgrade options are in Typesense's roadmap.

34mo

Retrieving All Documents: Challenges and Considerations

Masahiro asked about retrieving all documents in one query. Harisaran suggested increasing the `per_page` parameter. Kishore Nallan clarified that getting all results is possible via pagination or export, but implementing query + export feature is in the backlog. It was also discussed that extensively scrolling through search results can cause slower performance.

33mo

Increase Search Result Size and Filter Special Characters

Anton requested pagination beyond 250 hits per page and precise filtering for terms with special characters. Jason suggested multi-search as a workaround and planned to address special character filtering in an upcoming release. A Github issue was created to track this feature request.

32mo