#community-help

Optimizing Uploads to Typesense and Dealing with Hits Limit

TLDR Masahiro asked how to upload documents to Typesense faster and about their search feature limit_hits. Jason suggested batching documents and debugged the search query. Issue was resolved.

Powered by Struct AI

1

9
33mo
Solved
Join the chat
Apr 25, 2021 (33 months ago)
Masahiro
Photo of md5-366dff6b5f9b1a7d0f404fdc3261e573
Masahiro
06:18 AM
Hi,
I have 3 questions.
1. Faster way to upload documents to Typesense. (Iโ€™m okay with uploading speed now, but if possible I wanna make it faster.)
2. advantages over meilisearch (Typesense cloud and unlimited filters?) https://typesense.org/docs/overview/comparison-with-alternatives.html#typesense-vs-meilisearch
3. limit_hits did not work. search request retrieves all documents.
Iโ€™d appreciate it if you could answer my questions๐Ÿ˜„
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
08:42 PM
Masahiro 1. The fastest way to import documents is through the documents/import endpoint and ingesting a large batch of documents at once (assuming the server has sufficient capacity to handle the volume). I've been able to import 2M records, in 3 mins, importing 10K docs in one import call
08:44
Jason
08:44 PM
2. Looks like you found the comparison table! Besides the feature set, the biggest highlight I'd say is that Typesense can run in a clustered mode, whereas Meilisearch can only run on a single node and does not support a clustered configuration, which makes it a potential single point of failure in a production setting. So it's not production-ready yet.
08:45
Jason
08:45 PM
3. Could you share the exact search query you're using with limit_hits?
Apr 26, 2021 (33 months ago)
Masahiro
Photo of md5-366dff6b5f9b1a7d0f404fdc3261e573
Masahiro
12:18 AM
1. thanks! I will add this tips to documentation later. and besides, Do you have any plans to add FAQ section? It will help people to find answers more quickly!
2. wow-cool. Wny not mention in the table? ๐Ÿ˜†
3. This is the query I did
const searchRequests = {
    searches: [
      {
        collection: 'users',
        q: `${ids}`,
        query_by: 'userId',
        limit_hits: 10,
        page: 1,
        per_page: 10
      }
    ]
  };
  try {
    const result = await client.multiSearch.perform(searchRequests);
    console.log(result);
    // console.log(result['results'][0]['hits']);
  } catch (e) {
    cons
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
02:26 AM
1. No plans yet, but may be we should add one! May be to the overview section? Feel free to add it, and I can then also add a few questions that have come up in the past.
2. It's mentioned very lightly in the table as "Fault Tolerance" . I'll update it to make it clearer.
3. What happens when you change page to 2 in the same query?

1

Masahiro
Photo of md5-366dff6b5f9b1a7d0f404fdc3261e573
Masahiro
04:56 AM
because limit_hits: was set to 10, query failed.
so, I tried with this query, but did not work.
{
collection: โ€˜usersโ€™,
q: ${ids},
query_by: โ€˜userIdโ€™,
page: 2,
per_page: 10
}
05:11
Masahiro
05:11 AM
query found 450 documents,but hits were 10.
thanks it solved!!
Jason
Photo of md5-8813087cccc512313602b6d9f9ece19f
Jason
05:21 AM
Yup, it will still return the number of documents found, but it won't let you paginate through them based on limit_hits

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community

Similar Threads

Moving from Algolia to Typesense: Questions and Answers

Juan sought advice from Kishore Nallan about moving from Algolia to Typesense, handling MultiSearch, setting parameters, checking imported documents, and a specific syntax query.

1

22
22mo
Solved

Querying with Not-in in Typesense

Masahiro inquired about using not-in queries in Typesense. Kishore Nallan explained how to conduct such queries by using the "-" operator in the query string, and assisted Masahiro with issues stemming from a high number of exclusion tokens. The problem was eventually resolved by switching to the `multi_search` endpoint.

4

49
33mo
Solved

Typesense Capabilities and Troubleshooting Queries

A had issues with refinement lists and analytics in Typesense. Jason provided a possible solution and recommended the analytics widget. They clarified import size limits and helped identify a filter issue in A's query. Upgrade options are in Typesense's roadmap.

4

32
34mo
Solved

Retrieving All Documents: Challenges and Considerations

Masahiro asked about retrieving all documents in one query. Harisaran suggested increasing the `per_page` parameter. Kishore Nallan clarified that getting all results is possible via pagination or export, but implementing query + export feature is in the backlog. It was also discussed that extensively scrolling through search results can cause slower performance.

3

17
33mo
Solved

Increase Search Result Size and Filter Special Characters

Anton requested pagination beyond 250 hits per page and precise filtering for terms with special characters. Jason suggested multi-search as a workaround and planned to address special character filtering in an upcoming release. A Github issue was created to track this feature request.

3

7
32mo
Solved