#community-help

Ingesting JSON Data and UnhandledPromiseRejectionWarning Error.

TLDR vikram had an issue concerning ingestion of data from local sources and facing error. Jason helped them realize the error was due to an existing collection, and they fixed the problem.

14mo

Join the chat

Oct 23, 2022 (14 months ago)

Photo of md5-f930fdb99fd46477205fa1201164ea50

vikram

06:08 AM

Hi All, can anyone explain while running song repo. why i have to ingest first and then only i will able to see data.Also it is ingesting only 1k data but we see 30M data how?

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

07:49 AM

Can you please elaborate what you mean by "see 30M data" -- where are you seeing this?

vikram

08:24 AM

if we take example of book search engine in that we see 32M data in demo but when we run that repo in local we see we have to add 1k data then only we able to see 32M data. I don't understand how 32M data comes, if we are ingesting only 1k data on locally. let assume that remaining is store in cloud then why we need to ingest again to see the output.

Kishore Nallan

08:34 AM

Ah I see, the repo does not have all 32M docs, only 1K as sample dataset.

08:35

Kishore Nallan

08:35 AM

You can download the whole dataset from https://openlibrary.org/

vikram

08:35 AM

i am having difficulty to ingest 50M data directly to typsense.

Kishore Nallan

08:36 AM

What trouble are you facing?

vikram

08:38 AM

i have 50M data json but i have split it into 5 parts and i am ingesting one by one .but it is giving error on second.Will i able to ingest on same collection.

Kishore Nallan

08:39 AM

Again, can you please me more specific, as to what "error". In general, giving as much as information as you can will allow us to quickly get a sense of what's happening and help out.

vikram

08:43 AM

i have two json which contain 1lakh data each.when i ingest second i have to delete the first collection then only second get ingested

09:32

vikram

09:32 AM

Below is error :-

09:32

vikram

09:32 AM

(node:6650) UnhandledPromiseRejectionWarning: ObjectAlreadyExists: Request failed with HTTP code 409 | Server said: A collection with name artist already exists.
at ObjectAlreadyExists.TypesenseError [as constructor] (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28)
at new ObjectAlreadyExists (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/Errors/ObjectAlreadyExists.js:25:42)
at ApiCall.customErrorForResponse (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:337:21)
at /media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:195:98
at step (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:33:23)
at Object.next (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:14:53)
at step (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:18:139)
at Object.next (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:14:53)
at fulfilled (/media/sf_Backup/vikram/typesense-instantsearch-demo/node_modules/typesense/lib/Typesense/ApiCall.js:5:58)
at processTicksAndRejections (internal/process/task_queues.js:97:5)

Photo of md5-8813087cccc512313602b6d9f9ece19f

Jason

01:10 PM

It looks like the script you’re using to ingest is trying to create the collection even it already exists. You want to update it to only create the collection if it doesn’t already exist

vikram

01:30 PM

Thanks. working now

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Troubleshooting Typesense Docsearch Scraper Setup Issue

Vinicius experienced issues setting up typesense-docsearch-scraper locally. Jason identified a misconfiguration with the Typesense server after checking the .env file, and recommended using ngrok or port forwarding for development purposes. Vinicius successfully resolved the issue with port forwarding.

5mo

Unresolved High-Volume Write Issue in Typesense

Greg experienced issues with Typesense where it became unresponsive during high-volume write operations. Jason and Kishore Nallan suggested several solutions, but the issue remained unresolved. They suspect that the problem occurs when concurrent writes are happening to the same collection.

Large JSONL Documents Import Issue & Resolution

Suraj was having trouble loading large JSONL documents into Typesense server. After several discussions and attempts, it was discovered that the issue was due to data quality. Once the team extracted the data again, the upload process worked smoothly.

9mo

Issue Resolution and Upgrade Problems in Typesense Version 0.26rc16

Ankit reported an issue with Typesense, which was addressed by Kishore Nallan and Jason. However, Ankit experienced difficulties while trying to upgrade, with the server status showing as "Not ready or lagging" 503. The resolution remains incomplete.

2mo

Troubleshooting Issues with DocSearch Hits and Scraper Configuration

Rubai encountered issues with search result priorities and ellipsis. Jason helped debug the issue and suggested using different versions of typesense-docsearch.js, updating initialization parameters, and running the scraper on a Linux-based environment. The issues related to hits structure and scraper configuration were resolved.

131

8mo