Issues with Importing Typesense Collection to Different Server

We did the following -

1.) exported the localhost typesense collection

2.) changed the Docusaurus URLs in the collection JSON file to those of the Docusaurus site in the test environment

3.) created a collection in the Typesense server in the test environment

4.) converted the collection JSON file to a JSONL file and

5.) attempted to import the JSONL file to the newly created collection on the Typesense server on the test platform.

Unfortunately, nothing was imported. Here is a sample of the error messages displayed:

{"code":400,"document":"\"symbology\"","error":"Bad JSON: not a properly formed document.","success":false}
{"code":400,"document":"\"etc\"","error":"Bad JSON: not a properly formed document.","success":false}
{"code":400,"document":"\"etc\"","error":"Bad JSON: not a properly formed document.","success":false}
{"code":400,"document":"\"docs-default-current\"","error":"Bad JSON: not a properly formed document.","success":false}

Would anyone know if this is the correct approach? Typesense is a great tool, but it does not appear to be possible to import a collection by itself, at least via curl.

Or maybe this is a problem with scraped docusaurus sites? Maybe the exported JSON collection file needs to be modified in some way prior to conversion to JSONL?

Typesense meets our security needs, but we do need to test it thoroughly first.

Thank you all!

NOTE: If possible, we would have scraped the test Docusaurus site, but it is behind a login and password and Cloudflare Zero Trust (CF), Google Identity-Aware Proxy (IAP) and Keycloak (KC) are not used.

In retrospect, I suppose the structure of this file does seem illogical.

After giving the exported JSON file the extension'.jsonl', I was able to initiate an import. everything processed OK for 30 seconds or so, but then the message 502 Bad Gateway appeared.

The 502 problem has gone away after we re-installed the Typesense server.

Now when I attempt to create a collection through an import, the screen returns the following:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 11.4M    0    24  100 11.4M      1   817k  0:00:14  0:00:14 --:--:--  253k{"message": "Not Found"}

The 11.4M in the screen output corresponds to the size of the JSONL file.

Note that I ran curl on bash, if that matters.

The not found issue is separate - you need to first create the collection before importing documents into it

Thanks!

With, of course, the intention of using the exported schema to create a new collection on a differnt server.

You can then the output JSON of that endpoint to the create collection endpoint: https://typesense.org/docs/0.25.0/api/collections.html#create-a-collection

But when I attempted an import, I received lots of messages such as these:

{"code":400,"document":"    \"current\"","error":"Bad JSON: not a properly formed document.","success":false}
{"code":400,"document":"  ],","error":"Bad JSON: [json.exception.parse_error.101] parse error at line 1, column 3: syntax error while parsing value - unexpected ']'; expected '[', '{', or a literal","success":false}
{"code":400,"document":"  \"weight\": {","error":"Bad JSON: [json.exception.parse_error.101] parse error at line 1, column 11: syntax error while parsing value - unexpected ':'; expected end of input","success":false}
{"code":400,"document":"    \"level\": 0,","error":"Bad JSON: [json.exception.parse_error.101] parse error at line 1, column 12: syntax error while parsing value - unexpected ':'; expected end of input","success":false}
{"code":400,"document":"    \"page_rank\": 0,","error":"Bad JSON: [json.exception.parse_error.101] parse error at line 1, column 16: syntax error while parsing value - unexpected ':'; expected end of input","success":false}
{"code":400,"document":"    \"position\": 57,","error":"Bad JSON: [json.exception.parse_error.101] parse error at line 1, column 15: syntax error while parsing value - unexpected ':'; expected end of input","success":false}
{"code":400,"document":"    \"position_descending\": 1","error":"Bad JSON: [json.exception.parse_error.101] parse error at line 1, column 26: syntax error while parsing value - unexpected ':'; expected end of input","success":

Any idea why?

Do I need to removed EOL characters in the JSONL file?

Or rather this is the file exported from the other Typesense server.

This is how the Typesense export endpoint exports docs as well

It seems like there’s some additional processing you might be doing that’s outputting a formatted JSON object with line-breaks between key values

You are right. It works. I was able to import. Thanks much!

So whether you use curl / browser / client library - it’s the same API that gets called