Hey good evening 👋 - I was getting this exact error when using documents().import() with an array of documents https://typesense-community.slack.com/archives/C01P749MET0/p1626107101113000 With typesense.js 0.14.0 I was using batch_size as an option as described in the doc but in 1.0.0 with typings, batch_size is not an accepted option. Anyways, reducing it to 1 or using the default 40 didn't matter. The only way I was able to resolve it was by literally splitting the array in half and doing two imports one after the other

@Aljosa Asanovic That particular error actually comes from node, and not Typesense. It's essentially saying there are too many properties inside each object, for converting the entire array of objects into a JSON string. The solution for such large imports would be to convert to JSONL, and then send that JSONL string into the import method and you won't run into this issue. Now the

batch_size

parameter you mention is actually a Typesense server parameter which does something different - server-side batching, after ever X documents imported, it will pause and look at the search request queue services those and then get back to importing.

Hey Jason, thanks for the answer! I see that actually in typesense js it's converted to JSONL anyways https://github.com/typesense/typesense-js/blob/a21d4101bc21fe59e0e85b41e64ba14d6fe88667/src/Typesense/Documents.ts#L112

Yup, Typesense Server only accepts JSONL for import, so the client converts arrays to JSONL before making the API call

As for the

batch_size

parameter, I believe this is a documentation bug then ?

Yup, Typesense Server only accepts JSONL for import, so the client converts arrays to JSONL before making the API call

Also notice how that method calls JSON.stringify on the entire array object. That's what causes the issue. One thing we could do is to split large arrays into smaller ones, then call JSON.stringify on them indvidiually and then concat them together. So users of the client library don't have to do this themselves...

Also notice how that method calls JSON.stringify on the entire array object. That's what causes the issue. One thing we could do is to split large arrays into smaller ones, then call JSON.stringify on them indvidiually and then concat them together. So users of the client library don't have to do this themselves...

I guess I must be close to the limit with the additional manipulations I do on the raw json I post to my server since I'm able to split the json using the same

.map()

used in typesense js

The parameter still works from Typesense Server's perspective, we need to add it to Typescript types and clarify in the docs what it exactly means. It doesn't control client-side batching, only server-side batching

Understood 🙂 , I know what you mean now with regard to my initial question being about batching before sending. And I appreciate the issue having been created! Will

batch_size

actually be sent correctly then to the server if I add it to the options of the

import()

call ?

Will batch_size actually be sent correctly then to the server if I add it to the options of the import() call ?

Yup it should be sent. Do you want to create PR adding this to the types?

Sure thing, what would you call it ?

DocumentImportParameters

?

Yeah that sounds good