when indexing with embeddings and running into out...
# community-help
l
when indexing with embeddings and running into out_of_memory issues, we sometimes receive this error response instead:
Copy code
'SyntaxError: Unexpected non-whitespace character after JSON at position 16\n    at JSON.parse (<anonymous>)\n    at <anonymous> (..../node_modules/typesense/src/Typesense/Documents.ts:355:26)\n    at Array.map (<anonymous>)\n    at Documents.<anonymous>
It seems like there might be an internal bug where the JSONL parsing fails to properly handle the error message? i'm using
collection().import(data, {action=create})
with an array of objects.
also kind of related - our node has a base RAM consumption of 200MB, even after deleting all collections. Is this normal?
j
We fixed an issue with the import endpoint returning bad JSONL in recent RC builds. Could you try with 27.0.rc37?
re: RAM usage, that's expected because the process still holds on to memory to re-use, once you send data. Separately, the OS also needs some minimum memory to operate
1
j
Maybe someone can answer this: I have a query like:
Copy code
'q'              => '*',
            'include_fields' => 'product_id',
            'query_by'       => 'paragraph,tags,description,title,authors,isbn,isbn_print',
            'group_by'       => 'product_id',
            'group_limit'    => 1,
            'per_page'       => 24,
            'page'           => 1,
the response returns
found: 24
. As soon as I change the
q
param to something other than
*
the
found
value changes to the actual found groups, e.g.
found: 101
. This seems to be a bug. I would expect to get the total number of found groups even when the
q
param is
*
, it should not be identically to the
per_page
param. Can someone confirm? I'm not able to create a pagination otherwise, as
found: 24
and
per_page: 24
will always result in just one page for pagination.
j
Could you share a set of curl commands like this that replicates the issue with a small dataset: https://gist.github.com/jasonbosco/7c3432713216c378472f13e72246f46b#file-typesense-repro-steps-sh