Resolving Typesense 0.22.2 Syntax Error and Import Issues
TLDR John faced issues with Typesense 0.22.2 code syntax and import functionality. Jason advised to reformat the JSONL records into single lines, increase ulimit, and convert dates to int64 format manually in code. A typo in the code was also corrected.
7
1
Apr 05, 2022 (21 months ago)
John
07:08 PMI have raised a ticket #568
but main issue is I don’t know which records are supposed to be the “problem”
E20220405 18:49:30.707577 15581 collection.cpp:77] JSON error: [json.exception.parse_error.101] parse error at line 1, column 8: syntax error while parsing value - unexpected ‘:’; expected end of input
The errors all say line 1
Jason
07:09 PMJason
07:09 PMhead -1 allrec2.tmp
John
07:09 PM“published”: “2021-12-08T19:26:53-05:00",
“updated”: “2021-12-08T19:26:53-05:00",
“title”: “La Florentine Panettone”,
“vendor”: “La Florentine”,
“id”: “bc7bf87d3ee7de79f767dc0c6b1f4f827eb304ce”,
“price”: “10.00",
“currency”: “USD”,
“link”: “https://torronecandy.com/products/la-florentine-panettone”,
“tag”: “BrandLa Florentine Holiday Specialties new items Panettone TypePanettone “,
“image”: “https://cdn.shopify.com/s/files/1/0234/7301/products/La-Florentine-panettone-17-6.jpg?v=1639009499”,
“summary”: “Vendor La Florentine Type Cakes Dessert Bars Price 1000 La Florentine Panettone is packed in Italy and made with an original recipe t
hat yields a light yellow center adorned with candied fruits Product of Italy 176 oz Best if heated before serving”
},
{
“published”: “2021-12-03T17:07:09-05:00",
“updated”: “2021-12-03T17:07:09-05:00",
“title”: “Italy Lanyard”,
Jason
07:10 PMJason
07:10 PMhead -1 allrec2.tmp
return?John
07:10 PMJohn
07:11 PM1
John
07:11 PMJason
07:11 PM1
John
07:21 PM1
John
07:40 PMJohn
07:40 PMJohn
07:42 PMJohn
07:42 PM1
John
07:46 PMJohn
07:47 PMJohn
07:47 PMFile “type.py”, line 23, in <module>
client.collections[‘hoybuyer’].documents.import(jsonl_file.read().encode(‘utf-8’), {‘batch_size’: 50000})
File “/home/ubuntu/.local/lib/python3.8/site-packages/typesense/documents.py”, line 71, in import
api_response = self.api_call.post(self._endpoint_path(‘import’), documents, params, as_json=False)
File “/home/ubuntu/.local/lib/python3.8/site-packages/typesense/api_call.py”, line 143, in post
return self.make_request(requests.post, endpoint, as_json,
File “/home/ubuntu/.local/lib/python3.8/site-packages/typesense/api_call.py”, line 115, in make_request
raise ApiCall.get_exception(r.status_code)(r.status_code, error_message)
typesense.exceptions.ObjectNotFound: [Errno 404] Not Found
Jason
07:48 PMJohn
07:48 PM“name”: “hotbuyer”,
“fields”: [
{“name”: “.*“, “type”: “auto” }
]
}
John
07:48 PMJason
07:48 PMJohn
07:49 PMJohn
07:49 PME20220405 19:46:23.849153 16947 raft_server.cpp:613] 4153 queued writes > healthy write lag of 500
John
07:49 PM‘nodes’: [{
‘host’: ‘localhost’, # For Typesense Cloud use xxx.a1.typesense.net
‘port’: ‘8108’, # For Typesense Cloud use 443
‘protocol’: ‘http’ # For Typesense Cloud use https
}],
‘apikey’: ‘abc’,
‘connection_timeout_seconds’: 3000
})
schema = {
“name”: “hotbuyer”,
“fields”: [
{“name”: “.*“, “type”: “auto” }
]
}
client.collections.create(schema)
with open(‘allrec2.jsonl’) as jsonl_file:
client.collections[‘hoybuyer’].documents.import(jsonl_file.read().encode(‘utf-8’), {‘batch_size’: 50000})
Jason
07:51 PMhoybuyer
in the import call1
John
07:52 PM1
John
07:54 PMJason
07:54 PM1
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Understanding the Difference Between Query by and Filter by
satish asked the difference between Query by and filter by. Kishore Nallan explained that Query by supports fuzzy searching while filter by is a precise match restriction.
Overrides/Curation Query and Filtering Docs
Jameshwart sought for filtering assistance, and Kishore Nallan suggested checking the overrides/curation section in the docs.
Full Text Search Across Long Books: Chunking vs. Single Document
Epi asked about providing FTS for long books. Kishore Nallan suggested breaking the books into chunks for better performance and query results.
Collating Search Results from Different Collections.
Patrick inquired about collating search results on different collections based on text_match value. Jason confirmed it's possible, but must be done client-side.
Using Joins Without Filtering Results in Document Retrieval
Ahmed inquired whether joins could be used without filtering results, but needs to provide the schema of collections and a sample query as suggested by Harpreet.