Karl
06/04/2024, 11:27 AM> I can't load more than 7,000
Could you elaborate on this part? What error do you see?I don't see any specific error, just that everything is quite slow in the browser (when loading products from the local file into the Typesense server and indexing), and in 8 out of 10 cases, I receive a 504 error; I have to wait about 2-5 minutes before I can retry indexing/loading into the Typesense server.
Also, are you using single document writes or bulk imports?I'm using bulk import in this method: private function import(): void { $products = file_get_contents('....'); try { $this->client->collections['products']->documents->import($products, ['action' => 'upsert']); } catch (ConnectionError $e) { throw new ConnectionError("Import error: " . $e->getMessage()); } } PS. : After many attempts and time, I managed to import 26k products as follows: I retrieve the data (500 records) from the database (provided by the client), write them into the local file data.jsonl, and then upload them into Typesense. If I try to do them simultaneously (fetching from the database and uploading to Typesense), I'll most likely get a 504 error. I think I'm doing something wrong because I'm convinced it shouldn't take this long...