#community-help

Deletion Anomaly in Bulk Import Operations.

TLDR Mubashirullah found documents not returning post-deletion and re-run of a bulk import. Kishore Nallan suggested checking the operations and trying on a smaller dataset. The problem didn't recur for Mubashirullah.

Powered by Struct AI

1

1

12
25mo
Solved
Join the chat
Dec 22, 2021 (25 months ago)
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:06 AM
I just found this unusual anomaly. I deleted a few documents by id. Then I ran the bulk import again but those documents I deleted are still gone. Search no longer returns them.
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:10 AM
Check the responses from the deletion and bulk import operations to verify that the operations really went through.

Happy to investigate if you can reproduce it consistently on a small dataset which I can also run to see the problem.

1

Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:18 AM
How can I have fresh instance. When I delete and deploy the docker container again, it loads things from the hard disk
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:19 AM
Delete the contents of the data directory.
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:20 AM
this one right?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:20 AM
Yes
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:28 AM
im finding it hard to delete this folder
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:28 AM
Stop container, delete folder and start container back up.
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:32 AM
The response from bulk import prints like this
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
09:32 AM
Try it on a small dataset to reproduce first.
Mubashirullah
Photo of md5-cea7a558edb0c66a1c07dfdcf00bc141
Mubashirullah
09:45 AM
I tried it on a 40k dataset. The problem did not repeat. Interesting. Let me try it on a smaller dataset as well.
09:52
Mubashirullah
09:52 AM
yes. Everything is in order. Thank you

1

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3011 threads (79% resolved)

Join Our Community