#community-help

Document Overlapping Issue in Collections

TLDR Patrick faced issue with documents from one collection appearing in another when using the javascript library. Kishore Nallan suggested instantiating separate client objects and fixed an identified potential race condition in a newer build. Patrick confirmed the fix worked, resolving the issue.

17mo

Join the chat

Jul 19, 2022 (17 months ago)

Photo of md5-8072c418cbb65dda14994ff787349c1d

Patrick

03:05 AM

Is there some reason that documents from one collection might be appearing in another collection? I am using the javascript library and I think I've been very careful about keeping the collections separate, but searches on one collection are showing me documents that I can tell from the data should be in a different collection

Photo of md5-4e872368b2b2668460205b409e95c2ea

Kishore Nallan

03:14 AM

I think this might be because of the client object being shared across different parts of the code at the same time. Can you try instantiating separate client objects?

Patrick

03:16 AM

Interesting thanks

Yes I can, would I want to scope the client per collection?

Kishore Nallan

03:22 AM

Yes we can start with that to see if the problem gets resolved.

To be clear you will face this problem only when two async functions end up sharing the same client which accesses two different collections.

Patrick

03:38 AM

I am doing a huge amount of async work, and they all shared the same client.

I tried creating a client cache per collection name (actually alias name if that makes a difference) but I am still having the same problem

Kishore Nallan

03:46 AM

Would it be possible to create a standalone example that showcases the issue?

Patrick

03:50 AM

I think so, I'm not doing anything too crazy, just importing a bunch of things, then multi searching on them

Kishore Nallan

03:51 AM

Ok if you can share a snippet I can run and debug what's happening.

Patrick

04:05 AM

Okay, on a little more investigation it may be related to using import on the javascript client, replacing it with, roughly Promise.all(collection.upsert) seems to have solved my problem on a quick investigation

Kishore Nallan

04:06 AM

Oh I see, can you post the before/after of that segment?

Patrick

04:16 AM

Sure the before is just in comment here:

/*if (threads.length > 0) {
  await collection.documents().import(searchThreads, { action: "upsert" })
}*/
await Promise.all(searchThreads.map(c => collection.documents().upsert(c)))

I'll do a little more testing tomorrow and see if I can see what actually make a difference

04:21

Patrick

04:21 AM

Nevermind, on a second run the problem is still there using upsert, I think I just got a random successful query

Kishore Nallan

04:41 AM

Ok, we can take a look at a reproduceable snippet.

Patrick

04:54 AM

Okay great thanks, I'll put something together tomorrow

Kishore Nallan

11:19 AM

I've identified a potential race condition that could happen locally (but super rare when you connect to Typesense on another host) that I've fixed in 0.24.0.rc20 build. Could you please test against that to see if it helps? I've published to Docker.

Patrick

04:35 PM

Tried from scratch 3 times in a row and didn't see the error, before that it was happening consistently at least 1 out of 2 times

04:35

Patrick

04:35 PM

Thank you! Really appreciate it

04:36

Patrick

04:36 PM

Wasn't able to reproduce yet in a new repo, my production app is pretty complicated

Jul 20, 2022 (17 months ago)

Kishore Nallan

06:26 AM

Thanks for confirming!

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Threading Problem During Multiple Collection Creation and Batch Insertion in Typesense

Johan has a problem with creating multiple collections and batch-inserting documents into Typesense, which is returning results from different collections. Kishore Nallan helps troubleshoot the issue and suggests a potential local race condition, which is fixed in a later build.

17mo

Handling Kinesis Stream Event Batching with Typesense

Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.

24mo

Typesense Server Bulk Import/Upsert Issue Resolved

Adam was confused about the discrepancy between the successful responses and the actual indexed data while working with a custom WP plugin integrating with Typesense. The issue was a bug related to fetching documents in the wrong order, not a Typesense problem.

7mo

Troubleshooting Indexing Duration in Typesense Import

Alan asked about lengthy indexing times for importing documents to Typesense. Jason suggested various potential causes, including network connectivity and system resources. They later identified the problem to be an error in Alan's code.

15mo

Issue with Field Indexing and Multiple Data Types

Raymond encountered an issue where a field seemed to be indexed twice and hence couldn't be deleted. Jason advised upgrading to a patch version, but the problem remained. Kishore Nallan suspected a race condition and an issue with conflicting data types. An effective solution wasn't achieved.

17mo