#community-help

Document Overlapping Issue in Collections

TLDR Patrick faced issue with documents from one collection appearing in another when using the javascript library. Kishore Nallan suggested instantiating separate client objects and fixed an identified potential race condition in a newer build. Patrick confirmed the fix worked, resolving the issue.

Powered by Struct AI

2

Jul 19, 2022 (17 months ago)
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
03:05 AM
Is there some reason that documents from one collection might be appearing in another collection? I am using the javascript library and I think I've been very careful about keeping the collections separate, but searches on one collection are showing me documents that I can tell from the data should be in a different collection
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:14 AM
I think this might be because of the client object being shared across different parts of the code at the same time. Can you try instantiating separate client objects?
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
03:16 AM
Interesting thanks

Yes I can, would I want to scope the client per collection?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:22 AM
Yes we can start with that to see if the problem gets resolved.

To be clear you will face this problem only when two async functions end up sharing the same client which accesses two different collections.
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
03:38 AM
I am doing a huge amount of async work, and they all shared the same client.

I tried creating a client cache per collection name (actually alias name if that makes a difference) but I am still having the same problem
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:46 AM
Would it be possible to create a standalone example that showcases the issue?
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
03:50 AM
I think so, I'm not doing anything too crazy, just importing a bunch of things, then multi searching on them
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:51 AM
Ok if you can share a snippet I can run and debug what's happening.
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
04:05 AM
Okay, on a little more investigation it may be related to using import on the javascript client, replacing it with, roughly Promise.all(collection.upsert) seems to have solved my problem on a quick investigation
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:06 AM
Oh I see, can you post the before/after of that segment?
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
04:16 AM
Sure the before is just in comment here:


/*if (threads.length > 0) {
  await collection.documents().import(searchThreads, { action: "upsert" })
}*/
await Promise.all(searchThreads.map(c => collection.documents().upsert(c)))

I'll do a little more testing tomorrow and see if I can see what actually make a difference
04:21
Patrick
04:21 AM
Nevermind, on a second run the problem is still there using upsert, I think I just got a random successful query
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:41 AM
Ok, we can take a look at a reproduceable snippet.
Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
04:54 AM
Okay great thanks, I'll put something together tomorrow

1

Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
11:19 AM
I've identified a potential race condition that could happen locally (but super rare when you connect to Typesense on another host) that I've fixed in 0.24.0.rc20 build. Could you please test against that to see if it helps? I've published to Docker.

1

Patrick
Photo of md5-8072c418cbb65dda14994ff787349c1d
Patrick
04:35 PM
Tried from scratch 3 times in a row and didn't see the error, before that it was happening consistently at least 1 out of 2 times
04:35
Patrick
04:35 PM
Thank you! Really appreciate it
04:36
Patrick
04:36 PM
Wasn't able to reproduce yet in a new repo, my production app is pretty complicated
Jul 20, 2022 (17 months ago)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:26 AM
Thanks for confirming!

Typesense

Lightning-fast, open source search engine for everyone | Knowledge Base powered by Struct.AI

Indexed 3015 threads (79% resolved)

Join Our Community

Similar Threads

Threading Problem During Multiple Collection Creation and Batch Insertion in Typesense

Johan has a problem with creating multiple collections and batch-inserting documents into Typesense, which is returning results from different collections. Kishore Nallan helps troubleshoot the issue and suggests a potential local race condition, which is fixed in a later build.

35
17mo

Handling Kinesis Stream Event Batching with Typesense

Dui had questions about how to handle Kinesis stream events with Typesense. Kishore Nallan suggested using upsert mode for creation/update and differentiating with logical deletion. After various discussions including identifying and resolving a bug, they finalized to introduce an `emplace` action in Typesense v0.23.

8

91
24mo

Typesense Server Bulk Import/Upsert Issue Resolved

Adam was confused about the discrepancy between the successful responses and the actual indexed data while working with a custom WP plugin integrating with Typesense. The issue was a bug related to fetching documents in the wrong order, not a Typesense problem.

2

22
7mo

Troubleshooting Indexing Duration in Typesense Import

Alan asked about lengthy indexing times for importing documents to Typesense. Jason suggested various potential causes, including network connectivity and system resources. They later identified the problem to be an error in Alan's code.

5

43
15mo

Issue with Field Indexing and Multiple Data Types

Raymond encountered an issue where a field seemed to be indexed twice and hence couldn't be deleted. Jason advised upgrading to a patch version, but the problem remained. Kishore Nallan suspected a race condition and an issue with conflicting data types. An effective solution wasn't achieved.

5

51
17mo