Hey <@U09PMQ03M99> , yes I'll DM you with our clus...
# community-help
j
Hey @Jonathan King , yes I'll DM you with our cluster ids
j
Hi James. Could you verify that the issue is still occurring for you? I remember being able to reproduce it a week ago when I messaged, but I just peeked your clusters again and the counts seem to me lining up properly now. It may have just been a UI bug on the cloud website that got patched.
j
hey @Jonathan King, it looks like it's still happening, but with lower numbers. The following screenshot should say "4 results found from..."
cc @Terri Hounsom
👍 1
also @Kane Chang
👍 1
j
Hi @James Kirkby, I noticed a discrepancy in the retained memory on your cluster compared to the cloned one I've made. Retained memory is memory we used to be using, have since stopped using because of deleted records, but are holding onto just in case we want to re-allocate it. I just tried restarting your Typesense instance and it seems to have fixed the issue again (found is back to 4). This could suggest records are getting deleted, marked as de-allocated, but still being counted in found somehow, leading to a discrepancy over time. It would be good to have a solid way to reproduce this though before we create a bug report. Could you share any details about delete/upsert calls you are making to your testing cluster?
k
Hey @Jonathan King, apologies for the delay. Happy to send over the more details if it helps as I may not have the full context of what's already been discussed. For delete/upsert calls, we follow this process at a high level: • get documents from our API • get existing documents from typesense cluster • compare existing typesense documents and api documents: ◦ documents to upsert: ▪︎ api documents that are new (don't match any existing documents) ▪︎ api documents that match existing documents ◦ documents to delete: ▪︎ existing typesense documents that no longer exists in our API (no match) • upsert snippet:
Copy code
try { 
    await typesenseClient
      .collections(SEARCH_INDEX_PRODUCTS)
      .documents()
      .import(documentsToUpsert, { action: 'upsert' });
} catch ...
• delete snippet which runs immediately after upserting:
Copy code
for (const document of documentsToDelete) {
    try {
       await typesenseClient.collections(SEARCH_INDEX_PRODUCTS).documents(document.id).delete();
    } catch (error) { ...
Is this what you are looking for ?
j
Sorry for the late response on this. I've managed to follow the reproduction in this GitHub issue on the cloned we made of your testing cluster. The results look very similar to the bug you are experiencing and the method involves upserts followed by deletes, so it is probably the same issue. Its an open bug that should be fixed in upcoming release candidate builds, you'll want to follow the issue page and try out the fix when it comes out.
j
Hey @Jonathan King, really appreciate this, thanks for getting back to us. I've subscribed to the issue.