I ran into a problem when populating my collections on production. We have a membership collection which makes a reference to the users collection. When importing documents we got these errors:

{
      "code": 400,
      "document": "{\"id\":\"01J1EQR2DJ80SN82ATEF959N0R\",\"memberId\":\"01J050RKRFAERJ7VKW9FDDQ5S6\",\"memberType\":\"user\",\"groupId\":\"01J1EQR29WW139ZZDRPEWHMQBW\",\"groupType\":\"group\",\"roles\":[\"admin\",\"member\"],\"status\":\"active\",\"createdAt\":\"2024-06-28T06:15:28.417Z\",\"updatedAt\":\"2024-06-28T06:15:28.417Z\",\"groupEntityId\":\"01J1EQR29WW139ZZDRPEWHMQBW\",\"userId\":\"01J050RKRFAERJ7VKW9FDDQ5S6\"}",
      "error": "Referenced document having `id: 01J050RKRFAERJ7VKW9FDDQ5S6` not found in the collection `users`.",
      "success": false
    },

"message": "Failed to create collection: membership: 99 documents imported successfully, 1 documents failed during import. Use `error.importResults` from the raised exception to get a detailed error reason for each document.",

"stack": "ImportError: 99 documents imported successfully, 1 documents failed during import. Use `error.importResults` from the raised exception to get a detailed error reason for each document.\n    at ImportError.TypesenseError [as constructor] (/app/.yarn/__virtual__/typesense-virtual-c5bd18db09/0/cache/typesense-npm-1.8.2-932edd9511-29df11e5ba.zip/node_modules/typesense/lib/Typesense/Errors/TypesenseError.js:23:28)\n    at new ImportError (/app/.yarn/__virtual__/typesense-virtual-c5bd18db09/0/cache/typesense-npm-1.8.2-932edd9511-29df11e5ba.zip/node_modules/typesense/lib/Typesense/Errors/ImportError.js:25:28)\n    at Documents.<anonymous> (/app/.yarn/__virtual__/typesense-virtual-c5bd18db09/0/cache/typesense-npm-1.8.2-932edd9511-29df11e5ba.zip/node_modules/typesense/lib/Typesense/Documents.js:154:39)\n    at step (/app/.yarn/__virtual__/typesense-virtual-c5bd18db09/0/cache/typesense-npm-1.8.2-932edd9511-29df11e5ba.zip/node_modules/typesense/lib/Typesense/Documents.js:48:23)\n    at Object.next (/app/.yarn/__virtual__/typesense-virtual-c5bd18db09/0/cache/typesense-npm-1.8.2-932edd9511-29df11e5ba.zip/node_modules/typesense/lib/Typesense/Documents.js:29:53)\n    at fulfilled (/app/.yarn/__virtual__/typesense-virtual-c5bd18db09/0/cache/typesense-npm-1.8.2-932edd9511-29df11e5ba.zip/node_modules/typesense/lib/Typesense/Documents.js:20:58)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)",

So, a couple of questions. Does a failed document import stop the rest of documents from importing? Or does it continue importing the remaining ones? Additionally, I am populating the membership collection like this:

export const fetchAndDenormalizeMembershipsData = async () => {
  const membershipsData = await Membership.query().select(
    "id",
    "member_id",
    "member_type",
    "group_id",
    "group_type",
    "roles",
    "status",
    "created_at",
    "updated_at",
  );

  return membershipsData.map((membership) => {
    const baseData = {
      id: membership.id,
      memberId: membership.memberId,
      memberType: membership.memberType,
      groupId: membership.groupId,
      groupType: membership.groupType,
      roles: membership.roles,
      status: membership.status,
      createdAt: membership.createdAt,
      updatedAt: membership.updatedAt,
    };

    const additionalData: Record<string, string> = {};

    if (membership.groupType === "community") {
      additionalData.communityId = membership.groupId;
    } else if (membership.groupType === "project") {
      additionalData.projectId = membership.groupId;
    } else if (membership.groupType === "group") {
      additionalData.groupEntityId = membership.groupId;
    } else if (membership.groupType === "event") {
      additionalData.eventId = membership.groupId;
    }

    // Handle memberType
    if (membership.memberType === "user") {
      additionalData.userId = membership.memberId;
    } else if (membership.memberType === "community") {
      additionalData.communityId = membership.memberId;
    } else if (membership.memberType === "project") {
      additionalData.projectId = membership.memberId;
    } else if (membership.memberType === "group") {
      additionalData.groupEntityId = membership.memberId;
    }

    return { ...baseData, ...additionalData };
  });
};

Is there any way to check if a document in the collection being referenced exists before doing the import? As to avoid trying to create documents in the

membership

collection to documents that don't exist. Is there a better way to handle the errors when importing in production? I can provide further details if necessary.