Troubleshooting Typesense Issues in Kubectl v1.26

I had a problem with typesense. I use typesense in kubectl v1.26. After some days automatically all the collection delete also data. is there any solution of this problem

I faced the same problem Jason

This is likely because all Typesense nodes were reset by k8s

can you please elaborate 3-node typesense cluster means , we have only one docker file for typesense deployment in AWS

In a prod environment, you want to setup Typesense in HA mode with multiple nodes: https://typesense.org/docs/guide/high-availability.html

why it's reset after some time ? also when it's reset ?

You should be using a persistent volume for the data directory, aren't you not?

no we are not currently

As long as you use a persistent disk volume like ebs you will be fine.

if you save data at pvc, it will not save the data. After some day those are also deleted.

That should not happen. Many people use this setup so it's probably an issue with your configuration.

ok got it. if you share any reference configuration it will be helpful for me. I don't find any configuration in k8s.

You can try this: https://github.com/sai3010/Typesense-Kubernetes-Operator

cc Sai who maintains it can help if you run into any issues.

Arnob Please try that operator, it should solve your issue with managing typesense , but you will have to create a storage class manually and assign it with retain policy else the data will be lost. Happy to help if you need anything.

Kishore Nallan and Sai thank you for share this resource. I will try and give u the feedback.

Sai hello, after apply the typesense Kubernetes operator it giving some error.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/aiokits/aiotasks.py", line 108, in guard
    await coro
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/engines/peering.py", line 176, in keepalive
    await touch(
  File "/usr/local/lib/python3.9/site-packages/kopf/_core/engines/peering.py", line 221, in touch
    rsp = await patching.patch_obj(
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/patching.py", line 47, in patch_obj
    patched_body = await api.patch(
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/api.py", line 155, in patch
    response = await request(
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/auth.py", line 45, in wrapper
    return await fn(*args, **kwargs, context=context)
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/api.py", line 85, in request
    await errors.check_response(response)  # but do not parse it!
  File "/usr/local/lib/python3.9/site-packages/kopf/_cogs/clients/errors.py", line 150, in check_response
    raise cls(payload, status=response.status) from e
kopf._cogs.clients.errors.APIForbiddenError: (' "default" is forbidden: User "system:serviceaccount:typesense-rnd-v4:typesense-operator-sa" cannot patch resource "clusterkopfpeerings" in API group "" at the cluster scope', {'kind': 'Status', 'apiVersion': 'v1', 'metadata': {}, 'status': 'Failure', 'message': ' "default" is forbidden: User "system:serviceaccount:typesense-rnd-v4:typesense-operator-sa" cannot patch resource "clusterkopfpeerings" in API group "" at the cluster scope', 'reason': 'Forbidden', 'details': {'name': 'default', 'group': '', 'kind': 'clusterkopfpeerings'}, 'code': 403})

Can you share your operator config file ? Please DM me

ok