Issues deploying Typesense to AWS EKS
TLDR Pavan had issues when deploying Typesense to AWS EKS. Kishore Nallan suggested deployment to plain EC2 instances and provided the API key information. Eventually, Pavan resolved the issue with Helm.
1
Aug 03, 2022 (17 months ago)
Pavan
10:24 AMI deployed typesense to AWS EKS and it
/health
was giving {ok:true}
But when I tried to access, curl 'http://xyz:8108/keys' -X GET -H "X-TYPESENSE-API-KEY: MySuperstrongpassword#6876876578", all my pods went into Error. Here are the pod logs:
> kubectl logs typesense-dev-2
I20220803 10:20:41.334745 1 typesense_server_utils.cpp:352] Starting Typesense 0.23.1
I20220803 10:20:41.334774 1 typesense_server_utils.cpp:355] Typesense is using jemalloc.
I20220803 10:20:41.334929 1 typesense_server_utils.cpp:404] Thread pool size: 16
I20220803 10:20:41.336777 1 store.h:61] Initializing DB by opening state dir: /usr/share/typesense/data/db
I20220803 10:20:41.348294 1 store.h:61] Initializing DB by opening state dir: /usr/share/typesense/data/meta
I20220803 10:20:41.359787 1 typesense_server_utils.cpp:486] Starting API service...
I20220803 10:20:41.359961 1 http_server.cpp:177] Typesense has started listening on port 8108
I20220803 10:20:41.359997 103 batched_indexer.cpp:120] Starting batch indexer with 16 threads.
I20220803 10:20:41.360364 103 batched_indexer.cpp:126] BatchedIndexer skip_index: -9999
I20220803 10:20:41.363009 102 server.cpp:1045] Server[braft::RaftStatImpl+braft::FileServiceImpl+braft::RaftServiceImpl+braft::CliServiceImpl] is serving on port=8107.
I20220803 10:20:41.363023 102 server.cpp:1048] Check out in web browser.
E20220803 10:20:41.391083 102 raft_server.cpp:169] Unable to resolve host: typesense-0.ts.typesense.svc.cluster.local
E20220803 10:20:41.407008 102 raft_server.cpp:169] Unable to resolve host: typesense-1.ts.typesense.svc.cluster.local
E20220803 10:20:41.415683 102 raft_server.cpp:169] Unable to resolve host: typesense-2.ts.typesense.svc.cluster.local
E20220803 10:20:41.415729 102 configuration.cpp:43] Fail to parse typesense-0.ts.typesense.svc.cluster.local:8107:8108
E20220803 10:20:41.415732 102 raft_server.cpp:51] Failed to parse nodes configuration: `typesense-0.ts.typesense.svc.cluster.local:8107:8108,typesense-1.ts.typesense.svc.cluster.local:8107:8108,typesense-2.ts.typesense.svc.cluster.local:8107:8108` -- will retry shortly...
Kishore Nallan
10:26 AMPavan
10:30 AMKishore Nallan
10:31 AMPavan
10:32 AMPavan
10:40 AMI followed https://raw.githubusercontent.com/typesense/typesense-kubernetes/master/base/install.yaml
Kishore Nallan
10:50 AMPavan
11:03 AMKishore Nallan
01:33 PMAug 04, 2022 (17 months ago)
Pavan
07:09 AM1
Typesense
Indexed 3015 threads (79% resolved)
Similar Threads
Typesense Issues in Kubernetes Environment
siva.sunkara experienced Typesense issues in Kubernetes and shared logs. Kishore Nallan recommended stand-alone nodes and shared a Github issue thread. Sergio suggested fine-tuning Kubernetes deployment and linked to a working setup.
"Resolving Startup Issue with Typesense Single Replica Set in Local `K8s` Setup"
Vishal encountered issues starting a Typesense single replica set due to directory permission issues. Kishore Nallan suggested checking if the volume was accessible for writes. Vishal resolved the problem by adjusting permissions on the host.
Issues with Typesense and k8s Snapshot Restoration
Arnob experienced data loss and errors with Typesense in k8s. Kishore Nallan explained corruption could be from premature pod termination. To resolve, Kishore Nallan suggested deleting the data directory on the malfunctioning pod for automatic restoration from the leader.
Typesense Node Stuck in Segfault Loop After Stress Test
Adrian encountered a segfault loop when stress testing a Typesense cluster. Kishore Nallan recommended trying a newer RC build and suggested potential issues with hostname resolution.
Troubleshooting Typesense Issues in Kubectl v1.26
Arnob and Rubai face issues with Typesense erasing data. Jason and Kishore Nallan suggest HA setup and persistent volume usage. Sai recommends the Typesense Kubernetes operator but Arnob experiences new errors.