#community-help

Configuring a Typesense Cluster

TLDR Chetan had trouble configuring a Typesense cluster and facing issues with "peering-address". Kishore Nallan advised defaults for network settings, understanding Docker's role in networking, and provided tips for troubleshooting. Issue resolved when Chetan used a specific Docker argument.

Powered by Struct AI
34
1mo
Solved
Join the chat
Aug 23, 2023 (1 month ago)
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
06:05 AM
some questions regarding the HA guide:
• what should be put in "peering-address"? is it required?
◦ 0.0.0.0 /internal IPs/external IP addresses trigger "failed to listen" errors
◦ leaving it blank starts, but i think typesense sees its own IP in the peers as a different peer with whom it cannot communicate, so it doesn't elect itself leader
• do the instructions for HA assume all the processes are running in the same node? i ask because they all seem to use 192.168-based IPs which are all local to a single machine?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:12 AM
Due to some recent changes necessiated for cross cluster communication, peering address can only be 0.0.0.0 -- (default) so I recommend not setting it explicitly.
06:12
Kishore Nallan
06:12 AM
> do the instructions for HA assume all the processes are running in the same node? i ask because they all seem to use 192.168-based IPs which are all local to a single machine?
No, you could have internal IPs that begin with 192.168 for separate machines as well.
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
06:23 AM
i'm a bit stumped here then as to why i can't get this node to identify itself as the leader - it should according to the HA guide:
Force one of the nodes to become a single-node cluster by editing its nodes file to contain just its own IP address
Image 1 for i'm a bit stumped here then as to why i can't get this node to identify itself as the leader - it should according to the HA guide:
```Force one of the nodes to become a single-node cluster by editing its nodes file to contain just its own IP address```
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:24 AM
If you have some older configuration which was using peering address, you should probably stop, clear the data and dir and start fresh as some core configurations could persist and prevent a reset.
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
06:27 AM
clear all data? or just one of the folders?
06:28
Chetan
06:28 AM
Image 1 for
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:29 AM
State is the only directory that mattera. Other two are populated from state.
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
06:31 AM
didn't seem to resolve everything - at this point i'm just starting an empty server
Image 1 for didn't seem to resolve everything - at this point i'm just starting an empty server
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
06:40 AM
Can you post your full config file?
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
03:23 PM
Image 1 for
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
03:24 PM
Docker networking is the other variable here.
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
03:27 PM
yup, and VM networking. but i've verified those are working because i can ping port 8107 from one VM to the other and get back results that seem correct
03:28
Chetan
03:28 PM
here's a curl from one node to port :8107 of the others (and itself)
Image 1 for here's a curl from one node to port :8107 of the others (and itself)
08:23
Chetan
08:23 PM
just verified the curl commands work inside the container too
Image 1 for just verified the curl commands work inside the container tooImage 2 for just verified the curl commands work inside the container too
10:06
Chetan
10:06 PM
so they do seem to be talking to each other (i think), it's just the one node talks to another saying "X command is for node 10.0.0.5:8107" and the server responds "yo idk who 10.0.0.5:8107 is, i'm 172.17.0.2:8107:443" (which is a local IP - they're all that specific port funny enough).
Image 1 for so they do seem to be talking to each other (i think), it's just the one node talks to another saying "X command is for node 10.0.0.5:8107" and the server responds "yo idk who 10.0.0.5:8107 is, i'm 172.17.0.2:8107:443" (which is a local IP - they're all that specific port funny enough).
Aug 24, 2023 (1 month ago)
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
04:27 AM
hi Kishore Nallan - any chance the above screenshot gives any insights on what may be up here?
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:28 AM
Can you post the contents of the nodes file also
04:31
Kishore Nallan
04:31 AM
Nodes file should not contain 0.0.0.0 -- it should have internal IPs of the nodes in the cluster.
04:33
Kishore Nallan
04:33 AM
The other suggestion I have is to specify a peering address but given the internal IP. In the screenshots, I see both 10. and 172. getting used so I wonder if there are two networks and Typesense is not picking the correct IP for cluster peering.
04:33
Kishore Nallan
04:33 AM
So give a --peering-address=10.x.x.x explicitly.
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
04:47 AM
here's the current file - just trying to get the node to identify itself as the leader.
Image 1 for here's the current file - just trying to get the node to identify itself as the leader.
04:48
Chetan
04:48 AM
and yes it doesn't seem to recognize 10.0.0.4 as itself
Image 1 for and yes it doesn't seem to recognize 10.0.0.4 as itself
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:48 AM
Try using the 172.x address
04:49
Kishore Nallan
04:49 AM
Also delete data directory and start fresh
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
04:51 AM
the 172.x address is the same though across my VMs - all of them seem to self-identify is 172.17.0.2:8107
04:51
Chetan
04:51 AM
telling them all --peering-address 172.17.0.2:8107 seems wrong?
04:52
Chetan
04:52 AM
here's what happens when i try telling it its IP is 10.0.0.5 though
Image 1 for here's what happens when i try telling it its IP is 10.0.0.5 though
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
04:52 AM
Hmm, sorry I am not familiar with Docker networking enough to help further here.
04:53
Kishore Nallan
04:53 AM
Does ifconfig list that 10.x IP?
Chetan
Photo of md5-f5bf390d5711f8a2f96bd93d1bc97d20
Chetan
04:58 AM
ah no worries, i didn't realize that docker was causing the failure to listen to 10.0.0.5
04:58
Chetan
04:58 AM
i assumed it was typesense itself.
04:59
Chetan
04:59 AM
the fix for docker was simple - i just added a --network host argument (not the most security-ideal solution, but works for this case)
Kishore Nallan
Photo of md5-4e872368b2b2668460205b409e95c2ea
Kishore Nallan
05:01 AM
👍