Join over private ip

[StarpTech]

Hi, why is it necessary to pass the public IP of the master server?

• https://github.com/cicdteam/terraform-hcloud-k3s/blob/main/modules/node_group/templates/init.sh#L10

[mysticaltech]

Indeed, ideally, the private IP should be used. Also, ideally, we would not have a network overlay with flannel. We can use cilium's native routing over the private network, as the Hetzner team does in their tests here https://github.com/hetznercloud/hcloud-cloud-controller-manager/tree/master/e2etests

[systemstart]

The values don't seem to know about the server IP, it can be fetched after spinning up: hcloud server describe ${SE_DOMAIN}-master | grep -A5 "Private Net:" | grep -o "[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+\.[[:digit:]]\+"

That not possible to feed back into provisioning?

How would the overlay fix the issue?

[mysticaltech]

@systemstart Not sure I understand your question to the fullest, but definitely the CCM's job should be to bring all of that info into the cluster if I'm not mistaken. You shouldn't need to fetch the IPs manually at all.

[systemstart]

The original question was, how to use the private network for Advertise/Join.

You suggested flannel or some Go source files as possible solutions for this.

Would you go a bit more into details?

[jceb]

Fixed in multiple forks, e.g. identiops/terraform-hcloud-k3s@4edbb4a

[jceb]

Indeed, ideally, the private IP should be used. Also, ideally, we would not have a network overlay with flannel. We can use cilium's native routing over the private network, as the Hetzner team does in their tests here https://github.com/hetznercloud/hcloud-cloud-controller-manager/tree/master/e2etests

@mysticaltech Not quite sure what you mean by that and how to use the native network. Can you point me to an example? I replaced flannel by calico, maybe this is already helpful to you: identiops/terraform-hcloud-k3s@e98776b

These pieces of documentation were most helpful:

• https://github.com/hetznercloud/hcloud-cloud-controller-manager#networks-support -> and then go to the advanced documentation ->
• https://github.com/hetznercloud/hcloud-cloud-controller-manager/blob/master/docs/deploy_with_networks.md

Most of the configuration was already in place.

[mysticaltech]

Thanks, I've moved on too. I just make sure to configure the right interface to use by flannel, the eth1 with a Hetzner private network. It gives me what I needed.