Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install iptables rules to allow wireguard packets #7030

Merged
merged 1 commit into from
Mar 4, 2025
Merged

Install iptables rules to allow wireguard packets #7030

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion cmd/antrea-agent/agent.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,9 @@ func run(o *Options) error {
multicastEnabled,
o.config.SNATFullyRandomPorts,
*o.config.Egress.SNATFullyRandomPorts,
serviceCIDRProvider)
serviceCIDRProvider,
wireguardConfig.Port,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another concern is about the WireGuard port check, looks like we didn't check the range before:

	if o.config.WireGuard.Port == 0 {
		o.config.WireGuard.Port = apis.WireGuardListenPort
	}

I suppose we should go through and update the validation to make sure it's a valid port? @antoninbas

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good to me but we should do it in a separate PR, we have other ports that we could validate as well, e.g. the tunnelPort. Can you open an issue to track this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Issue created: #7035, I will take a look at it later today.

)
if err != nil {
return fmt.Errorf("error creating route client: %v", err)
}
Expand Down
27 changes: 15 additions & 12 deletions docs/network-requirements.md
View file
Open in desktop
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably try to improve this file in the future, as the configurable column is a bit generic and maybe should indicate how the port can be configured. we could also extract the kube-apiserver requirement from the table and mention it separately, as the requirement doesn't come from Antrea.

Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,24 @@
Antrea has a few network requirements to get started, ensure that your hosts and
firewalls allow the necessary traffic based on your configuration.

| Configuration | Host(s) | Protocols/Ports | Other |
|------------------------------------------------|---------------------------------------|--------------------------------------------|------------------------------|
| Antrea with VXLAN enabled | All | UDP 4789 | |
| Antrea with Geneve enabled | All | UDP 6081 | |
| Antrea with STT enabled | All | TCP 7471 | |
| Antrea with GRE enabled | All | IP Protocol ID 47 | No support for IPv6 clusters |
| Antrea with IPsec ESP enabled | All | IP protocol ID 50 and 51, UDP 500 and 4500 | |
| Antrea with WireGuard enabled | All | UDP 51820 | |
| Antrea Multi-cluster with WireGuard encryption | Multi-cluster Gateway Node | UDP 51821 | |
| Antrea with feature BGPPolicy enabled | Selected by user-provided BGPPolicies | TCP 179<sup>[1]</sup> | |
| All | Kube-apiserver host | TCP 443 or 6443<sup>[2]</sup> | |
| All | All | TCP 10349, 10350, 10351, UDP 10351 | |
| Configuration | Host(s) | Protocols/Ports | Configurable | Other |
Copy link
Member

@tnqn tnqn Feb 28, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The table is not displayed properly: https://github.com/antrea-io/antrea/blob/a209673dfb1f8e155f1f644d5295f56351986c48/docs/network-requirements.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

|------------------------------------------------|---------------------------------------|--------------------------------------------|--------------|------------------------------|
| Antrea with VXLAN enabled | All | UDP 4789 | Yes | |
| Antrea with Geneve enabled | All | UDP 6081 | Yes | |
| Antrea with STT enabled | All | TCP 7471 | Yes | |
| Antrea with GRE enabled | All | IP Protocol ID 47 | No | No support for IPv6 clusters |
| Antrea with IPsec ESP enabled | All | IP protocol ID 50 and 51, UDP 500 and 4500 | No | |
| Antrea with WireGuard enabled | All | UDP 51820<sup>[3]</sup> | Yes | |
| Antrea Multi-cluster with WireGuard encryption | Multi-cluster Gateway Node | UDP 51821 | Yes | |
| Antrea with feature BGPPolicy enabled | Selected by user-provided BGPPolicies | TCP 179<sup>[1]</sup> | Yes | |
| All | Kube-apiserver host | TCP 443 or 6443<sup>[2]</sup> | Yes | |
| All | All | TCP 10349, 10350, 10351, UDP 10351 | Yes | |

[1] _The default value is 179, but a user created BGPPolicy can assign a different
port number._

[2] _The value is passed to kube-apiserver `--secure-port` flag. You can find the port
number from the output of `kubectl get svc kubernetes -o yaml`._

[3] _Antrea automatically adds the firewall rules to allow the WireGuard packets
(starting from v2.4), so the manual configuration on the host is not needed._
91 changes: 69 additions & 22 deletions pkg/agent/route/route_linux.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ import (
"github.com/containernetworking/plugins/pkg/ip"
"github.com/vishvananda/netlink"
"golang.org/x/sys/unix"
"k8s.io/apimachinery/pkg/util/intstr"
"k8s.io/apimachinery/pkg/util/sets"
"k8s.io/apimachinery/pkg/util/wait"
"k8s.io/klog/v2"
Expand Down Expand Up @@ -158,10 +159,17 @@ type Client struct {
nodeNetworkPolicyIPTablesIPv4 sync.Map
// nodeNetworkPolicyIPTablesIPv6 caches all existing IPv6 iptables chains and rules for NodeNetworkPolicy.
nodeNetworkPolicyIPTablesIPv6 sync.Map
// wireguardIPTablesIPv4 caches all existing IPv4 iptables chains and rules for WireGuard.
wireguardIPTablesIPv4 sync.Map
// wireguardIPTablesIPv6 caches all existing IPv6 iptables chains and rules for WireGuard.
wireguardIPTablesIPv6 sync.Map
// deterministic represents whether to write iptables chains and rules for NodeNetworkPolicy deterministically when
// syncIPTables is called. Enabling it may carry a performance impact. It's disabled by default and should only be
// used in testing.
deterministic bool
// wireguardPort is the port used for the WireGuard UDP tunnels. When WireGuard is enabled (used as the encryption
// mode), we add iptables rules to the filter table to accept input and output UDP traffic destined to this port.
wireguardPort int
}

// NewClient returns a route client.
Expand All @@ -173,7 +181,8 @@ func NewClient(networkConfig *config.NetworkConfig,
multicastEnabled bool,
nodeSNATRandomFully bool,
egressSNATRandomFully bool,
serviceCIDRProvider servicecidr.Interface) (*Client, error) {
serviceCIDRProvider servicecidr.Interface,
wireguardPort int) (*Client, error) {
return &Client{
networkConfig: networkConfig,
noSNAT: noSNAT,
Expand All @@ -194,6 +203,7 @@ func NewClient(networkConfig *config.NetworkConfig,
antreaExternalIPIPSet: {},
antreaExternalIPIP6Set: {},
},
wireguardPort: wireguardPort,
}, nil
}

Expand Down Expand Up @@ -265,7 +275,9 @@ func (c *Client) Initialize(nodeConfig *config.NodeConfig, done func()) error {
if c.nodeNetworkPolicyEnabled {
c.initNodeNetworkPolicy()
}

if c.networkConfig.TrafficEncryptionMode == config.TrafficEncryptionModeWireGuard {
c.initWireguard()
}
return nil
}

Expand Down Expand Up @@ -675,7 +687,7 @@ func (c *Client) syncIPTables() error {
if c.proxyAll {
jumpRules = append(jumpRules, jumpRule{iptables.NATTable, iptables.OutputChain, antreaOutputChain, "Antrea: jump to Antrea output rules", true})
}
if c.nodeNetworkPolicyEnabled {
if c.nodeNetworkPolicyEnabled || c.networkConfig.TrafficEncryptionMode == config.TrafficEncryptionModeWireGuard {
jumpRules = append(jumpRules, jumpRule{iptables.FilterTable, iptables.InputChain, antreaInputChain, "Antrea: jump to Antrea input rules", false})
jumpRules = append(jumpRules, jumpRule{iptables.FilterTable, iptables.OutputChain, antreaOutputChain, "Antrea: jump to Antrea output rules", false})
}
Expand Down Expand Up @@ -711,20 +723,24 @@ func (c *Client) syncIPTables() error {
return true
})

nodeNetworkPolicyIPTablesIPv4 := map[string][]string{}
nodeNetworkPolicyIPTablesIPv6 := map[string][]string{}
c.nodeNetworkPolicyIPTablesIPv4.Range(func(key, value interface{}) bool {
chain := key.(string)
rules := value.([]string)
nodeNetworkPolicyIPTablesIPv4[chain] = rules
return true
})
c.nodeNetworkPolicyIPTablesIPv6.Range(func(key, value interface{}) bool {
chain := key.(string)
rules := value.([]string)
nodeNetworkPolicyIPTablesIPv6[chain] = rules
return true
})
addFilterRulesToChain := func(iptablesRulesByChain map[string][]string, m *sync.Map) {
m.Range(func(key, value interface{}) bool {
chain := key.(string)
rules := value.([]string)
iptablesRulesByChain[chain] = append(iptablesRulesByChain[chain], rules...)
return true
})
}

iptablesFilterRulesByChainV4 := make(map[string][]string)
// Install the static rules (WireGuard for now) before the dynamic rules (e.g., NodeNetworkPolicy)
// for performance reasons.
addFilterRulesToChain(iptablesFilterRulesByChainV4, &c.wireguardIPTablesIPv4)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add a comment explaining that we are intentionally adding static iptables rules (WireGuard for now) first for performance reasons.

addFilterRulesToChain(iptablesFilterRulesByChainV4, &c.nodeNetworkPolicyIPTablesIPv4)

iptablesFilterRulesByChainV6 := make(map[string][]string)
addFilterRulesToChain(iptablesFilterRulesByChainV6, &c.wireguardIPTablesIPv6)
addFilterRulesToChain(iptablesFilterRulesByChainV6, &c.nodeNetworkPolicyIPTablesIPv6)

// Use iptables-restore to configure IPv4 settings.
if c.networkConfig.IPv4Enabled {
Expand All @@ -737,7 +753,7 @@ func (c *Client) syncIPTables() error {
config.VirtualNodePortDNATIPv4,
config.VirtualServiceIPv4,
snatMarkToIPv4,
nodeNetworkPolicyIPTablesIPv4,
iptablesFilterRulesByChainV4,
false)

// Setting --noflush to keep the previous contents (i.e. non antrea managed chains) of the tables.
Expand All @@ -757,7 +773,7 @@ func (c *Client) syncIPTables() error {
config.VirtualNodePortDNATIPv6,
config.VirtualServiceIPv6,
snatMarkToIPv6,
nodeNetworkPolicyIPTablesIPv6,
iptablesFilterRulesByChainV6,
true)
// Setting --noflush to keep the previous contents (i.e. non antrea managed chains) of the tables.
if err := c.iptables.Restore(iptablesData.String(), false, true); err != nil {
Expand All @@ -777,7 +793,7 @@ func (c *Client) restoreIptablesData(podCIDR *net.IPNet,
nodePortDNATVirtualIP,
serviceVirtualIP net.IP,
snatMarkToIP map[uint32]net.IP,
nodeNetWorkPolicyIPTables map[string][]string,
iptablesFiltersRuleByChain map[string][]string,
isIPv6 bool) *bytes.Buffer {
// Create required rules in the antrea chains.
// Use iptables-restore as it flushes the involved chains and creates the desired rules
Expand Down Expand Up @@ -897,7 +913,7 @@ func (c *Client) restoreIptablesData(podCIDR *net.IPNet,
writeLine(iptablesData, iptables.MakeChainLine(antreaForwardChain))

var nodeNetworkPolicyIPTablesChains []string
for chain := range nodeNetWorkPolicyIPTables {
for chain := range iptablesFiltersRuleByChain {
nodeNetworkPolicyIPTablesChains = append(nodeNetworkPolicyIPTablesChains, chain)
}
if c.deterministic {
Expand Down Expand Up @@ -937,7 +953,7 @@ func (c *Client) restoreIptablesData(podCIDR *net.IPNet,
}...)
}
for _, chain := range nodeNetworkPolicyIPTablesChains {
for _, rule := range nodeNetWorkPolicyIPTables[chain] {
for _, rule := range iptablesFiltersRuleByChain[chain] {
writeLine(iptablesData, rule)
}
}
Expand Down Expand Up @@ -1198,6 +1214,37 @@ func (c *Client) initNodeNetworkPolicy() {
}
}

func (c *Client) initWireguard() {
wireguardPort := intstr.FromInt(c.wireguardPort)
antreaInputChainRules := []string{
iptables.NewRuleBuilder(antreaInputChain).
SetComment("Antrea: allow WireGuard input packets").
MatchTransProtocol(iptables.ProtocolUDP).
MatchPortDst(&wireguardPort, nil).
SetTarget(iptables.AcceptTarget).
Done().
GetRule(),
}
antreaOutputChainRules := []string{
iptables.NewRuleBuilder(antreaOutputChain).
SetComment("Antrea: allow WireGuard output packets").
MatchTransProtocol(iptables.ProtocolUDP).
MatchPortDst(&wireguardPort, nil).
SetTarget(iptables.AcceptTarget).
Done().
GetRule(),
}

if c.networkConfig.IPv6Enabled {
c.wireguardIPTablesIPv6.Store(antreaInputChain, antreaInputChainRules)
c.wireguardIPTablesIPv6.Store(antreaOutputChain, antreaOutputChainRules)
}
if c.networkConfig.IPv4Enabled {
c.wireguardIPTablesIPv4.Store(antreaInputChain, antreaInputChainRules)
c.wireguardIPTablesIPv4.Store(antreaOutputChain, antreaOutputChainRules)
}
}

// Reconcile removes orphaned podCIDRs from ipset and removes routes to orphaned podCIDRs
// based on the desired podCIDRs.
func (c *Client) Reconcile(podCIDRs []string) error {
Expand Down
Loading