wireguard: not all peers are returned in info when over a certain number of peers

[cmoiccool]

Hi,

When adding a certain number of peers to a wireguard interface (20 on some unbuntu servers, 26 on others for example) the returned informations for the interface gets incomplete returning only the first peers.

You can check running this test:

from pyroute2 import IPRoute, WireGuard
from cryptography.hazmat.primitives.asymmetric import x25519
from cryptography.hazmat.primitives import serialization
import base64
import time

def create_wireguard_interface(ifname):
    ip = IPRoute()
    ip.link("add", ifname=ifname, kind="wireguard")
    ip.link("set", ifname=ifname, state="up")
    ip.close()

def add_peer(wg, ifname, public_key, allowed_ips):
    wg.set(ifname, peer={
        'public_key': public_key,
        'allowed_ips': allowed_ips
    })

def get_peers(wg, ifname):
    info = wg.info(ifname)
    return info[0].get('WGDEVICE_A_PEERS')

def generate_key_pair():
    private_key = x25519.X25519PrivateKey.generate()
    public_key = private_key.public_key()
    public_key_bytes = public_key.public_bytes(
        encoding=serialization.Encoding.Raw,
        format=serialization.PublicFormat.Raw
    )
    public_key_base64 = base64.b64encode(public_key_bytes).decode('utf-8')
    return public_key_base64

def main():
    ifname = "wg0"
    wg = WireGuard()
    create_wireguard_interface(ifname)

    allowed_ips = ["10.0.0.1/32"]

    for i in range(1, 40):
        public_key_base64 = generate_key_pair()
        add_peer(wg, ifname, public_key_base64, allowed_ips)
        # Allow some time for the interface to update
        time.sleep(1)
        peers = get_peers(wg, ifname)
        if len(peers) != i:
            print(f"Expected {i} peers, but got {len(peers)}")
        print(f"Added peer {i}: {public_key_base64} - Total peers: {len(peers)}")

    print("All peers added successfully and verified.")

if __name__ == "__main__":
    main()

And this is what happens:

# sudo python wg_test.py
Added peer 1: dkLU2PO9rHIDVYxvfJ/OcJZXcXmZlblGe9GD0BGIzEI= - Total peers: 1
Added peer 2: Gm9hrOgCJjhmtltBXhcECzXU2fPNpXrqlnPu2lW/Z0s= - Total peers: 2
Added peer 3: v9HBJ/zqGT8E+r9adGKyom2JZgXZa/D2mgWlSkek5wY= - Total peers: 3
Added peer 4: B3Dj+M0+JaqZXMzavb7X+qGBT9Yk0Ipf4qZVJ0pOVBI= - Total peers: 4
Added peer 5: XGYO8+1KaXtHbhTHHAMIbKJ2SV2uPj48g92LXVSAABk= - Total peers: 5
Added peer 6: 0XeVVk+oIzp6NB8F7T6aNsfOUKjD+Wf4oqwkBnfjN2A= - Total peers: 6
Added peer 7: lFu/fQV61N+/wNWJfxrsN34B93T44lphC8CqWgrnzQc= - Total peers: 7
Added peer 8: q+ct3mPgQPjCsdlHCoXBcPxh5GxglMSnnJqUMUi9LCw= - Total peers: 8
Added peer 9: Gdd6yj1IlcI9Zh9Q8njhiLnwDE2y5QwIqtDK5bB2lmI= - Total peers: 9
Added peer 10: 8XgjxysojWZXoKmW/fGPiAwAquZhlnVyzob8wN5EJUw= - Total peers: 10
Added peer 11: 6T+k2Ax7IF8tzPr5EPm2pzWZre58r4UKoyJZepxf/Vk= - Total peers: 11
Added peer 12: lAqRGHw22BdiLXzyqPgRp409JclfOSmiQGIiQoxU1B0= - Total peers: 12
Added peer 13: iM8qpstc1ih3ItLx0Dx1wfqBt8LxjqCpcXphWbTdsWU= - Total peers: 13
Added peer 14: mVsCP2DQTF0zOmmzH9egWiQKo0nxxqglGmxTIj1BhX8= - Total peers: 14
Added peer 15: l1r7lZOFsgtnxYcZs856pNlAl6bOTCL9PVkeiJJjnjM= - Total peers: 15
Added peer 16: HsG5j4NlJhvIbgLj3Po++k0J4VduxGvV022aJvGTQzk= - Total peers: 16
Added peer 17: HxCfXFMJxsDgVSUJ7ecKiI0YKDTWUuH2T1KOjXkmqQ4= - Total peers: 17
Added peer 18: +5UAZAJUHfC7K1/XhNUiDz4w5LtDT/XmAM36+EFtM1c= - Total peers: 18
Added peer 19: bULWZm3asBn1FZJSzjuiTWxesakDNS4jZ9JgdFmlpzo= - Total peers: 19
Added peer 20: q/s7JN6XjRnCUp+AHlCiq0eZMBkhn9EdutxT5dowgUY= - Total peers: 20
Added peer 21: 4BZLXO9RXzGsmrgFRwRRpEmEbkdMjk1qFbmtxXv8FgY= - Total peers: 21
Added peer 22: 2kKH8QaEeZuu0sQQdzIsXBrgZjuonPB/2xf+QkvJIRg= - Total peers: 22
Added peer 23: Abplb6hjj8MGLSK3VpDKP4Oqowob8GFBqEYQMLdXOAI= - Total peers: 23
Added peer 24: bDEkTTLEf0N13SwtsPbZfqg/FmvPOOt3YWp5INYyIlU= - Total peers: 24
Added peer 25: XuaVZpQkhzp/LCig8QhpPqnmFdchOZU0Kwj4E/Qv+wQ= - Total peers: 25
Added peer 26: OdpEHdRC/zUaPuH5RzBut6fP0/emD7KT8AQufXG8LHc= - Total peers: 26
Added peer 27: twhIBbDlEzaj8hku0LjZKmELmwU5H2mEVGzVKZRVJVg= - Total peers: 27
Expected 28 peers, but got 27
Added peer 28: uK5zTwof7v7hsEQe/UMS2AhxxnYasXdvUs+Ad2lBFXc= - Total peers: 27
Expected 29 peers, but got 27
Added peer 29: o9nzD191XYR5lUCMI/x/e7bZUYfcBd587fyEdvZ/5UE= - Total peers: 27
Expected 30 peers, but got 27
Added peer 30: nu1jzKNPNjPBWq4MrKZlJYIkbHi18j0VPlhMYege5Ww= - Total peers: 27
Expected 31 peers, but got 27
Added peer 31: rq3S58zRvLh687fkH9sp8h5x7yJUQUMfpZBzVTHD4j8= - Total peers: 27
Expected 32 peers, but got 27
Added peer 32: cDwmT8wOo3YB/wl9s7G27BKp5FkN9ndaeDIcucJJ+x4= - Total peers: 27
Expected 33 peers, but got 27
Added peer 33: 0jDTNOnf56PjQXsZzsq93wGUlwYPJGCksieIdAmIuT8= - Total peers: 27
Expected 34 peers, but got 27
Added peer 34: xsLn16BaROUZS5dxoRjeJnfYA9RCES4nDb7xFhqyVR8= - Total peers: 27
Expected 35 peers, but got 27
Added peer 35: pv3k6jnhx6ZzepKspUWc5Nz2XYJOBmzkjWh7lkmYyEE= - Total peers: 27
Expected 36 peers, but got 27
Added peer 36: O+0oiF6QVJrYkoOPebbXiunK6aDhrQDoxhjgFuh8Y30= - Total peers: 27
Expected 37 peers, but got 27
Added peer 37: chWNnoeQH4RX828mpe/9k2RVEYY9HSgKFGvlIHhEdBE= - Total peers: 27
Expected 38 peers, but got 27
Added peer 38: Kw04bRUae1Ymng4rmcXNDxY/+wswiP7VU6x1QrwWDTY= - Total peers: 27
Expected 39 peers, but got 27
Added peer 39: G4Xk7Nmv6j8e+DIUVsFIfd5zmBEXbtcYDcQ+etFA1TE= - Total peers: 27
All peers added successfully and verified.

Any idea why this happens and how to solve it ?

Thanks for your help!

[cmoiccool]

After some more investigation, it seems that it is indeed related to pyroute2 wireguard implementation as with wireguard-py it does not happen.

Here is an updated version of the test that compares results from pyroute2 and wireguard-py:

from pyroute2 import IPRoute, WireGuard
from cryptography.hazmat.primitives.asymmetric import x25519
from cryptography.hazmat.primitives import serialization
import base64
import time
import wireguard_py

def create_wireguard_interface(ifname):
    ip = IPRoute()
    ip.link("add", ifname=ifname, kind="wireguard")
    ip.link("set", ifname=ifname, state="up")
    ip.close()

def add_peer(wg, ifname, public_key, allowed_ips):
    wg.set(ifname, peer={
        'public_key': public_key,
        'allowed_ips': allowed_ips
    })

def get_peers(wg, ifname):
    info = wg.info(ifname)
    return info[0].get('WGDEVICE_A_PEERS')

def generate_key_pair():
    private_key = x25519.X25519PrivateKey.generate()
    public_key = private_key.public_key()
    public_key_bytes = public_key.public_bytes(
        encoding=serialization.Encoding.Raw,
        format=serialization.PublicFormat.Raw
    )
    public_key_base64 = base64.b64encode(public_key_bytes).decode('utf-8')
    return public_key_base64

def main():
    ifname = "wg0"
    wg = WireGuard()
    create_wireguard_interface(ifname)

    allowed_ips = ["10.0.0.1/32"]

    for i in range(1, 40):
        public_key_base64 = generate_key_pair()
        add_peer(wg, ifname, public_key_base64, allowed_ips)
        # Allow some time for the interface to update
        time.sleep(1)
        peers = get_peers(wg, ifname)
        wg_py_peers = wireguard_py.list_peers(ifname.encode('utf-8'))
        if len(peers) != i:
            print(f"Expected {i} peers, but got {len(peers)}")
        if len(wg_py_peers) != i:
           print(f"Expected {i} peers, but wireguard-py got {len(peers)}")
        print(f"Added peer {i}: {public_key_base64} - Total peers: {len(peers)} - Total wireguard-py peers: {len(wg_py_peers)}")

    print("All peers added successfully and verified.")

if __name__ == "__main__":
    main()

This is what I get with this comparison test:

# python wg_test.py
Added peer 1: 77TKjHDWCBRcv3dJiDyYv+JLZ2IQUg0f2bKSPOCRTGo= - Total peers: 1 - Total wireguard-py peers: 1
Added peer 2: G+PoqYWYjD5CoElSjId4zDR3hsWCCoHWW+vREc3/z1I= - Total peers: 2 - Total wireguard-py peers: 2
Added peer 3: p3ZmFAEMfYKDb5hHh3f8Y7FEK2EUwuXO5GzWlxesSGM= - Total peers: 3 - Total wireguard-py peers: 3
Added peer 4: eFpbl4Sx0Hz6hdgHF7ZMVyStFsit5bwD6ipKsX5dC3E= - Total peers: 4 - Total wireguard-py peers: 4
Added peer 5: 2A0kZQsp54OcOveGcY5P4n42SdEvMx6qs7kW6IvzN3U= - Total peers: 5 - Total wireguard-py peers: 5
Added peer 6: EEMRNYoEm+VTO4rzmQ2sDG3J+v3Z0RDlkRdEk8rrWlg= - Total peers: 6 - Total wireguard-py peers: 6
Added peer 7: qtQlDWxPfJni6bxco3GOhJcF1Nf3Mt90hwMofLAKggw= - Total peers: 7 - Total wireguard-py peers: 7
Added peer 8: l5/MrmMgcnXWBUGULz65i2Kn2qyK50E8xZ5ZNyMWeF0= - Total peers: 8 - Total wireguard-py peers: 8
Added peer 9: ZQ+qmCHAoHkFOebAizRsh2TvoFgMwINR14FuEkNM1mw= - Total peers: 9 - Total wireguard-py peers: 9
Added peer 10: mmev99jQu2Yo6dHxz6sAUqaFVVzZRpRF/lSlRaIioVA= - Total peers: 10 - Total wireguard-py peers: 10
Added peer 11: HGClypZNzXqvlDjELDAAcJGMqV2f8CJ694tpzmj1JVc= - Total peers: 11 - Total wireguard-py peers: 11
Added peer 12: XzoXZ83OOJa80sEUzItQvsSOs1zZc+OqyTrmpjpDYFE= - Total peers: 12 - Total wireguard-py peers: 12
Added peer 13: omV3+fqYsN/VzjGRxQtc0Di+OL9slqZ9tWHoLNB3vBo= - Total peers: 13 - Total wireguard-py peers: 13
Added peer 14: B05sPxKjSYPxIhxDXvb8vlq4l1CtK3uAMU0uGZt2Qwc= - Total peers: 14 - Total wireguard-py peers: 14
Added peer 15: kUNWfmSUzkvm1bVStbYy7XcSBlysvNECwrHwMIvkQjE= - Total peers: 15 - Total wireguard-py peers: 15
Added peer 16: sN64VQ8qSuGelft41MCWJ2rbfd5s4ZIl0VZ0Xy5YJwQ= - Total peers: 16 - Total wireguard-py peers: 16
Added peer 17: Zcff69w8PT8lIsh9B/U04heWEBo3z61BreT779/0gjA= - Total peers: 17 - Total wireguard-py peers: 17
Added peer 18: clbAthLcIqSzC9daw72zBVl9UBSB0KdFvaA/fTKkAxE= - Total peers: 18 - Total wireguard-py peers: 18
Added peer 19: 0Q3Pcjnfig5/kVNtorU1UC1Hv8osxA1fj2D0oNIkV0w= - Total peers: 19 - Total wireguard-py peers: 19
Added peer 20: Wfz34DWIJySZ3WM73nie5SREuW33Jc1KKZxFP8s8al8= - Total peers: 20 - Total wireguard-py peers: 20
Added peer 21: WGCFv8eeDq3iFWnlco9pMzVBfEv0geYo3xRPBloNAhQ= - Total peers: 21 - Total wireguard-py peers: 21
Added peer 22: Yg1HBy1z4SP5qyjMu4ZTnPRp9ce4mJcIfWYG+mlHRk4= - Total peers: 22 - Total wireguard-py peers: 22
Added peer 23: tHPGG9buJ42YGMnY/3SSn/R3phzX7lUUtTCz/1YGz1s= - Total peers: 23 - Total wireguard-py peers: 23
Added peer 24: JQnBmCoBgaIF7ZGYrkTrK4cIYiXLe8R8BXBYhKxf5Eg= - Total peers: 24 - Total wireguard-py peers: 24
Added peer 25: PcrAyoCL182P5AJPpmGf/3vCctYCssDlpNMNCaqcgHk= - Total peers: 25 - Total wireguard-py peers: 25
Added peer 26: 428N818N4mWDsgDrfPQHMouFYX5GvQx4+SAl9oZcw1I= - Total peers: 26 - Total wireguard-py peers: 26
Added peer 27: r/g48+hriuq+jY580vRRvo/Jnh5SJ83PUFkIoXCcU2M= - Total peers: 27 - Total wireguard-py peers: 27
Expected 28 peers, but got 27
Added peer 28: T2QSr9c1YoDnABclU2KTH1jMrOiKGBKO7xXFV12JTCw= - Total peers: 27 - Total wireguard-py peers: 28
Expected 29 peers, but got 27
Added peer 29: s2Wd062RmLYZDT/wNT0SNmkiADTDiyH1c4hQ+ml/eFc= - Total peers: 27 - Total wireguard-py peers: 29
Expected 30 peers, but got 27
Added peer 30: 0oBYl8V56k7Y3p074+h6i29UyATQeDTqWkwKxKG4exI= - Total peers: 27 - Total wireguard-py peers: 30
Expected 31 peers, but got 27
Added peer 31: NaXMuC0Pntn37drfWOipkTbui33WeXXPJfBfFB8UQyA= - Total peers: 27 - Total wireguard-py peers: 31
Expected 32 peers, but got 27
Added peer 32: zxQ8VycVzVUqxrA/jN8XSgsMeQqA+CxOQZRAQAY1BlY= - Total peers: 27 - Total wireguard-py peers: 32
Expected 33 peers, but got 27
Added peer 33: h2Fe1upSASLs+5q+6UcZU8UZoiCJ2PHbQivf4PODt1I= - Total peers: 27 - Total wireguard-py peers: 33
Expected 34 peers, but got 27
Added peer 34: /brQkha0ewLTdSSy97dtT0fASEeNWW3lhXHcIBCpgTg= - Total peers: 27 - Total wireguard-py peers: 34
Expected 35 peers, but got 27
Added peer 35: GDojFKPcyDwAlqN6/rE9l1EChB8xTWPWGDMC7EVzXmk= - Total peers: 27 - Total wireguard-py peers: 35
Expected 36 peers, but got 27
Added peer 36: qN40euNxOm2mlgqe2N59/7/CUzY9B6qcRDX0bJStXjk= - Total peers: 27 - Total wireguard-py peers: 36
Expected 37 peers, but got 27
Added peer 37: 2e++K2iIgtqOrIwGnbij11RD/cM2pHOUSpX4yEv90Bo= - Total peers: 27 - Total wireguard-py peers: 37
Expected 38 peers, but got 27
Added peer 38: iTooRtWCjZF5gnts8C3GKEkk4DD8yPBSxgtTcKPLmkQ= - Total peers: 27 - Total wireguard-py peers: 38
Expected 39 peers, but got 27
Added peer 39: 3/7iKBJ2d59iKZoPFn+cW6kT7+3eL2zGY2XczDLkQic= - Total peers: 27 - Total wireguard-py peers: 39
All peers added successfully and verified.

[NiklasBeierl]

Hey @cmoiccool, the problem seems to be in your code for getting the peer info. wg.info(ifname) returns a a tuple of dicts for a reason: In the lower level api which pyroute2 uses (netlink) one has to deal with nasty details like binary format messages that have a fixed (or maximum) size. If the interface has too many peers, the list of peers doesn't fit into a single netlink response. Pyroute2 doesn't fully abstract this nastyness (aggregating multiple messages) away for you, but the fix is rather easy:

def get_peers(wg, ifname):
    info = wg.info(ifname)
    print(f"Info len: {len(info)}")
    return [ peer
             for sub_info in info
             for peer in sub_info.get('WGDEVICE_A_PEERS')
           ]

Best wishes