Packets and MBufs

NetBricks' Packets and MBufs give hints about how we should manage packet buffer memory in Sandstorm, but they aren't well documented so it's hard to see how well the approach will suit our needs. To understand them better I've documented a walkthrough of packet generation. The receive side is probably even more important, but I'm leaving that for later.

Packet Generation

Following along from net/test/packet_generation.

Key Structures

MBuf (net/framework/src/native/zcsi/mbuf.rs) represents a single packet buffer. In practice, most higher-level NetBricks NFs don't use MBufs, instead the are wrapped over with Packet documented below. The important fields are documented below as best as I can infer from the code.

Because each MBuf represents a packet buffer that must be NIC registered they also have special allocation. They are allocated with mbuf_alloc or mbuf_alloc_bulk (net/framework/src/native/zcsi/zcsi.rs) which effective in an FFI passthrough to (net/native/mempool.c) and returns a DPDK rte_mbuf* (or an array in the case of bulk). The layout of MBuf in NetBricks matches DPDK's rte_mbuf.

Generally, an rte_mbuf can represent multiple packets via next, but NetBricks doesn't use it.

pub struct MBuf {
    /// Pointer to the actual packet buffer memory.
    buf_addr: *mut u8,

    /// Completely unused within NB code.
    phys_addr: usize,

    /// 
    data_off: u16,
    refcnt: u16,
    nb_segs: u8, 
    port: u8, 
    ol_flags: u64,
    packet_type: u32,
    pkt_len: u32,
    data_len: u16,
    vlan_tci: u16,
    hash: u64,
    vlan_tci_outer: u32,
    buf_len: u16,
    timestamp: u64,
    userdata: u64,
    pool: u64,

    /// Next buffer in DPDK; unused in NetBricks.
    next: *mut MBuf,
    tx_offload: u64,
    priv_size: u16,
    timesync: u16,
    seqn: u32,
}