Testing FreeBSD ethernet Interfaces

This project is on pause for now, but contains work that I’ve done in late 2020 and early 2021 to try to create a test suite to make sure that FreeBSD ethernet drivers are programmed correctly, and have the correct behaviors.

It is easy to assume that if an interface passes traffic that things are working properly, but there is much more to it than that. As far as I can tell, there is no comprehensive test suite to validate that a driver works as expected, and that the features it claims to implement are properly implemented.

A non-exhaustive list of features that could be listed as supported, but not working are jumbo frames (large MTU), hardware VLAN tagging, multicast filters, and checksum offload. In the case of the last two, a driver can claim to support these, but if the underlying bits in ifnet structure are not set, nothing will happen. Another example is that a driver could just always set that the checksum is valid, even when it is not. In most cases, this won’t be noticed, and with many protocols doing their own verification (ssh and TLS), it will just result in a dropped connection, but in other cases it will cause data corruption.

As part of this project, two sets of patches were developed, one is the kvm_ctf branch which implements the begining of an ABI agnostic method for transfering data from kernel to userland (and possibly back w/ some adaptations). It uses Compact C Type Format to understand the layout of kernel data structures and the member sizes. The good thing about this is that it can, and will work against core dumps. It would allow for minimal changes existing programs that use libkvm to get forwards compatibility.

The other part is a custom DLT for fetching the mbuf csum flags on packets. This covers both TX and RX, so can be used to verify that transmitted checksums are correctly updated AND verify that the correct flags are set on received packets. The code for this is available on the dlt_mbuf branch. The bpf.py file contains interface code for opening the BPF device and returning the necessary information.

testinterfaces.sh

This is the initial shell script version. It uses a pair of FreeBSD jails w/ vnets, one with the interface under test, and another interface that is used to generate and receive the necessary test packets. This has the advantage that it tests the full network stack, but due to various bugs, both in the FreeBSD IP stack (IPv6 addresses don’t always get assigned properly), and possibly the driver (needing packets in the opposite direction to happen before they flow), it makes this a bit difficult to fully test.

testeth.py

This was started to allow direct inspection of the checksum flags. One advantage is that as packets can be generated and sent w/ scapy/BPF, jails+vnets are not required, nor is the IP stack involved. The disadvantage is that it does not (currently) test the full network stack.

Some progress has been made in replicating the features of the shell script, but this program is not as nearly complete, nor tested as the shell version.