Ethernet driver for xv6

Xv6 – educational OS – tells about the ideas that underlie operating systems.

We will teach xv6 to work in the network, get acquainted with the VirtIO virtual device standard, the DeviceTree device tree, Ethernet technology, network protocols, and build a network bridge between virtual machines.

The driver helps the OS communicate with the network adapter. The OS asks the driver to send packets to the network. The driver tells the OS when packets arrive from the network.
The driver and network adapter maintain queues of packets – outgoing and incoming. The driver places an outgoing packet in the queue so that the adapter sends the packet to the network. The adapter receives a packet from the network and places it in the incoming queue so that the driver reads the packet.
The network adapter sends Ethernet packets to the network, and the driver packs the information into Ethernet packets before sending them. The Ethernet network adapter forwards the packet to another adapter – it specifies the MAC address of the receiving adapter in the packet.
An Ethernet packet contains packets of other protocols – IP, TCP, UDP, ARP, etc. An Ethernet adapter communicates with other Ethernet adapters, but programs communicate with programs on other computers and do not know about Ethernet.

Let's connect the network adapter
Xv6 runs on a QEMU virtual machine. Let's specify the options device And netdevto connect the network adapter virtio-net to the machine and link it to the tap adapter on the host machine. The adapter virtio-net will pass packets to the tap adapter, and tap to the adapter virtio-net. Let's run Wireshark on the host and see the packets on tap.
QEMUOPTS += -device virtio-net-device,netdev=mynet,bus=virtio-mmio-bus.1,mac=52:54:00:78:76:36	# 'xv6' in ASCII
QEMUOPTS += -netdev tap,id=mynet
Makefile
Parameter bus=virtio-mmio-bus.1 means that QEMU will map the adapter registers into memory. Xv6 similarly connects the hard drive – specifies the parameter bus=virtio-mmio-bus.0.
QEMUOPTS += -device virtio-blk-device,drive=x0,bus=virtio-mmio-bus.0
The code determines the addresses of the device registers by the offset of the base address. Example: virtio disk base address – VIRTIO0 = 0x10001000. Find the base address of the network adapter. QEMU prints the device tree when it receives the option dumpdtb. Program dtc converts a tree from binary format to text.
qemu-system-riscv64 -M virt,dumpdtb=virt.dtb -nographic
dtc -I dtb -O dts virt.dtb >> virt.dts
Tree nodes virtio_mmio@XXXXXXXX – connectors on the bus virtio-mmio. Address XXXXXXXX – the base memory address from which the device will map registers. QEMU numbers the sockets in ascending order of addresses, so virtio-mmio-bus.1 will receive an address 0x10002000. File kernel/virtio.h defines the offsets of virtio device registers from the base address.
Devicetree fragment
The driver communicates with the device by writing and reading the device's registers.
Let's add the memory area with the network adapter registers to the kernel page table – otherwise the driver will get a memory access error when accessing the adapter register. Chapter 3 will tell you more about page tables.
kernel/vm.c:34
  // virtio mmio net interface
  kvmmap(kpgtbl, VIRTIO1, VIRTIO1, PGSIZE, PTE_R | PTE_W);
Let's configure the network adapter

Let's configure the adapter when the OS starts:
Let's reset the adapter.
Let's tell the OS that the adapter is visible – set the bit ACKNOWLEDGE status register.
Let's say that the OS can manage the adapter – set the bit DRIVER status register.
We agree on the set of adapter capabilities – the primitive network driver uses only VIRTIO_NET_F_MAC.
Confirm your choice – set the bit FEATURES_OK status register.
Let's read the beat again FEATURES_OK and make sure the bit is set, otherwise the device will reject that set of capabilities.
Let's read the MAC address of the adapter and prepare queues for incoming and outgoing packets.
Let's set the bit DRIVER_OK status register.
Function virtio_net_init configures the adapter.
VirtIO Queues
The driver and device pass memory buffers to each other using queues. Putting a buffer into a VirtIO queue is more complicated than adding an element to a singly linked list. Buckle up.
The VirtIO queue contains three arrays:
An array of descriptors. Each descriptor describes a memory buffer.
Array available contains the numbers of descriptors that the driver passed to the device.
Array used contains the descriptor numbers that the device passed to the driver.
The driver searches for a free descriptor, fills the buffer and writes the descriptor number to the array. availableto pass the buffer to the device. The device writes the descriptor number to the array usedwhen returning the buffer to the driver.
The device does not write to the descriptor array and works only with the buffers provided by the driver. The driver fills the incoming packet queue with buffers so that the adapter can accept packets from the network.
We send packages
Let's write a program ping and add a system call to the xv6 kernel pingwhich asks the driver to send a packet to the network.
We write tests. Tests save nerves and time.
We write functions after tests:
Let's declare the kernel function sys_ping and add to the array syscalls Under the number SYS_ping. Let's declare a system call ping V user/user.h and create an entry point into the kernel using a script user/usys.plChapter 4 covers system calls in more detail.
System call ping accepts the IP address of the host to which it will send the ICMP ECHO.
Function virtio_net_intr processes interrupts when the adapter sends or receives a packet. Chapter 5 covers device interrupts.
Let's run the program ping and we will see that the packet has gone into the network.
ping 192.168.56.100
We accept packages
The adapter interrupts the processor when it receives a packet from the network. Function virtio_net_intr processes incoming packets as well. The driver processes the incoming packet and returns the buffer to the incoming packet queue.
We write tests:
We write functions after tests:
Helper functions parse and print packet headers:
Let's run xv6 and see how random packets from the network get to the adapter, and the driver prints packet headers.
[Ethernet] EtherType: 0x86dd SourceAddress: 26:e3:ce:e6:aa:83 DestinationAddress: 33:33:0:0:0:fb
[IPv6] NextHeader: 17 PayloadSize: 149 SourceAddress: fe80:00:00:00:24e3:ceff:fee6:aa83 DestinationAddress: ff2:00:00:00:00:00:00:0fb
[UDP] SourcePort: 5353 DestinationPort: 5353 Length: 149 Checksum: 0xad89
We will send the package from another machine
Let's launch another VM under QEMU and create a bridge between network adapters tap0 And tap1 host. Let's send an ICMP ECHO and see that xv6 receives ARP requests. The machine sends an ARP request to the network to find out the MAC address of the owner of the IP address.
# /etc/network/interfaces.d/br0 on Debian host machine
auto br0
        iface br0 inet dhcp
        bridge_ports enp0s8
        bridge_fd 0
# Debian host
brctl addif br0 tap0 tap1
# Gentoo VM
ping 192.168.56.100
QEMU connects network interfaces tap0 and tap1 to the host after starting the virtual machines
Bridge br0 connects interfaces tap0, tap1 and enp0s8 into one network
Let's send a DHCP request
We write tests:
We write functions after tests:
The DHCP client performs the following steps:
Sends a DHCPDISCOVER message to discover DHCP servers on the network.
Receives DHCPOFFER offers from servers and selects one.
Sends a DHCPREQUEST to the server asking for an IP address.
Receives a DHCPACK acknowledgement or DHCPNAK failure from the server.
Let's add a kernel function sys_dhcp_request and system call dhcp_requestto get an IP address. The DHCP protocol runs on top of UDP, so dhcp_request sends a packet, waits for the server to respond, and resends the packet if it doesn't receive a response.
Let's add a kernel function sleep_for – analogue sys_sleep. The function sleeps for the specified number of timer ticks.
I don't understand why the DHCP server doesn't respond to DHCPDISCOVER requests from the driver. The server responds to requests from another VM with Gentoo on the same network. I compiled DHCPDISCOVER similar to what dhcpcd sends on Gentoo. Write in the comments what I'm doing wrong.
And besides…
The network adapter limits the size of the packet that it can transmit. The network adapter is able to divide packets into fragments that comply with the MTU – Maximum Transfer Unit limit.
The network adapter is capable of reading and checking checksums – this will unload the central processor.
VirtIO Specification will tell you how to enable these adapter options.
Berkeley sockets offer a set of functions for communication between programs over a network. Function socket creates a kernel object and returns a file descriptor that the program can call recv And send – analogues read And writeChapter 1 covers file descriptors.
We worked with split virtqueues. Now VirtIO offers a new format – packed virtqueue. Code for packed virtqueue works faster than split virtqueue.
The packed virtqueue allows both the driver and the device to write to the descriptor array. The driver and device no longer use arrays available And usedwhen passing buffers to each other. The packed virtqueue buffer is an array of elements, not a list element as in split virtqueue. The code works faster with a contiguous array than with a singly linked list.
Interrupt handling wastes CPU time when a driver responds to an interrupt and processes one request, but other requests come in during that time. A driver can better handle the flood of requests by responding to the interrupt, disabling interrupts, processing available requests, and then re-enabling interrupts.
Links