Ethernet driver for xv6
Ethernet driver for xv6
Xv6 – educational OS – tells about the ideas that underlie operating systems.
We will teach xv6 to work in the network, get acquainted with the VirtIO virtual device standard, the DeviceTree device tree, Ethernet technology, network protocols, and build a network bridge between virtual machines.
The driver helps the OS communicate with the network adapter. The OS asks the driver to send packets to the network. The driver tells the OS when packets arrive from the network.
The driver and network adapter maintain queues of packets – outgoing and incoming. The driver places an outgoing packet in the queue so that the adapter sends the packet to the network. The adapter receives a packet from the network and places it in the incoming queue so that the driver reads the packet.
The network adapter sends Ethernet packets to the network, and the driver packs the information into Ethernet packets before sending them. The Ethernet network adapter forwards the packet to another adapter – it specifies the MAC address of the receiving adapter in the packet.
An Ethernet packet contains packets of other protocols – IP, TCP, UDP, ARP, etc. An Ethernet adapter communicates with other Ethernet adapters, but programs communicate with programs on other computers and do not know about Ethernet.
Let's connect the network adapter
Xv6 runs on a QEMU virtual machine. Let's specify the options device
And netdev
to connect the network adapter virtio-net
to the machine and link it to the tap adapter on the host machine. The adapter virtio-net
will pass packets to the tap adapter, and tap to the adapter virtio-net
. Let's run Wireshark on the host and see the packets on tap.
QEMUOPTS += -device virtio-net-device,netdev=mynet,bus=virtio-mmio-bus.1,mac=52:54:00:78:76:36 # 'xv6' in ASCII
QEMUOPTS += -netdev tap,id=mynet
Parameter bus=virtio-mmio-bus.1
means that QEMU will map the adapter registers into memory. Xv6 similarly connects the hard drive – specifies the parameter bus=virtio-mmio-bus.0
.
QEMUOPTS += -device virtio-blk-device,drive=x0,bus=virtio-mmio-bus.0
The code determines the addresses of the device registers by the offset of the base address. Example: virtio disk base address – VIRTIO0 = 0x10001000
. Find the base address of the network adapter. QEMU prints the device tree when it receives the option dumpdtb
. Program dtc
converts a tree from binary format to text.
qemu-system-riscv64 -M virt,dumpdtb=virt.dtb -nographic
dtc -I dtb -O dts virt.dtb >> virt.dts
Tree nodes virtio_mmio@XXXXXXXX
– connectors on the bus virtio-mmio
. Address XXXXXXXX
– the base memory address from which the device will map registers. QEMU numbers the sockets in ascending order of addresses, so virtio-mmio-bus.1
will receive an address 0x10002000
. File kernel/virtio.h defines the offsets of virtio device registers from the base address.
![Devicetree fragment](https://habrastorage.org/webt/ga/kf/ef/gakfefx0pjnwbeawgnxxn9-q7uo.png)
The driver communicates with the device by writing and reading the device's registers.
Let's add the memory area with the network adapter registers to the kernel page table – otherwise the driver will get a memory access error when accessing the adapter register. Chapter 3 will tell you more about page tables.
// virtio mmio net interface
kvmmap(kpgtbl, VIRTIO1, VIRTIO1, PGSIZE, PTE_R | PTE_W);
Let's configure the network adapter
Let's configure the adapter when the OS starts:
Let's reset the adapter.
Let's tell the OS that the adapter is visible – set the bit
ACKNOWLEDGE
status register.Let's say that the OS can manage the adapter – set the bit
DRIVER
status register.We agree on the set of adapter capabilities – the primitive network driver uses only
VIRTIO_NET_F_MAC
.Confirm your choice – set the bit
FEATURES_OK
status register.Let's read the beat again
FEATURES_OK
and make sure the bit is set, otherwise the device will reject that set of capabilities.Let's read the MAC address of the adapter and prepare queues for incoming and outgoing packets.
Let's set the bit
DRIVER_OK
status register.
Function virtio_net_init configures the adapter.
VirtIO Queues
The driver and device pass memory buffers to each other using queues. Putting a buffer into a VirtIO queue is more complicated than adding an element to a singly linked list. Buckle up.
The VirtIO queue contains three arrays:
An array of descriptors. Each descriptor describes a memory buffer.
Array
available
contains the numbers of descriptors that the driver passed to the device.Array
used
contains the descriptor numbers that the device passed to the driver.
The driver searches for a free descriptor, fills the buffer and writes the descriptor number to the array. available
to pass the buffer to the device. The device writes the descriptor number to the array used
when returning the buffer to the driver.
The device does not write to the descriptor array and works only with the buffers provided by the driver. The driver fills the incoming packet queue with buffers so that the adapter can accept packets from the network.
We send packages
Let's write a program ping and add a system call to the xv6 kernel ping
which asks the driver to send a packet to the network.
We write tests. Tests save nerves and time.
We write functions after tests:
Let's declare the kernel function sys_ping and add to the array syscalls Under the number SYS_ping. Let's declare a system call ping
V user/user.h and create an entry point into the kernel using a script user/usys.plChapter 4 covers system calls in more detail.
System call ping
accepts the IP address of the host to which it will send the ICMP ECHO.
Function virtio_net_intr processes interrupts when the adapter sends or receives a packet. Chapter 5 covers device interrupts.
Let's run the program ping
and we will see that the packet has gone into the network.
ping 192.168.56.100
We accept packages
The adapter interrupts the processor when it receives a packet from the network. Function virtio_net_intr
processes incoming packets as well. The driver processes the incoming packet and returns the buffer to the incoming packet queue.
We write tests:
We write functions after tests:
Helper functions parse and print packet headers:
Let's run xv6 and see how random packets from the network get to the adapter, and the driver prints packet headers.
[Ethernet] EtherType: 0x86dd SourceAddress: 26:e3:ce:e6:aa:83 DestinationAddress: 33:33:0:0:0:fb
[IPv6] NextHeader: 17 PayloadSize: 149 SourceAddress: fe80:00:00:00:24e3:ceff:fee6:aa83 DestinationAddress: ff2:00:00:00:00:00:00:0fb
[UDP] SourcePort: 5353 DestinationPort: 5353 Length: 149 Checksum: 0xad89
![](https://habrastorage.org/webt/7o/ec/7z/7oec7zblm_5kjkv8u4dupelq2bs.png)
We will send the package from another machine
Let's launch another VM under QEMU and create a bridge between network adapters tap0
And tap1
host. Let's send an ICMP ECHO and see that xv6 receives ARP requests. The machine sends an ARP request to the network to find out the MAC address of the owner of the IP address.
# /etc/network/interfaces.d/br0 on Debian host machine
auto br0
iface br0 inet dhcp
bridge_ports enp0s8
bridge_fd 0
# Debian host
brctl addif br0 tap0 tap1
# Gentoo VM
ping 192.168.56.100
![QEMU connects network interfaces tap0 and tap1 to the host after starting the virtual machines](https://habrastorage.org/webt/w-/ky/w2/w-kyw2vjnmzeuskabosfkqqyr9y.png)
![Bridge br0 connects interfaces tap0, tap1 and enp0s8 into one network](https://habrastorage.org/webt/zn/ui/sc/znuisce980andqonnxaegbyz6qm.png)
Let's send a DHCP request
We write tests:
We write functions after tests:
The DHCP client performs the following steps:
Sends a DHCPDISCOVER message to discover DHCP servers on the network.
Receives DHCPOFFER offers from servers and selects one.
Sends a DHCPREQUEST to the server asking for an IP address.
Receives a DHCPACK acknowledgement or DHCPNAK failure from the server.
Let's add a kernel function sys_dhcp_request and system call dhcp_requestto get an IP address. The DHCP protocol runs on top of UDP, so dhcp_request sends a packet, waits for the server to respond, and resends the packet if it doesn't receive a response.
Let's add a kernel function sleep_for – analogue sys_sleep. The function sleeps for the specified number of timer ticks.
I don't understand why the DHCP server doesn't respond to DHCPDISCOVER requests from the driver. The server responds to requests from another VM with Gentoo on the same network. I compiled DHCPDISCOVER similar to what dhcpcd sends on Gentoo. Write in the comments what I'm doing wrong.
And besides…
The network adapter limits the size of the packet that it can transmit. The network adapter is able to divide packets into fragments that comply with the MTU – Maximum Transfer Unit limit.
The network adapter is capable of reading and checking checksums – this will unload the central processor.
VirtIO Specification will tell you how to enable these adapter options.
Berkeley sockets offer a set of functions for communication between programs over a network. Function socket
creates a kernel object and returns a file descriptor that the program can call recv
And send
– analogues read
And write
Chapter 1 covers file descriptors.
We worked with split virtqueues. Now VirtIO offers a new format – packed virtqueue. Code for packed virtqueue works faster than split virtqueue.
The packed virtqueue allows both the driver and the device to write to the descriptor array. The driver and device no longer use arrays available
And used
when passing buffers to each other. The packed virtqueue buffer is an array of elements, not a list element as in split virtqueue. The code works faster with a contiguous array than with a singly linked list.
Interrupt handling wastes CPU time when a driver responds to an interrupt and processes one request, but other requests come in during that time. A driver can better handle the flood of requests by responding to the interrupt, disabling interrupts, processing available requests, and then re-enabling interrupts.