Recently, I had a need to understand IP fragmentation better, so I built a lab to experiment with. I wanted it to be easy to setup, and built in a way that is repeatable by others, so I designed automation to stand it up with virtual machines in a hypervisor. The lab network topology is shown in Figure 1.
Figure 1 – Network Diagram for IP Fragmentation Lab
The IP MTU on the WAN link is reduced to 1280 bytes to create IP fragmentation (under some circumstances, more on this later).
Setting the Lab Up
Vagrant is used as the automation suite and VirtualBox is the hypervisor. The code is on GitHub at https://github.com/mattjhayes/Vagrant/tree/master/Network_Test_Labs/Fragmentation_IPv4 and could easily be modified to other topologies, or different scenarios. I’ve covered Vagrant and VirtualBox setup in previous posts Vagrant for SDN Labs and Vagrant for SDN Labs pt2.
Lab Design
The main setup is done by Vagrantfile, with shell files run inside the guests for further configuration. There are also aliases included on the client (pc1) guest to carry out some simple tests and the server (sv1) runs a basic web server on port 80.
A top tip is to use SSH to connect to the guests, as the user experience is better than the VirtualBox console, here are the details for connecting:
| Guest | Address | Port | Default Creds |
| pc1 | localhost | 2222 | vagrant/vagrant |
| rt1 | localhost | 2200 | vagrant/vagrant |
| rt2 | localhost | 2201 | vagrant/vagrant |
| sv1 | localhost | 2202 | vagrant/vagrant |
SFTP can also be used to transfer files (i.e. packet captures) from the guests with a program such as WinSCP.
Searching for Fragmentation in the Lab
Surprisingly, I found that IP fragmentation is not that easy to induce (which may be a good thing, as IP fragmentation stinks, for many reasons). The reason for this is Path MTU Discovery (PMTUD) which works by sending an ICMP Destination unreachable (fragmentation needed) message back from the router to the endpoint that sent the too large packet, as per packet 25 in Figure 2.
Figure 2 – Example of PMTUD at work
Note how packets 28 and 29 are now sent as replacements for packet 24 so that packet size does not exceed MTU and fragmentation is thus not needed.
PMTUD should be embraced – do not block ICMP destination unreachable messages! The downside for this lab is that it works too well for the HTTP connection, so we need to find another way to create fragmentation, PINGs to the rescue…
Fragmenting PINGs
Fragmentation can be induced in the lab by sending PINGs with a size over the WAN MTU and without DF set. The result from a packet capture on the WAN showing fragmentation is:
When we drill into the IP layer of the first of these two packets, we see the ‘More fragments’ flag is set:
The second packet has ‘Fragment offset’ set to 185:
Multiplying 185 by 8 (as per RFC791) we get an offset of 1480 bytes, which makes sense as adding on 20 bytes of IP header and 14 bytes of Ethernet header takes us to the 1514 length of the first packet.
Is there a Downside to a Virtual Lab?
Virtual labs are very convenient, especially when paired with build automation. There is however a potential downside – some behaviour can differ from physical hosts. As an example, in shaking down this lab I found the default VirtualBox network adapter (Intel PRO/1000 MT Desktop (82540EM)) caused issues where MTU was not respected. I fixed this in the Vagrant file by specifying the Am79C973 NIC Adapter instead, however it is possible that other side effects of virtualisation could skew results, so care should be taken to not rely fully on virtual lab results.
The lab is fairly rough and ready, and more could be done with it. If you do improve it then do send a pull request with the updated and improved code.
Bits 'n Bytes 