IP Fragmentation Lab with Vagrant

Recently, I had a need to understand IP fragmentation better, so I built a lab to experiment with.  I wanted it to be easy to setup, and built in a way that is repeatable by others, so I designed automation to stand it up with virtual machines in a hypervisor. The lab network topology is shown in Figure 1.


Figure 1 – Network Diagram for IP Fragmentation Lab

The IP MTU on the WAN link is reduced to 1280 bytes to create IP fragmentation (under some circumstances, more on this later).

Setting the Lab Up

Vagrant is used as the automation suite and VirtualBox is the hypervisor. The code is on GitHub at https://github.com/mattjhayes/Vagrant/tree/master/Network_Test_Labs/Fragmentation_IPv4 and could easily be modified to other topologies, or different scenarios. I’ve covered Vagrant and VirtualBox setup in previous posts Vagrant for SDN Labs and Vagrant for SDN Labs pt2.

Lab Design

The main setup is done by Vagrantfile, with shell files run inside the guests for further configuration. There are also aliases included on the client (pc1) guest to carry out some simple tests and the server (sv1) runs a basic web server on port 80.

A top tip is to use SSH to connect to the guests, as the user experience is better than the VirtualBox console, here are the details for connecting:

Guest Address Port Default Creds
pc1 localhost 2222 vagrant/vagrant
rt1 localhost 2200 vagrant/vagrant
rt2 localhost 2201 vagrant/vagrant
sv1 localhost 2202 vagrant/vagrant

SFTP can also be used to transfer files (i.e. packet captures) from the guests with a program such as WinSCP.

Searching for Fragmentation in the Lab

Surprisingly, I found that IP fragmentation is not that easy to induce (which may be a good thing, as IP fragmentation stinks, for many reasons). The reason for this is Path MTU Discovery (PMTUD) which works by sending an ICMP Destination unreachable (fragmentation needed) message back from the router to the endpoint that sent the too large packet, as per packet 25 in Figure 2.

frag2Figure 2 – Example of PMTUD at work

Note how packets 28 and 29 are now sent as replacements for packet 24 so that packet size does not exceed MTU and fragmentation is thus not needed.

PMTUD should be embraced – do not block ICMP destination unreachable messages! The downside for this lab is that it works too well for the HTTP connection, so we need to find another way to create fragmentation, PINGs to the rescue…

Fragmenting PINGs

Fragmentation can be induced in the lab by sending PINGs with a size over the WAN MTU and without DF set. The result from a packet capture on the WAN showing fragmentation is:


When we drill into the IP layer of the first of these two packets, we see the ‘More fragments’ flag is set:


The second packet has ‘Fragment offset’ set to 185:


Multiplying 185 by 8 (as per RFC791) we get an offset of 1480 bytes, which makes sense as adding on 20 bytes of IP header and 14 bytes of Ethernet header takes us to the 1514 length of the first packet.

Is there a Downside to a Virtual Lab?

Virtual labs are very convenient, especially when paired with build automation. There is however a potential downside – some behaviour can differ from physical hosts. As an example, in shaking down this lab I found the default VirtualBox network adapter (Intel PRO/1000 MT Desktop (82540EM)) caused issues where MTU was not respected. I fixed this in the Vagrant file by specifying the Am79C973 NIC Adapter instead, however it is  possible that other side effects of virtualisation could skew results, so care should be taken to not rely fully on virtual lab results.

The lab is fairly rough and ready, and more could be done with it. If you do improve it then do send a pull request with the updated and improved code.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s