Cloudlab Disk Image Tutorial
Cloudlab is a cloud computing platform for researchers that relies on the notion of profiles, which are written specifications for hardware, software, and network connections, to easily reproduce the testing environment that other researchers use.
Disk images are used to specify the software the computers in an experiment will run. Cloudlab provides pre-made disk images that come with an operating system like Ubuntu.
Using a Disk Image
There are a few different ways to utilize a disk image, but we will be working with geni-lib scripts. These are python scripts that are used to generate an RSpec, which is a tree-like structure of code that describes what hardware we will use, what software is in the disk drive of the hardware, and how the hardware is connected to each other.
Below is an example of a geni-lib script:
import geni.portal as portal
request = portal.context.makeRequestRSpec()# Here we assign constants that will be used later to describe our machines.
DEFAULT_SVR_HARDWARE_TYPE = "m510"
DEFAULT_CLI_HARDWARE_TYPE = "d6515"
DEFAULT_DISK_IMAGE = "urn:publicid:IDN+emulab.net+image+emulab-ops:UBUNTU18-64-STD"
DEFAULT_LAN_SOCKET = "eth1"
lan = request.LAN()# Here we say that we want to use 4 machines for our experiment. We give unique names to each of these machines. Then we assign them to variables so we can give more information later on the hardware, software, and network connections of each machine.
svr_1 = request.RawPC("svr_1")
svr_2 = request.RawPC("svr_2")
svr_3 = request.RawPC("svr_3")
cli_1 = request.RawPC("cli_1")# Here we group our machines into lists so that we can give multiple machines the same specifications.
svrs = [svr_1, svr_2, svr_3]
clis = [cli_1]# Every machine we define in the "svrs" list will have the following characteristics:
# 1. Each machine will be an m510. You can check http://docs.cloudlab.us/hardware.html to see what computers are available in Cloudlab
# 2. Each machine will come loaded with Ubuntu 18.04 created by the Cloudlab team.
# 3. Each machine will have a connection to a LAN interface.
for svr in svrs:
svr.hardware_type = DEFAULT_SVR_HARDWARE_TYPE
svr.disk_image = DEFAULT_DISK_IMAGE
s_iface = svr.addInterface(DEFAULT_LAN_SOCKET)
lan.addInterface(s_iface)# Similar for clients.
for cli in clis:
cli.hardware_type = DEFAULT_CLI_HARDWARE_TYPE
cli.disk_image = DEFAULT_DISK_IMAGE
c_iface = cli.addInterface(DEFAULT_LAN_SOCKET)
lan.addInterface(c_iface)
portal.context.printRequestRSpec(request)
As seen through the example above, using a disk image is as simple as creating a machine object, storing it in a variable, then assigning the machine object’s disk image field to a URN (similar to a URL) of your choice.
Making a Disk Image
We can do more with disk images than download operating systems to our hardware. To make it easier for reviewers, we could download all of the dependencies and software required to run our research codebase into our hard drive, and then create a disk image of the operating system we used + everything we downloaded into it.
Get Neo Zhou’s stories in your inbox
Join Medium for free to get updates from this writer.
First, we need to run an experiment with a profile. There are many ways to do this, but we’ll do this by instantiating our profile from its description.
We’ll select all of the default settings and load our experiment up.
Now that our experiment is loaded up, we can ssh to any of our machines, or use the Cloudlab shell, and start downloading our codebase and its dependencies.
After we’ve finished downloading what we need, we simply click the Create Disk Image button.
We select the machine we want to make a disk image from, give the disk image a name we haven’t used before, and give a description of the image. Then we wait ~30 minutes for the disk imaging process to complete.
You may have noticed the warning in the Create a Disk Image message box. This is very important. If your downloads are in your default home directory, Cloudlab won’t save those. Before you download anything, make sure you `cd` into /usr/local or any other directory that is not your home directory. Even /root works. Then whatever you install there will be saved. You’ll probably have to use sudo su to be able to download files in these locations.
Using Your Own Disk Image
Once you’ve made your disk image, all of the images you’ve ever created will be visible when you click the Storage tab at the top left corner of your screen, and then click My Disk Images.
Here, we can see our images are organized by the cluster we created them in. When you create a disk image with a machine, that disk image can only be used in the cluster the machine is in.
To use a saved disk image in one of your profiles, simply click the link icon for the disk image you want to use, copy the URN, and assign it to a machine’s disk image field in your geni-lib script like we did above.
Conclusion
If you’re using a profile with a custom disk image, it’s likely that you won’t have to do much work to get a research codebase to run. Disk images are a convenient way to share your experiment environment in a streamlined fashion.
Further Reading
The cloudlab manual is a very useful guide on the many features of cloudlab. If you’re just getting started, take an hour to read sections 1–8 and 14 to understand the basics of cloudlab. It won’t take you very long and it’ll save you a lot of time in the long run. Skim through the other sections and take a mental note on what they describe so you can come back to them later if needed.
