The main reason for why we created the RVL Cloud is that
practically every research project these days has some sort
of a big-data and deep-learning component to it. A cloud
based execution of these projects makes it much easier to
customize the resources needed for solving such
problems. Additionally, some of our research projects
now require deliverables in the form of virtual machines
that must run on a commercial cloud platform like the Amazon
These factors have heavily influenced how computing is done in RVL at Purdue. Several of our research projects involve large datasets of satellite imagery. These datasets can be terabytes in size and are typically processed by GPU enabled VMs. Other examples include processing of video for tracking humans and objects, detection of hazardous materials in airport security systems, retrieval of information from large textual datasets and software libraries, automatic bug localization in large software systems, etc.
RVL Cloud runs on the OpenStack framework. This is the home page for our RVL Cloud where we hope to share our experience and knowledge in setting up a powerful open-source cloud computing platform in a university setting.
We are frequently asked as to why we created our own cloud platform. Why not just rent time on, say, Amazon Web Services? To answer, an in-house cloud platform allows the students to experiment with the cloud architecture itself and thus develop deeper insights into the workings of a cloud. Additionally, the inter-node communication in an in-house cloud can be much faster than in a large-scale commercial operation.
In the process of building and maintaining our own OpenStack cloud, we have learnt quite a few important lessons along the way. For instance, it might seem counterintuitive but some times one can obtain faster execution times for optimized programs by using VMs with fewer VCPUs and smaller memory than by greedily hogging all the VCPUs and memory on a host machine. More information about such interesting experiences can be found on our RVL Blog.
If you are interested in setting up your own OpenStack cloud, here are some bash script that you might find helpful. A common strategy in setting up a new cloud platform is to first install a cloud controller in one of the physical nodes. Subsequently, you create your first general-purpose compute node for the cloud on another machine. Finally, you duplicate the compute node in the other machines you are deploying for the cloud. In the cloud platform thus created, you can then start creating virtual machines for whatever application you have in mind. The scripts that are listed below should help you with all these phases of an OpenStack cloud set-up.
your physical computer into a
VM for an OpenStack
Yes, you got it right. You can create a VM out of your personal computer. This VM running in an OpenStack cloud (such as the RVL Cloud) will behave just like the physical computer you are currently using. NOTE: When one creates a VM out of a physical machine in this manner, typically one does not copy over one's home directory. But, if you want, you can include your home directory also in the VM.
a physical OpenStack node as a
Let's say you want to add a new physical node to an OpenStack cloud. An efficient way to do that is to create a tarball from one of your current nodes and copy it over to the new node. This script shows how you can do that.
the tarball for an existing
OpenStack physical node in a
new physical node.
After you have created a tarball from an existing OpenStack node, you can use this script to install the tarball in your new node.