It's a CloudCase now - a portable demo of the XtreemFS Cloud file system
Demonstrating how the Cloud works is difficult, isn't it? It's all somewhere out there, isn't it? Well, it was. Thanks to the new Raspberry Pi credit size computer we can now build a fully functional Cloud system in a briefcase. We can carry it around to workshops and meetings just to tell about the workings of the XtreemFS Cloud file system, only by means of this CloudCase.
XtreemFS is a Cloud file system, designed to be distributed over many computers. The description of a file is separated from the file content itself, and XtreemFS can maintain several copies of the file data. What if the computer fails or is unreachable? No worries, XtreemFS just gets the data from another copy. Sounds complicated? Well, we can show it if we take our CloudCase.
This is the CloudCase closed.Just a briefcase we do not use anymore these days. Documents are on a tablet. So the briefcase was just gaining dust. But that changed. When we open the briefcase, now we see the actual miniature Cloud:
There are eight credit card sized Raspberry Pi computers. Despite their small size they are fully functional computers that run a standard Linux operating system. Three of them have a USB drive and are acting as storage for the files.
We included a network switch in the CloudCase that connects the Raspberries into one cluster of computers. Just like in any Cloud.
There are also some small USB Hubs that actually only serve to provide the Raspberry computers and the network with power.
And there are cables. Nineteen in total. Makes it all look pretty cool. There are small lights on the computers, the switch and the power supplies. Looks nice in the dark as you can see.
The only thing to get the micro XtreemFS Cloud started is hook it up, start the system, and your Cloud is running.
One of the Raspberry Pi computers acts as a network server, the router for the complete system.
Of the other 7 Raspberries six computers are configured to run XtreemFS.
First, there are three file data servers, or OSDs as the XtreemFS guys call them. These hold the data and we have three so there can be three copies to create a very resilient file system. Our favorite demo is the one where you play a video from the XtreemFS file system and unplug the file data server that is currently active. XtreemFS notices that, and redirects the video stream to another file data server, and the video continues.
Then there is one file catalogue server or MRC. This is where the file seems to be for the user, but actually this file catalogue server only knows where the content actually is. If you request a file from the file catalogue server it looks up were the data is, sends you a pointer to a file data server and off you go.
The fifth Raspberry Pi knows where the catalogue is and where the file data servers are. It is the directory server.
So these five servers are the core of the XtreemFS Cloud file system.
To access the file system we put an XtreemFS client on the sixth Raspberry Pi. From this machine you can as a user, access the file system, get a file - a document, video, anything - and use it. This is the boring part: files on XtreemFS look like any other file, and you access an XtreemFS Cloud file system as any other file system on your computer.
So we can also connect other computers, PCs, laptops to the XtreemFS in the CloudCase, and they can access the files in there.
XtreemFS has a lot of other nice features. Which ones? Well, we can come to you and show it in the CloudCase.
How did we build it?
We were looking for a reasonable number or computers to put in the CloudCase. Eight is a nice number. It allows for enough complexity to create a small Cloud, Grid or parallel computer. Four would be too small to run the complete distributed XtreemFS for instance.
Eight is also a typical computer number. So there are 8 port network switches that can connect eight computers. We opted for a fast 100 Mbit/s Ethernet switch, fast enough for nice demos.
Outside computers can also connect to the switch (but then you can only use 7 Raspberry Pis) or you can connect outside computers to the "network" computer: A Raspberry Pi that acts as a router and as a DHCP server giving networking addresses to the other computers.
The whole system consumes about 25W.
The software took a bit longer to put together.
Each Raspberry Pi runs a full Debian Linux operating system. The latest version has good support for Java, something you need for XtreemFS.
The XtreemFS server software (DIR, MRC, OSD) is available for Debian at the XtreemFS website. Because it is implemented in Java, it runs on any hardware that supports Debian.
Just follow the instructions and it is up and running in no time.
Some additional tweaks were needed to take care the USB storage is automatically attached to the system. Also some tweaks to get XtreemFS to automatically start when the Raspberries power up.
Installing the XtreemFS client takes a little more work. It is not built in Java, so you need a version that is specific to the ARM processor of the Raspberry Pi. The XtreemFS website has clients for Windows, MacOSX, and even Debian Linux. But unfortunately not one suited for the ARM processor of the Raspberry Pi. So we had to build that ourselves. The computer source code is available on the XtreemFS web site. So you download that computer code, download a number of libraries and some tools and you start compiling the software and building the client. We did all that on the Raspberry Pi itself. That takes patience, because the Rapberry Pi is not the fastest computer in the world. But after many hours, the XtreemFS client was successfully built.
After that we had the CloudCase XtreemFS Cloud file system fully functional: we can store files from the Rasperry Pi client computer into the XtreemFS file system and retrieve them. When we connect an outside computer, we can also access the files on XtreemFS.
It is a little bit like building your own DropBox or OneDrive, although these work slightly different. And as demos with the CloudCase may show, it can be as secure and user friendly as those commercial Cloud file systems, but more resilient and better controllable.
XtreemFS has a website at http://xtreemfs.org
More info on the Raspberry Pi: http://www.raspberrypi.org
(This work was done as part of the Contrail project.)