Parallel Processing on the Pi (Bramble)

Parallel processing on the Raspberry Pi is possible, thanks to the portable MPICH2 (Message Passing Interface). I was keen to try this out as soon as I managed to get hold of two of these brilliant little computers. Here I'm going to show you how I managed to get it all working, and I will also show you the results I obtained. (Bramble was a name an ingenious Raspberry Pi forum member made up, not myself!)

There are three ways which you can install MPICH2 (in case one of them doesn't seem to work for you). Compiling and installing from source, using my .deb package then following the rest of the tutorial, or using the Python scripts I wrote. Installing from source takes a while on the little Pi when not cross compiling.

Install - Choose Method 1, 2 or 3

1) Simply download the script;
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/install.py

and then run it as root with the command;
sudo python install.py

Follow the instructions on the screen entering all of the necessary info. The script will do a full install and configuration like the steps in the tutorial below. Do this for every node on your Bramble.

I have also written an RSA key script which sets up a secure public and private key between the master and all of the worker nodes. This means that you don't have to enter the password into SSH every time, but still maintaining a secure connection;
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/rsa.py

python rsa.py

If you want to set it up for the root user, you can run as root, and enter the username "root" for the other worker(s).

Finally you can join onto the Testing phase of the tutorial if you want to, otherwise you are all set!

Please let me know of any problems with the scripts and I will fix it. Thanks!

2) From .deb package

First, download the .deb package;

wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/mpich2_1.4.1p1-1_armel.deb

De-package and install;

sudo dpkg -i mpich2_1.4.1p1-1_armel.deb 

Now proceed to the setup phase of this tutorial.

3) From source;

First, download the source tar ball:

wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.4.1p1/mpich2-1.4.1p1.tar.gz

Now extract the files;

tar zxfv mpich2-1.4.1p1.tar.gz

Go into the source folder;

cd mpich2-1.4.1p1

Now we must run ./configure to configure the files ready for compilation;

sudo ./configure --disable-f77 --disable-fc

If you have a FORTRAN compiler then you can run it without "--disable-f77" and "--disable-fc".

If you do not, and you don't want to run FORTRAN programs on your cluster then make sure you do configure with them disabled as the error occurs a good 20 mins in and you have to start all over again.

Now that you have configured without error, compile then install the compiled program;

sudo make

sudo make install

Now proceed to the setup phase of this tutorial below.

Setup

Now you should have MPICH2 installed on both (all) of your Raspberry Pis. On your Bramble, you are going to need one  node and the rest are worker nodes.

First create this file in both the pi and root user home directories, 

touch ~/.rhosts

sudo touch /root/.rhosts

add the following lines to the normal user file (pi is the default username for both of my nodes);

Master pi

Slave pi

For the root user;

Master root

Slave root

Do this for every Pi/node on the Bramble.

Now edit the /etc/hosts file to contain the IP of each node and the master node like so;

192.168.0.14 Master

192.168.0.11 Slave

Edit /etc/hosts.allow to contain just "ALL+", or each IP address of the other nodes.

(To find out the IP on each node, type ifconfig)

Now edit /etc/securetty to contain the line to allow root access;

rsh, rlogin, rexec, pts/0, pts/1

Create the file /etc/pam.d/rsh if it is not there already and put the following in;

auth       sufficient   /lib/security/pam_nologin.so

auth       optional     /lib/security/pam_securetty.so auth       sufficient   /lib/security/pam_env.so auth       sufficient   /lib/security/pam_rhosts_auth.so account    sufficient   /lib/security/pam_stack.so service=system-auth session    sufficient   /lib/security/pam_stack.so service=system-auth

SSH Public Key Authentication - Optional

This section is optional but advised! The use of a public and private key for SSH will mean you won't have to type in the SSH password between the master and the worker every time you want to run a program.

Use the Python script I have written;
wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/rsa.py

python rsa.py

Or if it does not work properly, follow the instructions at this link: http://steve.dynedge.co.uk/2012/05/30/logging-into-a-rasberry-pi-using-publicprivate-keys/

Thanks to Steve Smith who wrote the tutorial!

Testing

Now you should be ready to rock and roll! Lets test the cluster out.

If you installed from source, you will have this file to test in the ~/mpich2-1.4.1p1/examples directory (you will need to compile it). If you installed from the .deb package then download this test file to the same directory in each of the nodes (or even better a centralised dir, perhaps with NFS);

wget https://github.com/downloads/philleonard/MPICH2-Armel-Raspberry-Pi/cpi_test.tar.gz

Extract the tar ball and make sure the permissions are set;

tar zxfv cp_test.tar.gz

sudo chmod -R 777 cpi_test

Enter the cpi_test directory. This cpi program is a little program written in C which outputs Pi to 16 decimal places and outputs run time. I think this program is quite apt given what is running it.

Now we have to set up a "machinefile". This tells MPICH2 where to delegate the processes to. Yes, more IP addresses to come! Create the machine file on the Master node;

touch machinefile

Now edit the file to contain the IP addresses of the node and a colon followed by the number process to follow. For example my machine file looks like this;

192.168.0.14

192.168.0.11:2

This tells MPICH2 to pass the first process to my master node at 192.168.0.14 and the next (second) process to my only worker node at 192.168.0.11. There is no process number after the first IP because empty is the same to the first. (i.e. 192.168.0.14 is the same as 192.168.0.14:1).

It's easier command wise if the machinefile and the program you are executing are in the same folder to reduce the length of the next command.

Now it's finally time to run our first program! Se sure that you are in the cpi_test folder and that the same folder is located in the same place in all the other nodes on the Bramble. Execute the following line;

mpiexec -f machinefile -n 100 ./cpi

-f shows mpiexec where the machinefile is. -n is the number of times to repeat the process. This is set at 100 to execute cpi 100 times so that we can see the difference in power easier. 

If it gives you an error about file not found, again make sure all the cpi files are in the same location on each node.

If the command executes successfully then you should get a record of the processes running and an output like;

pi is approximately 3.1415926544231274, Error is 0.0000000008333343

wall clock time = 0.728116

You can remove the worker IPs from the machinefile to compare running this program on just the Master node. 

My Results

I ran the previous command 10 times, with both nodes enables and again with just the Master and took an average both times. The average I got from both nodes was; 0.731 Seconds – with a minimum of around 0.6 and a limit of 0.8. The average I got from one node enabled was; 1.418 Seconds –with a minimum of 0.85 and a limit of 5.0. 

Finish 

You should now have your Bramble up and running!

I just compiled and created a deb file, and found the best way to set the program up for the Pi, all thanks go to the team who created the awesome and portable MPICH2, and any sources I used to make the tutorial.

Further reading: http://www.mcs.anl.gov/research/projects/mpich2/documentation/files/mpich2-1.4.1-installguide.pdf

This shows a lot of other useful applications of MPICH2 including compiling across your Bramble, it would be nice to have a compiler farm made of Pis.

Thanks for reading! Don't hesitate to ask for help below!

Phil

MPICH2 on the RPi