= MPI-Utils =
<<TableOfContents>>

== Installation ==
MPI-Utils is a meta package that depends on MPI-Start and the yaim mpi module for configuration of the MPI support in the CE and WN. Administrators must install a MPI implementation and configure it at the site. Most Linux distributions provide ready to use packages for Open MPI and MPICH implmentations.

MPI-Utils can be installed from the EMI repositories, and should be as easy as:

{{{
yum install glite-mpi
}}}

== Configuration ==

Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see SiteConfig for details). This is performed by the yaim MPI module which should be run on both types of nodes.


=== Configure MPI in site-info.def ===
Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable Open MPI, add the following:

{{{#!highlight sh
MPI_OPENMPI_ENABLE="yes"
}}}
You may set the path and version to non default locations for a flavour of MPI as follows:

{{{#!highlight sh
MPI_OPENMPI_PATH="/opt/openmpi-1.3/"
MPI_OPENMPI_VERSION="1.3"
}}}

Possible flavours are:
 * OPENMPI, for Open MPI
 * MPICH for MPICH-1
 * MPICH2 for MPICH-2
 * LAM for LAM-MPI

The remaining variables are:

|| Variable         || Value ||
||`MPI_SHARED_HOME` ||set this to "yes" if you have a shared home area between WNs. ||
||`MPI_SSH_HOST_BASED_AUTH` || set it to "yes" if you have SSH based authentication between WNs ||
||`MPI_<FLAVOUR>_MPIEXEC` ||If you are using OSC mpiexec (only in PBS/Torque sites), set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec" ||
||`MPI_MPI_START` ||Location of mpi-start if not installed in standard location /opt/mpi-start/bin/mpi-start ||
||`MPI_DEFAULT_FLAVOUR` || Set it to the default flavour for your site, if undefined, the first defined flavour will be used ||


Here is an example configuration:

{{{#!highlight sh
#----------------------------------
# MPI-related configuration:
#----------------------------------
# Several MPI implementations (or "flavours") are available.
# If you do NOT want a flavour to be installed/configured, set its variable
# to "no". Else, set it to "yes" (default). If you want to use an
# already installed version of an implementation, set its "_PATH" and
# "_VERSION" variables to match your setup (examples below).
#
# NOTE 1: the CE_RUNTIMEENV will be automatically updated in the file
# functions/config_mpi, so that the CE advertises the MPI implementations
# you choose here - you do NOT have to change it manually in this file.
# It will become something like this:
#
#   CE_RUNTIMEENV="$CE_RUNTIMEENV
#              MPI_MPICH
#              MPI_MPICH2
#              MPI_OPENMPI
#              MPI_LAM"
#
# NOTE 2: it is currently NOT possible to configure multiple concurrent
# versions of the same implementations (e.g. MPICH-1.2.3 and MPICH-1.2.7)
# using YAIM. Customize "/opt/glite/yaim/functions/config_mpi" file
# to do so.

MPI_MPICH_ENABLE="yes"
MPI_MPICH2_ENABLE="yes"
MPI_OPENMPI_ENABLE="yes"
MPI_LAM_ENABLE="yes"

# set Open MPI as default flavour
MPI_DEFAULT_FLAVOUR=OPENMPI


#---
# Example for using an already installed version of MPI.
# Setting "_PATH" and "_VERSION" variables will prevent YAIM
# from downloading and installing the gLite-provided packages.
# Just fill in the path to its current installation (e.g. "/usr")
# and which version it is (e.g. "6.5.9").
#---
MPI_MPICH_PATH="/opt/mpich-1.2.7p1/"
MPI_MPICH_VERSION="1.2.7p1"
MPI_MPICH2_PATH="/opt/mpich2-1.0.4/"
MPI_MPICH2_VERSION="1.0.4"
MPI_OPENMPI_VERSION="1.1"
MPI_LAM_VERSION="7.1.2"

# If you do NOT provide a shared home, set $MPI_SHARED_HOME to "no" (default).
#
MPI_SHARED_HOME="yes"

#
# If you do NOT have SSH Hostbased Authentication between your WNs, set the below
# variable to "no" (default). Else, set it to "yes".
#
MPI_SSH_HOST_BASED_AUTH="no"

#
# If you provide an 'mpiexec' for MPICH or MPICH2, please state the full path to
# that file here (http://www.osc.edu/~pw/mpiexec/index.php). Else, leave empty.
#
#MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
}}}


=== Configure CE ===
{{{/opt/glite/yaim/bin/yaim -c -s site-info.def  -n MPI_CE}}}

=== Configure WN ===
For a Torque worker node: 
{{{/opt/glite/yaim/bin/yaim -c -s site-info.def  -n MPI_WN}}}

== Testing ==
You can do some basic tests by logging in on a WN as a pool user and running the following:

{{{
[dte056@cagnode48 dte056]$ env|grep MPI_
}}}
You should see something like this:

{{{
MPI_MPICC_OPTS=-m32
MPI_SSH_HOST_BASED_AUTH=yes
MPI_OPENMPI_PATH=/opt/openmpi/1.1
MPI_LAM_VERSION=7.1.2
MPI_MPICXX_OPTS=-m32
MPI_LAM_PATH=/usr
MPI_OPENMPI_VERSION=1.1
MPI_MPIF77_OPTS=-m32
MPI_MPICH_VERSION=1.2.7
MPI_MPIEXEC_PATH=/opt/mpiexec-0.80
MPI_MPICH2_PATH=/opt/mpich2-1.0.4
MPI_MPICH2_VERSION=1.0.4
I2G_MPI_START=/opt/mpi-start/bin/mpi-start
MPI_MPICH_PATH=/opt/mpich-1.2.7p1
}}}

You can also try submitting a job to your site, please read [[../UserDocumentation | MPI-Start user documentation]]