= MPI-Utils =
<<TableOfContents>>

== Functional Description ==
MPI-Utils is a meta package that provide [[../ | mpi-start]] and a yaim mpi module in order to ease the installation of MPI support in the nodes.

== Installation ==
MPI-Utils is a meta package that depends on MPI-Start and the yaim mpi module for configuration of the MPI support in the CE and WN. Administrators must install a MPI implementation and configure it at the site. Most Linux distributions provide ready to use packages for Open MPI and MPICH implmentations.

MPI-Utils can be installed from the EMI repositories, and should be as easy as:

{{{
yum install glite-mpi
}}}

In the WN, a MPI implementation must be installed also. Open MPI is recommended (the devel package allows users to compile their applications):

{{{
yum install openmpi openmpi-devel
}}}

== Configuration ==

Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see [[../SiteConfiguration | Site Configuration]] for details). This is performed by the yaim MPI module which should be run on both types of nodes.


=== Configure MPI in site-info.def ===
Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable Open MPI, add the following:

{{{#!highlight sh
MPI_OPENMPI_ENABLE="yes"
}}}
You may set the path and version to non default locations for a flavour of MPI as follows:

{{{#!highlight sh
MPI_OPENMPI_PATH="/opt/openmpi-1.3/"
MPI_OPENMPI_VERSION="1.3"
}}}

Possible flavours are:
 * OPENMPI, for Open MPI
 * MPICH for MPICH-1
 * MPICH2 for MPICH-2
 * LAM for LAM-MPI

The remaining variables are:

|| Variable         || Value ||
||`MPI_SHARED_HOME` ||set this to "yes" if you have a shared home area between WNs. ||
||`MPI_SSH_HOST_BASED_AUTH` || set it to "yes" if you have SSH based authentication between WNs ||
||`MPI_<FLAVOUR>_MPIEXEC` ||If you are using OSC mpiexec (only in PBS/Torque sites), set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec" ||
||`MPI_MPI_START` ||Location of mpi-start if not installed in standard location /opt/mpi-start/bin/mpi-start ||
||`MPI_DEFAULT_FLAVOUR` || Set it to the default flavour for your site, if undefined, the first defined flavour will be used ||


Here is an example configuration:

{{{#!highlight sh
#----------------------------------
# MPI-related configuration:
#----------------------------------
# Several MPI implementations (or "flavours") are available.
# If you do NOT want a flavour to be installed/configured, set its variable
# to "no". Else, set it to "yes" (default). If you want to use an
# already installed version of an implementation, set its "_PATH" and
# "_VERSION" variables to match your setup (examples below).
#
# NOTE 1: the CE_RUNTIMEENV will be automatically updated in the file
# functions/config_mpi, so that the CE advertises the MPI implementations
# you choose here - you do NOT have to change it manually in this file.
# It will become something like this:
#
#   CE_RUNTIMEENV="$CE_RUNTIMEENV
#              MPI_MPICH
#              MPI_MPICH2
#              MPI_OPENMPI
#              MPI_LAM"
#
# NOTE 2: it is currently NOT possible to configure multiple concurrent
# versions of the same implementations (e.g. MPICH-1.2.3 and MPICH-1.2.7)
# using YAIM. Customize "/opt/glite/yaim/functions/config_mpi" file
# to do so.

MPI_MPICH_ENABLE="yes"
MPI_MPICH2_ENABLE="yes"
MPI_OPENMPI_ENABLE="yes"
MPI_LAM_ENABLE="yes"

# set Open MPI as default flavour
MPI_DEFAULT_FLAVOUR=OPENMPI


#---
# Example for using an already installed version of MPI.
# Setting "_PATH" and "_VERSION" variables will prevent YAIM
# from downloading and installing the gLite-provided packages.
# Just fill in the path to its current installation (e.g. "/usr")
# and which version it is (e.g. "6.5.9").
#---
MPI_MPICH_PATH="/opt/mpich-1.2.7p1/"
MPI_MPICH_VERSION="1.2.7p1"
MPI_MPICH2_PATH="/opt/mpich2-1.0.4/"
MPI_MPICH2_VERSION="1.0.4"
MPI_OPENMPI_VERSION="1.1"
MPI_LAM_VERSION="7.1.2"

# If you do NOT provide a shared home, set $MPI_SHARED_HOME to "no" (default).
#
MPI_SHARED_HOME="yes"

#
# If you do NOT have SSH Hostbased Authentication between your WNs, set the below
# variable to "no" (default). Else, set it to "yes".
#
MPI_SSH_HOST_BASED_AUTH="no"

#
# If you provide an 'mpiexec' for MPICH or MPICH2, please state the full path to
# that file here (http://www.osc.edu/~pw/mpiexec/index.php). Else, leave empty.
#
#MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
}}}

=== Configure CE ===
{{{/opt/glite/yaim/bin/yaim -c -s site-info.def  -n MPI_CE}}}

=== Configure WN ===
For a Torque worker node: 
{{{/opt/glite/yaim/bin/yaim -c -s site-info.def  -n MPI_WN}}}

== Testing ==
You can do some basic tests by logging in on a WN as a pool user and running the following:

{{{
[dte056@cagnode48 dte056]$ env|grep MPI_
}}}
You should see something like this:

{{{
MPI_MPICC_OPTS=-m32
MPI_SSH_HOST_BASED_AUTH=yes
MPI_OPENMPI_PATH=/opt/openmpi/1.1
MPI_LAM_VERSION=7.1.2
MPI_MPICXX_OPTS=-m32
MPI_LAM_PATH=/usr
MPI_OPENMPI_VERSION=1.1
MPI_MPIF77_OPTS=-m32
MPI_MPICH_VERSION=1.2.7
MPI_MPIEXEC_PATH=/opt/mpiexec-0.80
MPI_MPICH2_PATH=/opt/mpich2-1.0.4
MPI_MPICH2_VERSION=1.0.4
I2G_MPI_START=/opt/mpi-start/bin/mpi-start
MPI_MPICH_PATH=/opt/mpich-1.2.7p1
}}}

You can also try submitting a job to your site, please read [[../UserDocumentation | MPI-Start user documentation]]