Differences between revisions 3 and 5 (spanning 2 versions)

MPI-Utils

Contents

MPI-Utils

Installation

MPI-Utils is a meta package that depends on MPI-Start and the yaim mpi module for configuration of the MPI support in the CE and WN. Administrators must install a MPI implementation and configure it at the site. Most Linux distributions provide ready to use packages for Open MPI and MPICH implmentations.

MPI-Utils can be installed from the EMI repositories, and should be as easy as:

yum install glite-MPI_utils

Configuration

Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see SiteConfig for details). This is performed by the yaim MPI module which should be run on both types of nodes.

Configure MPI in site-info.def

Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable Open MPI, add the following:

   1 MPI_OPENMPI_ENABLE="yes"

You may set the path and version to non default locations for a flavour of MPI as follows:

   1 MPI_OPENMPI_PATH="/opt/openmpi-1.3/"
   2 MPI_OPENMPI_VERSION="1.3"

Possible flavours are:

OPENMPI, for Open MPI
MPICH for MPICH-1
MPICH2 for MPICH-2
LAM for LAM-MPI

The remaining variables are:

Variable	Value
`MPI_SHARED_HOME`	set this to "yes" if you have a shared home area between WNs.
`MPI_SSH_HOST_BASED_AUTH`	set it to "yes" if you have SSH base authentication between WNs
`MPI_<FLAVOUR>_MPIEXEC`	If you are using OSC mpiexec (only in PBS/Torque sites), set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec"
`MPI_MPI_START`	Location of mpi-start if not installed in standard location /opt/mpi-start/bin/mpi-start
`MPI_DEFAULT_FLAVOUR`	Set it to the default flavour for your site, if undefined, the first defined flavour will be used

Here is an example configuration:

   1 #----------------------------------
   2 # MPI-related configuration:
   3 #----------------------------------
   4 # Several MPI implementations (or "flavours") are available.
   5 # If you do NOT want a flavour to be installed/configured, set its variable
   6 # to "no". Else, set it to "yes" (default). If you want to use an
   7 # already installed version of an implementation, set its "_PATH" and
   8 # "_VERSION" variables to match your setup (examples below).
   9 #
  10 # NOTE 1: the CE_RUNTIMEENV will be automatically updated in the file
  11 # functions/config_mpi, so that the CE advertises the MPI implementations
  12 # you choose here - you do NOT have to change it manually in this file.
  13 # It will become something like this:
  14 #
  15 #   CE_RUNTIMEENV="$CE_RUNTIMEENV
  16 #              MPI_MPICH
  17 #              MPI_MPICH2
  18 #              MPI_OPENMPI
  19 #              MPI_LAM"
  20 #
  21 # NOTE 2: it is currently NOT possible to configure multiple concurrent
  22 # versions of the same implementations (e.g. MPICH-1.2.3 and MPICH-1.2.7)
  23 # using YAIM. Customize "/opt/glite/yaim/functions/config_mpi" file
  24 # to do so.
  25 
  26 MPI_MPICH_ENABLE="yes"
  27 MPI_MPICH2_ENABLE="yes"
  28 MPI_OPENMPI_ENABLE="yes"
  29 MPI_LAM_ENABLE="yes"
  30 
  31 # set Open MPI as default flavour
  32 MPI_DEFAULT_FLAVOUR=OPENMPI
  33 
  34 
  35 #---
  36 # Example for using an already installed version of MPI.
  37 # Setting "_PATH" and "_VERSION" variables will prevent YAIM
  38 # from downloading and installing the gLite-provided packages.
  39 # Just fill in the path to its current installation (e.g. "/usr")
  40 # and which version it is (e.g. "6.5.9").
  41 #---
  42 MPI_MPICH_PATH="/opt/mpich-1.2.7p1/"
  43 MPI_MPICH_VERSION="1.2.7p1"
  44 MPI_MPICH2_PATH="/opt/mpich2-1.0.4/"
  45 MPI_MPICH2_VERSION="1.0.4"
  46 MPI_OPENMPI_VERSION="1.1"
  47 MPI_LAM_VERSION="7.1.2"
  48 
  49 # If you do NOT provide a shared home, set $MPI_SHARED_HOME to "no" (default).
  50 #
  51 MPI_SHARED_HOME="yes"
  52 
  53 #
  54 # If you do NOT have SSH Hostbased Authentication between your WNs, set the below
  55 # variable to "no" (default). Else, set it to "yes".
  56 #
  57 MPI_SSH_HOST_BASED_AUTH="no"
  58 
  59 #
  60 # If you provide an 'mpiexec' for MPICH or MPICH2, please state the full path to
  61 # that file here (http://www.osc.edu/~pw/mpiexec/index.php). Else, leave empty.
  62 #
  63 #MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
  64 MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"

Configure CE

/opt/glite/yaim/bin/yaim -c -s site-info.def -n MPI_CE

Configure WN

For a Torque worker node: /opt/glite/yaim/bin/yaim -c -s site-info.def -n MPI_WN

Testing

You can do some basic tests by logging in on a WN as a pool user and running the following:

[dte056@cagnode48 dte056]$ env|grep MPI_

You should see something like this:

MPI_MPICC_OPTS=-m32
MPI_SSH_HOST_BASED_AUTH=yes
MPI_OPENMPI_PATH=/opt/openmpi/1.1
MPI_LAM_VERSION=7.1.2
MPI_MPICXX_OPTS=-m32
MPI_LAM_PATH=/usr
MPI_OPENMPI_VERSION=1.1
MPI_MPIF77_OPTS=-m32
MPI_MPICH_VERSION=1.2.7
MPI_MPIEXEC_PATH=/opt/mpiexec-0.80
MPI_MPICH2_PATH=/opt/mpich2-1.0.4
MPI_MPICH2_VERSION=1.0.4
I2G_MPI_START=/opt/mpi-start/bin/mpi-start
MPI_MPICH_PATH=/opt/mpich-1.2.7p1

You can also try submitting a job to your site, please read MPI-Start user documentation

eciencia: Middleware/MpiStart/MpiUtils (last edited 2012-02-17 14:34:36 by enol)

-  ⇤ ← Revision 3 as of 2011-02-17 10:28:44 → 
  Size: 5452
  Editor: enol
  Comment:
+   ← Revision 5 as of 2011-02-18 08:33:40 → ⇥
  Size: 5195
  Editor: enol
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 5:
-The YAIM module for MPI configuration is now available as part of the gLite distribution (http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-MPI_utils/sl4/i386/RPMS.release/glite-yaim-mpi-0.1.6-3.noarch.rpm). If you install the meta-package glite-MPI_utils-3.1.1-0.i386.rpm it will pull in this YAIM module as well as other packages needed for MPI.
+MPI-Utils is a meta package that depends on MPI-Start and the yaim mpi module for configuration of the MPI support in the CE and WN. Administrators must install a MPI implementation and configure it at the site. Most Linux distributions provide ready to use packages for Open MPI and MPICH implmentations.
 Line 7:
-To install this using YUM, add a file (e.g. glite-MPI_utils.repo) with the following contents to your YUM repositories directory (/etc/yum/repos.d):

{{{
[glite-MPI_utils]
name=glite 3.1 MPI
enabled=1
gpgcheck=0
baseurl=http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-MPI_utils/sl4/i386/
}}}
Then run this:
+MPI-Utils can be installed from the EMI repositories, and should be as easy as:
-Line 22:
+Line 13:
-= yaim Configuration =
== Overview ==
Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see SiteConfig for details). This is performed by the gLite YAIM module glite-yaim-mpi which should be run on both the CE and WNs.
+== Configuration ==

Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see SiteConfig for details). This is performed by the yaim MPI module which should be run on both types of nodes.
-Line 28:
+Line 19:
-Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable MPICH, add the following:
+Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable Open MPI, add the following:
-Line 31:
+Line 22:
-MPI_MPICH_ENABLE="yes"
+MPI_OPENMPI_ENABLE="yes"
-Line 33:
+Line 24:
-You may set the path and version for a flavour of MPI as follows:
+You may set the path and version to non default locations for a flavour of MPI as follows:
-Line 36:
+Line 27:
-MPI_MPICH_PATH="/opt/mpich-1.2.7p1/"
MPI_MPICH_VERSION="1.2.7p1"
+MPI_OPENMPI_PATH="/opt/openmpi-1.3/"
MPI_OPENMPI_VERSION="1.3"
-Line 40:
+Line 31:
+Possible flavours are:
 * OPENMPI, for Open MPI
 * MPICH for MPICH-1
 * MPICH2 for MPICH-2
 * LAM for LAM-MPI
-Line 41:
+Line 38:
-||{{{MPI_SHARED_HOME}}} ||set this to "yes" if you have a shared home area between WNs. ||
||{{{MPI_SSH_HOST_BASED_AUTH}}} || ||
||{{{MPI_<FLAVOUR>_MPIEXEC}}} ||If you are using OSC mpiexec with mpich, set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec" ||
||{{{MPI_MPI_START}}} ||Location of mpi-start if not installed in standard location /opt/i2g/bin/mpi-start ||
+|| Variable         || Value ||
||`MPI_SHARED_HOME` ||set this to "yes" if you have a shared home area between WNs. ||
||`MPI_SSH_HOST_BASED_AUTH` || set it to "yes" if you have SSH base authentication between WNs ||
||`MPI_<FLAVOUR>_MPIEXEC` ||If you are using OSC mpiexec (only in PBS/Torque sites), set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec" ||
||`MPI_MPI_START` ||Location of mpi-start if not installed in standard location /opt/mpi-start/bin/mpi-start ||
||`MPI_DEFAULT_FLAVOUR` || Set it to the default flavour for your site, if undefined, the first defined flavour will be used ||
 Line 80:
+# set Open MPI as default flavour
MPI_DEFAULT_FLAVOUR=OPENMPI
-Line 113:
+Line 117:
-This should cause the variable GlueCEInfoLRMSType to be set correctly in  /opt/lcg/var/gip/lcg-info-static-ce.conf and /opt/lcg/var/gip/ldif/static-file-CE.ldif

=== Configure CE ===
+==== Configure CE ====
-Line 118:
+Line 120:
-=== Configure WN ===
For a Torque worker node: {{{/opt/glite/yaim/bin/yaim -c -s site-info.def  -n MPI_WN -n glite-WN  -n TORQUE_client}}}
+==== Configure WN ====
For a Torque worker node: 
{{{/opt/glite/yaim/bin/yaim -c -s site-info.def  -n MPI_WN}}}
-Line 121:
+Line 124:
-=== Testing ===
+== Testing ==
-Line 142:
+Line 145:
-I2G_MPI_START=/opt/i2g/bin/mpi-start
+I2G_MPI_START=/opt/mpi-start/bin/mpi-start
-Line 145:
+Line 148:
-You can also try submitting a job to your site using the instructions found via the page JobSubmission.
-Line 147:
+Line 149:
-SAM tests are run daily to verify MPI functionality. You can see the results by going to the SAM tests homepage (https://lxn1181.cern.ch:8443/sam/sam.py) and selecting to display the test "mpi-all" for CEs in the dteam VO.
+You can also try submitting a job to your site, please read [[../UserDocumentation | MPI-Start user documentation]]

Quick Links

Search Wiki

Page Tools