|
⇤ ← Revision 1 as of 2011-02-16 15:09:10
Size: 53
Comment:
|
Size: 5841
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 3: | Line 3: |
| Under construction. | <<TableOfContents>> |
| Line 5: | Line 5: |
| == yaim? == | == yaim Configuration == === Overview === Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see SiteConfig for details). This is performed by the gLite YAIM module glite-yaim-mpi which should be run on both the CE and WNs. === Install packages === The YAIM module for MPI configuration is now available as part of the gLite distribution (http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-MPI_utils/sl4/i386/RPMS.release/glite-yaim-mpi-0.1.6-3.noarch.rpm). If you install the meta-package glite-MPI_utils-3.1.1-0.i386.rpm it will pull in this YAIM module as well as other packages needed for MPI. To install this using YUM, add a file (e.g. glite-MPI_utils.repo) with the following contents to your YUM repositories directory (/etc/yum/repos.d): {{{ [glite-MPI_utils] name=glite 3.1 MPI enabled=1 gpgcheck=0 baseurl=http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-MPI_utils/sl4/i386/ }}} Then run this: {{{ yum install glite-MPI_utils }}} === Configure MPI in site-info.def === Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable MPICH, add the following: {{{#!highlight sh MPI_MPICH_ENABLE="yes" }}} You may set the path and version for a flavour of MPI as follows: {{{#!highlight sh MPI_MPICH_PATH="/opt/mpich-1.2.7p1/" MPI_MPICH_VERSION="1.2.7p1" }}} The remaining variables are: || {{{MPI_SHARED_HOME}}} || set this to "yes" if you have a shared home area between WNs.|| || {{{MPI_SSH_HOST_BASED_AUTH}}} || || || {{{MPI_<FLAVOUR>_MPIEXEC}}} || If you are using OSC mpiexec with mpich, set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec" || || {{{MPI_MPI_START}}} || Location of mpi-start if not installed in standard location /opt/i2g/bin/mpi-start || Here is an example configuration: {{{#!highlight sh #---------------------------------- # MPI-related configuration: #---------------------------------- # Several MPI implementations (or "flavours") are available. # If you do NOT want a flavour to be installed/configured, set its variable # to "no". Else, set it to "yes" (default). If you want to use an # already installed version of an implementation, set its "_PATH" and # "_VERSION" variables to match your setup (examples below). # # NOTE 1: the CE_RUNTIMEENV will be automatically updated in the file # functions/config_mpi, so that the CE advertises the MPI implementations # you choose here - you do NOT have to change it manually in this file. # It will become something like this: # # CE_RUNTIMEENV="$CE_RUNTIMEENV # MPI_MPICH # MPI_MPICH2 # MPI_OPENMPI # MPI_LAM" # # NOTE 2: it is currently NOT possible to configure multiple concurrent # versions of the same implementations (e.g. MPICH-1.2.3 and MPICH-1.2.7) # using YAIM. Customize "/opt/glite/yaim/functions/config_mpi" file # to do so. MPI_MPICH_ENABLE="yes" MPI_MPICH2_ENABLE="yes" MPI_OPENMPI_ENABLE="yes" MPI_LAM_ENABLE="yes" #--- # Example for using an already installed version of MPI. # Setting "_PATH" and "_VERSION" variables will prevent YAIM # from downloading and installing the gLite-provided packages. # Just fill in the path to its current installation (e.g. "/usr") # and which version it is (e.g. "6.5.9"). #--- MPI_MPICH_PATH="/opt/mpich-1.2.7p1/" MPI_MPICH_VERSION="1.2.7p1" MPI_MPICH2_PATH="/opt/mpich2-1.0.4/" MPI_MPICH2_VERSION="1.0.4" MPI_OPENMPI_VERSION="1.1" MPI_LAM_VERSION="7.1.2" # If you do NOT provide a shared home, set $MPI_SHARED_HOME to "no" (default). # MPI_SHARED_HOME="yes" # # If you do NOT have SSH Hostbased Authentication between your WNs, set the below # variable to "no" (default). Else, set it to "yes". # MPI_SSH_HOST_BASED_AUTH="no" # # If you provide an 'mpiexec' for MPICH or MPICH2, please state the full path to # that file here (http://www.osc.edu/~pw/mpiexec/index.php). Else, leave empty. # #MPI_MPICH_MPIEXEC="/usr/bin/mpiexec" MPI_MPICH_MPIEXEC="/usr/bin/mpiexec" }}} You should also set the variable CE_BATCH_SYS to "pbs" not "torque": {{{#!highlight sh # Jobmanager specific settings # If WNs and CE share /home, set this: JOB_MANAGER=pbs # If WNs and CE do not use the same shared /home (although WNs may use shared home), set this: #JOB_MANAGER=lcgpbs # If you are supporting MPI, set this: CE_BATCH_SYS=pbs }}} This should cause the variable GlueCEInfoLRMSType to be set correctly in /opt/lcg/var/gip/lcg-info-static-ce.conf and /opt/lcg/var/gip/ldif/static-file-CE.ldif === Configure CE === {{{/opt/glite/yaim/bin/yaim -c -s site-info.def -n MPI_CE}}} === Configure WN === For a Torque worker node: {{{/opt/glite/yaim/bin/yaim -c -s site-info.def -n MPI_WN -n glite-WN -n TORQUE_client}}} === Testing === You can do some basic tests by logging in on a WN as a pool user and running the following: {{{ [dte056@cagnode48 dte056]$ env|grep MPI_ }}} You should see something like this: {{{ MPI_MPICC_OPTS=-m32 MPI_SSH_HOST_BASED_AUTH=yes MPI_OPENMPI_PATH=/opt/openmpi/1.1 MPI_LAM_VERSION=7.1.2 MPI_MPICXX_OPTS=-m32 MPI_LAM_PATH=/usr MPI_OPENMPI_VERSION=1.1 MPI_MPIF77_OPTS=-m32 MPI_MPICH_VERSION=1.2.7 MPI_MPIEXEC_PATH=/opt/mpiexec-0.80 MPI_MPICH2_PATH=/opt/mpich2-1.0.4 MPI_MPICH2_VERSION=1.0.4 I2G_MPI_START=/opt/i2g/bin/mpi-start MPI_MPICH_PATH=/opt/mpich-1.2.7p1 }}} You can also try submitting a job to your site using the instructions found via the page JobSubmission. SAM tests are run daily to verify MPI functionality. You can see the results by going to the SAM tests homepage (https://lxn1181.cern.ch:8443/sam/sam.py) and selecting to display the test "mpi-all" for CEs in the dteam VO. |
MPI-Utils
Contents
yaim Configuration
Overview
Configuration is necessary on both the CE and WNs in order to support and advertise MPI correctly (see SiteConfig for details). This is performed by the gLite YAIM module glite-yaim-mpi which should be run on both the CE and WNs.
Install packages
The YAIM module for MPI configuration is now available as part of the gLite distribution (http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-MPI_utils/sl4/i386/RPMS.release/glite-yaim-mpi-0.1.6-3.noarch.rpm). If you install the meta-package glite-MPI_utils-3.1.1-0.i386.rpm it will pull in this YAIM module as well as other packages needed for MPI.
To install this using YUM, add a file (e.g. glite-MPI_utils.repo) with the following contents to your YUM repositories directory (/etc/yum/repos.d):
[glite-MPI_utils] name=glite 3.1 MPI enabled=1 gpgcheck=0 baseurl=http://glitesoft.cern.ch/EGEE/gLite/R3.1/glite-MPI_utils/sl4/i386/
Then run this:
yum install glite-MPI_utils
Configure MPI in site-info.def
Individual "flavours" of MPI are enabled by setting the associated variable to "yes". For example, to enable MPICH, add the following:
1 MPI_MPICH_ENABLE="yes"
You may set the path and version for a flavour of MPI as follows:
The remaining variables are:
MPI_SHARED_HOME |
set this to "yes" if you have a shared home area between WNs. |
MPI_SSH_HOST_BASED_AUTH |
|
MPI_<FLAVOUR>_MPIEXEC |
If you are using OSC mpiexec with mpich, set this to the location of the mpiexec program, e.g. "/usr/bin/mpiexec" |
MPI_MPI_START |
Location of mpi-start if not installed in standard location /opt/i2g/bin/mpi-start |
Here is an example configuration:
1 #----------------------------------
2 # MPI-related configuration:
3 #----------------------------------
4 # Several MPI implementations (or "flavours") are available.
5 # If you do NOT want a flavour to be installed/configured, set its variable
6 # to "no". Else, set it to "yes" (default). If you want to use an
7 # already installed version of an implementation, set its "_PATH" and
8 # "_VERSION" variables to match your setup (examples below).
9 #
10 # NOTE 1: the CE_RUNTIMEENV will be automatically updated in the file
11 # functions/config_mpi, so that the CE advertises the MPI implementations
12 # you choose here - you do NOT have to change it manually in this file.
13 # It will become something like this:
14 #
15 # CE_RUNTIMEENV="$CE_RUNTIMEENV
16 # MPI_MPICH
17 # MPI_MPICH2
18 # MPI_OPENMPI
19 # MPI_LAM"
20 #
21 # NOTE 2: it is currently NOT possible to configure multiple concurrent
22 # versions of the same implementations (e.g. MPICH-1.2.3 and MPICH-1.2.7)
23 # using YAIM. Customize "/opt/glite/yaim/functions/config_mpi" file
24 # to do so.
25
26 MPI_MPICH_ENABLE="yes"
27 MPI_MPICH2_ENABLE="yes"
28 MPI_OPENMPI_ENABLE="yes"
29 MPI_LAM_ENABLE="yes"
30
31 #---
32 # Example for using an already installed version of MPI.
33 # Setting "_PATH" and "_VERSION" variables will prevent YAIM
34 # from downloading and installing the gLite-provided packages.
35 # Just fill in the path to its current installation (e.g. "/usr")
36 # and which version it is (e.g. "6.5.9").
37 #---
38 MPI_MPICH_PATH="/opt/mpich-1.2.7p1/"
39 MPI_MPICH_VERSION="1.2.7p1"
40 MPI_MPICH2_PATH="/opt/mpich2-1.0.4/"
41 MPI_MPICH2_VERSION="1.0.4"
42 MPI_OPENMPI_VERSION="1.1"
43 MPI_LAM_VERSION="7.1.2"
44
45 # If you do NOT provide a shared home, set $MPI_SHARED_HOME to "no" (default).
46 #
47 MPI_SHARED_HOME="yes"
48
49 #
50 # If you do NOT have SSH Hostbased Authentication between your WNs, set the below
51 # variable to "no" (default). Else, set it to "yes".
52 #
53 MPI_SSH_HOST_BASED_AUTH="no"
54
55 #
56 # If you provide an 'mpiexec' for MPICH or MPICH2, please state the full path to
57 # that file here (http://www.osc.edu/~pw/mpiexec/index.php). Else, leave empty.
58 #
59 #MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
60 MPI_MPICH_MPIEXEC="/usr/bin/mpiexec"
You should also set the variable CE_BATCH_SYS to "pbs" not "torque":
This should cause the variable GlueCEInfoLRMSType to be set correctly in /opt/lcg/var/gip/lcg-info-static-ce.conf and /opt/lcg/var/gip/ldif/static-file-CE.ldif
Configure CE
/opt/glite/yaim/bin/yaim -c -s site-info.def -n MPI_CE
Configure WN
For a Torque worker node: /opt/glite/yaim/bin/yaim -c -s site-info.def -n MPI_WN -n glite-WN -n TORQUE_client
Testing
You can do some basic tests by logging in on a WN as a pool user and running the following:
[dte056@cagnode48 dte056]$ env|grep MPI_
You should see something like this:
MPI_MPICC_OPTS=-m32 MPI_SSH_HOST_BASED_AUTH=yes MPI_OPENMPI_PATH=/opt/openmpi/1.1 MPI_LAM_VERSION=7.1.2 MPI_MPICXX_OPTS=-m32 MPI_LAM_PATH=/usr MPI_OPENMPI_VERSION=1.1 MPI_MPIF77_OPTS=-m32 MPI_MPICH_VERSION=1.2.7 MPI_MPIEXEC_PATH=/opt/mpiexec-0.80 MPI_MPICH2_PATH=/opt/mpich2-1.0.4 MPI_MPICH2_VERSION=1.0.4 I2G_MPI_START=/opt/i2g/bin/mpi-start MPI_MPICH_PATH=/opt/mpich-1.2.7p1
You can also try submitting a job to your site using the instructions found via the page JobSubmission.
SAM tests are run daily to verify MPI functionality. You can see the results by going to the SAM tests homepage (https://lxn1181.cern.ch:8443/sam/sam.py) and selecting to display the test "mpi-all" for CEs in the dteam VO.