welcome: please sign in

Upload page content

You can upload content for the page named below. If you change the page name, you can also upload content for another page. If the page name is empty, we derive the page name from the file name.

File to load page content from
Page name
Comment

Revision 9 as of 2013-02-26 08:37:21

location: Middleware / MpiStart / ParallelEnvironment

MPI-Start as EMI-ES ParallelEnvironment backend

Contents

  1. UNICORE
  2. ARC
    1. Explicit mpi-start
    2. mpi-start backend
  3. CREAM

1. UNICORE

The UNICORE EMI-ES ParallelEnvironment is a complete implementation of the specification. The configuration is done via xml files where the mapping from the job specification to the final executable is specified.

For each mpi-flavor that needs to be supported by mpi-start A sample file is: 

   1 <a>
   2 </a>

2. ARC

ARC supports MPI jobs through the RunTimeEnvironments. There are two options for integrating them with mpi-start

  1. Explicit mpi-start usage by users.
  2. mpi-start as backend of parallel jobs

2.1. Explicit mpi-start

In this use case, the only requirements is that mpi-start is available at the nodes that will execute the job. There is no need for a RuntimeEnvironment and the user must set as executable for the jobs mpi-start. A minimal job description is shown below: 

   1 <ActivityDescription xmlns="http://www.eu-emi.eu/es/2010/12/adl">
   2     <ActivityIdentification>
   3         <Name>MPI Test job</Name>
   4         <Description>A test job showing the features of EMI-ES</Description>
   5         <Type>single</Type>
   6     </ActivityIdentification>
   7     <Application>
   8         <Executable>
   9             <Path>/usr/bin/mpi-start</Path>
  10             <Argument>-t</Argument>
  11             <Argument>openmpi</Argument>
  12             <Argument>-pre</Argument>
  13             <Argument>pre.sh</Argument>
  14             <Argument>cpi</Argument>
  15         </Executable>
  16         <Error>std.err</Error>
  17         <Output>std.out</Output>
  18     </Application>
  19     <Resources>
  20         <SlotRequirement>
  21             <NumberOfSlots>2</NumberOfSlots>
  22             <SlotsPerHost>2</SlotsPerHost>
  23             <ExclusiveExecution>true</ExclusiveExecution>
  24         </SlotRequirement>
  25     </Resources>
  26 
  27     <DataStaging>
  28         <InputFile><Name>pre.sh</Name><Source><URI>pre.sh</URI></Source></InputFile>
  29         <InputFile><Name>cpi.c</Name><Source><URI>cpi.c</URI></Source>
  30         </InputFile>
  31 </DataStaging>
  32 </ActivityDescription>

Note that the -t openmpi option is passed explicitly to indicate to mpi-start which implementation to use.

2.2. mpi-start backend

In this case, a RuntimeEnvironment prepares the execution with mpi-start, hiding from the user the details of the implementation. Any additional arguments to mpi-start must be passed as environment variables.

A sample runtime environment for running !OpenMPI applications would be: 

   1 #!/bin/sh
   2 
   3 export I2G_MPI_TYPE=openmpi
   4 
   5 case "$1" in
   6 0)
   7         joboption_args="mpi-start -- $joboption_args"
   8 ;;
   9 1)
  10 ;;
  11 2)
  12 ;;
  13 *)
  14         return 1
  15 ;;
  16 esac

The job description would look as follows: 

   1 <ActivityDescription xmlns="http://www.eu-emi.eu/es/2010/12/adl">
   2     <ActivityIdentification>
   3         <Name>test job</Name>
   4         <Description>A test job showing the features of EMI-ES</Description>
   5         <Type>single</Type>
   6         <Annotation>test</Annotation>
   7     </ActivityIdentification>
   8     <Application>
   9         <Executable><Path>cpi</Path></Executable>
  10         <Error>std.err</Error>
  11         <Output>std.out</Output>
  12         <Environment><Name>I2G_MPI_PRE_RUN_HOOK</Name><Value>pre.sh</Value></Environment>
  13     </Application>
  14     <Resources>
  15         <SlotRequirement>
  16             <NumberOfSlots>2</NumberOfSlots>
  17             <SlotsPerHost>2</SlotsPerHost>
  18             <ExclusiveExecution>true</ExclusiveExecution>
  19         </SlotRequirement>
  20         <RuntimeEnvironment><Name>OPENMPI</Name></RuntimeEnvironment>
  21     </Resources>
  22     <DataStaging>
  23         <InputFile> <Name>pre.sh</Name> <Source><URI>pre.sh</URI></Source> </InputFile>
  24         <InputFile> <Name>cpi.c</Name> <Source><URI>cpi.c</URI></Source> </InputFile>
  25         <!-- Fake input file -->
  26         <InputFile> <Name>cpi</Name> <Source><URI>cpi</URI></Source> </InputFile>
  27     </DataStaging>
  28 </ActivityDescription>

3. CREAM

Cream does not support the ParallelEnvironment. Jobs may be sumbmitted using /usr/bin/mpi-start as executable. See the previous ARC example for details.