welcome: please sign in

Upload page content

You can upload content for the page named below. If you change the page name, you can also upload content for another page. If the page name is empty, we derive the page name from the file name.

File to load page content from
Page name
Comment

Revision 11 as of 2013-02-27 09:09:44

location: Middleware / MpiStart / ParallelEnvironment

MPI-Start as EMI-ES ParallelEnvironment backend

Contents

  1. UNICORE
  2. ARC
    1. Explicit mpi-start
    2. mpi-start backend
  3. CREAM

1. UNICORE

The UNICORE EMI-ES ParallelEnvironment is a complete implementation of the specification. The configuration is done via xml files where the mapping from the job specification to the final executable is specified.

For each mpi-flavor that needs to be supported by mpi-start an entry in the idb must be included. For example for OpenMPI:

Toggle line numbers 
   1 <ee:ExecutionEnvironment xmlns:ee="http://www.unicore.eu/unicore/jsdl-extensions">
   2     <ee:Name>OpenMPI</ee:Name>
   3     <ee:Version>1.5.4</ee:Version>
   4     <ee:Description>Runs OpenMPI job using mpi-start to start the job</ee:Description>
   5     <ee:ExecutableName>/usr/bin/mpi-start</ee:ExecutableName>
   6     <ee:CommandlineTemplate>#EXECUTABLE -t openmpi #ARGS #USERCOMMAND #USERARGS</ee:CommandlineTemplate>
   7     <ee:Argument>
   8       <ee:Name>Output</ee:Name>
   9       <ee:IncarnatedValue>-o </ee:IncarnatedValue>
  10       <ee:ArgumentMetadata>
  11         <ee:Type>string</ee:Type>
  12         <ee:Description>Write the job output to a FILE instead of to the standard output stream</ee:Description>
  13       </ee:ArgumentMetadata>
  14     </ee:Argument>
  15     <ee:Argument>
  16       <ee:Name>PostHook</ee:Name>
  17       <ee:IncarnatedValue>-post </ee:IncarnatedValue>
  18       <ee:ArgumentMetadata>
  19         <ee:Type>string</ee:Type>
  20         <ee:Description>Use the file as post hook</ee:Description>
  21       </ee:ArgumentMetadata>
  22     </ee:Argument>
  23     <ee:Argument>
  24       <ee:Name>PreHook</ee:Name>
  25       <ee:IncarnatedValue>-pre </ee:IncarnatedValue>
  26       <ee:ArgumentMetadata>
  27         <ee:Type>string</ee:Type>
  28         <ee:Description>Use the file as pre hook</ee:Description>
  29       </ee:ArgumentMetadata>
  30     </ee:Argument>
  31     <ee:Argument>
  32       <ee:Name>Error</ee:Name>
  33       <ee:IncarnatedValue>-e </ee:IncarnatedValue>
  34       <ee:ArgumentMetadata>
  35         <ee:Type>string</ee:Type>
  36         <ee:Description>Write the job error to a FILE instead of to the standard error stream</ee:Description>
  37       </ee:ArgumentMetadata>
  38     </ee:Argument>
  39     <ee:Option> 
  40       <ee:Name>Verbose</ee:Name> 
  41       <ee:IncarnatedValue>-v</ee:IncarnatedValue> 
  42       <ee:OptionMetadata> 
  43         <ee:Description>Enable verbose mode</ee:Description> 
  44       </ee:OptionMetadata> 
  45     </ee:Option> 
  46     <ee:Option> 
  47       <ee:Name>PerNode</ee:Name> 
  48       <ee:IncarnatedValue>-pnode</ee:IncarnatedValue> 
  49       <ee:OptionMetadata> 
  50         <ee:Description>Start one process per available node</ee:Description> 
  51       </ee:OptionMetadata> 
  52     </ee:Option> 
  53     <ee:Option> 
  54       <ee:Name>PerSocket</ee:Name> 
  55       <ee:IncarnatedValue>-psocket</ee:IncarnatedValue> 
  56       <ee:OptionMetadata> 
  57         <ee:Description>Start one process per available socket</ee:Description> 
  58       </ee:OptionMetadata> 
  59     </ee:Option> 
  60     <ee:Option> 
  61       <ee:Name>PerCore</ee:Name> 
  62       <ee:IncarnatedValue>-pcore</ee:IncarnatedValue> 
  63       <ee:OptionMetadata> 
  64         <ee:Description>Start one process per available core</ee:Description> 
  65       </ee:OptionMetadata> 
  66     </ee:Option> 
  67 </ee:ExecutionEnvironment>

Sample job:

Toggle line numbers 
   1 <ActivityDescription xmlns="http://www.eu-emi.eu/es/2010/12/adl">
   2     <ActivityIdentification>
   3         <Name>test job</Name>
   4         <Description>A test job showing the features of EMI-ES</Description>
   5         <Type>single</Type>
   6         <Annotation>test</Annotation>
   7     </ActivityIdentification>
   8     <Application>
   9         <Executable><Path>cpi</Path></Executable>
  10         <Error>std.err</Error>
  11         <Output>std.out</Output>
  12     </Application>
  13     <Resources>
  14         <SlotRequirement>
  15             <NumberOfSlots>2</NumberOfSlots>
  16             <SlotsPerHost>2</SlotsPerHost>
  17             <ExclusiveExecution>true</ExclusiveExecution>
  18         </SlotRequirement>
  19         <ParallelEnvironment>
  20            <Type>OpenMPI</Type>
  21         </ParallelEnvironment>
  22     </Resources>
  23     <DataStaging>
  24         <InputFile> <Name>pre.sh</Name> <Source><URI>pre.sh</URI></Source> </InputFile>
  25         <InputFile> <Name>cpi.c</Name> <Source><URI>cpi.c</URI></Source> </InputFile>
  26     </DataStaging>
  27 </ActivityDescription>

2. ARC

ARC supports MPI jobs through the RunTimeEnvironments. There are two options for integrating them with mpi-start

  1. Explicit mpi-start usage by users.
  2. mpi-start as backend of parallel jobs

2.1. Explicit mpi-start

In this use case, the only requirements is that mpi-start is available at the nodes that will execute the job. There is no need for a RuntimeEnvironment and the user must set as executable for the jobs mpi-start. A minimal job description is shown below:

Toggle line numbers 
   1 <ActivityDescription xmlns="http://www.eu-emi.eu/es/2010/12/adl">
   2     <ActivityIdentification>
   3         <Name>MPI Test job</Name>
   4         <Description>A test job showing the features of EMI-ES</Description>
   5         <Type>single</Type>
   6     </ActivityIdentification>
   7     <Application>
   8         <Executable>
   9             <Path>/usr/bin/mpi-start</Path>
  10             <Argument>-t</Argument>
  11             <Argument>openmpi</Argument>
  12             <Argument>-pre</Argument>
  13             <Argument>pre.sh</Argument>
  14             <Argument>cpi</Argument>
  15         </Executable>
  16         <Error>std.err</Error>
  17         <Output>std.out</Output>
  18     </Application>
  19     <Resources>
  20         <SlotRequirement>
  21             <NumberOfSlots>2</NumberOfSlots>
  22             <SlotsPerHost>2</SlotsPerHost>
  23             <ExclusiveExecution>true</ExclusiveExecution>
  24         </SlotRequirement>
  25     </Resources>
  26 
  27     <DataStaging>
  28         <InputFile><Name>pre.sh</Name><Source><URI>pre.sh</URI></Source></InputFile>
  29         <InputFile><Name>cpi.c</Name><Source><URI>cpi.c</URI></Source>
  30         </InputFile>
  31 </DataStaging>
  32 </ActivityDescription>

Note that the -t openmpi option is passed explicitly to indicate to mpi-start which implementation to use.

2.2. mpi-start backend

In this case, a RuntimeEnvironment prepares the execution with mpi-start, hiding from the user the details of the implementation. Any additional arguments to mpi-start must be passed as environment variables.

A sample runtime environment for running !OpenMPI applications would be:

Toggle line numbers 
   1 #!/bin/sh
   2 
   3 export I2G_MPI_TYPE=openmpi
   4 
   5 case "$1" in
   6 0)
   7         joboption_args="mpi-start -- $joboption_args"
   8 ;;
   9 1)
  10 ;;
  11 2)
  12 ;;
  13 *)
  14         return 1
  15 ;;
  16 esac

The job description would look as follows:

Toggle line numbers 
   1 <ActivityDescription xmlns="http://www.eu-emi.eu/es/2010/12/adl">
   2     <ActivityIdentification>
   3         <Name>test job</Name>
   4         <Description>A test job showing the features of EMI-ES</Description>
   5         <Type>single</Type>
   6         <Annotation>test</Annotation>
   7     </ActivityIdentification>
   8     <Application>
   9         <Executable><Path>cpi</Path></Executable>
  10         <Error>std.err</Error>
  11         <Output>std.out</Output>
  12         <Environment><Name>I2G_MPI_PRE_RUN_HOOK</Name><Value>pre.sh</Value></Environment>
  13     </Application>
  14     <Resources>
  15         <SlotRequirement>
  16             <NumberOfSlots>2</NumberOfSlots>
  17             <SlotsPerHost>2</SlotsPerHost>
  18             <ExclusiveExecution>true</ExclusiveExecution>
  19         </SlotRequirement>
  20         <RuntimeEnvironment><Name>OPENMPI</Name></RuntimeEnvironment>
  21     </Resources>
  22     <DataStaging>
  23         <InputFile> <Name>pre.sh</Name> <Source><URI>pre.sh</URI></Source> </InputFile>
  24         <InputFile> <Name>cpi.c</Name> <Source><URI>cpi.c</URI></Source> </InputFile>
  25         <!-- Fake input file -->
  26         <InputFile> <Name>cpi</Name> <Source><URI>cpi</URI></Source> </InputFile>
  27     </DataStaging>
  28 </ActivityDescription>

3. CREAM

Cream does not support the ParallelEnvironment. Jobs may be sumbmitted using /usr/bin/mpi-start as executable. See the previous ARC example for details.