eciencia: Middleware/MpiStart/DeveloperDocumentation

Developer information

This section gives an overview about the interfaces of different components of MPI-Start for developing new plugins.

Variables

All necessary information of the mpi-start program are provided via environment variables. See ../UserDocumentation and ../SiteConfiguration for a list of those variables.

Logging and debugging

The I2G_MPI_START_VERBOSE variable can be set to 1 to turn on the additional output. For debugging purpose the I2G_MPI_START_DEBUG variable can be set to 1 to enable the debugging output. The variable I2G_MPI_START_TRACE can be set to 1 to trace every operation that is performed by mpi-start (goes to stderr).

The functions for logging messages to output are the following:

The dump_vars function dumps the complete environment to the output.

Scheduler plugins

A scheduler plugin is responsible for detection of a runtime system and the definition of process distribution variables. A scheduler plugin must provide the following functions and variables:

The process distribution variables that the scheduler plugin must set are:

Variable

Meaning

MPI_START_MACHINEFILE

This variable points a machinefile in the standard format. In the standard format there is one host per line. If multiple processes should run on a host this host must be inserted multiple times.

MPI_START_HOSTFILE

Variable pointing to a file with a list of available hosts with one host per line without host duplicates

MPI_START_HOST_SLOTS_FILE

Variable pointing to a file where for each line there is a hostname and the number of slots allocated in that host.

MPI_START_NHOSTS

Number of hosts allocated (number of lines in the MPI_START_HOSTFILE file)

MPI_START_NSLOTS

Number of slots allocated (number of lines in the MPI_START_MACHINEFILE file)

MPI_START_NSLOTS_PER_HOST

Number of slots allocated in each host. If the slots are not equally distributed, put here the maximum number of slots available on every host.

Execution environment plugins

A Execution Environment plugin is responsible for setting up the environment for the specified execution environment (MPI) version. It has to provide the following functions:

Several internal variables may change the behavior of the execution environment:

Variable

Meaning

MPI_START_SCHEDULER

Name of the scheduler plugin loaded

MPI_START_MPI_PREFIX

Path of the execution environment

In order to provide a different environment for the user application, MPI-Start uses a wrapper that is executed as a different process. In this wrapper, the variables defined in the MPI_START_ENV_VARIABLES are exported to the environment. The use of the wrapper can be controlled with the functions:

Clean up

When mpi-start finishes its execution the clean_up function is called. This will remove any temporary files created by MPI-Start by default (e.g. host files) and any other file appended to the MPI_START_CLEANUP_FILES variable.

Hooks

See Hooks Frameworkfor a complete description of the hooks mechanism and how to develop new hooks.

Internals

Global Configuration Variables

Variable

Default

Description

MPI_START_DUMMY_SCHEDULER

1

Enables or disables the dummy scheduler.

I2G_MPI_START_KEEP_FILES

0

Enables or disables the removal of temporary files at the end of execution.

I2G_MPI_START_FULL_TRACE

-

Enables or disables full trace of mpi-start.

MPI_START_DO_NOT_USE_WRAPPER

-

Enables or disables the use of a wrapper for the executable

MPI_START_SOCKETS

-

Number of sockets in the host (if not defined, mpi-start tries to detect them).

MPI_START_COREPERSOCKET

-

Number of cores per socket in the host (if not defined, mpi-start tries to detect them).

MPI_START_COREPERSOCKET

-

Number of cores per socket in the host (if not defined, mpi-start tries to detect them).

I2G_MPI_START_ENABLE_TESTING

-

If equal to "TEST", then do not call the main function. Used for sourcing the mpi-start file.

Scheduler plugin variables

Variable

Description

MPI_START_SCHEDULER

Name of the scheduler.

MPI_START_HOSTFILE

File containing one line per slot available.

MPI_START_MACHINEFILE

File containing one line per host available.

MPI_START_HOST_SLOTS_FILE

File containing one line with a name of host, and the number of slots available in that host.

MPI_START_NP

Total number of processors to use.

MPI_START_NPHOST

Number of processes per host, may be undefined if slot allocation is used.

MPI Execution Variables

Variable

Description

MPIEXEC

Defined by each flavour, mpiexec executable

MPI_GLOBAL_PARAMS

Global parameters for mpiexec

MPI_LOCAL_PARAMS

Local parameters for mpiexec

MPI_START_SSH_AGENT

User (or system) specified ssh agent (used mostly by condor).

MPI_START_DISABLE_LRMS_INTEGRATION

If set to "yes", do not use any LRMS integration available in the MPI flavour.

MPI_MPICH2_DISABLE_HYDRA

If set to 1, disable the use of hydra launcher for mpich2.

OSC_MPIEXEC

Set to 1 if OSC mpiexec is found.

HYDRA_MPIEXEC

Set to 1 if hydra mpiexec is found.

OPENMPI_VERSION_MAJOR

Set to Open MPI major version.

OPENMPI_VERSION_MINOR

Set to Open MPI minor version.

OPENMPI_VERSION_RELEASE

Set to Open MPI release version.

eciencia: Middleware/MpiStart/DeveloperDocumentation (last edited 2011-11-24 15:55:00 by enol)