= Hooks Framework =
The MPI-Start Hooks Framework allow the extension of MPI-Start features without changing the core functionality.

Several hooks are included in the default distribution of MPI-Start for dealing with [[#File_distribution_hooks|file distribution]] and some [[#Extensions_hooks|MPI extensions]].

Site admins can check the [[#Local_site_hooks|local hooks]] description.

Users probably are interested in [[#Developing_User_Hooks|developing their own hooks]].

== File distribution hooks ==
File distribution hooks are responsible for providing a common set of files prior to the execution of the application in all the hosts involved in that execution. Two steps are taken for file distribution:

 * Detection of shared filesystem: MPI-Start will try to detect if the current working directory is in a network file system (currently considered as such are: nfs, gfs, afs, smb, gpfs and lustre). If the detection is positive, the distribution of files is not performed. Detection can be totally skipped by setting: `MPI_START_SHARED_FS` to 0 or 1 (0 means that MPI-Start will try distribution in the next step, and 1 that it won't).

 * File distribution: in this step, MPI-Start copies files from the current host to the other hosts involved in the execution. It uses the most suitable of the available distribution methods. Distribution methods are plugins that are detected at runtime by checking all the files with `.filedist` extension in the MPI-Start `etc` directory.

The file distribution method can be fixed by using the `I2G_MPI_FILE_DIST` variable.

=== File Distribution Method Plugins ===
A file distribution plugin must contain the following functions:

 * `check_distribution_method()`: called during initialization to check if the distribution method is suitable. It returns a number, the lower the number it returns, the higher priority it will have. If the method is not suitable, then it should return 255 or larger.
 * `copy()`: perform the actual copy of files between hosts. Files are in a gzipped tarball pointed by the `$TARBALL` environment variable.
 * `clean()`: clean up any files once the execution is finished.

These distribution methods are included in MPI-Start:

 * ssh: uses scp to copy files, needs passwordless ssh properly configured.
 * mpiexec: uses OSC mpiexec for copying, needs OSC mpiexec installed.
 * cptoshared_area: copies files to a shared area that is not the current working directory. Needs the following variables:
  * `MPI_SHARED_HOME`: set to "yes".
  * `MPI_SHARED_HOME_PATH`: path of the shared area that will be used for execution.
 * mpi_mt: uses the mpi_mt for copying the files. Needs mpi_mt available in all machines.

== Extensions hooks ==
Extension hooks are local site hooks that come in the default MPI-Start distribution. These 3 hooks are available:

 * MPI_TRACE
 * MARMOT
 * Compiler flags

These hooks can be removed by deleting the mpitrace.hook, marmot.hook, or compiler.hook in the mpi-start configuration directory.

=== MPI_TRACE ===
The `MPI_TRACE` is enabled by setting the `I2G_USE_MPITRACE` variable to `1`. It adds to the execution the mpitrace utility, assuming it is installed at `MPITRACE_INSTALLATION`. Once the execution is finished, it gathers and creates the output files at the first host.

=== MARMOT ===
[[http://www.hlrs.de/organization/av/amt/projects/marmot/|Marmot]] is a tool for analysing and checking MPI programs. This hook enables the use of the tool if the variable `I2G_USE_MARMOT` is set to `1`. It also copies the analysis output to the first host.

=== Compiler flags ===
Sites that do have various compilers and support various architectures may not have proper compiler flags (`MPI_MPIxx_OPTS`) for all the possible combination. This plugin tries to avoid compilation errors by checking the current compiler options and change them to the default architecture of the compiler if the binaries are not generated.

== Local site hooks ==
Site admins can define their own hooks by creating `.hook` files in the configuration directory of MPI-Start (by default /opt/i2g/etc/mpi-start). The file must contain one of the following functions:

 * `pre_run_hook ()`: it will be executed before the user hooks and the user application gets executed.
 * `post_run_hook ()`: it will be executed after the user application gets executed.

== Developing User Hooks ==
Users can also customize the MPI-Start behavior defining their own hooks by setting the `I2G_MPI_PRE_RUN_HOOK` or `I2G_MPI_POST_RUN_HOOK` variables.

 * `I2G_MPI_PRE_RUN_HOOK`: path of the file containing the pre-hook, in this file a `pre_run_hook()` function must be available. This function will be called before the application execution. The pre-hook can be used, for example, to compile the executable itself or download data.

 * `I2G_MPI_POST_RUN_HOOK`: path of the file containing the post-hook, in this file a `post_run_hook()` function must be available. This function will be called once the applications finishes its execution. The post-hook can be used to analyze results or to save the results on the grid.

Both pre and post hooks can be in the same file.

In the next sections there are some hooks examples

=== Compilation ===
Pre-run hook can be used for generating the binaries of the application that will be run by MPI-Start. The following sample shows a hook that compiles an application using `mpicc`. It assumes that the source code is called like the application binary, but with a `.c` extension. Use of complex compilation commands like configure, make, etc is also possible. This code is only executed in the first host. The results of the compilation will be available to all hosts thanks to the file distribution mechanisms.

{{{#!highlight sh
#!/bin/sh

# This function will be called before the execution of MPI application
pre_run_hook () {

  # Compile the program.
  echo "Compiling ${I2G_MPI_APPLICATION}"
  mpicc $MPI_MPICC_OPTS -o ${I2G_MPI_APPLICATION} ${I2G_MPI_APPLICATION}.c
  if [ ! $? -eq 0 ]; then
    echo "Error compiling program.  Exiting..."
    return 1
  fi
  echo "Successfully compiled ${I2G_MPI_APPLICATION}"
  return 0
}
}}}
=== Input Preprocessing ===
Some applications require some input preprocessing before the application gets executed. For example, [[http://www.gromacs.org/|gromacs]] has a `grompp` tool that prepares the input for the actual `mdrun` application. In the following example the `grompp` tool prepares the input for gromacs:

{{{#!highlight sh
#!/bin/sh

pre_run_hook()
{
   echo "pre_run_hook called"

   # Here comes the pre-mpirun actions of gromacs
   export PATH=$PATH:/$VO_COMPCHEM_SW_DIR/gromacs-3.3/bin
   grompp -v -f full -o full -c after_pr -p speptide -np 4

   return 0
}
}}}
=== Output Gathering ===
Applications that write output files in each of the hosts involved in the execution may need to fetch all those files to transfer them back to the user once the execution is finished. The following example copies all the `mydata.*` files to the first host. It uses the `mpi_start_foreach_host` function of MPI-Start that will call the first argument for each of the hosts passing the name of the host as parameter.

{{{#!highlight sh
# the first paramter is the name of a host in the
my_copy () {
    CMD="scp . \$1:\$PWD/mydata.*"
    echo \$CMD
}

post_run_hook () {
    echo "post_run_hook called"
    if [ "x\$MPI_START_SHARED_FS" = "x0" ] ; then
        echo "gather output from remote hosts"
        mpi_start_foreach_host my_copy
    fi
    return 0
}
}}}