4.6. Required support libraries

Open MPI requires the following support libraries with the minimum listed versions:

Library

Minimum version

Notes

Hardware Locality

1.11.0
This library is required; Open MPI will not build without it.

Libevent

2.0.21
This library is required; Open MPI will not build without it.

PMIx

4.1.2
This library is required; Open MPI will not build without it.

PRRTE

2.0.2
This library is optional in some environments. PRRTE provides
Open MPI’s full-featured mpirun / mpiexec MPI
application launchers (the two are identical; they are symbolic
links to the same executable).
  • If your environment uses another MPI application launcher
    (e.g., Slurm users can use the srun launcher to “direct
    launch” Open MPI applications), then the use of PRRTE is
    optional.
  • If your environment has no other MPI application launcher, then
    you need to install PRRTE and build Open MPI with PRRTE
    support.
  • Open MPI can use the copy of PRRTE embedded in its source code
    tree, or compile/link against an external PRRTE installation.
    See this section for details about how to specify each method.

Since these support libraries are fundamental to Open MPI’s operation and not universally available in all environments, they are directly incorporated into Open MPI’s configure, build, and installation process. More on this below.

4.6.1. Library dependencies

These support libraries have dependencies upon each other:

../_images/required-support-libraries-dependency-graph.png

Open MPI required support library dependency graph.

The higher-level boxes depend on the lower-level boxes. Specifically:

  • Open MPI depends on PRRTE, PMIx, Hwloc, and Libevent (i.e., everything).

  • PRRTE depends on PMIx, Hwloc, and Libevent (i.e., everything except Open MPI).

  • PMIx depends on Hwloc and Libevent.

  • Hwloc does not depend on anything.

  • Libevent does not depend on anything.

At run time, it is critical that the run-time linker loads exactly one copy of each of these libraries.

Note

The required support libraries can have other dependencies, but for simplicity and relevance to building Open MPI, those other dependencies are not discussed here.

4.6.2. Potential problems

Problems can (will) arise if multiple different copies of the above shared libraries are loaded into a single process. For example, consider if:

  • Loading the Open MPI shared library causes the loading of Libevent shared library vA.B.C.

  • But then the subsequent loading of the PMIx shared library causes the loading of Libevent shared library vX.Y.Z.

Since there are now two different versions of the Libevent shared library loaded into the same process (yes, this can happen!), unpredictable behavior can (will) occur.

Many variations on this same basic erroneous scenario are possible. All of them are bad, and can be extremely difficult to diagnose.

4.6.3. Avoiding the problems

A simple way to avoid these problems is to configure your system such that it has exactly one copy of each of the required support libraries.

Important

If possible, use your OS / environment’s package manager to install as many of these support libraries — including their development headers — as possible before invoking Open MPI’s configure script.

Not all package managers provide all of the required support libraries. But even if your package manager installs — for example — only Libevent and Hwloc, that somewhat simplifies the final Open MPI configuration, and therefore avoids some potentially erroneous configurations.

4.6.4. How configure finds the required libraries

In an attempt to strike a balance between end-user convenience and flexibility, Open MPI bundles these four required support libraries in its official distribution tarball.

Generally, if Open MPI cannot find a required support library, it will automatically configure, build, install, and use its bundled version as part of the main Open MPI configure, build, and installation process.

Put differently: Open MPI’s configure script will examine the build machine and see if it can find each of the required support header files and libraries. If it cannot find them, it will attempt to fall back and use the corresponding bundled support library instead.

Important

Note, however, that configure is smart enough to understand the dependencies between the required support libraries.

Specifically: If configure finds the development headers and libraries for a given support library already installed on the system, then it will ignore both the corresponding bundled support library, and it will also ignore all bundled support libraries that are below it in the dependency graph shown above.

4.6.4.1. Build example 1

configure finds the PRRTE development headers and libraries in /usr/local. This will cause the following to occur:

  1. configure will ignore the PRRTE library that is bundled in the Open MPI source tree and will use the PRRTE that is already installed in /usr/local.

  2. configure will also ignore the bundled PMIx, Hwloc, and Libevent libraries in the Open MPI source tree.

    • If configure is unable to find header files and libraries for PMIx, Hwloc, and Libevent elsewhere on the build machine (i.e., assumedly the same PMIx, Hwloc, and Libevent than the PRRTE in /usr/local is using), this is an error: configure will abort, and therefore refuse to build Open MPI.

4.6.4.2. Build example 2

configure does not find PRRTE on the build machine, but does find PMIx development headers and libraries in /opt/local. This will cause the following to occur:

  1. configure will set up to build the PRRTE library that is bundled in the Open MPI source tree.

  2. configure will ignore the PMIx library that is bundled in the Open MPI source tree and will use the PMIx that is already installed in /opt/local.

  3. configure will also ignore the bundled Hwloc and Libevent libraries in the Open MPI source tree.

    • If configure is unable to find header files and libraries for Hwloc and Libevent elsewhere on the build machine (i.e., assumedly the same Hwloc and Libevent than the PMIx in /opt/local is using), this is an error: configure will abort, and therefore refuse to build Open MPI.

4.6.4.3. Build example 3

configure only finds the development headers and libraries for Libevent on the build machine. This will cause the following to occur:

  1. configure will set up to build the PRRTE, PMIx, and Hwloc libraries that are bundled in the Open MPI source tree.

  2. configure will ignore the Libevent library that is bundled in the Open MPI source tree and will use the Libevent that is already installed.

4.6.5. Overriding configure behavior

If configure’s default searching behavior is not sufficient for your environment, you can use command line options to override its default behavior.

For example, if PMIx and/or PRRTE are installed such that the default header file and linker search paths will not find them, you can provide command line options telling Open MPI’s configure where to search. Here’s an example configure invocation where PMIx and PRRTE have both been installed to /opt/open-mpi-stuff:

./configure --prefix=$HOME/openmpi-install \
    --with-pmix=/opt/open-mpi-stuff \
    --with-prrte=/opt/open-mpi-stuff ...

As another example, if you do not have root-level privileges to use the OS / environment package manager, and if you have a simple MPI application (e.g., that has no external library dependencies), you may wish to configure Open MPI something like this:

./configure --prefix=$HOME/openmpi-install \
    --with-libevent=internal --with-hwloc=internal \
    --with-pmix=internal --with-prrte=internal ...

The internal keywords force configure to use all four bundled versions of the required libraries.

Danger

Be very, very careful when overriding configure’s default search behavior for these libraries. Remember the critical requirement: that Open MPI infrastructure and applications load exactly one copy of each support library. For simplicity, it may be desirable to ensure to use exactly the support libraries that Open MPI was compiled and built against.

For example, using the Open MPI installed from the sample configure line (above), you may want to prefix your run-time linker search path (e.g., LD_LIBRARY_PATH on Linux) with $HOME/openmpi-install/lib. This will ensure that linker finds the four support libraries from your Open MPI installation tree, even if other copies of the same support libraries are present elsewhere on your system.

4.6.6. (Strong) Advice for packagers

If you are an Open MPI packager, we strongly suggest that your Open MPI package should not include Hwloc, Libevent, PMIx, or PRRTE. Instead, it should depend on external, independently-built versions of these packages.

See the Advice for packagers section for more details.