13.6. Open MPI terminology
Open MPI is a large project containing many different sub-systems and a relatively large code base. Let’s first cover some fundamental terminology in order to make the rest of the discussion easier.
13.6.1. Modular Component Architecture (MCA)
See this section for a discussion of the Modular Component Architecture (MCA). Seriously. Go read it now. From reading that section, you should understand the following terms before continuing reading these docs:
Project
Framework
Component
Module
Parameters (variables)
13.6.2. Notes on projects
Projects are strict abstraction barriers in the code. That is, they
are compiled into separate libraries: liboshmem
, libmpi
,
libopen-pal
with a strict dependency order: OSHMEM depends on
OMPI, OMPI depends on OPAL. For example, MPI executables are linked
with:
shell$ mpicc myapp.c -o myapp
# This actually turns into:
shell$ cc myapp.c -o myapp -lmpi ...
More system-level libraries may listed after -lmpi
, but you get
the idea. libmpi
will implicitly pull libopen-pal
into the
overall link step.
Strictly speaking, these are not “layers” in the classic software engineering sense (even though it is convenient to refer to them as such). They are listed above in dependency order, but that does not mean that, for example, the OMPI code must go through the OPAL code in order to reach the operating system or a network interface.
As such, this code organization more reflects abstractions and software engineering, not a strict hierarchy of functions that must be traversed in order to reach a lower layer. For example, OMPI can directly call the operating system as necessary (and not go through OPAL). Indeed, many top-level MPI API functions are quite performance sensitive; it would not make sense to force them to traverse an arbitrarily deep call stack just to move some bytes across a network.
Frameworks, components, and modules can be dynamic or static. That is,
they can be available as plugins or they may be compiled statically
into libraries (e.g., libmpi
).
In Open MPI head of development, configure
defaults to:
Building projects as dynamic libraries
Linking all components into their parent project libraries (vs. compiling them as independent DSOs)
Although these defaults can be modified by command line arguments to configure.
13.6.3. Required 3rd party libraries
Note that Open MPI also uses some third-party libraries for core functionality:
PMIx
PRRTE
Libevent
Hardware Locality (“hwloc”)
These are discussed in detail in the required support libraries section.