2. Getting help
If you have a problem or question, it is highly recommended that you execute the following steps in order. Many people have similar problems with configuration and initial setup of Open MPI — it is possible that your question has already been answered.
Validate your Open MPI installation. This ensures that you have a nominally-correct Open MPI installation.
Check prior GitHub issues and see if others have asked the same question and had it answered.
Check the mailing list archives “search” features (or use Google) to check old posts and see if others have asked the same question and had it answered.
If you do not find a solution to your problem in the above resources, proceed to the Where to send? section.
2.1. Where to send?
Different types of questions and problems should be sent to different places. If you have:
A general end-user question or problem: you should probably subscribe to the Open MPI user’s mailing list and post it there.
Because of spam, only subscribers to the mailing list are allowed to post to the mailing list. Specifically: you must subscribe to the mailing list before posting.
If you have a compile-time question or problem, see the For problems building or installing Open MPI section below for the content of what to include in your email.
If you have problems launching your MPI or OpenSHMEM application successfully, see the For problems launching MPI or OpenSHMEM applications section below for the content of what to include in your email.
If you have other questions or problems about running your MPI or OpenSHMEM application, see the For problems running MPI or OpenSHMEM applications section below for the content of what to include in your email.
The more information you include in your report, the better. E-mails/bug reports simply stating, “It doesn’t work!” are not helpful; we need to know as much information about your environment as possible in order to provide meaningful assistance.
The best way to get help is to provide a “recipe” for reproducing the problem. This will allow the Open MPI developers to see the error for themselves, and therefore be able to fix it.
Please use a descriptive “subject” line in your email! Some Open MPI question-answering people decide whether to read a mail based on its subject line (e.g., to see if it’s a question that they can answer). So please please please use a good subject line that succinctly describes your problem.
A bug report: you should probably post it to Open MPI’s Github issue tracker. Follow the template to submit all the requested information.
A patch, bug fix, or other code submission: please post a Github Pull Request to the Open MPI Github repository.
A developer-level / internal question about Open MPI itself: you should probably subscribe to the Open MPI developer’s mailing list and post it there.
If you’re unsure where to send your question, subscribe and send an email to the user’s mailing list (i.e., option #1, above).
2.2. For problems building or installing Open MPI
If you cannot successfully configure, build, or install Open MPI, please provide all of the following information:
The version of Open MPI that you’re using.
The stdout and stderr from running
config.logfiles from the Open MPI build tree.
Output from when you ran
make V=1 allto build Open MPI.
Output from when you ran
make installto install Open MPI.
The script below may be helpful to gather much of the above information (adjust as necessary for your specific environment):
#!/usr/bin/env bash set -euxo pipefail # Make a directory for the output files dir="`pwd`/ompi-output" mkdir $dir # Fill in the options you want to pass to configure here options="" ./configure $options 2>&1 | tee $dir/config.out tar -cf - `find . -name config.log` | tar -x -C $dir - # Build and install Open MPI make V=1 all 2>&1 | tee $dir/make.out make install 2>&1 | tee $dir/make-install.out # Bundle up all of these files into a tarball filename="ompi-output.tar.bz2" tar -jcf $filename `basename $dir` echo "Tarball $filename created"
Then attach the resulting
ompi-output.tar.bz2 file to your report.
The mailing lists have a 150 KB size limit on messages (this is a limitation of the mailing list web archives). If attaching the tarball makes your message larger than 150 KB, you may need to post the tarball elsewhere and include a link to that tarball in your mail to the list.
2.3. For problems launching MPI or OpenSHMEM applications
If you cannot successfully launch simple applications across multiple
nodes (e.g., the non-MPI
hostname command, or the MPI “hello world”
or “ring” sample applications in the
examples/ directory), please
provide all of the information from the For problems building
or installing Open MPI section, and
all of the following additional information:
The output of the
ompi_info --allcommand from the node where you are invoking mpirun(1).
If you have questions or problems about process mapping or binding, send the output from running the
lstopo --of xmlcommands from a recent version of Hwloc.
If running on more than one node, also include the output of the
ompi_info --versioncommand from each node on which you are trying to run.
The output of running
mpirun --map-by ppr:1:node --prtemca plm_base_verbose 100 --prtemca rmaps_base_verbose 100 --display alloc hostname. Add in a
--hostfileargument if needed for your environment.
The script below may be helpful to gather much of the above information (adjust as necessary for your specific environment).
It is safe to run this script after running the script from the building and installing section.
#!/usr/bin/env bash set -euxo pipefail # Make a directory for the output files dir="`pwd`/ompi-output" mkdir -p $dir # Get installation and system information ompi_info --all 2>&1 | tee $dir/ompi-info-all.out lstopo -v | tee $dir/lstopo-v.txt lstopo --of xml | tee $dir/lstopo.xml # Have a text file "my_hostfile.txt" containing the hostnames on # which you are trying to launch for host in `cat my_hostfile.txt`; do ssh $host ompi_info --version 2>&1 | tee $dir/ompi_info-version-$host.out ssh $host lstopo -v | tee $dir/lstopo-v-$host.txt ssh $host lstopo --of xml | tee $dir/lstopo-$host.xml done # Have a my_hostfile.txt file if needed for your environment, or # remove the --hostfile argument altogether if not needed. set +e mpirun \ --hostfile my_hostfile.txt \ --map-by ppr:1:node \ --prtemca plm_base_verbose 100 \ --prtemca rmaps_base_verbose 100 \ --display alloc \ hostname 2>&1 | tee $dir/mpirun-hostname.out # Bundle up all of these files into a tarball filename="ompi-output.tar.bz2" tar -jcf $filename `basename $dir` echo "Tarball $filename created"
2.4. For problems running MPI or OpenSHMEM applications
If you can successfully launch parallel MPI or OpenSHMEM applications, but the jobs fail during the run, please provide all of the information from the For problems building or installing Open MPI section, all of the information from the For problems launching MPI or OpenSHMEM applications section, and then all of the following additional information:
A detailed description of what is failing. The more details that you provide, the better. Please include at least the following information:
The exact command used to run your application.
Any relevant MCA parameters that were set (or unset) when you ran (from either the command line, environment, parameter file, etc.).
The value of the
LD_LIBRARY_PATHenvironment variables (did you set them correctly to point to all relevant executables, the Open MPI libraries, and any required support libraries, such as libraries required for high-speed networks such as InfiniBand).
The source code of a short sample program (preferably in C or Fortran) that exhibits the problem.
If you are experiencing networking problems, include detailed information about your network.
TODO Update link to IB FAQ entry.
For RoCE- or InfiniBand-based networks, include the information in this FAQ entry.
For Ethernet-based networks (including RoCE-based networks), include the output of the
ip addrcommand (or the legacy
ifconfigcommand) on all relevant nodes.
Some Linux distributions do not put
ifconfigin the default
PATHof normal users. Try looking for it in