EXPERMENTAL
FORTRAN 90 MPI DEBUGGING PACKAGE, mpiSim
This file’s URL is http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/mpiSim.html
H.
Skip Knoble
mpiSim.zip File Table of Contents
-----------------------------------------------------------------------------------------
mpiSim.f90 -> mpiSim Fortran 90/95 source code (over 5000 lines of code).
useSim.f90 -> USE statements that need to be INCLUDEd in an application.
mpiSim.html -> The web page that you are now reading.
*_ex.f90 -> Other authors' MPI examples (20 of them); these have been
set up to compile and run with mpiSim; see How To below.
lfdebug.sh -> Linux BASH script to invoke Lahey compile/link with debug options.
lfdebug.bat -> Windows BAT file to invoke Lahey compile/link with debug options.
done.txt -> A list of MPI(1.1) subprograms currently supported by mpiSim.
-----------------------------------------------------------------------------------------
mpiSim includes over 75 unique subprograms (over 200 overloaded routines).
View/download above source code files: http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/
Download the mpiSim package: http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/mpiSim.zip
OVERVIEW: The purpose of mpiSim is to debug MPI applications at compile-time;
run-time debugging is also possible using compilers that support run-time compiler
debug options (for example, for the Linux Lahey compiler, please see the Bash script
lfdebug.sh or ifdebug.bat). At compile-time mpiSim essentially uses a F90/95 compiler to scan
an application’s MPI calls for validity.
All of the mpiSim code is Standard Fortran 90/95, low overhead, easy to use; just insert
Include statement(s) in each program/subprogram that calls MPI routines (see How To
section below). mpiSim runs under Linux or Windows using any available F90/95 compiler;
and does not require an OS batch job nor an “MPI compiler” nor a parallel library like MPICH.
For more specifics see the sections: CAPABILITIES and LIMITATIONS below.
ACKNOWLEDGMENTS/REFERENCES: Nothing is done in a vacuum. Special thanks to:
Barbara Garrison for sharing her "fake MPI" routines which inspired this work;
James Van Buskirk for overloading code examples posted at comp.lang.fortran.
Mike Metcalf for his postings at comp.lang.fortran and book "Fortran 95/2003
Explained". John Burkardt, et al, and Hans Joachim Pflug, for their excellent
work and many of the *_ex*.f90 example codes. Some of these original codes are
available at web pages: http://people.scs.fsu.edu/~burkardt/f_src/mpi/mpi.html
An MPI Course is available on John Burkardt's web page:
http://people.scs.fsu.edu/~burkardt/pdf/mpi_course.pdf
Descriptions of MPI subprograms and special variables may be found at:
http://people.scs.fsu.edu/~burkardt/html/mpi_glossary.html and
http://www.mpi-forum.org/docs/mpi-11-html/node141.html#Node141
Finally, there are other MPI debugging tools, such as:
MPI-CHECK for MPI Fortran 90 and Fortran 77. This tool reportedly can diagnose
some cases where actual or potential deadlocks occur. See:
http://andrew.ait.iastate.edu/HPC/Papers/mpicheck/mpicheck1.htm
MPI Newsgroups: comp.parallel, comp.sys.super
CAPABILITIES: mpiSim can be used in a non-batch job environment to quickly
find compile-time errors,
including MPI subprogram calling argument errors, using
any Fortran 90/95 compiler (like free g95). In addition, true run-time debugging
is possible using run-time debugging options of compilers like Lahey LF95, NAG f95,
or Silverfrost ftn95(free) for Windows. Yes, mpiSim
will work under Linux or Windows.
Please see the mpiSim examples,*_ex.f90; (four of these example source programs do have
uninitialized variables or subscripts out of range: day1ex3_ex, ring_ex, simple_ex,
and split_ex; these are commented as such.)
Time to Run and compile for mpiSim debugging are not usually a concern since
the MPI subprograms are simulated on one processor (but with correct argument
interfaces). Every mpiSim MPI simulated subprogram uses INTENT for every argument;
thus application calling argument errors can be quickly located at compile-time.
The Fortran code, for mpiSim.f90 is Standard Fortran 90/95. Many mpiSim modules use
overloading for subprogram arguments that could be Integer, Real, Double Precision or
Character arguments; these have subprogram names ending in _I, _R, _D, and _C. arguments that
could be Scalar, but normally are matrix/vector, have names ending in _IS, _RS, _DS. and _CS.
LIMITATIONS: MPI applications must be self-contained Fortran 90/95 source code;
mixed-language applications (C++ subprograms that call MPI subprograms, for example)
are not supported by mpiSim.
When MPI Fortran applications (like *_ex.f90 examples) are run using mpiSim, correct "answers"
will NOT obtained. Rather mpiSim provides a single-processor command line debugging
environment by simulating several routines named after their real MPI Library counterparts.
Again, the purpose of mpiSim is to find compile-time and run-time code errors, NOT
verification of computed results. Finally, mpiSim does not catch all bugs; in particular
mpiSim does not detect MPI deadlock; see the examples deadlock_ex.f90 and nodeadlock_ex.f90.
For true run-time debugging (detecting for example, undefined variables, subscripts out of
range), MpiSim is dependent on having a Fortran 90/95 compiler with run-time diagnostics.
For example compilers shown above under CAPABILITIES have compiler options that cause run-time
debugging to be activated. Please see the section below HOW TO USE.
MPI applications written in Fortran 77 syntax should be converted using a conversion utility;
please see Step (1) below. However, carefully note mpiSim accepts vector and scalar data arguments only.
That is, multi-dimensional data arrays or array sections, like X(:,:) or X, or scalar X(1,k), passed to MPI
subprogram, by F90 conventions, will be passed as a matrix or scalar; thus a workaround would be needed:
if appropriate, use vector section array notation like X(1:n,k); or save and restore array sections to a vector
work array for Intent(out) arguments; or for Intent(in) use the Transfer intrinsic to convert to a vector;
e.g., Transfer(X(1,k),X(1,k),n). please see example poisson_band2_ex.f90 for examples of using array
section workaround for calls to MPI_Irecv and MPI_Send.
MpiSim is an experiment presently limited for illustrative purposes to the MPI
library simulated subprograms given in file: mpiSim.f90 and listed by name in file done.txt.
There are likely bugs in mpiSim; please report them.
HOW TO USE THE Lahey
FORTRAN COMPILER TO DEBUG Using mpiSim):
0) Get the mpiSim Source Code
Download the file: http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/mpiSim.zip
Unzip this file to a separate directory. Copy your Fortran 90 MPI application
to this same directory - all of the Fortran source, main program and all subprograms
and any necessary data files.
1) If Necessary Convert your Fortran 77 code to Fortran 90/95 code.
Since mpiSim is Fortran 90, any MPI code that uses it must be Fortran 90.
One may convert Fortran 77 to Fortran 90 with Michael Metcalf's convert.f90 tool;
This tool converts fixed form f77 source code to free-form loop structured f90:
ftp://ftp.numerical.rl.ac.uk/pub/MandR/convert.f90
2a) Comment any
existing Include statements pointing to any other MPI library.
For example, comment out (insert a leading ! for lines like: include 'mpif.h’
2b) Put INCLUDE
statements in your MPI application.
To facilitate true debugging, mpiSim must be compiled with your
MPI application code (see Include statements below).
In each main program insert two INCLUDE statements as shown below, one before
the Program statement, one after; after each independent subroutine or function
statement, that calls an MPI subprogram, insert the line: include "useSim.f90"
include "mpiSim.f90"
program main (or other program and subprogram statement(s))
include "useSim.f90"
...
end program main (or other end program and end subprogram statement(s))
subroutine foo (that calls MPI subprogram(s))
include "useSim.f90"
...
end subroutine foo
....subroutine bar (that does not call MPI subprograms)
end subroutine bar
Please see the poisson_band2_ex.f90 code for a working example of use of these mpiSim
INCLUDE statements.
3) Compile and
Link mpiSim and your MPI application.
This can be done in either of two modes: Compile only or Compile, link, and run.
Example: compile only with any compiler, like g95 for example: g95 matvec_ex.f90
Example: compile, link,and run using a compiler with compiler options that causes
run-time debugging to occur. In this example we use the PSU interactive processor,
Hammer console (for example), compile/link mpiSim and your MPI Fortran application
(matvec_ex.f90) using, for example, Lahey (run-time) debugging options (specified as
the script lfdebug.sh):
sh lfdebug.sh matvec_ex.f90
or Intel Fortran debugging options:
ifort -check all -d3 -g -fp -O0 -traceback -WB -ftrapuv
-fltconsistency -fpe0 -no-ftz matvec_ex.f90
where matvec_ex.f90 is a fileid of an example program.
By "debugging options" we mean running from a command prompt to compile, link,
and run interactivly (not batch jobs); using compiler debug options will cause
run-time checks for undefined variables, subscripts/strings out of range,
invalid subprogram calls (argument/dummy argument mismatches), etc. Error messages
would then be displayed with traceback and line numbers where such errors occur
in the source code.
4) Optionally Run your application with mpiSim.
Run the compiled/linked program from the Hammer console: ./a.out
Or Run it from a Windows command prompt: matvec_ex.exe
Keep running using mpiSim until the debugging process shows no more coding bugs.
WINDOWS NOTE: The above example will also run using the Windows platform if a
Compiler with run-time debug options is available under Windows. For example
lfdebug.bat matvec_ex.f90
matvec_ex.exe
Or the g95 compiler for Windows could also be used either to debug compile-time
problems or to partially debug run-time problems thus:
SET G95_FPU_ZERODIV=1
SET G95_FPU_OVERFLOW=1
SET G95_FPU_UNDERFLOW=1
SET G95_FPU_EXCEPTIONS=1
SET G95_FPU_DENORMAL=1
SET G95_FPU_INVALID=1
g95 matvec_ex.f90
-Wall -pedantic -fbounds-check -ftrace=full -Wuninitialized -Wunset-vars -Wline-truncation
For g95 compile-time checking only simply issue a command like: g95 matvec_ex.f90
Note: The Lahey, NAG, or Silverfrost compilers presently diagnose far more run-time
errors than either G95 or Intel ifort; see the Polyhedron diagnostic benchmarks:
http://www.polyhedron.com/compare0html
or the
http://ftp.aset.psu.edu/pub/ger/fortran/test/results.txt
Again, when using mpiSim, any F90/95 compiler can be used without special compiler options
to debug compile-time problems which includes argument/dummy argument inconsistencies.
5) When done debugging comment out mpiSim Include statements and uncomment the original MPI Include
statements. Then run your application as usual using MPI.
5) PORTABLE NAMING
CONVENSIONS
For Windows the case of fields are case insensitive and case preserving.
For Linux (Unix) fields like Include files, subprogram and module names are
case sensitive. For Mac OS/X fields are case sensitive case preserving. Thus naming
conventions are important when using mpiSim under Linux or OS/X.
mpiSim subprogram naming conventions have the form: MPI_Capitalized_lowercase .
That is, the first part of a subprogram name (token) after the leading "MPI_" is
capitalized; if there are any more tokens, these are lower case.
An example: mpiSim subroutine MPI_Get_processor_name . The naming conventions for
INCLUDEd filenames are capitalized as shown in Step (2) above and in module names of
The USE statements in the file useSim.f90. Please see mpiSim source code in file
--------------------------------------------------------------------------------