This file’s URL is  http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/mpiSim.html


H. Skip Knoble                                                    30 May 2008


mpiSim.zip            File Table of Contents


   mpiSim.f90      -> mpiSim Fortran 90/95 source code (over 5000 lines of code).

   useSim.f90      -> USE statements that need to be INCLUDEd in an application.

   mpiSim.html     -> The web page that you are now reading.

   *_ex.f90        -> Other authors' MPI examples (20 of them); these have been

                      set up to compile and run with mpiSim; see How To below.

   lfdebug.sh      -> Linux BASH script to invoke Lahey compile/link with debug options.

   lfdebug.bat     -> Windows BAT file to invoke Lahey compile/link with debug options.

   done.txt        -> A list of MPI(1.1) subprograms currently supported by mpiSim.


mpiSim includes over 75 unique subprograms (over 200 overloaded routines).


View/download above source code files: http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/

Download the mpiSim package: http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/mpiSim.zip


OVERVIEW: The purpose of mpiSim is to debug MPI applications at compile-time;

run-time debugging is also possible using compilers that support run-time compiler

debug options (for example, for the Linux Lahey compiler, please see the Bash script

lfdebug.sh or ifdebug.bat). At compile-time mpiSim essentially uses a F90/95 compiler to scan

an application’s MPI calls for validity.


All of the mpiSim code is Standard Fortran 90/95, low overhead, easy to use; just insert

Include statement(s) in each program/subprogram that calls MPI routines (see How To

section below). mpiSim runs under Linux or Windows using any available F90/95 compiler;

and does not require an OS batch job nor an “MPI compiler” nor a parallel library like MPICH.

For more specifics see the sections: CAPABILITIES and LIMITATIONS below.



ACKNOWLEDGMENTS/REFERENCES: Nothing is done in a vacuum. Special thanks to:

Barbara Garrison for sharing her "fake MPI" routines which inspired this work;

James Van Buskirk for overloading code examples posted at comp.lang.fortran.

Mike Metcalf for his postings at comp.lang.fortran and book "Fortran 95/2003

Explained". John Burkardt, et al, and Hans Joachim Pflug, for their excellent

work and many of the *_ex*.f90 example codes. Some of these original codes are

available at web pages: http://people.scs.fsu.edu/~burkardt/f_src/mpi/mpi.html


An MPI Course is available on John Burkardt's web page:


Descriptions of MPI subprograms and special variables may be found at:

http://people.scs.fsu.edu/~burkardt/html/mpi_glossary.html and



Finally, there are other MPI debugging tools, such as:

MPI-CHECK for MPI Fortran 90 and Fortran 77. This tool reportedly can diagnose

some cases where actual or potential deadlocks occur. See:



MPI Newsgroups: comp.parallel,  comp.sys.super



CAPABILITIES: mpiSim can be used in a non-batch job environment to quickly

find compile-time errors, including MPI subprogram calling argument errors, using

any Fortran 90/95 compiler (like free g95). In addition, true run-time debugging

is possible using run-time debugging options of compilers like Lahey LF95, NAG f95,

or Silverfrost ftn95(free) for Windows. Yes, mpiSim will work under Linux or Windows.

Please see the mpiSim examples,*_ex.f90; (four of these example source programs do have

uninitialized variables or subscripts out of range: day1ex3_ex, ring_ex, simple_ex,

and split_ex; these are commented as such.)


Time to Run and compile for mpiSim debugging are not usually a concern since

the MPI subprograms are simulated on one processor (but with correct argument

interfaces). Every mpiSim MPI simulated subprogram uses INTENT for every argument;

thus application calling argument errors can be quickly located at compile-time.


The Fortran code, for mpiSim.f90 is Standard Fortran 90/95. Many mpiSim modules use

overloading for subprogram arguments that could be Integer, Real, Double Precision or

Character  arguments; these have subprogram names ending in _I, _R, _D, and _C. arguments that

could be Scalar, but normally are matrix/vector, have names ending in _IS, _RS, _DS. and _CS.



LIMITATIONS: MPI applications must be self-contained Fortran 90/95 source code;

mixed-language applications (C++ subprograms that call MPI subprograms, for example)

are not supported by mpiSim.


When MPI Fortran applications (like *_ex.f90 examples) are run using mpiSim, correct "answers"

will NOT obtained. Rather mpiSim provides a single-processor command line debugging

environment by simulating several routines named after their real MPI Library counterparts.

Again, the purpose of mpiSim is to find compile-time and run-time code errors, NOT

verification of computed results. Finally, mpiSim does not catch all bugs; in particular

mpiSim does not detect MPI deadlock; see the examples deadlock_ex.f90 and nodeadlock_ex.f90.


For true run-time debugging (detecting for example, undefined variables, subscripts out of

range), MpiSim is dependent on having a Fortran 90/95 compiler with run-time diagnostics.

For example compilers shown above under CAPABILITIES have compiler options that cause run-time

debugging to be activated. Please see the section below HOW TO USE.


MPI applications written in Fortran 77 syntax should be converted using a conversion utility;

please see Step (1) below. However, carefully note mpiSim accepts vector and scalar data arguments only.

That is, multi-dimensional data arrays or array sections, like X(:,:) or X, or scalar X(1,k), passed to MPI

subprogram, by F90 conventions, will be passed as a matrix or scalar; thus a workaround would be needed:

if appropriate, use vector section array notation like X(1:n,k); or save and restore array sections to a vector

work array for Intent(out) arguments; or for Intent(in) use the Transfer intrinsic to convert to a vector;

e.g., Transfer(X(1,k),X(1,k),n). please see example poisson_band2_ex.f90 for examples of using array

section workaround for calls to MPI_Irecv and MPI_Send.


MpiSim is an experiment presently limited for illustrative purposes to the MPI

library simulated subprograms given in file: mpiSim.f90 and listed by name in file done.txt.

There are likely bugs in mpiSim; please report them.





0) Get the mpiSim Source Code

Download the file: http://ftp.aset.psu.edu/pub/ger/fortran/mpiSim/mpiSim.zip

Unzip this file to a separate directory. Copy your Fortran 90 MPI application

to this same directory - all of the Fortran source, main program and all subprograms

and any necessary data files.


1) If Necessary Convert your Fortran 77 code to Fortran 90/95 code.

Since mpiSim is Fortran 90, any MPI code that uses it must be Fortran 90.

One may convert Fortran 77 to Fortran 90 with Michael Metcalf's convert.f90 tool;

This tool converts fixed form f77 source code to free-form loop structured f90:



2a) Comment any existing Include statements pointing to any other MPI library.

For example, comment out (insert a leading ! for lines like:  include 'mpif.h


2b) Put INCLUDE statements in your MPI application.

To facilitate true debugging, mpiSim must be compiled with your

MPI application code (see Include statements below).

In each main program insert two INCLUDE statements as shown below, one before

the Program statement, one after; after each independent subroutine or function

statement, that calls an MPI subprogram, insert the line: include "useSim.f90"


include "mpiSim.f90"

      program main  (or other program and subprogram statement(s))

include "useSim.f90"


      end program main  (or other end program and end subprogram statement(s))

      subroutine foo  (that calls MPI subprogram(s))

include "useSim.f90"


      end subroutine foo


....subroutine bar  (that does not call MPI subprograms)

    end subroutine bar


Please see the poisson_band2_ex.f90 code for a working example of use of these mpiSim

INCLUDE statements.


3) Compile and Link mpiSim and your MPI application.

This can be done in either of two modes: Compile only or Compile, link, and run.


Example:  compile only with any compiler, like g95 for example: g95 matvec_ex.f90



Example:  compile, link,and run using a compiler with compiler options that causes

run-time debugging to occur. In this example we use the PSU interactive processor,

Hammer console (for example), compile/link mpiSim and your MPI Fortran application

(matvec_ex.f90) using, for example, Lahey (run-time) debugging options (specified as

the script lfdebug.sh):


sh lfdebug.sh matvec_ex.f90


or Intel Fortran debugging options:

ifort -check all -d3 -g -fp -O0 -traceback -WB -ftrapuv

-fltconsistency -fpe0 -no-ftz matvec_ex.f90


where matvec_ex.f90 is a fileid of an example program.


By "debugging options" we mean running from a command prompt to compile, link,

and run interactivly (not batch jobs); using compiler debug options will cause

run-time checks for undefined variables, subscripts/strings out of range,

invalid subprogram calls (argument/dummy argument mismatches), etc. Error messages

would then be displayed with traceback and line numbers where such errors occur

in the source code.


4) Optionally Run your application with mpiSim.

Run the compiled/linked program from the Hammer console: ./a.out

Or Run it from a Windows command prompt: matvec_ex.exe

Keep running using mpiSim until the debugging process shows no more coding bugs.


WINDOWS NOTE:  The above example will also run using the Windows platform if a

Compiler with run-time debug options is available under Windows. For example

lfdebug.bat matvec_ex.f90



Or the g95 compiler for Windows could also be used either to debug compile-time

problems or to partially debug run-time problems thus:








g95 matvec_ex.f90

-Wall -pedantic -fbounds-check -ftrace=full -Wuninitialized -Wunset-vars -Wline-truncation


For g95 compile-time checking only simply issue a command like:  g95 matvec_ex.f90


Note: The Lahey, NAG, or Silverfrost compilers presently diagnose far more run-time

errors than either G95 or Intel ifort; see the Polyhedron diagnostic benchmarks:

http://www.polyhedron.com/compare0html or the Penn State diagnostic benchmarks at:



Again, when using mpiSim, any F90/95 compiler can be used without special compiler options

to debug compile-time problems which includes argument/dummy argument inconsistencies.



5) When done debugging comment out mpiSim Include statements and uncomment the original MPI Include

statements. Then run your application as usual using MPI.





For Windows the case of fields are case insensitive and case preserving.

For Linux (Unix) fields like Include files, subprogram and module names are

case sensitive. For Mac OS/X fields are case sensitive case preserving. Thus naming

conventions are important when using mpiSim under Linux or OS/X.


mpiSim subprogram naming conventions have the form: MPI_Capitalized_lowercase .

That is, the first part of a subprogram name (token) after the leading "MPI_" is

capitalized; if there are any more tokens, these are lower case.


An example:  mpiSim subroutine MPI_Get_processor_name . The naming conventions for

INCLUDEd filenames are capitalized as shown in Step (2) above and in module names of

The USE statements in the file useSim.f90. Please see mpiSim source code in file