Parallel Programming

On the IBM SMP machine, there are three basic parallelization methods:

 

MPI

The standard "Message Passing Interface" providing a maximum of portability.

 

OpenMP

A standard set of directives to be inserted in your code to control its parallel execution on shared memory computers. It does not require a complete rewrite of your program - adding a few lines at the most time-consuming loops might suffice. However, with OpenMP you can utilize only the processors of a single node.

 

LAPI

An IBM-specific "Low-Level Application Programming Interface", offering the potential of fast communication between the processors at the expense of confining your code to IBM SP machines. Please turn to IBM's web pages for an introduction (with examples), and for reference information about the function call parameters.

 

 

Parallel Operating Environment

An attempt was made by IBM to hide away the details of how several instances of your program are started on different processors - presumably this is how the poe command came into existence. POE stands for "Parallel Operating Environment", and it is, as it were, the cornerstone of parallel computing under AIX.

If you have never used POE, you may find the Introduction to IBM's Parallel Environment interesting to read. If you have used mpirun on other systems, you should already have a pretty good idea of what poe does, and all you need are some details about options and environment variables to control its behaviour.

 

Example

To run a simple MPI program, please follow these steps:

• Compile your program, e.g.

        mpxlf90_r myprog.f -o myprog

• Create a host.list file with an entry "localhost" for each processor on which your program is to run. To use, say, 4 processors of an SMP node, you must enter "localhost" (without quotes) 4 times.
• Start your program using

        poe ./myprog -procs 4

• Unix tradition would lead you to believe that

        poe -procs 4 ./a.out

  would be just as valid as

        poe ./a.out -procs 4

  However, only the second form is correct. poe's options must appear after the name of your binary.

Environment Variables

There are many environment variables and command line flags to the poe command that you can set to influence the operation of the PE tools and the execution of parallel programs. A complete list of these environment variables can be found in the poe documentation. (Just in case it is not clear what is meant by "environment variables": As a csh or tcsh user you would type, for example,

    setenv MP_PROCS 4

and as a Korn shell (ksh) user

    export MP_PROCS=4

before invoking poe.)

Critical settings with strong effects on program execution should always be made:

 

Variable Name	Command Line Switch	Explanation	Recommended Values
MP_PROCS	-procs <num>	The number of nodes to allocate for your partition	Number between 1 and the size of the SMP
MP_HOSTFILE	-hostfile <filename>	Determines the name of a host list file for node allocation.	Default setting is host.list.
MP_SHARED_MEMORY	-shared_memory <value>	If set to yes, the MPI processes will communicate via the shared memory interface	Default setting is no

 

Helpful settings for controlling output and getting additional debug information:

 

Variable Name	Command Line Switch	Explanation	Recommended Values
MP_STDOUTMODE	-stdoutmode <value>	Enables you to manage the STDOUT from your parallel tasks. Possible values are unordered all tasks write output data to STDOUT asynchronously. ordered output data from each parallel task is written its own buffer. Later, all buffers are flushed, in task order, to STDOUT. [task id] only the task indicated writes output data to STDOUT.	[task id] must be a number between 0 and MP_PROCS-1; default setting is unordered
MP_INFOLEVEL	-infolevel <value>	Determines the level of message reporting. Valid values are: 0 error 1 warning and error 2 informational, warning, and error 3 informational, warning, and error. Also reports diagnostic messages for use by the IBM Support Center. 4,5,6 Informational, warning, and error. Also reports high- and low-level diagnostic messages for use by the IBM Support Center.	If not set, the default is 1 (warning and error).
MP_EUIDEVELOP	-euidevelop <value>	Determines whether the message passing interface performs more detailed checking during execution. This functionality is intended for developing applications, and can significantly slow performance. Valid values are yes, no, deb (for "debug"), and nor (for "normal"). The debug and noc values are used to start and stop parameter checking.	Default value is no