Docs:  Installation

  Quick instructions:

  Invoke the following commands from the toplevel PETSc directory [using bash shell] 
export PETSC_DIR=$PWD
./config/configure.py --with-cc=gcc --with-fc=g77 --download-f-blas-lapack=1 --download-mpich=1
make all test

  Detailed instructions:

Example Usages:

  • Examples are at config/*.py. We use some of these scripts locally for testing - for example config/linux-gnu-amd64.py
  • Using the bash shell, assuming BLAS, LAPACK, MPICH are not currently installed ./config/configure.py will download & install BLAS, LAPACK, MPICH if they are not already installed on the system) :
            export PETSC_DIR=/home/petsc/petsc-2.3.2-p0
            cd $PETSC_DIR
            ./config/configure.py --download-f-blas-lapack=1 --download-mpich=1
            make
            make test
  • Same as above - but build Complex version of PETSc [using c++ compiler]:
            export PETSC_DIR=/home/petsc/petsc-2.3.2-p0
            cd $PETSC_DIR
            ./config/configure.py --download-f-blas-lapack=1 --download-mpich=1 --with-scalar-type=complex --with-clanguage=cxx
            make
            make test
  • Using the bash shell, assuming BLAS, LAPACK, MPICH software are installed at the specified locations:
            export PETSC_DIR=/home/petsc/petsc-2.3.2-p0
            cd $PETSC_DIR
            ./config/configure.py --with-blas-lapack-dir=/usr/local/lib --with-mpi-dir=/usr/local/mpich
            make
            make test
  • Using tcsh shell, install PETSc with default BLAS and LAPACK and no MPI:
            setenv PETSC_DIR /home/petsc/petsc-2.3.2-p0
            cd $PETSC_DIR
            ./config/configure.py --with-mpi=0
            make
            make test
  • Install PETSc with default BLAS, LAPACK and MPI using the GNU compilers, build the optimized version of the libraries:
            ./config/configure.py --with-vendor-compilers=0
            make
            make test
  • Using csh shell, install PETSc on Linux with two different sets of compilers:
            setenv PETSC_ARCH linux-gnu
            ./config/configure.py --with-vendor-compilers=0 --with-mpi-dir=/usr/local/mpich-gnu
            make
            make test
            setenv PETSC_ARCH linux-gnu-intel
            ./config/configure.py --with-vendor-compilers=intel --with-gnu-compilers=0 --with-blas-lapack-dir=/usr/local/mkl --with-mpi-dir=/usr/local/mpich-intel
            make
            make test

Several variables control the configuration and build process of PETSc. They can either be given as arguments to make or be set as environment variables.

PETSC_DIR: - this environment/make variable should point to the location of the PETSc installation that is used. You can add export PETSC_DIR=value in your .profile or .sh file or setenv PETSC_DIR value in your .cshrc or .tcshrc file. Multiple PETSc versions can coexist on the same file-system. By changing PETSC_DIR one can switch between the versions

PETSC_ARCH: this environment/make variable is used to specify the configuration that should currently be used. It corresponds to the configuration files in bmake/${PETSC_ARCH}. Multiple variants of PETSc libraries can be installed - each variant corresponding to a different PETSC_ARCH. One can switch between using these variants - by changing the value of PETSC_ARCH. If PETSC_ARCH is not set, the configuration from the last time you ran ./config/configure.py will be used.

Return to Installation Instructions     


Cygwin  provides UNIX tools on Microsoft Windows. When installing Cygwin make sure you install the following additional packages

  • make
  • python

Cygwin also has GNU compilers ( gcc, g++, g77) which can be used  if Microsoft, Intel, or Borland Group compilers are not available

Note: Previously we recommanded doing 'rebaseall' to avoid python problems. However 'rebaseall' has its own problems - so if there any python issues, the current recommendation is to reinstall cygwin from scratch.

 Cygwin  http://www.cygwin.com

Return to Installation Instructions    


BLAS/LAPACK: these packages provide some basic numeric kernels used by PETSc.

  • If ./config/configure.py cannot locate BLAS and LAPACK you can use the options --with-blas-lapack-dir=directory or --with-blas-lib=library and --with-lapack-lib=library or --with-blas-lapack-lib=library to indicate the location of the libraries.
    • on Microsoft Windows, Linux and Intel based Apple Macs, one can use Intel MKL
  • Use the ./config/configure.py options --download-f-blas-lapack=yes or --download-f-blas-lapack=ifneeded, then PETSc will download and install a BLAS and LAPACK for you.
  • Use --download-c-blas-lapack (as opposed to --download-f-blas-lapack) only if there is no fortran compiler.
  • Sadly, IBM's ESSL does not have all the private routines of BLAS that some packages, such as SuperLU expect; in particular slmach, dlmach and xerbla. Therefor you need to either use --download-f-blas-lapack=yes or install a copy of BLAS/LAPACK from Netlib instead of using ESSL directly.

Return to Installation Instructions    


MPI: This software provides the parallel functionality for PETSc.

  • If ./config/configure.py cannot locate MPI you can indicate its location with --with-mpi-dir=directory or --with-mpi-include=directory --with-mpi-lib=libraryname --with-mpirun=pathofmpirun
    • On parallel machines, Vendor provided MPI might already be installed.  IBM, SGI, Cray etc provide their own. 
    • If vendor provided MPI is not available, it can be installed from sources. We recommend MPICH
  • Use the ./config/configure.py option --download-mpich=yes then PETSc will download and install MPICH for you.
 MPI     http://www.mpi-forum.org
 MPICH  http://www.mcs.anl.gov/mpi/mpich
 LAM  http://www.lam-mpi.org
 MPICH-NT (Microsoft Windows)  http://www-unix.mcs.anl.gov/~ashton/mpich.nt
 MPI/Pro (Microsoft Windows)  http://www.mpi-softtech.com/products/mpi_pro
 HPVM (Microsoft Windows)  http://www-csag.ucsd.edu/projects/hpvm.html
 WMPI (Microsoft Windows)  http://www.criticalsoftware.com/hpc

Return to Installation Instructions    


I don't want to use MPI: You can build (sequential) PETSc without an MPI.

Run ./config/configure.py --with-mpi=0 ....

Return to Installation Instructions    


Additional Microsoft Windows Notes:

Microsoft Compilers: Use configure with the following options to build for Microsoft C/C++ & Compaq Fortran [assuming MPICH & Intel MLK are already installed in the default locations]
        ./config/configure.py --with-cc='win32fe cl' --with-fc='win32fe f90'  --with-cxx='win32fe cl'

If fortran usage is not required - and you don't have Intel MLK - just use:

       ./config/configure.py --with-cc='win32fe cl' --with-fc=0 --download-c-blas-lapack=1

If fortran usage is required [with Digital/Compaq f90 compiler] - and you don't have Intel MLK - just use the following [assuming the path to compaq fortran libraries is correct]:

       ./config/configure.py --with-cc='win32fe cl' --with-fc='win32fe f90' --download-f-blas-lapack=1 LIBS="-L'/cygdrive/c/Program Files/Microsoft Visual Studio/DF98/LIB'"

Note the usage of both single quotes and double quotes in the above line.

Configure: --download-package option does not work with many external packages [for eg: mpich]

Project Files: We provide templates for Microsoft Visual Studio project files at ${PETSC_DIR}/projects. These work for us - with our configure build [config/cygwin-ms.py with MPICH1 and Intel MKL 5], However they will REQURE MODIFICATIONS to work with a user build of PETSc - as the locations of packages & configure options used by user could be different from our defult build.

To get these project files working for your installation of PETSc, please do the following:

  • try compiling the example from cygwin shell - using makefile - for eg:
    cd src/ksp/ksp/examples/tutorials
    make ex2
  • if the above works - then make sure all the compiler/linker options used by make are also present in the project file in the correct notation.
  • if errors - redo the above step. [if all the options are correctly specified - then the example should compile from MSDev.

Debugger:  Running PETSc probrams with -start_in_debugger is not supported on this platform, so debuggers will need to be initiated manually. Make sure your environment is properly configured to use the appropriate debugger for your compiler. The debuggers can be initiated using Microsoft Visual Studio 6: msdev ex1.exe, Microsoft Visual Studio .NET: devenv ex1.exe, Intel Enhanced Debugger: edb ex1.exe, or GNU Debugger gdb ex1.exe.

PETSc Win32 front end - win32fe:  This tool is used as a wrapper to Microsoft/ Borland/ Intel compilers and associated tools - to enable building PETSc libraries using make and other UNIX tools. For additional info, run ${PETSC_DIR}/bin/win32/win32fe without any options.

Return to Installation Instructions       


Installing on machine requiring cross compiler or a job scheduler:

If one has to use a cross compiler - or go through the job scheduler to use MPI on a given machine - use the configure option '--with-batch=1' as follows:

  • run configure with the additional option '--with-batch=1' on the frontend node [perhpas with the additional option '--with-mpi-shared=0']
  • the above configure run will create a binary 'conftest'. Run this binary 'conftest' on one compute node using the job scheduler.
  • The above run of conftest will create a new python script 'reconfigure'. Run 'python reconfigure'  to complete the configure process

Installing on IBM SP:

To run any code compiled with MPI on IBM SP - one has to go through the job scheduler.  [So follow the instructions from the above section 'Installing on machine requiring job scheduler']. Alternatively check/run config/aix5.1.0.0.py for all configure options required on the SP with compilers defaulting to 32bit mode, and config/aix5.1.0.0-64.py with compilers defaulting to 64bit mode.

Installing on IBM Blue Gene:

IBM Blue Gene [as of now] requires modified compiler script wrappers to compile PETSc-2.3.2. Please download ftp://ftp.mcs.anl.gov/pub/petsc/tmp/petsc-bgl-tools.tar.gz and follow instructions provided in the README file.

Installing with TAU Instrumentation package:

TAU package and the prerequisite PDT packages need to be installed separately. PETSc provides a wrapper compiler for TAU. This needs to be invoked with the correct information for the installed TAU and PDT packages. For eg:

 ./config/configure.py --with-mpi-dir=/home/petsc/soft/linux-rh73/mpich-1.2.4 --with-fc=0 --with-cxx=0 -PETSC_ARCH=linux-tau -with-cpp=cpp --with-cc="`pwd`/bin/taucc.py -cc=gcc -pdt_parse=/homes/petsc/soft/linux-rh73/pdtoolkit-2.2b1/linux/bin/cxxparse -tau_lib_dir=/homes/petsc/soft/linux-rh73/tau-2.11.18/i386_linux/lib"

Installing TOPS components:

To install TOPS components one has to download and install CCA Tools.  [linux only]

  • Get the CCA Tools package from http://www.cca-forum.org/download/cca-tools/cca-tools-0.5.9.tar.gz
  • Configure cca-tools using '-with-mpi' option [prerequisites might be Java SDK, c++, mpich1]
  • Configure PETSc with C++, babel, ccafe options. Check config/asterix-tops.py for an example.
  • Build PETSc libraries & TOPS components with 'make all'
  • To test  do: 'cd src/tops; make examples'

Return to Installation Instructions