Skip to content
Commits on Source (6)
Show whitespace changes
Inline Side-by-side
gpaw/README.md
View file @ b57ca544
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
## Summary version ## Summary version
0.1 0.9
## Purpose of the benchmark ## Purpose of the benchmark
...@@ -37,16 +37,23 @@ There is currently no active support for non-CUDA accelerator platforms. ...@@ -37,16 +37,23 @@ There is currently no active support for non-CUDA accelerator platforms.
For the UEABS benchmark version 2.2, the following versions of GPAW were tested: For the UEABS benchmark version 2.2, the following versions of GPAW were tested:
* CPU-based: * CPU-based:
* Version 1.5.2 as this one is the last of the 1.5 branch and since the GPU version * Version 20.1.0, as this one is the version on which the most recent GPU commits
is derived from this version. are based.
* Version 20.1.0, the most recent version during the development of the UEABS * Version 20.10.0, as it was the most recent version during the development of
2.2 benchmark suite. the IEABS 2.2. benchmark suite.
* GPU-based: There is no official release or version number. The UEABS 2.2 benchmark * GPU-based: As there is no official release of the GPU version and as it is
suite was tested using commit TODO of at the moment of the release of the UEABS version 2.2 under heavy development
[the cuda branch of the GitLab for CUDA development](https://gitlab.com/mlouhivu/gpaw/tree/cuda). to also support AMD GPUs, there is no official support for the GPU version
([the cuda branch of the GitLab for CUDA development](https://gitlab.com/mlouhivu/gpaw/tree/cuda))
in UEABS version 2.2.
Versions 1.5.2 and 19.8.1 were also considered but are not compatible with the regular
input files provided here. Hence support for those versions of GPAW was dropped in
this version of the UEABS.
There are three benchmark cases, denotes S, M and L. There are three benchmark cases, denotes S, M and L.
### Case S: Carbon nanotube ### Case S: Carbon nanotube
A ground state calculation for a carbon nanotube in vacuum. By default uses a A ground state calculation for a carbon nanotube in vacuum. By default uses a
...@@ -57,21 +64,25 @@ on 100 cores of a modern x86 cluster. ...@@ -57,21 +64,25 @@ on 100 cores of a modern x86 cluster.
Input file: [benchmark/1_S_carbon-nanotube/input.py](benchmark/1_S_carbon-nanotube/input.py) Input file: [benchmark/1_S_carbon-nanotube/input.py](benchmark/1_S_carbon-nanotube/input.py)
This input file still works with version 1.5.2 and 19.8.1 of GPAW.
### Case M: Copper filament ### Case M: Copper filament
A ground state calculation for a copper filament in vacuum. By default uses a A ground state calculation for a copper filament in vacuum. By default uses a
3x4x4 FCC lattice with 71 atoms (freely adjustable through the variables `x`, 3x4x4 FCC lattice with 71 atoms (freely adjustable through the variables `x`,
`y` and `z` in the input file) and ScaLAPACK for `y` and `z` in the input file) and ScaLAPACK for
parallellisation. Expected to scale up to 100 nodes and/or 1000 MPI tasks. parallelisation. Expected to scale up to 100 nodes and/or 1000 MPI tasks.
Input file: [benchmark/2_M_copper-filament/input.py](benchmark/2_M_copper-filament/input.py) Input file: [benchmark/2_M_copper-filament/input.py](benchmark/2_M_copper-filament/input.py)
The benchmark was tested using 1000 and 1024 cores. For some core configurations, one may This input file does not work with GPAW 1.5.2 and 19.8.1. It requires GPAW
get error messages similar to ``gpaw.grid_descriptor.BadGridError: Grid ... to small 20.1.0 or 20.10.0. Please try older versions of the UEABS if you want to use
for ... cores``. If one really wants to run the benchmark for those number of cores, these versions of GPAW.
one needs to adapt the values of `x`, `y` and `z` in `input.py`. However, this
changes the benchmark so results cannot be compared easily with benchmark runs for The benchmark runs best when using full nodes. Expect a
different values of these variables. performance drop on other configurations.
### Case L: Silicon cluster ### Case L: Silicon cluster
...@@ -82,10 +93,16 @@ and/or 10000 MPI tasks. ...@@ -82,10 +93,16 @@ and/or 10000 MPI tasks.
Input file: [benchmark/3_L_silicon-cluster/input.py](benchmark/3_L_silicon-cluster/input.py) Input file: [benchmark/3_L_silicon-cluster/input.py](benchmark/3_L_silicon-cluster/input.py)
This input file does not work with GPAW 1.5.2 and 19.8.1. It requires GPAW
20.1.0 or 20.10.0. Please try older versions of the UEABS if you want to use
these versions of GPAW.
## Mechanics of building the benchmark ## Mechanics of building the benchmark
Note that GPAW version numbering changed in 2019. Version 1.5.3 is the Installing and running GPAW has changed a lot in the since the previous
versions of the UEABS. GPAW version numbering changed in 2019. Version 1.5.3 is the
last version with the old numbering. In 2019 the development team switched last version with the old numbering. In 2019 the development team switched
to a version numbering scheme based on year, month and patchlevel, e.g., to a version numbering scheme based on year, month and patchlevel, e.g.,
19.8.1 for the second version released in August 2019. 19.8.1 for the second version released in August 2019.
...@@ -94,16 +111,11 @@ Another change is in the Python packages used to install GPAW. Versions up to ...@@ -94,16 +111,11 @@ Another change is in the Python packages used to install GPAW. Versions up to
and including 19.8.1 use the `distutils` package while versions 20.1.0 and later and including 19.8.1 use the `distutils` package while versions 20.1.0 and later
are based on `setuptools`. This does affect the installation process. are based on `setuptools`. This does affect the installation process.
GPAW for a while supports two different ways to run in parallel distributed memory mode: Running GPAW is no longer done via a wrapper executable `gpaw-python` that
* Using a wrapper executable `gpaw-python` that replaces the Python interpreter (it internally replaces the Python interpreter (it internally links to the libpython library)
links to the libpython library) and that provides the MPI functionality. and that provides the MPI functionality. Since version 20.1.0, the standard Python
* Using the standard Python interpreter, including the MPI functionality in the interpreter is used and the MPI functionality is included in the `_gpaw.so` shared library.
`_gpaw.so` shared library.
In the `distutils`-based versions, the wrapper script approach is the default behaviour,
while in the `setuptools`-based versions, the approach using the standard Python interpreter
is the preferred one in the manual. Even though the code in the `setuptools`-based
versions still includes the option to use the wrapper script approach, it does not
work in the tested version 20.1.0.
### Available instructions ### Available instructions
...@@ -128,26 +140,23 @@ Hence GPAW has the following requirements: ...@@ -128,26 +140,23 @@ Hence GPAW has the following requirements:
* BLAS, LAPACK, BLACS and ScaLAPACK. ScaLAPACK is optional for GPAW, but mandatory * BLAS, LAPACK, BLACS and ScaLAPACK. ScaLAPACK is optional for GPAW, but mandatory
for the UEABS benchmarks. It is used by the medium and large cases and optional for the UEABS benchmarks. It is used by the medium and large cases and optional
for the small case. for the small case.
* Python. GPAW 1.5.2 requires * Python. GPAW 20.1.0 requires Python 3.5-3.8 and GPAW 20.10.0 Python 3.6-3.9.
Python 2.7 or 3.4-3.7, GPAW 19.8.1 requires 3.4-3.7, GPAW 20.1.0 Python 3.5-3.8
and GPAW 20.10.0 Python 3.6-3.9.
* Mandatory Python packages: * Mandatory Python packages:
* [NumPY](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) * [NumPY](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 20.1.0/20.10.0)
* [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) GPAW versions before 20.10.0 produce warnings when used with NumPy 1.19.x.
* [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 20.1.0/20.10.0)
* [FFTW](http://www.fftw.org) is highly recommended. As long as the optional libvdwxc * [FFTW](http://www.fftw.org) is highly recommended. As long as the optional libvdwxc
component is not used, the MKL FFTW wrappers can also be used. Recent versions of component is not used, the MKL FFTW wrappers can also be used. Recent versions of
GPAW also show good performance using just the NumPy-provided FFT routines provided GPAW also show good performance using just the NumPy-provided FFT routines provided
that NumPy has been built with a highly optimized FFT library. that NumPy has been built with a highly optimized FFT library.
* [LibXC](https://www.tddft.org/programs/libxc/) 2.X or newer for GPAW 1.5.2, * [LibXC](https://www.tddft.org/programs/libxc/) 3.X or 4.X for GPAW 20.1.0 and 20.10.0.
3.X or 4.X for GPAW 19.8.1, 20.1.0 and 20.10.0. LibXC is a library LibXC is a library of exchange-correlation functions for density-functional theory.
of exchange-correlation functions for density-functional theory. None of the None of the versions currently mentions LibXC 5.X as officially supported.
versions currently mentions LibXC 5.X as officially supported.
* [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package * [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package
from the same group that develops GPAW from the same group that develops GPAW
* Check the release notes of GPAW as the releases of ASE and GPAW should match. * Check the release notes of GPAW as the releases of ASE and GPAW should match.
E.g., during the development of the UEABS version 2.2 benchamark suite, The benchmarks were tested using ASE 3.19.3 with GPAW 20.1.0 and ASE 3.20.1
version 20.1.0 was the most up-to-date release of GPAW with 3.19.1 the matching ASE version with GPAW 20.1.0.
(though 3.18.0 should also work).
* ASE has some optional dependencies that are not needed for the benchmarking: Matplotlib (2.0.0 or newer), * ASE has some optional dependencies that are not needed for the benchmarking: Matplotlib (2.0.0 or newer),
tkinter (Tk interface, part of the Standard Python Library) and Flask. tkinter (Tk interface, part of the Standard Python Library) and Flask.
* Optional components of GPAW that are not used by the UEABS benchmarks: * Optional components of GPAW that are not used by the UEABS benchmarks:
...@@ -194,17 +203,15 @@ git clone -b cuda https://gitlab.com/mlouhivu/gpaw.git ...@@ -194,17 +203,15 @@ git clone -b cuda https://gitlab.com/mlouhivu/gpaw.git
### Install ### Install
Crucial for the configuration of GPAW is a proper `customize.py` (GPAW 19.8.1 and Crucial for the configuration of GPAW is a proper `siteconfig.py` file (GPAW 20.1.0 and later,
earlier) or `siteconfig.py` (GPAW 20.1.0 and later) file. The defaults used by GPAW earlier versions used `customize.py`). The defaults used by GPAW
may not offer optimal performance and the automatic detection of the libraries also may not offer optimal performance and the automatic detection of the libraries also
fails on some systems. fails on some systems.
The UEABS repository contains additional instructions: The UEABS repository contains additional instructions:
* [general instructions](build/build-cpu.md) * [general instructions](build/build-cpu.md)
* [GPGPUs](build/build-cuda.md) - To check
Example [build scripts](build/examples/) are also available for some PRACE and non-PRACE Example [build scripts](build/examples/) are also available.
systems.
## Mechanics of Running the Benchmark ## Mechanics of Running the Benchmark
...@@ -221,26 +228,9 @@ right from this repository. ...@@ -221,26 +228,9 @@ right from this repository.
### Running the benchmarks ### Running the benchmarks
#### Using the `gpaw-python` wrapper script These instructions are exclusively for GPAW 20.1.0 and later.
This is the default approach for versions up to and including 19.8.1 of GPAW
These versions of GPAW come with their own wrapper executable, `gpaw-python`, There are two different ways to start GPAW.
to start a MPI-based GPAW run.
No special command line options or environment variables are needed to run the
benchmarks if your MPI process starter (`mpirun`, Slurm `srun`, ...) communicates
properly with the resource manager. E.g., on Slurm systems, use
```
srun gpaw-python input.py
```
#### Using the regular Python interpreter and parallel GPAW shared library
This is the default method for GPAW 20.1.0 (and likely later).
The wrapper executable `gpaw-python` is no longer available in the default parallel
build of GPAW. There are now two different ways to start GPAW.
One way is through `mpirun`, `srun` or an equivalent process starter and the One way is through `mpirun`, `srun` or an equivalent process starter and the
`gpaw python` command: `gpaw python` command:
...@@ -263,13 +253,6 @@ That option however doesn't do the imports in the same way that the `gpaw` scrip ...@@ -263,13 +253,6 @@ That option however doesn't do the imports in the same way that the `gpaw` scrip
would do. would do.
### Examples
Example [job scripts](scripts/) (`scripts/job-*.sh`) are provided for
different PRACE systems that may offer a helpful starting point.
*TODO: Update the examples as testing on other systems goes on.*
## Verification of Results ## Verification of Results
### Case S: Carbon nanotube ### Case S: Carbon nanotube
......
gpaw/build/build-CPU.md
View file @ b57ca544
...@@ -7,11 +7,11 @@ These instructions are in addition to the brief instructions in [README.md](../R ...@@ -7,11 +7,11 @@ These instructions are in addition to the brief instructions in [README.md](../R
### Libraries and Python interpreter ### Libraries and Python interpreter
GPAW needs (for the UEABS benchmarks) GPAW needs (for the UEABS benchmarks)
* [Python](https://www.python.org/): GPAW 1.5.2 supports Python 2.7 and 3.4-3.7. * [Python](https://www.python.org/): GPAW 20.1.0 requires Python 3.5-3.8, and
GPAW 19.8.1 needs Python 3.4-3.7 and GPAW 20.1.0 requires Python 3.5-3.8. GPAW 20.10.0 and 21.1.0 require Python 3.6-3.9.
* [MPI library](https://www.mpi-forum.org/) * [MPI library](https://www.mpi-forum.org/)
* [LibXC](https://www.tddft.org/programs/libxc/). GPAW 1.5.2 requires LibXC 1.5.2 * [LibXC](https://www.tddft.org/programs/libxc/). GPAW 20.1.0,
or later. GPAW 19.8.1 and 20.1.0 need LibXC 3.x or 4.x. 20.10.0 and 21.1.0 all need LibXC 3.x or 4.x.
* (Optimized) [BLAS](http://www.netlib.org/blas/) and * (Optimized) [BLAS](http://www.netlib.org/blas/) and
[LAPACK](http://www.netlib.org/lapack/) libraries. [LAPACK](http://www.netlib.org/lapack/) libraries.
There are both commercial and free and open source versions of these libraries. There are both commercial and free and open source versions of these libraries.
...@@ -48,15 +48,15 @@ Optional components of GPAW that are not used by the UEABS benchmarks: ...@@ -48,15 +48,15 @@ Optional components of GPAW that are not used by the UEABS benchmarks:
GPAW needs GPAW needs
* [wheel](https://pypi.org/project/wheel/) is needed in most (if not all) ways of * [wheel](https://pypi.org/project/wheel/) is needed in most (if not all) ways of
installing the packages from source. installing the packages from source.
* [NumPy](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) * [NumPy](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 20.1.0/20.10.0/21.1.0)
* Installing NumPy from source will also require * Installing NumPy from source will also require
[Cython](https://pypi.org/project/Cython/) [Cython](https://pypi.org/project/Cython/)
* GPAW 1.5.2 is not fully compatible with NumPy 1.19.x. Warnings about the use * GPAW 20.1.0 is not fully compatible with NumPy 1.19.x or later. Warnings about the use
of deprecated constructs will be shown. of deprecated constructs will be shown.
* [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) * [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 20.1.0/20.10.0/21.1.0)
* [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package * [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package
from the same group that develops GPAW. Required versions are 3.17.0 or later for from the same group that develops GPAW. The required versions is 3.18.0 or later for
GPAW 1.5.2 and 3.18.0 or later for GPAW 19.8.1 or 20.1.0. GPAW 20.1.0, 20.10.0 and 21.1.0.
ASE has a couple of dependendencies ASE has a couple of dependendencies
that are not needed for running the UEABS benchmarks. However, several Python that are not needed for running the UEABS benchmarks. However, several Python
package install methods will trigger the installation of those packages, and package install methods will trigger the installation of those packages, and
...@@ -119,11 +119,11 @@ GPAW needs ...@@ -119,11 +119,11 @@ GPAW needs
The table below give the combinations of major packages Python, NumPy, SciPy, ASE and The table below give the combinations of major packages Python, NumPy, SciPy, ASE and
GPAW that were tested: GPAW that were tested:
| Python | NumPy | SciPy | ASE | GPAW | | GPAW | ASE | Python | NumPy | SciPy |
|:-------|:-------|:------|:-------|:--------| |:--------|:--------|:-------|:-------|:------|
| 3.7.9 | 1.18.5 | 1.4.1 | 3.17.0 | 1.5.2 | | 20.1.0 | 3.19.3 | 3.8.7 | 1.18.5 | 1.5.4 |
| 3.7.9 | 1.18.5 | 1.4.1 | 3.18.2 | 19.8.1 | | 20.10.0 | 3.20.1 | 3.9.1 | 1.19.5 | 1.5.4 |
| 3.8.6 | 1.18.5 | 1.4.1 | 3.19.3 | 20.1.0 | | 21.1.0 | 3.21.1 | 3.9.1 | 1.19.5 | 1.5.4 |
## Installing all prerequisites ## Installing all prerequisites
...@@ -213,43 +213,10 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -213,43 +213,10 @@ of NumPy, SciPy and GPAW itself proves much more important.
## Configuring and installing GPAW ## Configuring and installing GPAW
### GPAW 1.5.2 * GPAW 20.1.0 and later use `setuptools`. Customization of the installation process is possible
* GPAW 1.5.2 uses `distutils`. Customization of the installation process is possible
through the `customize.py` file.
* The FFT library: According to the documentation, the following strategy is used
* The compile process searches (in this order) for ``libmkl_rt.so``,
``libmkl_intel_lp64.so`` and ``libfftw3.so`. First one found will be
loaded.
* If none is found, the built-in FFT from NumPy will be used. This does not need
to be a problem if NumPy provides a properly optimized FFT library.
* The choice can also be overwritten using the GPAW_FFTWSO environment variable.
* With certain compilers, the GPAW test suite produced crashes in `xc/xc.py`. The
patch for GPAW 1.5.2 included in the [pathces](patches) subdirectory solved these
problems on the systems tested.
### GPAW 19.8.1
* GPAW 19.8.1 uses `distutils`. Customization of the installation process is possible
through a `customize.py` file.
* The selection process of the FFT library has changed from version 1.5.2. It is
now possible to specify the FFT library in `customize.py` or to simply select to
use the NumPy FFT routines.
### GPAW 20.1.0 and 20.10.0
* GPAW 20.1.0 uses `setuptools`. Customization of the installation process is possible
through the `siteconfig.py` file. through the `siteconfig.py` file.
* The selection process of the FFT library is the same as in version 19.8.1, except * It is possible to specify the FFT library in `siteconfrig.py or to simply select to
that the settings are now in `siteconfrig.py` rather than `customize.py`. use the NumPy FFT routines.
### All versions
* GPAW also needs a number of so-called "Atomic PAW Setup" files. The latest files * GPAW also needs a number of so-called "Atomic PAW Setup" files. The latest files
can be found on the [GPAW website, Atomic PAW Setups page](https://wiki.fysik.dtu.dk/gpaw/setups/setups.html). can be found on the [GPAW website, Atomic PAW Setups page](https://wiki.fysik.dtu.dk/gpaw/setups/setups.html).
For the testing we used []`gpaw-setups-0.9.20000.tar.gz`](https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz) For the testing we used []`gpaw-setups-0.9.20000.tar.gz`](https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz)
...@@ -257,8 +224,7 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -257,8 +224,7 @@ of NumPy, SciPy and GPAW itself proves much more important.
the file and set the environment variable GPAW_SETUP_PATH to point to that directory. the file and set the environment variable GPAW_SETUP_PATH to point to that directory.
In the examples provided we use the `share/gpaw-setups` subdirectory of the install In the examples provided we use the `share/gpaw-setups` subdirectory of the install
directory for this purpose. directory for this purpose.
* Up to and including version 20.1.0, GPAW does comes with a test suite which can be * GPAW 20.1.0 comes with a test suite which can be used after installation.
used after installation.
* Running the sequential tests: * Running the sequential tests:
gpaw test gpaw test
...@@ -288,7 +254,7 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -288,7 +254,7 @@ of NumPy, SciPy and GPAW itself proves much more important.
* Depending on the Python installation, some tests may fail with error messages that point * Depending on the Python installation, some tests may fail with error messages that point
to a package in the Standard Python Library that is not present. Some of these errors have no to a package in the Standard Python Library that is not present. Some of these errors have no
influence on the benchmarks as that part of the code is not triggered by the benchmark. influence on the benchmarks as that part of the code is not triggered by the benchmark.
* The full test suite is missing in GPAW 20.10.0. There is a brief sequential test * The full test suite is missing in GPAW 20.10.0 and later. There is a brief sequential test
that can be run with that can be run with
gpaw test gpaw test
...@@ -298,7 +264,7 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -298,7 +264,7 @@ of NumPy, SciPy and GPAW itself proves much more important.
gpaw -P 4 test gpaw -P 4 test
* Multiple versions of GPAW likely contain a bug in `c/bmgs/fd.c` (around line 44 * Multiple versions of GPAW likely contain a bug in `c/bmgs/fd.c` (around line 44
in GPAW 1.5.2). The code enforces vectorization on OpenMP 4 compilers by using in GPAW 20.1.0). The code enforces vectorization on OpenMP 4 compilers by using
`#pragma omp simd`. However, it turns out that the data is not always correctly `#pragma omp simd`. However, it turns out that the data is not always correctly
aligned, so if the reaction of the compiler to `#pragma omp simd` is to fully vectorize aligned, so if the reaction of the compiler to `#pragma omp simd` is to fully vectorize
and use load/store instructions for aligned data, crashes may occur. It did happen and use load/store instructions for aligned data, crashes may occur. It did happen
...@@ -312,6 +278,6 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -312,6 +278,6 @@ of NumPy, SciPy and GPAW itself proves much more important.
wrappers in the 2020 compilers. Downgrading to the MKL libraries of the 2018 wrappers in the 2020 compilers. Downgrading to the MKL libraries of the 2018
compilers or using the FFTW libraries solves the problem. compilers or using the FFTW libraries solves the problem.
This has been observed not only in GPAW, but also in some other DFT packages. This has been observed not only in GPAW, but also in some other DFT packages.
* The GPAW test code in versions 1.5.2 till 20.1.0 detects that matplotlib is not installed * The GPAW test code in version 20.1.0 detects if matplotlib is not installed
and will skip this test. We did however observe a failed test when Python could not find and will skip this test. We did however observe a failed test when Python could not find
the SQLite package as the Python standard library sqlite3 package is used. the SQLite package as the Python standard library sqlite3 package is used.
gpaw/build/examples/CalcUA-leibniz-broadwell/README.md 0 → 100644
View file @ b57ca544
# Example build scripts for CalcUA-leibniz-broadwell
Cluster characteristics:
* CPU architecture: dual-socket 14-core Intel Xeon E5-2680v4 "broadwell" CPUs
* Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
* Package manager: Most software on the cluster is installed through [
EasyBuild](https://easybuilders.github.io/).
Unless otherwise mentioned below, the following combinations of versions of GPAW, ASE, Python, NumPy
and SciPy were used:
| GPAW | ASE | Python | NumPy | SciPy |
|:--------|:--------|:-------|:-------|:------|
| 20.1.0 | 3.19.3 | 3.8.7 | 1.18.5 | 1.5.4 |
| 20.10.0 | 3.20.1 | 3.9.1 | 1.19.5 | 1.5.4 |
| 21.1.0 | 3.21.1 | 3.9.1 | 1.19.5 | 1.5.4 |
The following example build scripts do a complete build, including a build of Python
and a minimal set of dependencies that is needed to build a large enough part of the
Python standard library to build GPAW and run the benchmarks: zlib,
ncurses, readline, libffi, SQLite (used by one of the GPAW tests but likely
not needed for the benchmarks), FFTW, libxc, Python and the Python packages
NumPy, SciPy, ase and GPAW. The included libraries are not sufficient for a
full Python installation with all standard library packages or a full ASE/GPAW
installation with all optional packages but should be enough to run the benchmarks.
All software is build with the Intel 2020 compilers.
* `build_20.1.0_Python38_FFTW_icc`
* `build_20.10.0_Python39_FFTW_icc`
* `build_21.1.0_Python39_FFTW_icc`
The following build example build scripts install GPAW on top of the Intel Python
distribution. They rely as much as possible on modules already in the Intel Python
distribution or software that should be installed in the OS to run Intel Python.
The configuration accesses the FFT library available through
NumPy rather then linking directly to a FFT library.
The example script does use a module that provides up-to-date build
tools (buildtools/2020a) but on recent OSes it can be omitted.
Rather than loading the Intel and IntelPython3 modules one could as well
run the scripts provided by Intel to initialize the environment variables
for the compilers and Python distribution, and provided the necessary build
tools are also available in the system OS, this build could be done without
relying on any EasyBuild or Spack-generated modules.
* `build_20.1.0_IntelPython3_icc.sh`
* `build_20.10.0_IntelPython3_icc.sh`
* `build_21.1.0_IntelPython3_icc.sh`
gpaw/build/examples/CalcUA-vaughan-rome/build_1.5.2_IntelPython3_icc.sh → gpaw/build/examples/CalcUA-leibniz-broadwell/build_20.1.0_IntelPython3_icc.sh
View file @ b57ca544
#!/bin/bash #!/bin/bash
# #
# Installation script for GPAW 1.5.2: # Installation script for GPAW 20.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized # * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included. # NumPy and SciPy included.
# * Using the matching version of ase, 3.17.0 # * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers # * Compiling with the Intel compilers
# #
# The FFT library is discovered at runtime. With the settings used in this script # The FFT library is discovered at runtime. With the settings used in this script
...@@ -12,40 +12,80 @@ ...@@ -12,40 +12,80 @@
# (link below). # (link below).
# #
# The original installation instructions for GPAW can be found at # The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/1.5.2/doc/install.rst # https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
# #
packageID='1.5.2-IntelPython3-icc' packageID='20.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
download_dir=$install_root/Downloads module purge
install_dir=$install_root/$systemID/Packages/GPAW-manual/$packageID module use $install_root/$systemID/Modules
modules_dir=$install_root/$systemID/Modules/GPAW-manual module load UEABS/$UEABS_version
build_dir="/dev/shm/$USER/GPAW-manual/$packageID"
patch_dir=$VSC_DATA/Projects/PRACE/GPAW-experiments/UEABS/build/patches # Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.02'
libxc_version='4.3.4' libxc_version='4.3.4'
ase_version='3.17.0' ase_version='3.19.3'
GPAW_version='1.5.2' GPAW_version='20.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7' py_maj_min='3.7'
################################################################################ ################################################################################
# #
# Prepare the system # Prepare the system
# #
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
# #
# Load modules # Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
# #
module purge module purge
module load calcua/2020a MODULEPATH=/apps/antwerpen/modules/centos7/calcua-admin-broadwell/
module load intel/2020a module load calcua/admin
module load IntelPython3/2020a module load $compiler_module
module load buildtools/2020a module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
# #
# Create the directories and make sure they are clean if that matters # Create the directories and make sure they are clean if that matters
...@@ -63,7 +103,6 @@ module load buildtools/2020a ...@@ -63,7 +103,6 @@ module load buildtools/2020a
/usr/bin/mkdir -p $build_dir /usr/bin/mkdir -p $build_dir
################################################################################ ################################################################################
# #
# Download components # Download components
...@@ -78,17 +117,14 @@ libxc_file="libxc-$libxc_version.tar.bz2" ...@@ -78,17 +117,14 @@ libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version" libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file" [[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# We do not download ase in this script. As it is pure python and doesn't need # Downloading ase so that we can gather all sources for reproducibility
# customization, we will install it using pip right away. ase_file="ase-$ase_version.tar.gz"
## https://files.pythonhosted.org/packages/d9/08/35969da23b641d3dfca46ba7559f651fcfdca81dbbc00b9058c934e75769/ase-3.17.0.tar.gz ase_url="https://pypi.python.org/packages/source/a/ase"
#ase_file="ase-$ase_version.tar.gz" [[ -f $ase_file ]] || wget "$ase_url/$ase_file"
#ase_url="https://files.pythonhosted.org/packages/d9/08/35969da23b641d3dfca46ba7559f651fcfdca81dbbc00b9058c934e75769"
#[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources. # GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz # https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz" GPAW_file="gpaw-$GPAW_version.tar.gz"
#GPAW_url="https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw" GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file" [[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
...@@ -117,16 +153,15 @@ cd libxc-$libxc_version ...@@ -117,16 +153,15 @@ cd libxc-$libxc_version
autoreconf -i autoreconf -i
export CC=icc export CC=icc
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -fPIC" export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC"
#export CFLAGS="-O0 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC"
./configure --prefix="$install_dir" \ ./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran --disable-static --enable-shared --disable-fortran
# Build # Build
make -j 16 make -j $parallel
# Install # Install
make -j 16 install make install
# Add bin, lib and include to the PATH variables # Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH PATH=$install_dir/bin:$PATH
...@@ -134,6 +169,37 @@ LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH ...@@ -134,6 +169,37 @@ LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################ ################################################################################
# #
...@@ -142,10 +208,12 @@ CPATH=$install_dir/include:$CPATH ...@@ -142,10 +208,12 @@ CPATH=$install_dir/include:$CPATH
echo -e "\nInstalling ase...\n" echo -e "\nInstalling ase...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages" # IntelPython3 does not define pip3 or pip3.7.
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages" pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
pip install --prefix=$install_dir --no-deps ase==$ase_version # Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################ ################################################################################
...@@ -172,47 +240,38 @@ cd $build_dir ...@@ -172,47 +240,38 @@ cd $build_dir
# Uncompress # Uncompress
tar -xf $download_dir/$GPAW_file tar -xf $download_dir/$GPAW_file
# Apply patches
patch -p0 <$patch_dir/gpaw-1.5.2.patch
cd gpaw-$GPAW_version cd gpaw-$GPAW_version
# Make the customize.py script # Make the customize.py script
mv customize.py customize.py.orig cat >siteconfig.py <<EOF
cat >customize.py <<EOF print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW build INFO: Starting execution of the customization script' ) print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW build INFO: Variables at the start of the customization script' ) print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW build INFO: libraries = ', libraries ) print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW build INFO: mpi_libaries = ', mpi_libraries ) print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW build INFO: library_dirs = ', library_dirs ) print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW build INFO: mpi_libary_dirs = ', mpi_library_dirs ) print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW build INFO: runtime_library_dirs = ', runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW build INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs ) print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW build INFO: include_dirs = ', include_dirs ) print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW build INFO: mpi_include_dirs = ', mpi_include_dirs ) print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW build INFO: compiler = ', compiler ) print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW build INFO: mpicompiler = ', mpicompiler ) print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW build INFO: mpilinker = ', mpilinker ) print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW build INFO: extra_compile_args = ', extra_compile_args ) print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW build INFO: extra_link_args = ', extra_link_args ) print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW build INFO: define_macros = ', define_macros ) print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW build INFO: mpi_define_macros = ', mpi_define_macros ) print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW build INFO: undef_macros = ', undef_macros ) print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW build INFO: scalapack = ', scalapack ) print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW build INFO: libvdwxc = ', libvdwxc ) print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW build INFO: elpa = ', elpa ) print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
# Reset the lists of libraries as often the wrong BLAS library is picked up. print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
libraries = []
mpi_libraries = [] # LibXC
include_dirs.append('$install_dir/include')
# LibXC. Re-add the library (removed by resetting libraries). #libraries.append('xc')
# There is no need to add the library directory as we do set library_dirs to the
# content of LIBRARY_PATH further down.
# There should be no need to add the include directory as it is in CPATH which is
# set when we install gpaw.
#include_dirs.append('$install_dir/include')
libraries.append('xc')
# libvdwxc # libvdwxc
libvdwxc = False libvdwxc = False
...@@ -220,17 +279,17 @@ libvdwxc = False ...@@ -220,17 +279,17 @@ libvdwxc = False
# ELPA # ELPA
elpa = False elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK # ScaLAPACK
scalapack = True scalapack = True
mpi_libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64'] libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
mpi_define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
mpi_define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1')]
# Add EasyBuild LAPACK/BLAS libs # MKL BLAS
# This should also enable MKL FFTW according to the documentation of GPAW 1.5.2 libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
libraries += ['mkl_intel_lp64', 'mkl_sequential', 'mkl_core']
# Add other EasyBuild library directories. # Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':') library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers # Set the compilers
...@@ -238,44 +297,49 @@ compiler = os.environ['CC'] ...@@ -238,44 +297,49 @@ compiler = os.environ['CC']
mpicompiler = os.environ['MPICC'] mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC'] mpilinker = os.environ['MPICC']
# We need extra_compile_args to have the right compiler options when re-compiling print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
# files for gpaw-python. It does imply double compiler options for the other print( 'GPAW EasyBuild INFO: libraries = ', libraries )
# compiles though. print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
extra_compile_args = os.environ['CFLAGS'].split(' ') print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW build INFO: Variables at the end of the customization script' ) print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW build INFO: libraries = ', libraries ) print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW build INFO: mpi_libaries = ', mpi_libraries ) print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW build INFO: library_dirs = ', library_dirs ) print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW build INFO: mpi_libary_dirs = ', mpi_library_dirs ) print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW build INFO: runtime_library_dirs = ', runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW build INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW build INFO: include_dirs = ', include_dirs ) print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW build INFO: mpi_include_dirs = ', mpi_include_dirs ) print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW build INFO: compiler = ', compiler ) print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW build INFO: mpicompiler = ', mpicompiler ) print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW build INFO: mpilinker = ', mpilinker ) print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW build INFO: extra_compile_args = ', extra_compile_args ) print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW build INFO: extra_link_args = ', extra_link_args ) print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW build INFO: define_macros = ', define_macros ) print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW build INFO: mpi_define_macros = ', mpi_define_macros ) print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW build INFO: undef_macros = ', undef_macros ) print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW build INFO: scalapack = ', scalapack ) print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW build INFO: libvdwxc = ', libvdwxc ) print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
print( 'GPAW build INFO: elpa = ', elpa )
print( 'GPAW build INFO: Ending execution of the customization script' )
EOF EOF
# Now install gpaw # Now install gpaw
export CC=icc export CC=icc
export MPICC=mpiicc export MPICC=mpiicc
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -qno-openmp-simd" export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -qno-openmp-simd" python$py_maj_min setup.py build -j $parallel
#export CFLAGS="-O0 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -qno-openmp-simd"
python setup.py build -j 32
# Install GPAW # Install GPAW
python setup.py install --prefix="$install_dir" pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################ ################################################################################
...@@ -283,14 +347,17 @@ python setup.py install --prefix="$install_dir" ...@@ -283,14 +347,17 @@ python setup.py install --prefix="$install_dir"
# Finish the install # Finish the install
# #
echo -e "\nCleaning up and making the LUA-module GPAW-manual/$packageID...\n" echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory # Go to a different directory before cleaning up the build directory
cd $modules_dir cd $modules_dir
#/bin/rm -rf $build_dir /bin/rm -rf $build_dir
# Create a module file # Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF cat >$packageID.lua <<EOF
help([==[ help([==[
...@@ -300,14 +367,23 @@ Description ...@@ -300,14 +367,23 @@ Description
GPAW $GPAW_version for the UEABS benchmark. GPAW $GPAW_version for the UEABS benchmark.
Configuration: Configuration:
* IntelPython3/2020a module, also for NumPy and SciPy
* Parallel GPAW $GPAW_version with ase $ase_version * Parallel GPAW $GPAW_version with ase $ase_version
* FFT library selected at runtime. The default with the path * On top of the system IntelPython3 $IntelPython3_version module, which provides
as set through this module should be MKL but it can be changed Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
by setting GPAW_FFTWSO as indicated in the install instructions at * NumPy is used for FFT, so the performance of the code
https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/install.rst will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers * libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information More information
================ ================
...@@ -316,31 +392,25 @@ More information ...@@ -316,31 +392,25 @@ More information
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/ - GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/ - Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/ - ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
Included extensions
===================
ase-$ase_version, gpaw-$GPAW_version
]==]) ]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration.]==]) whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
conflict("GPAW") family("GPAW")
conflict("GPAW-manual")
if not ( isloaded("calcua/2020a") ) then prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
load("calcua/2020a")
end
if not ( isloaded("intel/2020a") ) then if not ( isloaded("calcua/admin") ) then
load("intel/2020a") load("calcua/admin")
end end
if not ( isloaded("IntelPython3/2020a") ) then if not ( isloaded("$compiler_module") ) then
load("IntelPython3/2020a") load("$compiler_module")
end end
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
prepend_path("PATH", "$install_dir/bin") prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib") prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
......
gpaw/build/examples/CalcUA-leibniz-broadwell/build_20.1.0_Python38_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.1.0:
# * We compile our own Python as this is the best guarantee to not have to
# struggle with compatibility problems between various compilers used for
# various components
# * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
#
packageID='20.1.0-Python38-FFTW-icc'
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
download_dir=$install_root/Downloads
install_dir=$install_root/$systemID/Packages/GPAW-UEABS/$packageID
modules_dir=$install_root/$systemID/Modules/GPAW-UEABS
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
patch_dir=$VSC_DATA/Projects/PRACE/GPAW-experiments/UEABS/build/patches
python_version='3.8.6'
zlib_version='1.2.11'
ncurses_version='6.2'
readline_version='8.0'
sqlite_version='3.33.0'
sqlite_download='3330000'
libffi_version='3.3'
fftw_version='3.3.8'
libxc_version='4.3.4'
wheel_version='0.35.1'
cython_version='0.29.21'
numpy_version='1.18.5'
scipy_version='1.5.4'
ase_version='3.19.3'
GPAW_version='20.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-xHost'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.8'
################################################################################
#
# Prepare the system
#
#
# Load modules
#
module purge
#module load personal/UEABS
MODULEPATH=$install_root/$systemID/Modules:$install_root/$systemID/Modules-link
module load $compiler_module
module load buildtools/$UEABS_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# zlib: https://www.zlib.net/zlib-1.2.11.tar.gz
zlib_file="zlib-$zlib_version.tar.gz"
zlib_url="https://www.zlib.net"
[[ -f $zlib_file ]] || wget "$zlib_url/$zlib_file"
# ncurses: https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz
ncurses_file="ncurses-$ncurses_version.tar.gz"
ncurses_url="https://ftp.gnu.org/pub/gnu/ncurses"
[[ -f $ncurses_file ]] || wget "$ncurses_url/$ncurses_file"
# readline: https://ftp.gnu.org/pub/gnu/readline/readline-8.0.tar.gz
readline_file="readline-$readline_version.tar.gz"
readline_url="https://ftp.gnu.org/pub/gnu/readline"
[[ -f $readline_file ]] || wget "$readline_url/$readline_file"
# sqlite: https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
sqlite_file="sqlite-autoconf-$sqlite_download.tar.gz"
sqlite_url="https://www.sqlite.org/2020"
[[ -f $sqlite_file ]] || wget "$sqlite_url/$sqlite_file"
# libffi: https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
libffi_file="libffi-$libffi_version.tar.gz"
libffi_url="https://github.com/libffi/libffi/releases/download/v$libffi_version"
[[ -f $libffi_file ]] || wget "$libffi_url/$libffi_file"
# FFTW: http://www.fftw.org/fftw-3.3.8.tar.gz
fftw_file="fftw-$fftw_version.tar.gz"
fftw_url="http://www.fftw.org"
[[ -f $fftw_file ]] || wget "$fftw_url/$fftw_file"
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Python: https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz
python_file="Python-$python_version.tar.xz"
python_url="https://www.python.org/ftp/python/$python_version"
[[ -f $python_file ]] || wget "$python_url/$python_file"
# Downloading wheel so that we can gather all sources for reproducibility.
wheel_file="wheel-$wheel_version.tar.gz"
wheel_url="https://pypi.python.org/packages/source/w/wheel"
[[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install ncurses
#
# We mirror the two-step EasyBuild install process. This may be overkill, but
# we know it works.
#
echo -e "\nInstalling ncurses...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$ncurses_file
cd ncurses-$ncurses_version
# Configure step 1
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CXX=icpc
export CXXFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite
# Build step 1
make -j $parallel
# Install step 1
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Configure step 2
make distclean
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite \
--enable-ext-colors --enable-widec \
--includedir=$install_dir/include/ncursesw/
# Build step 2
make -j $parallel
# Install step 2
make install
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
################################################################################
#
# Install readline
#
echo -e "\nInstalling readline...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$readline_file
cd readline-$readline_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CPPFLAGS="-I$install_dir/include"
export LDFLAGS="-L$install_dir/lib -lncurses"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
unset LDFLAGS
################################################################################
#
# Install zlib
#
echo -e "\nInstalling zlib...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$zlib_file
cd zlib-$zlib_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install libffi
#
echo -e "\nInstalling libffi...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libffi_file
cd libffi-$libffi_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-multi-os-directory
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install SQLite
#
echo -e "\nInstalling SQLite...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$sqlite_file
cd sqlite-autoconf-$sqlite_download
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC -DSQLITE_DISABLE_INTRINSIC"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
################################################################################
#
# Install FFTW
#
echo -e "\nInstalling FFTW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$fftw_file
# Patch the sources to compile with icc
cat >fftw.patch <<EOF
avoid using -no-gcc when compiling FFTW with Intel compilers,
since that fails on CentOS 8
see https://github.com/easybuilders/easybuild-easyconfigs/issues/10932
and https://github.com/FFTW/fftw3/issues/184
--- fftw-3.3.8/configure.orig
+++ fftw-3.3.8/configure
@@ -14861,6 +14861,9 @@
intel) # Stop icc from defining __GNUC__, except on MacOS where this fails
case "\${host_os}" in
*darwin*) ;; # icc -no-gcc fails to compile some system headers
+ # using -no-gcc with recent Intel compilers fails on CentOS 8,
+ # and was only needed as a workaround for old Intel compilers anyway
+ *linux*) ;;
*)
{ \$as_echo "\$as_me:\${as_lineno-\$LINENO}: checking whether C compiler accepts -no-gcc" >&5
$as_echo_n "checking whether C compiler accepts -no-gcc... " >&6; }
EOF
cd fftw-$fftw_version
patch -p1 <../fftw.patch
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export MPICC=mpiicc
export F77=ifort
export F77FLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-sse --disable-sse2 --disable-avx --enable-avx2 --disable-avx512 --disable-avx-128-fma --disable-kcvi \
--enable-fma \
--disable-openmp --disable-threads \
--enable-mpi \
-disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset MPICC
unset F77
unset F77FLAGS
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make -j install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install Python
#
echo -e "\nInstalling Python...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$python_file
cd Python-$python_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC"
export LD=icc
export LDFLAGS="-L$install_dir/lib"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir" \
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
--enable-shared --disable-ipv6 \
--enable-optimizations \
--with-ensurepip=upgrade
# --with-icc \
# Build
make -j 32
# Install
make install
cd $install_dir/bin
ln -s python$py_maj_min python
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset LD
unset LDFLAGS
unset CPPFLAGS
################################################################################
#
# Install wheel
#
echo -e "\nInstalling wheel...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $download_dir/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $install_dir/lib:$MKLROOT/lib/intel64_lin
include_dirs = $install_dir/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix="$install_dir" --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Minimal use of system libraries or libraries that are installed
through package managers such as EasyBuild or Spack.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* Python dependencies:
* zlib $zlib_version
* ncurses $ncurses_version
* readline $readline_version
* SQLite $sqlite_version
* libffi $libffi_version
* FFTW $fftw_version
* libxc $libxc_version
* Python $python_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration with Python $python_version, numpy-$numpy_version, scipy-$scipy_version, libxc $libxc_version and FFTW $fftw_version.]==])
conflict("GPAW")
conflict("GPAW-UEABS")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-leibniz-broadwell/build_20.10.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.10.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.20.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.10.0/doc/install.rst
#
packageID='20.10.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.02'
libxc_version='4.3.4'
ase_version='3.20.1'
GPAW_version='20.10.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos7/calcua-admin-broadwell/
module load calcua/admin
module load $compiler_module
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-leibniz-broadwell/build_20.10.0_Python39_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.10.0:
# * We compile our own Python as this is the best guarantee to not have to
# struggle with compatibility problems between various compilers used for
# various components
# * Using the matching version of ase, 3.20.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.10.0/doc/install.rst
#
packageID='20.10.0-Python39-FFTW-icc'
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
download_dir=$install_root/Downloads
install_dir=$install_root/$systemID/Packages/GPAW-UEABS/$packageID
modules_dir=$install_root/$systemID/Modules/GPAW-UEABS
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
patch_dir=$VSC_DATA/Projects/PRACE/GPAW-experiments/UEABS/build/patches
python_version='3.9.1'
zlib_version='1.2.11'
ncurses_version='6.2'
readline_version='8.0'
sqlite_version='3.33.0'
sqlite_download='3330000'
libffi_version='3.3'
fftw_version='3.3.8'
libxc_version='4.3.4'
wheel_version='0.36.2'
cython_version='0.29.21'
numpy_version='1.19.5'
scipy_version='1.5.4'
ase_version='3.20.1'
GPAW_version='20.10.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-xHost'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.9'
################################################################################
#
# Prepare the system
#
#
# Load modules
#
module purge
#module load personal/UEABS
MODULEPATH=$install_root/$systemID/Modules:$install_root/$systemID/Modules-link
module load $compiler_module
module load buildtools/$UEABS_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# zlib: https://www.zlib.net/zlib-1.2.11.tar.gz
zlib_file="zlib-$zlib_version.tar.gz"
zlib_url="https://www.zlib.net"
[[ -f $zlib_file ]] || wget "$zlib_url/$zlib_file"
# ncurses: https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz
ncurses_file="ncurses-$ncurses_version.tar.gz"
ncurses_url="https://ftp.gnu.org/pub/gnu/ncurses"
[[ -f $ncurses_file ]] || wget "$ncurses_url/$ncurses_file"
# readline: https://ftp.gnu.org/pub/gnu/readline/readline-8.0.tar.gz
readline_file="readline-$readline_version.tar.gz"
readline_url="https://ftp.gnu.org/pub/gnu/readline"
[[ -f $readline_file ]] || wget "$readline_url/$readline_file"
# sqlite: https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
sqlite_file="sqlite-autoconf-$sqlite_download.tar.gz"
sqlite_url="https://www.sqlite.org/2020"
[[ -f $sqlite_file ]] || wget "$sqlite_url/$sqlite_file"
# libffi: https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
libffi_file="libffi-$libffi_version.tar.gz"
libffi_url="https://github.com/libffi/libffi/releases/download/v$libffi_version"
[[ -f $libffi_file ]] || wget "$libffi_url/$libffi_file"
# FFTW: http://www.fftw.org/fftw-3.3.8.tar.gz
fftw_file="fftw-$fftw_version.tar.gz"
fftw_url="http://www.fftw.org"
[[ -f $fftw_file ]] || wget "$fftw_url/$fftw_file"
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Python: https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz
python_file="Python-$python_version.tar.xz"
python_url="https://www.python.org/ftp/python/$python_version"
[[ -f $python_file ]] || wget "$python_url/$python_file"
# Downloading wheel so that we can gather all sources for reproducibility.
wheel_file="wheel-$wheel_version.tar.gz"
wheel_url="https://pypi.python.org/packages/source/w/wheel"
[[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install ncurses
#
# We mirror the two-step EasyBuild install process. This may be overkill, but
# we know it works.
#
echo -e "\nInstalling ncurses...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$ncurses_file
cd ncurses-$ncurses_version
# Configure step 1
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CXX=icpc
export CXXFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite
# Build step 1
make -j $parallel
# Install step 1
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Configure step 2
make distclean
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite \
--enable-ext-colors --enable-widec \
--includedir=$install_dir/include/ncursesw/
# Build step 2
make -j $parallel
# Install step 2
make install
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
################################################################################
#
# Install readline
#
echo -e "\nInstalling readline...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$readline_file
cd readline-$readline_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CPPFLAGS="-I$install_dir/include"
export LDFLAGS="-L$install_dir/lib -lncurses"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
unset LDFLAGS
################################################################################
#
# Install zlib
#
echo -e "\nInstalling zlib...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$zlib_file
cd zlib-$zlib_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install libffi
#
echo -e "\nInstalling libffi...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libffi_file
cd libffi-$libffi_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-multi-os-directory
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install SQLite
#
echo -e "\nInstalling SQLite...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$sqlite_file
cd sqlite-autoconf-$sqlite_download
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC -DSQLITE_DISABLE_INTRINSIC"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
################################################################################
#
# Install FFTW
#
echo -e "\nInstalling FFTW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$fftw_file
# Patch the sources to compile with icc
cat >fftw.patch <<EOF
avoid using -no-gcc when compiling FFTW with Intel compilers,
since that fails on CentOS 8
see https://github.com/easybuilders/easybuild-easyconfigs/issues/10932
and https://github.com/FFTW/fftw3/issues/184
--- fftw-3.3.8/configure.orig
+++ fftw-3.3.8/configure
@@ -14861,6 +14861,9 @@
intel) # Stop icc from defining __GNUC__, except on MacOS where this fails
case "\${host_os}" in
*darwin*) ;; # icc -no-gcc fails to compile some system headers
+ # using -no-gcc with recent Intel compilers fails on CentOS 8,
+ # and was only needed as a workaround for old Intel compilers anyway
+ *linux*) ;;
*)
{ \$as_echo "\$as_me:\${as_lineno-\$LINENO}: checking whether C compiler accepts -no-gcc" >&5
$as_echo_n "checking whether C compiler accepts -no-gcc... " >&6; }
EOF
cd fftw-$fftw_version
patch -p1 <../fftw.patch
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export MPICC=mpiicc
export F77=ifort
export F77FLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-sse --disable-sse2 --disable-avx --enable-avx2 --disable-avx512 --disable-avx-128-fma --disable-kcvi \
--enable-fma \
--disable-openmp --disable-threads \
--enable-mpi \
-disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset MPICC
unset F77
unset F77FLAGS
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make -j install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install Python
#
echo -e "\nInstalling Python...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$python_file
cd Python-$python_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC"
export LD=icc
export LDFLAGS="-L$install_dir/lib"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir" \
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
--enable-shared --disable-ipv6 \
--enable-optimizations \
--with-ensurepip=upgrade
# --with-icc \
# Build
make -j 32
# Install
make install
cd $install_dir/bin
ln -s python$py_maj_min python
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset LD
unset LDFLAGS
unset CPPFLAGS
################################################################################
#
# Install wheel
#
echo -e "\nInstalling wheel...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $download_dir/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $install_dir/lib:$MKLROOT/lib/intel64_lin
include_dirs = $install_dir/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix="$install_dir" --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Minimal use of system libraries or libraries that are installed
through package managers such as EasyBuild or Spack.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* Python dependencies:
* zlib $zlib_version
* ncurses $ncurses_version
* readline $readline_version
* SQLite $sqlite_version
* libffi $libffi_version
* FFTW $fftw_version
* libxc $libxc_version
* Python $python_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration with Python $python_version, numpy-$numpy_version, scipy-$scipy_version, libxc $libxc_version and FFTW $fftw_version.]==])
conflict("GPAW")
conflict("GPAW-UEABS")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-leibniz-broadwell/build_21.1.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 21.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.21.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/21.1.0/doc/install.rst
#
packageID='21.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.02'
libxc_version='4.3.4'
ase_version='3.21.1'
GPAW_version='21.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos7/calcua-admin-broadwell/
module load calcua/admin
module load $compiler_module
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-leibniz-broadwell/build_21.1.0_Python39_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 21.1.0:
# * We compile our own Python as this is the best guarantee to not have to
# struggle with compatibility problems between various compilers used for
# various components
# * Using the matching version of ase, 3.21.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/21.1.0/doc/install.rst
#
packageID='21.1.0-Python39-FFTW-icc'
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
download_dir=$install_root/Downloads
install_dir=$install_root/$systemID/Packages/GPAW-UEABS/$packageID
modules_dir=$install_root/$systemID/Modules/GPAW-UEABS
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
patch_dir=$VSC_DATA/Projects/PRACE/GPAW-experiments/UEABS/build/patches
python_version='3.9.1'
zlib_version='1.2.11'
ncurses_version='6.2'
readline_version='8.0'
sqlite_version='3.33.0'
sqlite_download='3330000'
libffi_version='3.3'
fftw_version='3.3.8'
libxc_version='4.3.4'
wheel_version='0.36.2'
cython_version='0.29.21'
numpy_version='1.19.5'
scipy_version='1.5.4'
ase_version='3.21.1'
GPAW_version='21.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-xHost'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.9'
################################################################################
#
# Prepare the system
#
#
# Load modules
#
module purge
#module load personal/UEABS
MODULEPATH=$install_root/$systemID/Modules:$install_root/$systemID/Modules-link
module load $compiler_module
module load buildtools/$UEABS_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# zlib: https://www.zlib.net/zlib-1.2.11.tar.gz
zlib_file="zlib-$zlib_version.tar.gz"
zlib_url="https://www.zlib.net"
[[ -f $zlib_file ]] || wget "$zlib_url/$zlib_file"
# ncurses: https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz
ncurses_file="ncurses-$ncurses_version.tar.gz"
ncurses_url="https://ftp.gnu.org/pub/gnu/ncurses"
[[ -f $ncurses_file ]] || wget "$ncurses_url/$ncurses_file"
# readline: https://ftp.gnu.org/pub/gnu/readline/readline-8.0.tar.gz
readline_file="readline-$readline_version.tar.gz"
readline_url="https://ftp.gnu.org/pub/gnu/readline"
[[ -f $readline_file ]] || wget "$readline_url/$readline_file"
# sqlite: https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
sqlite_file="sqlite-autoconf-$sqlite_download.tar.gz"
sqlite_url="https://www.sqlite.org/2020"
[[ -f $sqlite_file ]] || wget "$sqlite_url/$sqlite_file"
# libffi: https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
libffi_file="libffi-$libffi_version.tar.gz"
libffi_url="https://github.com/libffi/libffi/releases/download/v$libffi_version"
[[ -f $libffi_file ]] || wget "$libffi_url/$libffi_file"
# FFTW: http://www.fftw.org/fftw-3.3.8.tar.gz
fftw_file="fftw-$fftw_version.tar.gz"
fftw_url="http://www.fftw.org"
[[ -f $fftw_file ]] || wget "$fftw_url/$fftw_file"
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Python: https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz
python_file="Python-$python_version.tar.xz"
python_url="https://www.python.org/ftp/python/$python_version"
[[ -f $python_file ]] || wget "$python_url/$python_file"
# Downloading wheel so that we can gather all sources for reproducibility.
wheel_file="wheel-$wheel_version.tar.gz"
wheel_url="https://pypi.python.org/packages/source/w/wheel"
[[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install ncurses
#
# We mirror the two-step EasyBuild install process. This may be overkill, but
# we know it works.
#
echo -e "\nInstalling ncurses...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$ncurses_file
cd ncurses-$ncurses_version
# Configure step 1
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CXX=icpc
export CXXFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite
# Build step 1
make -j $parallel
# Install step 1
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Configure step 2
make distclean
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite \
--enable-ext-colors --enable-widec \
--includedir=$install_dir/include/ncursesw/
# Build step 2
make -j $parallel
# Install step 2
make install
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
################################################################################
#
# Install readline
#
echo -e "\nInstalling readline...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$readline_file
cd readline-$readline_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CPPFLAGS="-I$install_dir/include"
export LDFLAGS="-L$install_dir/lib -lncurses"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
unset LDFLAGS
################################################################################
#
# Install zlib
#
echo -e "\nInstalling zlib...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$zlib_file
cd zlib-$zlib_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install libffi
#
echo -e "\nInstalling libffi...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libffi_file
cd libffi-$libffi_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-multi-os-directory
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install SQLite
#
echo -e "\nInstalling SQLite...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$sqlite_file
cd sqlite-autoconf-$sqlite_download
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC -DSQLITE_DISABLE_INTRINSIC"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
################################################################################
#
# Install FFTW
#
echo -e "\nInstalling FFTW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$fftw_file
# Patch the sources to compile with icc
cat >fftw.patch <<EOF
avoid using -no-gcc when compiling FFTW with Intel compilers,
since that fails on CentOS 8
see https://github.com/easybuilders/easybuild-easyconfigs/issues/10932
and https://github.com/FFTW/fftw3/issues/184
--- fftw-3.3.8/configure.orig
+++ fftw-3.3.8/configure
@@ -14861,6 +14861,9 @@
intel) # Stop icc from defining __GNUC__, except on MacOS where this fails
case "\${host_os}" in
*darwin*) ;; # icc -no-gcc fails to compile some system headers
+ # using -no-gcc with recent Intel compilers fails on CentOS 8,
+ # and was only needed as a workaround for old Intel compilers anyway
+ *linux*) ;;
*)
{ \$as_echo "\$as_me:\${as_lineno-\$LINENO}: checking whether C compiler accepts -no-gcc" >&5
$as_echo_n "checking whether C compiler accepts -no-gcc... " >&6; }
EOF
cd fftw-$fftw_version
patch -p1 <../fftw.patch
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export MPICC=mpiicc
export F77=ifort
export F77FLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-sse --disable-sse2 --disable-avx --enable-avx2 --disable-avx512 --disable-avx-128-fma --disable-kcvi \
--enable-fma \
--disable-openmp --disable-threads \
--enable-mpi \
-disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset MPICC
unset F77
unset F77FLAGS
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make -j install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install Python
#
echo -e "\nInstalling Python...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$python_file
cd Python-$python_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC"
export LD=icc
export LDFLAGS="-L$install_dir/lib"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir" \
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
--enable-shared --disable-ipv6 \
--enable-optimizations \
--with-ensurepip=upgrade
# --with-icc \
# Build
make -j 32
# Install
make install
cd $install_dir/bin
ln -s python$py_maj_min python
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset LD
unset LDFLAGS
unset CPPFLAGS
################################################################################
#
# Install wheel
#
echo -e "\nInstalling wheel...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $download_dir/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $install_dir/lib:$MKLROOT/lib/intel64_lin
include_dirs = $install_dir/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $download_dir/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix="$install_dir" --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Minimal use of system libraries or libraries that are installed
through package managers such as EasyBuild or Spack.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* Python dependencies:
* zlib $zlib_version
* ncurses $ncurses_version
* readline $readline_version
* SQLite $sqlite_version
* libffi $libffi_version
* FFTW $fftw_version
* libxc $libxc_version
* Python $python_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration with Python $python_version, numpy-$numpy_version, scipy-$scipy_version, libxc $libxc_version and FFTW $fftw_version.]==])
conflict("GPAW")
conflict("GPAW-UEABS")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/README.md 0 → 100644
View file @ b57ca544
# Example build scripts for CalcUA-vaughan-rome
Cluster characteristics:
* CPU architecture: dual-socket 32-core AMD EPYC 7452 "Rome" CPUs
* Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
* Package manager: Most software on the cluster is installed through [
EasyBuild](https://easybuilders.github.io/).
Unless otherwise mentioned below, the following combinations of versions of GPAW, ASE, Python, NumPy
and SciPy were used:
| GPAW | ASE | Python | NumPy | SciPy |
|:--------|:--------|:-------|:-------|:------|
| 20.1.0 | 3.19.3 | 3.8.7 | 1.18.5 | 1.5.4 |
| 20.10.0 | 3.20.1 | 3.9.1 | 1.19.5 | 1.5.4 |
| 21.1.0 | 3.21.1 | 3.9.1 | 1.19.5 | 1.5.4 |
The following example build scripts do a complete build, including a build of Python
and a minimal set of dependencies that is needed to build a large enough part of the
Python standard library to build GPAW and run the benchmarks: zlib,
ncurses, readline, libffi, SQLite (used by one of the GPAW tests but likely
not needed for the benchmarks), FFTW, libxc, Python and the Python packages
NumPy, SciPy, ase and GPAW. The included libraries are not sufficient for a
full Python installation with all standard library packages or a full ASE/GPAW
installation with all optional packages but should be enough to run the benchmarks.
All software is build with the Intel 2020 compilers.
* `build_20.1.0_Python38_FFTW_icc`
* `build_20.10.0_Python39_FFTW_icc`
* `build_21.1.0_Python39_FFTW_icc`
The following build example build scripts install GPAW on top of the Intel Python
distribution. They rely as much as possible on modules already in the Intel Python
distribution or software that should be installed in the OS to run Intel Python.
The configuration accesses the FFT library available through
NumPy rather then linking directly to a FFT library.
The example script does use a module that provides up-to-date build
tools (buildtools/2020a) but on recent OSes it can be omitted.
Rather than loading the Intel and IntelPython3 modules one could as well
run the scripts provided by Intel to initialize the environment variables
for the compilers and Python distribution, and provided the necessary build
tools are also available in the system OS, this build could be done without
relying on any EasyBuild or Spack-generated modules.
* `build_20.1.0_IntelPython3_icc.sh`
* `build_20.10.0_IntelPython3_icc.sh`
* `build_21.1.0_IntelPython3_icc.sh`
The following example build scripts depend on other modules (Python and FFTW) that
were installed on the cluster through EasyBuild beyond the compiler modules and a
module providing some up-to-date basic build tools:
* `build_20.1.0_Python39icc_icc.sh`
* `build_20.10.0_Python39icc_icc.sh`
* `build_21.1.0_Python39icc_icc.sh`
gpaw/build/examples/CalcUA-vaughan-rome/build_20.1.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
#
packageID='20.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.00'
libxc_version='4.3.4'
ase_version='3.19.3'
GPAW_version='20.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a.00'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos8/calcua-admin-rome/
module load calcua/admin
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
module load $compiler_module
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
--
-- The order of loading is a deliberate choice. It takes some libraries from
-- the compiler directories rather than from the Python directories and has
-- solved some unexplained issues we had with SLURM.
--
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_20.1.0_Python38_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.1.0:
# * We compile our own Python as this is the best guarantee to not have to
# struggle with compatibility problems between various compilers used for
# various components
# * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
#
packageID='20.1.0-Python38-FFTW-icc'
packageName='GPAW-UEABS'
echo -e "\n### Building $packageName/$packageID from $0\n\n"
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Of course these variables can be set by hand also.
# Note that the UEABS module also completes the MODULEPATH set below.
#
# The next three variables are only used to load the right UEABS module
# and to give example values for variable settings in comments.
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
module purge
MODULEPATH=$install_root/$systemID/Modules
module load UEABS/$UEABS_version
#
# The following UEABS_ variables are needed:
#
# Directory to put the downloaded sources of the packages.
# UEABS_DOWNLOADS=$install_root/Downloads
# Directory where packages should be installed.
# UEABS_PACKAGES=$install_root/$systemID/Packages
# Directory where modules are installed
# UEABS_MODULES=$install_root/$systemID/Modules
install_dir=$UEABS_PACKAGES/$packageName/$packageID
modules_dir=$UEABS_MODULES/$packageName
build_dir="/dev/shm/$USER/$packageName/$packageID"
# Software versions
python_version='3.8.7'
zlib_version='1.2.11'
ncurses_version='6.2'
readline_version='8.0'
sqlite_version='3.33.0'
sqlite_download='3330000'
libffi_version='3.3'
fftw_version='3.3.8'
libxc_version='4.3.4'
wheel_version='0.36.2'
cython_version='0.29.21'
numpy_version='1.18.5'
scipy_version='1.4.1'
ase_version='3.19.3'
GPAW_version='20.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.8'
################################################################################
#
# Prepare the system
#
#
# Load modules
# - Compiler module: This is a system-provide module
# - buildtools module: Contains a number of updated build tools already available
# on most systems. We use it to get more consistent builds across systems, but
# the module is not currently included in the UEABS repository. In most cases,
# it will not be needed at all except for more exactly reproducing our test
# results.
#
module load $compiler_module
module load buildtools/$UEABS_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $UEABS_DOWNLOADS
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $UEABS_DOWNLOADS
# zlib: https://www.zlib.net/zlib-1.2.11.tar.gz
zlib_file="zlib-$zlib_version.tar.gz"
zlib_url="https://www.zlib.net"
[[ -f $zlib_file ]] || wget "$zlib_url/$zlib_file"
# ncurses: https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz
ncurses_file="ncurses-$ncurses_version.tar.gz"
ncurses_url="https://ftp.gnu.org/pub/gnu/ncurses"
[[ -f $ncurses_file ]] || wget "$ncurses_url/$ncurses_file"
# readline: https://ftp.gnu.org/pub/gnu/readline/readline-8.0.tar.gz
readline_file="readline-$readline_version.tar.gz"
readline_url="https://ftp.gnu.org/pub/gnu/readline"
[[ -f $readline_file ]] || wget "$readline_url/$readline_file"
# sqlite: https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
sqlite_file="sqlite-autoconf-$sqlite_download.tar.gz"
sqlite_url="https://www.sqlite.org/2020"
[[ -f $sqlite_file ]] || wget "$sqlite_url/$sqlite_file"
# libffi: https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
libffi_file="libffi-$libffi_version.tar.gz"
libffi_url="https://github.com/libffi/libffi/releases/download/v$libffi_version"
[[ -f $libffi_file ]] || wget "$libffi_url/$libffi_file"
# FFTW: http://www.fftw.org/fftw-3.3.8.tar.gz
fftw_file="fftw-$fftw_version.tar.gz"
fftw_url="http://www.fftw.org"
[[ -f $fftw_file ]] || wget "$fftw_url/$fftw_file"
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Python: https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz
python_file="Python-$python_version.tar.xz"
python_url="https://www.python.org/ftp/python/$python_version"
[[ -f $python_file ]] || wget "$python_url/$python_file"
# Downloading wheel so that we can gather all sources for reproducibility.
wheel_file="wheel-$wheel_version.tar.gz"
wheel_url="https://pypi.python.org/packages/source/w/wheel"
[[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install ncurses
#
# We mirror the two-step EasyBuild install process. This may be overkill, but
# we know it works.
#
echo -e "\nInstalling ncurses...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$ncurses_file
cd ncurses-$ncurses_version
# Configure step 1
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CXX=icpc
export CXXFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite
# Build step 1
make -j $parallel
# Install step 1
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Configure step 2
make distclean
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite \
--enable-ext-colors --enable-widec \
--includedir=$install_dir/include/ncursesw/
# Build step 2
make -j $parallel
# Install step 2
make install
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
################################################################################
#
# Install readline
#
echo -e "\nInstalling readline...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$readline_file
cd readline-$readline_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CPPFLAGS="-I$install_dir/include"
export LDFLAGS="-L$install_dir/lib -lncurses"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
unset LDFLAGS
################################################################################
#
# Install zlib
#
echo -e "\nInstalling zlib...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$zlib_file
cd zlib-$zlib_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install libffi
#
echo -e "\nInstalling libffi...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libffi_file
cd libffi-$libffi_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-multi-os-directory
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install SQLite
#
echo -e "\nInstalling SQLite...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$sqlite_file
cd sqlite-autoconf-$sqlite_download
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC -DSQLITE_DISABLE_INTRINSIC"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
################################################################################
#
# Install FFTW
#
echo -e "\nInstalling FFTW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$fftw_file
# Patch the sources to compile with icc
cat >fftw.patch <<EOF
avoid using -no-gcc when compiling FFTW with Intel compilers,
since that fails on CentOS 8
see https://github.com/easybuilders/easybuild-easyconfigs/issues/10932
and https://github.com/FFTW/fftw3/issues/184
--- fftw-3.3.8/configure.orig
+++ fftw-3.3.8/configure
@@ -14861,6 +14861,9 @@
intel) # Stop icc from defining __GNUC__, except on MacOS where this fails
case "\${host_os}" in
*darwin*) ;; # icc -no-gcc fails to compile some system headers
+ # using -no-gcc with recent Intel compilers fails on CentOS 8,
+ # and was only needed as a workaround for old Intel compilers anyway
+ *linux*) ;;
*)
{ \$as_echo "\$as_me:\${as_lineno-\$LINENO}: checking whether C compiler accepts -no-gcc" >&5
$as_echo_n "checking whether C compiler accepts -no-gcc... " >&6; }
EOF
cd fftw-$fftw_version
patch -p1 <../fftw.patch
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export MPICC=mpiicc
export F77=ifort
export F77FLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-sse --disable-sse2 --disable-avx --enable-avx2 --disable-avx512 --disable-avx-128-fma --disable-kcvi \
--enable-fma \
--disable-openmp --disable-threads \
--enable-mpi \
-disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset MPICC
unset F77
unset F77FLAGS
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make -j install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install Python
#
echo -e "\nInstalling Python...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$python_file
cd Python-$python_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC"
export LD=icc
export LDFLAGS="-L$install_dir/lib"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir" \
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
--enable-shared --disable-ipv6 \
--enable-optimizations \
--with-ensurepip=upgrade
# --with-icc \
# Build
make -j 32
# Install
make install
cd $install_dir/bin
ln -s python$py_maj_min python
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset LD
unset LDFLAGS
unset CPPFLAGS
################################################################################
#
# Initialising for installing Python packages
#
echo -e "\nInitialising for installing Python packages...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
################################################################################
#
# Install wheel
#
echo -e "\nInstalling wheel...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $UEABS_DOWNLOADS/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $install_dir/lib:$MKLROOT/lib/intel64_lin
include_dirs = $install_dir/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $UEABS_DOWNLOADS/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$GPAW_file
cd gpaw-$GPAW_version
# Make the siteconfig.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module $packageName/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Minimal use of system libraries or libraries that are installed
through package managers such as EasyBuild or Spack.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* Python dependencies:
* zlib $zlib_version
* ncurses $ncurses_version
* readline $readline_version
* SQLite $sqlite_version
* libffi $libffi_version
* FFTW $fftw_version
* libxc $libxc_version
* Python $python_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration with Python $python_version, numpy-$numpy_version, scipy-$scipy_version, libxc $libxc_version and FFTW $fftw_version.]==])
conflict("GPAW")
conflict("$packageName")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_1.5.2_Python37icc_icc.sh → gpaw/build/examples/CalcUA-vaughan-rome/build_20.1.0_Python38icc_FFTW_icc.sh
View file @ b57ca544
#!/bin/bash #!/bin/bash
# #
# Installation script for GPAW 1.5.2: # Installation script for GPAW 20.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized # * Using a Python and FFTW module compiled with EasyBuild, but libxc and
# NumPy and SciPy included. # NumPy/SciPy compiled by hand.
# * Using the matching version of ase, 3.17.0 # * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers # * Compiling with the Intel compilers
# #
# The FFT library is discovered at runtime. With the settings used in this script # The FFT library is discovered at runtime. With the settings used in this script
...@@ -12,31 +12,68 @@ ...@@ -12,31 +12,68 @@
# (link below). # (link below).
# #
# The original installation instructions for GPAW can be found at # The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/1.5.2/doc/install.rst # https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
# #
packageID='1.5.2-Python37icc-icc' #set -e
packageID='20.1.0-Python38icc-FFTW-icc'
packageName='GPAW-UEABS'
echo -e "\n### Building $packageName/$packageID from $0\n\n"
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Of course these variables can be set by hand also.
# Note that the UEABS module also completes the MODULEPATH set below.
#
# The next three variables are only used to load the right UEABS module
# and to give example values for variable settings in comments.
install_root=$VSC_SCRATCH/UEABS install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
download_dir=$install_root/Downloads module purge
install_dir=$install_root/$systemID/Packages/GPAW-manual/$packageID MODULEPATH=$install_root/$systemID/Modules
modules_dir=$install_root/$systemID/Modules/GPAW-manual module load UEABS/$UEABS_version
build_dir="/dev/shm/$USER/GPAW-manual/$packageID"
patch_dir=$VSC_DATA/Projects/PRACE/GPAW-experiments/UEABS/build/patches
#
# The following UEABS_ variables are needed:
#
# Directory to put the downloaded sources of the packages.
# UEABS_DOWNLOADS=$install_root/Downloads
# Directory where packages should be installed.
# UEABS_PACKAGES=$install_root/$systemID/Packages
# Directory where modules are installed
# UEABS_MODULES=$install_root/$systemID/Modules
install_dir=$UEABS_PACKAGES/$packageName/$packageID
modules_dir=$UEABS_MODULES/$packageName
build_dir="/dev/shm/$USER/$packageName/$packageID"
# Software versions
python_version='3.8.7'
fftw_version='3.3.8'
libxc_version='4.3.4' libxc_version='4.3.4'
#wheel_version='0.35.1'
cython_version='0.29.21' cython_version='0.29.21'
numpy_version='1.18.5' numpy_version='1.18.5'
#numpy_version='1.19.2' scipy_version='1.5.4'
scipy_version='1.4.1' ase_version='3.19.3'
#scipy_version='1.5.3' GPAW_version='20.1.0'
ase_version='3.17.0'
GPAW_version='1.5.2'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
py_maj_min='3.7' # Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.8'
################################################################################ ################################################################################
# #
...@@ -45,19 +82,28 @@ py_maj_min='3.7' ...@@ -45,19 +82,28 @@ py_maj_min='3.7'
# #
# Load modules # Load modules
# - Compiler module: This is a system-provide module
# - buildtools module: Contains a number of updated build tools already available
# on most systems. We use it to get more consistent builds across systems, but
# the module is not currently included in the UEABS repository. In most cases,
# it will not be needed at all except for more exactly reproducing our test
# results.
# - An EasyBuild-generated module containing a suitable FFTW implementation.
# - An EasyBuild-generated module with Python 3.9 and the basic installation tools
# (pip, setuptools, wheel)
# In some cases we used modules build specifically for this project, but this was
# to determine the minimum set of libraries etc. that are needed. Modules provided
# on the system would have done the job as well.
# #
module purge module load $compiler_module
module load calcua/2020a module load buildtools/$UEABS_version
module load personal/UEABS module load FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW
module load intel/2020a module load Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW
module load buildtools/2020a
module load FFTW/3.3.8-intel-2020a
module load Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW
# #
# Create the directories and make sure they are clean if that matters # Create the directories and make sure they are clean if that matters
# #
/usr/bin/mkdir -p $download_dir /usr/bin/mkdir -p $UEABS_DOWNLOADS
/usr/bin/mkdir -p $install_dir /usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir /usr/bin/rm -rf $install_dir
...@@ -70,7 +116,6 @@ module load Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW ...@@ -70,7 +116,6 @@ module load Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW
/usr/bin/mkdir -p $build_dir /usr/bin/mkdir -p $build_dir
################################################################################ ################################################################################
# #
# Download components # Download components
...@@ -78,14 +123,23 @@ module load Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW ...@@ -78,14 +123,23 @@ module load Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW
echo -e "\nDownloading files...\n" echo -e "\nDownloading files...\n"
cd $download_dir cd $UEABS_DOWNLOADS
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2 # https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2" libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version" libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file" [[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# We do not download Cython but simply install it using pip. # As wheel is already present in the Python module we don't need to download it again.
# Downloading wheel so that we can gather all sources for reproducibility.
# wheel_file="wheel-$wheel_version.tar.gz"
# wheel_url="https://pypi.python.org/packages/source/w/wheel"
# [[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources # NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip" numpy_file="numpy-$numpy_version.zip"
...@@ -97,12 +151,10 @@ scipy_file="scipy-$scipy_version.tar.gz" ...@@ -97,12 +151,10 @@ scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy" scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file" [[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# We do not download ase in this script. As it is pure python and doesn't need # Downloading ase so that we can gather all sources for reproducibility
# customization, we will install it using pip right away. ase_file="ase-$ase_version.tar.gz"
## https://files.pythonhosted.org/packages/d9/08/35969da23b641d3dfca46ba7559f651fcfdca81dbbc00b9058c934e75769/ase-3.17.0.tar.gz ase_url="https://pypi.python.org/packages/source/a/ase"
#ase_file="ase-$ase_version.tar.gz" [[ -f $ase_file ]] || wget "$ase_url/$ase_file"
#ase_url="https://files.pythonhosted.org/packages/d9/08/35969da23b641d3dfca46ba7559f651fcfdca81dbbc00b9058c934e75769"
#[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources. # GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz # https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
...@@ -117,7 +169,6 @@ GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files" ...@@ -117,7 +169,6 @@ GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file" [[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################ ################################################################################
# #
# Install libxc # Install libxc
...@@ -128,7 +179,7 @@ echo -e "\nInstalling libxc...\n" ...@@ -128,7 +179,7 @@ echo -e "\nInstalling libxc...\n"
cd $build_dir cd $build_dir
# Uncompress # Uncompress
tar -xf $download_dir/$libxc_file tar -xf $UEABS_DOWNLOADS/$libxc_file
cd libxc-$libxc_version cd libxc-$libxc_version
...@@ -136,16 +187,14 @@ cd libxc-$libxc_version ...@@ -136,16 +187,14 @@ cd libxc-$libxc_version
autoreconf -i autoreconf -i
export CC=icc export CC=icc
#export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -fPIC" export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC"
#export CFLAGS="-O0 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC"
./configure --prefix="$install_dir" \ ./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran --disable-static --enable-shared --disable-fortran
# Build # Build
make -j 16 make -j $parallel
# Install # Install
make -j 16 install make install
# Add bin, lib and include to the PATH variables # Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH PATH=$install_dir/bin:$PATH
...@@ -153,22 +202,62 @@ LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH ...@@ -153,22 +202,62 @@ LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################ ################################################################################
# #
# Install Cython # Initialising for installing Python packages
# #
echo -e "\nInstalling Cython...\n" echo -e "\nInitialising for installing Python packages...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages" /usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir cd $install_dir
/usr/bin/ln -s lib lib64 /usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages" PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
################################################################################
#
# No need to install wheel as it would erase the wheel version already present in
# the Python module.
#
echo -e "\nNot installing wheel...\n"
#cd $build_dir
#pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
pip3 install --prefix=$install_dir --no-deps cython==$cython_version
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$cython_file
################################################################################ ################################################################################
...@@ -181,7 +270,7 @@ echo -e "\nInstalling NumPy...\n" ...@@ -181,7 +270,7 @@ echo -e "\nInstalling NumPy...\n"
cd $build_dir cd $build_dir
# Uncompress # Uncompress
unzip $download_dir/$numpy_file unzip $UEABS_DOWNLOADS/$numpy_file
cd numpy-$numpy_version cd numpy-$numpy_version
...@@ -198,25 +287,38 @@ libraries = fftw3 ...@@ -198,25 +287,38 @@ libraries = fftw3
EOF EOF
export CC=icc export CC=icc
export CFLAGS='-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890' export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc export CXX=icpc
export FC=ifort export FC=ifort
export F90=$FC export F90=$FC
export FFLAGS='-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC' export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc export LD=icc
# Build NumPy. # Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the # Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly, # choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there. # the wrong linker was used leading to problems there.
python setup.py build -j 32 # Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy # Install NumPy
python setup.py install --prefix $install_dir pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory... # Brief test: Should not be run in the NumPy build directory...
cd $build_dir cd $build_dir
python -c "import numpy" python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################ ################################################################################
...@@ -229,30 +331,41 @@ echo -e "\nInstalling SciPy...\n" ...@@ -229,30 +331,41 @@ echo -e "\nInstalling SciPy...\n"
cd $build_dir cd $build_dir
# Uncompress # Uncompress
tar -xf $download_dir/$scipy_file tar -xf $UEABS_DOWNLOADS/$scipy_file
cd scipy-$scipy_version cd scipy-$scipy_version
export CC=icc export CC=icc
export CFLAGS='-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438' export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc export CXX=icpc
export FC=ifort export FC=ifort
export F90=$FC export F90=$FC
# FFLAGS is used in the fitpack Makefile # FFLAGS is used in the fitpack Makefile
export FFLAGS='-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC' export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile # FOPT is used in the odrpack Makefile
export FOPT='-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC' export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc export LD=icc
# Build scipy # Build scipy
python setup.py build -j 32 # SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy # Install scipy
python setup.py install --prefix $install_dir pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory... # Brief test: Should not be run in the NumPy build directory...
cd $build_dir cd $build_dir
python -c "import scipy" python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################ ################################################################################
...@@ -263,8 +376,12 @@ python -c "import scipy" ...@@ -263,8 +376,12 @@ python -c "import scipy"
echo -e "\nInstalling ase...\n" echo -e "\nInstalling ase...\n"
cd $build_dir cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
pip3 install --prefix=$install_dir --no-deps ase==$ase_version # Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################ ################################################################################
...@@ -276,7 +393,7 @@ echo -e "\nInstalling gpaw-setups...\n" ...@@ -276,7 +393,7 @@ echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1 tar -xf $UEABS_DOWNLOADS/$GPAWsetups_file --strip-components=1
################################################################################ ################################################################################
...@@ -289,49 +406,40 @@ echo -e "\nInstalling GPAW...\n" ...@@ -289,49 +406,40 @@ echo -e "\nInstalling GPAW...\n"
cd $build_dir cd $build_dir
# Uncompress # Uncompress
tar -xf $download_dir/$GPAW_file tar -xf $UEABS_DOWNLOADS/$GPAW_file
# Apply patches
patch -p0 <$patch_dir/gpaw-1.5.2.patch
cd gpaw-$GPAW_version cd gpaw-$GPAW_version
# Make the customize.py script # Make the siteconfig.py script
mv customize.py customize.py.orig cat >siteconfig.py <<EOF
cat >customize.py <<EOF print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW build INFO: Starting execution of the customization script' ) print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW build INFO: Variables at the start of the customization script' ) print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW build INFO: libraries = ', libraries ) print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW build INFO: mpi_libaries = ', mpi_libraries ) print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW build INFO: library_dirs = ', library_dirs ) print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW build INFO: mpi_libary_dirs = ', mpi_library_dirs ) print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW build INFO: runtime_library_dirs = ', runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW build INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs ) print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW build INFO: include_dirs = ', include_dirs ) print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW build INFO: mpi_include_dirs = ', mpi_include_dirs ) print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW build INFO: compiler = ', compiler ) print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW build INFO: mpicompiler = ', mpicompiler ) print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW build INFO: mpilinker = ', mpilinker ) print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW build INFO: extra_compile_args = ', extra_compile_args ) print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW build INFO: extra_link_args = ', extra_link_args ) print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW build INFO: define_macros = ', define_macros ) print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW build INFO: mpi_define_macros = ', mpi_define_macros ) print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW build INFO: undef_macros = ', undef_macros ) print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW build INFO: scalapack = ', scalapack ) print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW build INFO: libvdwxc = ', libvdwxc ) print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW build INFO: elpa = ', elpa ) print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
# Reset the lists of libraries as often the wrong BLAS library is picked up. print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
libraries = []
mpi_libraries = [] # LibXC
include_dirs.append('$install_dir/include')
# LibXC. Re-add the library (removed by resetting libraries). #libraries.append('xc')
# There is no need to add the library directory as we do set library_dirs to the
# content of LIBRARY_PATH further down.
# There should be no need to add the include directory as it is in CPATH which is
# set when we install gpaw.
#include_dirs.append('$install_dir/include')
libraries.append('xc')
# libvdwxc # libvdwxc
libvdwxc = False libvdwxc = False
...@@ -339,17 +447,18 @@ libvdwxc = False ...@@ -339,17 +447,18 @@ libvdwxc = False
# ELPA # ELPA
elpa = False elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK # ScaLAPACK
scalapack = True scalapack = True
mpi_libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64'] libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
mpi_define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
mpi_define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1')]
# Add EasyBuild LAPACK/BLAS libs # MKL BLAS
# This should also enable MKL FFTW according to the documentation of GPAW 1.5.2 libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
libraries += ['mkl_intel_lp64', 'mkl_sequential', 'mkl_core']
# Add other EasyBuild library directories. # Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':') library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers # Set the compilers
...@@ -357,44 +466,49 @@ compiler = os.environ['CC'] ...@@ -357,44 +466,49 @@ compiler = os.environ['CC']
mpicompiler = os.environ['MPICC'] mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC'] mpilinker = os.environ['MPICC']
# We need extra_compile_args to have the right compiler options when re-compiling print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
# files for gpaw-python. It does imply double compiler options for the other print( 'GPAW EasyBuild INFO: libraries = ', libraries )
# compiles though. print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
extra_compile_args = os.environ['CFLAGS'].split(' ') print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW build INFO: Variables at the end of the customization script' ) print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW build INFO: libraries = ', libraries ) print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW build INFO: mpi_libaries = ', mpi_libraries ) print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW build INFO: library_dirs = ', library_dirs ) print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW build INFO: mpi_libary_dirs = ', mpi_library_dirs ) print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW build INFO: runtime_library_dirs = ', runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW build INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW build INFO: include_dirs = ', include_dirs ) print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW build INFO: mpi_include_dirs = ', mpi_include_dirs ) print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW build INFO: compiler = ', compiler ) print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW build INFO: mpicompiler = ', mpicompiler ) print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW build INFO: mpilinker = ', mpilinker ) print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW build INFO: extra_compile_args = ', extra_compile_args ) print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW build INFO: extra_link_args = ', extra_link_args ) print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW build INFO: define_macros = ', define_macros ) print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW build INFO: mpi_define_macros = ', mpi_define_macros ) print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW build INFO: undef_macros = ', undef_macros ) print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW build INFO: scalapack = ', scalapack ) print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW build INFO: libvdwxc = ', libvdwxc ) print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
print( 'GPAW build INFO: elpa = ', elpa )
print( 'GPAW build INFO: Ending execution of the customization script' )
EOF EOF
# Build gpaw
export CC=icc export CC=icc
export MPICC=mpiicc export MPICC=mpiicc
#export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -qno-openmp-simd" export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -qno-openmp-simd" python$py_maj_min setup.py build -j $parallel
#export CFLAGS="-O0 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -qno-openmp-simd"
python setup.py build -j 32
# Install GPAW # Install GPAW
python setup.py install --prefix="$install_dir" #python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################ ################################################################################
...@@ -402,12 +516,12 @@ python setup.py install --prefix="$install_dir" ...@@ -402,12 +516,12 @@ python setup.py install --prefix="$install_dir"
# Finish the install # Finish the install
# #
echo -e "\nCleaning up and making the LUA-module GPAW-manual/$packageID...\n" echo -e "\nCleaning up and making the LUA-module $packageName/$packageID...\n"
# Go to a different directory before cleaning up the build directory # Go to a different directory before cleaning up the build directory
cd $modules_dir cd $modules_dir
#/bin/rm -rf $build_dir /bin/rm -rf $build_dir
# Create a module file # Create a module file
cat >$packageID.lua <<EOF cat >$packageID.lua <<EOF
...@@ -419,16 +533,19 @@ Description ...@@ -419,16 +533,19 @@ Description
GPAW $GPAW_version for the UEABS benchmark. GPAW $GPAW_version for the UEABS benchmark.
Configuration: Configuration:
* Python 3 compiled with the Intel compilers pre-installed on the system
* NumPy/SciPy using MKL for linear algebra and FFTW for FFT.
* Parallel GPAW $GPAW_version with ase $ase_version * Parallel GPAW $GPAW_version with ase $ase_version
* FFT library selected at runtime. The default with the path * Using FFTW for the FFT computations and MKL for the BLAS/Lapack
as set through this module should be MKL. However, we set GPAW_FFTWSO * Python, FFTW and the dependencies of Python build with EasyBuild.
in this module to select the regular FFTW libraries as we have
experienced problems with the Intel MKL FFTW routines on some systems. Detailed configuration:
Setting GPAW_FFTWSO is documented in the install instructions at * Compiler module: $compiler_module, used for all packages
https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/install.rst * EasyBuild modules (special build for the benchmarks, not taken from the system)
* libxc and GPAW compiled with the Intel compilers * FFTW $fftw_version
* Python $python_version
* libxc $libxc_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information More information
...@@ -442,32 +559,24 @@ More information ...@@ -442,32 +559,24 @@ More information
Included extensions Included extensions
=================== ===================
ase-$ase_version, gpaw-$GPAW_version ase-$ase_version, gpaw-$GPAW_version, numpy-$numpy_version, scipy-$scipy_version, cython-$cython_version
]==]) ]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration.]==]) whatis([==[Description: GPAW $GPAW_version with ase $ase_version, numpy $numpy_version and scipty $scipy_version: UEABS benchmark configuration.]==])
conflict("GPAW") conflict("GPAW")
conflict("GPAW-manual") conflict("$packageName")
if not ( isloaded("calcua/2020a") ) then
load("calcua/2020a")
end
-- if not ( isloaded("personal/UEABS") ) then
-- load("personal/UEABS")
-- end
if not ( isloaded("intel/2020a") ) then if not ( isloaded("$compiler_module") ) then
load("intel/2020a") load("$compiler_module")
end end
if not ( isloaded("FFTW/3.3.8-intel-2020a") ) then if not ( isloaded("FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW") ) then
load("FFTW/3.3.8-intel-2020a") load("FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW")
end end
if not ( isloaded("Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW") ) then if not ( isloaded("Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW") ) then
load("Python/3.7.9-intel-2020a-Cfg5-UEABS-GPAW") load("Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW")
end end
prepend_path("PATH", "$install_dir/bin") prepend_path("PATH", "$install_dir/bin")
...@@ -476,5 +585,4 @@ prepend_path("LIBRARY_PATH", "$install_dir/lib") ...@@ -476,5 +585,4 @@ prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages") prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups") setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
setenv("GPAW_FFTWSO", "libfftw3.so")
EOF EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_20.10.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.10.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.20.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.10.0/doc/install.rst
#
packageID='20.10.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.00'
libxc_version='4.3.4'
ase_version='3.20.1'
GPAW_version='20.10.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a.00'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos8/calcua-admin-rome/
module load calcua/admin
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
module load $compiler_module
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
--
-- The order of loading is a deliberate choice. It takes some libraries from
-- the compiler directories rather than from the Python directories and has
-- solved some unexplained issues we had with SLURM.
--
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_20.10.0_Python39_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.10.0:
# * We compile our own Python as this is the best guarantee to not have to
# struggle with compatibility problems between various compilers used for
# various components
# * Using the matching version of ase, 3.20.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.10.0/doc/install.rst
#
packageID='20.10.0-Python39-FFTW-icc'
packageName='GPAW-UEABS'
echo -e "\n### Building $packageName/$packageID from $0\n\n"
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Of course these variables can be set by hand also.
# Note that the UEABS module also completes the MODULEPATH set below.
#
# The next three variables are only used to load the right UEABS module
# and to give example values for variable settings in comments.
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
module purge
MODULEPATH=$install_root/$systemID/Modules
module load UEABS/$UEABS_version
#
# The following UEABS_ variables are needed:
#
# Directory to put the downloaded sources of the packages.
# UEABS_DOWNLOADS=$install_root/Downloads
# Directory where packages should be installed.
# UEABS_PACKAGES=$install_root/$systemID/Packages
# Directory where modules are installed
# UEABS_MODULES=$install_root/$systemID/Modules
install_dir=$UEABS_PACKAGES/$packageName/$packageID
modules_dir=$UEABS_MODULES/$packageName
build_dir="/dev/shm/$USER/$packageName/$packageID"
# Software versions
python_version='3.9.1'
zlib_version='1.2.11'
ncurses_version='6.2'
readline_version='8.0'
sqlite_version='3.33.0'
sqlite_download='3330000'
libffi_version='3.3'
fftw_version='3.3.8'
libxc_version='4.3.4'
wheel_version='0.36.2'
cython_version='0.29.21'
numpy_version='1.19.5'
scipy_version='1.5.4'
ase_version='3.20.1'
GPAW_version='20.10.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.9'
################################################################################
#
# Prepare the system
#
#
# Load modules
# - Compiler module: This is a system-provide module
# - buildtools module: Contains a number of updated build tools already available
# on most systems. We use it to get more consistent builds across systems, but
# the module is not currently included in the UEABS repository. In most cases,
# it will not be needed at all except for more exactly reproducing our test
# results.
#
module load $compiler_module
module load buildtools/$UEABS_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $UEABS_DOWNLOADS
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $UEABS_DOWNLOADS
# zlib: https://www.zlib.net/zlib-1.2.11.tar.gz
zlib_file="zlib-$zlib_version.tar.gz"
zlib_url="https://www.zlib.net"
[[ -f $zlib_file ]] || wget "$zlib_url/$zlib_file"
# ncurses: https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz
ncurses_file="ncurses-$ncurses_version.tar.gz"
ncurses_url="https://ftp.gnu.org/pub/gnu/ncurses"
[[ -f $ncurses_file ]] || wget "$ncurses_url/$ncurses_file"
# readline: https://ftp.gnu.org/pub/gnu/readline/readline-8.0.tar.gz
readline_file="readline-$readline_version.tar.gz"
readline_url="https://ftp.gnu.org/pub/gnu/readline"
[[ -f $readline_file ]] || wget "$readline_url/$readline_file"
# sqlite: https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
sqlite_file="sqlite-autoconf-$sqlite_download.tar.gz"
sqlite_url="https://www.sqlite.org/2020"
[[ -f $sqlite_file ]] || wget "$sqlite_url/$sqlite_file"
# libffi: https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
libffi_file="libffi-$libffi_version.tar.gz"
libffi_url="https://github.com/libffi/libffi/releases/download/v$libffi_version"
[[ -f $libffi_file ]] || wget "$libffi_url/$libffi_file"
# FFTW: http://www.fftw.org/fftw-3.3.8.tar.gz
fftw_file="fftw-$fftw_version.tar.gz"
fftw_url="http://www.fftw.org"
[[ -f $fftw_file ]] || wget "$fftw_url/$fftw_file"
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Python: https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz
python_file="Python-$python_version.tar.xz"
python_url="https://www.python.org/ftp/python/$python_version"
[[ -f $python_file ]] || wget "$python_url/$python_file"
# Downloading wheel so that we can gather all sources for reproducibility.
wheel_file="wheel-$wheel_version.tar.gz"
wheel_url="https://pypi.python.org/packages/source/w/wheel"
[[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install ncurses
#
# We mirror the two-step EasyBuild install process. This may be overkill, but
# we know it works.
#
echo -e "\nInstalling ncurses...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$ncurses_file
cd ncurses-$ncurses_version
# Configure step 1
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CXX=icpc
export CXXFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite
# Build step 1
make -j $parallel
# Install step 1
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Configure step 2
make distclean
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite \
--enable-ext-colors --enable-widec \
--includedir=$install_dir/include/ncursesw/
# Build step 2
make -j $parallel
# Install step 2
make install
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
################################################################################
#
# Install readline
#
echo -e "\nInstalling readline...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$readline_file
cd readline-$readline_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CPPFLAGS="-I$install_dir/include"
export LDFLAGS="-L$install_dir/lib -lncurses"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
unset LDFLAGS
################################################################################
#
# Install zlib
#
echo -e "\nInstalling zlib...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$zlib_file
cd zlib-$zlib_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install libffi
#
echo -e "\nInstalling libffi...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libffi_file
cd libffi-$libffi_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-multi-os-directory
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install SQLite
#
echo -e "\nInstalling SQLite...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$sqlite_file
cd sqlite-autoconf-$sqlite_download
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC -DSQLITE_DISABLE_INTRINSIC"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
################################################################################
#
# Install FFTW
#
echo -e "\nInstalling FFTW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$fftw_file
# Patch the sources to compile with icc
cat >fftw.patch <<EOF
avoid using -no-gcc when compiling FFTW with Intel compilers,
since that fails on CentOS 8
see https://github.com/easybuilders/easybuild-easyconfigs/issues/10932
and https://github.com/FFTW/fftw3/issues/184
--- fftw-3.3.8/configure.orig
+++ fftw-3.3.8/configure
@@ -14861,6 +14861,9 @@
intel) # Stop icc from defining __GNUC__, except on MacOS where this fails
case "\${host_os}" in
*darwin*) ;; # icc -no-gcc fails to compile some system headers
+ # using -no-gcc with recent Intel compilers fails on CentOS 8,
+ # and was only needed as a workaround for old Intel compilers anyway
+ *linux*) ;;
*)
{ \$as_echo "\$as_me:\${as_lineno-\$LINENO}: checking whether C compiler accepts -no-gcc" >&5
$as_echo_n "checking whether C compiler accepts -no-gcc... " >&6; }
EOF
cd fftw-$fftw_version
patch -p1 <../fftw.patch
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export MPICC=mpiicc
export F77=ifort
export F77FLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-sse --disable-sse2 --disable-avx --enable-avx2 --disable-avx512 --disable-avx-128-fma --disable-kcvi \
--enable-fma \
--disable-openmp --disable-threads \
--enable-mpi \
-disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset MPICC
unset F77
unset F77FLAGS
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make -j install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install Python
#
echo -e "\nInstalling Python...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$python_file
cd Python-$python_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC"
export LD=icc
export LDFLAGS="-L$install_dir/lib"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir" \
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
--enable-shared --disable-ipv6 \
--enable-optimizations \
--with-ensurepip=upgrade
# --with-icc \
# Build
make -j 32
# Install
make install
cd $install_dir/bin
ln -s python$py_maj_min python
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset LD
unset LDFLAGS
unset CPPFLAGS
################################################################################
#
# Initialising for installing Python packages
#
echo -e "\nInitialising for installing Python packages...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
################################################################################
#
# Install wheel
#
echo -e "\nInstalling wheel...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $UEABS_DOWNLOADS/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $install_dir/lib:$MKLROOT/lib/intel64_lin
include_dirs = $install_dir/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $UEABS_DOWNLOADS/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$GPAW_file
cd gpaw-$GPAW_version
# Make the siteconfig.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module $packageName/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Minimal use of system libraries or libraries that are installed
through package managers such as EasyBuild or Spack.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* Python dependencies:
* zlib $zlib_version
* ncurses $ncurses_version
* readline $readline_version
* SQLite $sqlite_version
* libffi $libffi_version
* FFTW $fftw_version
* libxc $libxc_version
* Python $python_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration with Python $python_version, numpy-$numpy_version, scipy-$scipy_version, libxc $libxc_version and FFTW $fftw_version.]==])
conflict("GPAW")
conflict("$packageName")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_20.10.0_Python39icc_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.10.0:
# * Using a Python and FFTW module compiled with EasyBuild, but libxc and
# NumPy/SciPy compiled by hand.
# * Using the matching version of ase, 3.20.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.10.0/doc/install.rst
#
#set -e
packageID='20.10.0-Python39icc-FFTW-icc'
packageName='GPAW-UEABS'
echo -e "\n### Building $packageName/$packageID from $0\n\n"
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Of course these variables can be set by hand also.
# Note that the UEABS module also completes the MODULEPATH set below.
#
# The next three variables are only used to load the right UEABS module
# and to give example values for variable settings in comments.
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
module purge
MODULEPATH=$install_root/$systemID/Modules
module load UEABS/$UEABS_version
#
# The following UEABS_ variables are needed:
#
# Directory to put the downloaded sources of the packages.
# UEABS_DOWNLOADS=$install_root/Downloads
# Directory where packages should be installed.
# UEABS_PACKAGES=$install_root/$systemID/Packages
# Directory where modules are installed
# UEABS_MODULES=$install_root/$systemID/Modules
install_dir=$UEABS_PACKAGES/$packageName/$packageID
modules_dir=$UEABS_MODULES/$packageName
build_dir="/dev/shm/$USER/$packageName/$packageID"
# Software versions
python_version='3.9.1'
fftw_version='3.3.8'
libxc_version='4.3.4'
#wheel_version='0.35.1'
cython_version='0.29.21'
numpy_version='1.19.5'
scipy_version='1.5.4'
ase_version='3.20.1'
GPAW_version='20.10.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.9'
################################################################################
#
# Prepare the system
#
#
# Load modules
# - Compiler module: This is a system-provide module
# - buildtools module: Contains a number of updated build tools already available
# on most systems. We use it to get more consistent builds across systems, but
# the module is not currently included in the UEABS repository. In most cases,
# it will not be needed at all except for more exactly reproducing our test
# results.
# - An EasyBuild-generated module containing a suitable FFTW implementation.
# - An EasyBuild-generated module with Python 3.9 and the basic installation tools
# (pip, setuptools, wheel)
# In some cases we used modules build specifically for this project, but this was
# to determine the minimum set of libraries etc. that are needed. Modules provided
# on the system would have done the job as well.
#
module load $compiler_module
module load buildtools/$UEABS_version
module load FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW
module load Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $UEABS_DOWNLOADS
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $UEABS_DOWNLOADS
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# As wheel is already present in the Python module we don't need to download it again.
# Downloading wheel so that we can gather all sources for reproducibility.
# wheel_file="wheel-$wheel_version.tar.gz"
# wheel_url="https://pypi.python.org/packages/source/w/wheel"
# [[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Initialising for installing Python packages
#
echo -e "\nInitialising for installing Python packages...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
################################################################################
#
# No need to install wheel as it would erase the wheel version already present in
# the Python module.
#
echo -e "\nNot installing wheel...\n"
#cd $build_dir
#pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $UEABS_DOWNLOADS/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $EBROOTFFTW/lib:$MKLROOT/lib/intel64_lin
include_dirs = $EBROOTFFTW/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $UEABS_DOWNLOADS/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$GPAW_file
cd gpaw-$GPAW_version
# Make the siteconfig.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module $packageName/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Python, FFTW and the dependencies of Python build with EasyBuild.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* EasyBuild modules (special build for the benchmarks, not taken from the system)
* FFTW $fftw_version
* Python $python_version
* libxc $libxc_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
Included extensions
===================
ase-$ase_version, gpaw-$GPAW_version, numpy-$numpy_version, scipy-$scipy_version, cython-$cython_version
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, numpy $numpy_version and scipty $scipy_version: UEABS benchmark configuration.]==])
conflict("GPAW")
conflict("$packageName")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
if not ( isloaded("FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW") ) then
load("FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW")
end
if not ( isloaded("Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW") ) then
load("Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_21.1.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 21.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.21.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/21.1.0/doc/install.rst
#
packageID='21.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.00'
libxc_version='4.3.4'
ase_version='3.21.1'
GPAW_version='21.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a.00'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos8/calcua-admin-rome/
module load calcua/admin
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
module load $compiler_module
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
--
-- The order of loading is a deliberate choice. It takes some libraries from
-- the compiler directories rather than from the Python directories and has
-- solved some unexplained issues we had with SLURM.
--
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_21.1.0_Python39_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 21.1.0:
# * We compile our own Python as this is the best guarantee to not have to
# struggle with compatibility problems between various compilers used for
# various components
# * Using the matching version of ase, 3.21.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/21.1.0/doc/install.rst
#
packageID='21.1.0-Python39-FFTW-icc'
packageName='GPAW-UEABS'
echo -e "\n### Building $packageName/$packageID from $0\n\n"
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Of course these variables can be set by hand also.
# Note that the UEABS module also completes the MODULEPATH set below.
#
# The next three variables are only used to load the right UEABS module
# and to give example values for variable settings in comments.
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
module purge
MODULEPATH=$install_root/$systemID/Modules
module load UEABS/$UEABS_version
#
# The following UEABS_ variables are needed:
#
# Directory to put the downloaded sources of the packages.
# UEABS_DOWNLOADS=$install_root/Downloads
# Directory where packages should be installed.
# UEABS_PACKAGES=$install_root/$systemID/Packages
# Directory where modules are installed
# UEABS_MODULES=$install_root/$systemID/Modules
install_dir=$UEABS_PACKAGES/$packageName/$packageID
modules_dir=$UEABS_MODULES/$packageName
build_dir="/dev/shm/$USER/$packageName/$packageID"
# Software versions
python_version='3.9.1'
zlib_version='1.2.11'
ncurses_version='6.2'
readline_version='8.0'
sqlite_version='3.33.0'
sqlite_download='3330000'
libffi_version='3.3'
fftw_version='3.3.8'
libxc_version='4.3.4'
wheel_version='0.36.2'
cython_version='0.29.21'
numpy_version='1.19.5'
scipy_version='1.5.4'
ase_version='3.21.1'
GPAW_version='21.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.9'
################################################################################
#
# Prepare the system
#
#
# Load modules
# - Compiler module: This is a system-provide module
# - buildtools module: Contains a number of updated build tools already available
# on most systems. We use it to get more consistent builds across systems, but
# the module is not currently included in the UEABS repository. In most cases,
# it will not be needed at all except for more exactly reproducing our test
# results.
#
module load $compiler_module
module load buildtools/$UEABS_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $UEABS_DOWNLOADS
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $UEABS_DOWNLOADS
# zlib: https://www.zlib.net/zlib-1.2.11.tar.gz
zlib_file="zlib-$zlib_version.tar.gz"
zlib_url="https://www.zlib.net"
[[ -f $zlib_file ]] || wget "$zlib_url/$zlib_file"
# ncurses: https://ftp.gnu.org/pub/gnu/ncurses/ncurses-6.2.tar.gz
ncurses_file="ncurses-$ncurses_version.tar.gz"
ncurses_url="https://ftp.gnu.org/pub/gnu/ncurses"
[[ -f $ncurses_file ]] || wget "$ncurses_url/$ncurses_file"
# readline: https://ftp.gnu.org/pub/gnu/readline/readline-8.0.tar.gz
readline_file="readline-$readline_version.tar.gz"
readline_url="https://ftp.gnu.org/pub/gnu/readline"
[[ -f $readline_file ]] || wget "$readline_url/$readline_file"
# sqlite: https://www.sqlite.org/2020/sqlite-autoconf-3330000.tar.gz
sqlite_file="sqlite-autoconf-$sqlite_download.tar.gz"
sqlite_url="https://www.sqlite.org/2020"
[[ -f $sqlite_file ]] || wget "$sqlite_url/$sqlite_file"
# libffi: https://github.com/libffi/libffi/releases/download/v3.3/libffi-3.3.tar.gz
libffi_file="libffi-$libffi_version.tar.gz"
libffi_url="https://github.com/libffi/libffi/releases/download/v$libffi_version"
[[ -f $libffi_file ]] || wget "$libffi_url/$libffi_file"
# FFTW: http://www.fftw.org/fftw-3.3.8.tar.gz
fftw_file="fftw-$fftw_version.tar.gz"
fftw_url="http://www.fftw.org"
[[ -f $fftw_file ]] || wget "$fftw_url/$fftw_file"
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Python: https://www.python.org/ftp/python/3.7.9/Python-3.7.9.tar.xz
python_file="Python-$python_version.tar.xz"
python_url="https://www.python.org/ftp/python/$python_version"
[[ -f $python_file ]] || wget "$python_url/$python_file"
# Downloading wheel so that we can gather all sources for reproducibility.
wheel_file="wheel-$wheel_version.tar.gz"
wheel_url="https://pypi.python.org/packages/source/w/wheel"
[[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install ncurses
#
# We mirror the two-step EasyBuild install process. This may be overkill, but
# we know it works.
#
echo -e "\nInstalling ncurses...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$ncurses_file
cd ncurses-$ncurses_version
# Configure step 1
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CXX=icpc
export CXXFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite
# Build step 1
make -j $parallel
# Install step 1
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Configure step 2
make distclean
./configure --prefix="$install_dir" \
--with-shared --enable-overwrite \
--enable-ext-colors --enable-widec \
--includedir=$install_dir/include/ncursesw/
# Build step 2
make -j $parallel
# Install step 2
make install
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
################################################################################
#
# Install readline
#
echo -e "\nInstalling readline...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$readline_file
cd readline-$readline_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CPPFLAGS="-I$install_dir/include"
export LDFLAGS="-L$install_dir/lib -lncurses"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
unset LDFLAGS
################################################################################
#
# Install zlib
#
echo -e "\nInstalling zlib...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$zlib_file
cd zlib-$zlib_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install libffi
#
echo -e "\nInstalling libffi...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libffi_file
cd libffi-$libffi_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-multi-os-directory
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install SQLite
#
echo -e "\nInstalling SQLite...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$sqlite_file
cd sqlite-autoconf-$sqlite_download
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC -DSQLITE_DISABLE_INTRINSIC"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir"
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset CPPFLAGS
################################################################################
#
# Install FFTW
#
echo -e "\nInstalling FFTW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$fftw_file
# Patch the sources to compile with icc
cat >fftw.patch <<EOF
avoid using -no-gcc when compiling FFTW with Intel compilers,
since that fails on CentOS 8
see https://github.com/easybuilders/easybuild-easyconfigs/issues/10932
and https://github.com/FFTW/fftw3/issues/184
--- fftw-3.3.8/configure.orig
+++ fftw-3.3.8/configure
@@ -14861,6 +14861,9 @@
intel) # Stop icc from defining __GNUC__, except on MacOS where this fails
case "\${host_os}" in
*darwin*) ;; # icc -no-gcc fails to compile some system headers
+ # using -no-gcc with recent Intel compilers fails on CentOS 8,
+ # and was only needed as a workaround for old Intel compilers anyway
+ *linux*) ;;
*)
{ \$as_echo "\$as_me:\${as_lineno-\$LINENO}: checking whether C compiler accepts -no-gcc" >&5
$as_echo_n "checking whether C compiler accepts -no-gcc... " >&6; }
EOF
cd fftw-$fftw_version
patch -p1 <../fftw.patch
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export MPICC=mpiicc
export F77=ifort
export F77FLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-sse --disable-sse2 --disable-avx --enable-avx2 --disable-avx512 --disable-avx-128-fma --disable-kcvi \
--enable-fma \
--disable-openmp --disable-threads \
--enable-mpi \
-disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Clean-up
unset CC
unset CFLAGS
unset MPICC
unset F77
unset F77FLAGS
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make -j install
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Install Python
#
echo -e "\nInstalling Python...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$python_file
cd Python-$python_version
# Configure
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC"
export LD=icc
export LDFLAGS="-L$install_dir/lib"
export CPPFLAGS="-I$install_dir/include"
./configure --prefix="$install_dir" \
--build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu \
--enable-shared --disable-ipv6 \
--enable-optimizations \
--with-ensurepip=upgrade
# --with-icc \
# Build
make -j 32
# Install
make install
cd $install_dir/bin
ln -s python$py_maj_min python
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset LD
unset LDFLAGS
unset CPPFLAGS
################################################################################
#
# Initialising for installing Python packages
#
echo -e "\nInitialising for installing Python packages...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
################################################################################
#
# Install wheel
#
echo -e "\nInstalling wheel...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $UEABS_DOWNLOADS/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $install_dir/lib:$MKLROOT/lib/intel64_lin
include_dirs = $install_dir/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $UEABS_DOWNLOADS/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$GPAW_file
cd gpaw-$GPAW_version
# Make the siteconfig.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module $packageName/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Minimal use of system libraries or libraries that are installed
through package managers such as EasyBuild or Spack.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* Python dependencies:
* zlib $zlib_version
* ncurses $ncurses_version
* readline $readline_version
* SQLite $sqlite_version
* libffi $libffi_version
* FFTW $fftw_version
* libxc $libxc_version
* Python $python_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration with Python $python_version, numpy-$numpy_version, scipy-$scipy_version, libxc $libxc_version and FFTW $fftw_version.]==])
conflict("GPAW")
conflict("$packageName")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/build_21.1.0_Python39icc_FFTW_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 21.1.0:
# * Using a Python and FFTW module compiled with EasyBuild, but libxc and
# NumPy/SciPy compiled by hand.
# * Using the matching version of ase, 3.21.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/21.1.0/doc/install.rst
#
#set -e
packageID='21.1.0-Python39icc-FFTW-icc'
packageName='GPAW-UEABS'
echo -e "\n### Building $packageName/$packageID from $0\n\n"
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Of course these variables can be set by hand also.
# Note that the UEABS module also completes the MODULEPATH set below.
#
# The next three variables are only used to load the right UEABS module
# and to give example values for variable settings in comments.
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
module purge
MODULEPATH=$install_root/$systemID/Modules
module load UEABS/$UEABS_version
#
# The following UEABS_ variables are needed:
#
# Directory to put the downloaded sources of the packages.
# UEABS_DOWNLOADS=$install_root/Downloads
# Directory where packages should be installed.
# UEABS_PACKAGES=$install_root/$systemID/Packages
# Directory where modules are installed
# UEABS_MODULES=$install_root/$systemID/Modules
install_dir=$UEABS_PACKAGES/$packageName/$packageID
modules_dir=$UEABS_MODULES/$packageName
build_dir="/dev/shm/$USER/$packageName/$packageID"
# Software versions
python_version='3.9.1'
fftw_version='3.3.8'
libxc_version='4.3.4'
#wheel_version='0.35.1'
cython_version='0.29.21'
numpy_version='1.19.5'
scipy_version='1.5.4'
ase_version='3.21.1'
GPAW_version='21.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.9'
################################################################################
#
# Prepare the system
#
#
# Load modules
# - Compiler module: This is a system-provide module
# - buildtools module: Contains a number of updated build tools already available
# on most systems. We use it to get more consistent builds across systems, but
# the module is not currently included in the UEABS repository. In most cases,
# it will not be needed at all except for more exactly reproducing our test
# results.
# - An EasyBuild-generated module containing a suitable FFTW implementation.
# - An EasyBuild-generated module with Python 3.9 and the basic installation tools
# (pip, setuptools, wheel)
# In some cases we used modules build specifically for this project, but this was
# to determine the minimum set of libraries etc. that are needed. Modules provided
# on the system would have done the job as well.
#
module load $compiler_module
module load buildtools/$UEABS_version
module load FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW
module load Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $UEABS_DOWNLOADS
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $UEABS_DOWNLOADS
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# As wheel is already present in the Python module we don't need to download it again.
# Downloading wheel so that we can gather all sources for reproducibility.
# wheel_file="wheel-$wheel_version.tar.gz"
# wheel_url="https://pypi.python.org/packages/source/w/wheel"
# [[ -f $wheel_file ]] || wget "$wheel_url/$wheel_file"
# Downloading Cython so that we can gather all sources for reproducibility.
cython_file="Cython-$cython_version.tar.gz"
cython_url="https://pypi.python.org/packages/source/c/cython"
[[ -f $cython_file ]] || wget "$cython_url/$cython_file"
# NumPy needs customizations, so we need to download and unpack the sources
numpy_file="numpy-$numpy_version.zip"
numpy_url="https://pypi.python.org/packages/source/n/numpy"
[[ -f $numpy_file ]] || wget "$numpy_url/$numpy_file"
# SciPy
scipy_file="scipy-$scipy_version.tar.gz"
scipy_url="https://pypi.python.org/packages/source/s/scipy"
[[ -f $scipy_file ]] || wget "$scipy_url/$scipy_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Initialising for installing Python packages
#
echo -e "\nInitialising for installing Python packages...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
cd $install_dir
/usr/bin/ln -s lib lib64
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages"
################################################################################
#
# No need to install wheel as it would erase the wheel version already present in
# the Python module.
#
echo -e "\nNot installing wheel...\n"
#cd $build_dir
#pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$wheel_file
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir pytest
################################################################################
#
# Install Cython
#
echo -e "\nInstalling Cython...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$cython_file
################################################################################
#
# Install NumPy
#
echo -e "\nInstalling NumPy...\n"
cd $build_dir
# Uncompress
unzip $UEABS_DOWNLOADS/$numpy_file
cd numpy-$numpy_version
cat >site.cfg <<EOF
[DEFAULT]
library_dirs = $EBROOTFFTW/lib:$MKLROOT/lib/intel64_lin
include_dirs = $EBROOTFFTW/include:$MKLROOT/include
search_static_first=True
[mkl]
lapack_libs = mkl_rt
mkl_libs = mkl_rt
[fftw]
libraries = fftw3
EOF
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438,2650,3175,1890"
export CXX=icpc
export FC=ifort
export F90=$FC
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build NumPy.
# Note: Try python setup.py build --help-compiler and python setup.py build --help-fcompiler to list the
# choice of compilers. We did experience though that if the Intel compilers are selected explicitly,
# the wrong linker was used leading to problems there.
# Parallel build seems dangereous? We got some random failures.
python$py_maj_min setup.py config
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install NumPy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import numpy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install SciPy
#
echo -e "\nInstalling SciPy...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$scipy_file
cd scipy-$scipy_version
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fwrapv -fPIC -diag-disable=1678,10148,111,188,3438"
export CXX=icpc
export FC=ifort
export F90=$FC
# FFLAGS is used in the fitpack Makefile
export FFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
# FOPT is used in the odrpack Makefile
export FOPT="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export LD=icc
# Build scipy
# SciPy contains Fortran code so a parallel build may be dangerous.
python$py_maj_min setup.py build --compiler=intel --fcompiler=intelem
# Install scipy
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test: Should not be run in the NumPy build directory...
cd $build_dir
python$py_maj_min -c "import scipy"
# Clean-up
unset CC
unset CFLAGS
unset CXX
unset FC
unset F90
unset FFLAGS
unset FOPT
unset LD
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
cd $build_dir
pip$py_maj_min install --prefix=$install_dir --no-deps $UEABS_DOWNLOADS/$ase_file
#pip install --prefix=$install_dir --no-deps ase==$ase_version
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $UEABS_DOWNLOADS/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $UEABS_DOWNLOADS/$GPAW_file
cd gpaw-$GPAW_version
# Make the siteconfig.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use regular FFTW
fftw = True
libraries += ['fftw3']
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
#python$py_maj_min setup.py install --prefix="$install_dir"
pip$py_maj_min install --prefix $install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module $packageName/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* Using FFTW for the FFT computations and MKL for the BLAS/Lapack
* Python, FFTW and the dependencies of Python build with EasyBuild.
Detailed configuration:
* Compiler module: $compiler_module, used for all packages
* EasyBuild modules (special build for the benchmarks, not taken from the system)
* FFTW $fftw_version
* Python $python_version
* libxc $libxc_version
* Important Python packages: numpy-$numpy_version, scipy-$scipy_version, ase-$ase_version, gpaw-$GPAW_version
* Additional Python packages: wheel-$wheel_version, cython-$cython_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
Included extensions
===================
ase-$ase_version, gpaw-$GPAW_version, numpy-$numpy_version, scipy-$scipy_version, cython-$cython_version
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, numpy $numpy_version and scipty $scipy_version: UEABS benchmark configuration.]==])
conflict("GPAW")
conflict("$packageName")
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
if not ( isloaded("FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW") ) then
load("FFTW/$fftw_version-intel-2020a-Cfg5-UEABS-GPAW")
end
if not ( isloaded("Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW") ) then
load("Python/$python_version-intel-2020a-Cfg5-UEABS-GPAW")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/README.md 0 → 100644
View file @ b57ca544
# Example build scripts
## CalcUA-vaughan-rome
Vaughan is a cluster installed at the University of Antwerpen
as one of the VSC (Vlaams Supercomputer Centrum) Tier-2 clusters.
The cluster has 144 nodes with dual 32-core AMD EPYC 7452 "Rome" CPUs and 256GB of RAM memory
per node. Nodes are interconnected through an InfiniBand HDR100 network. The system
does not contain accelerators. The cluster uses CentOS 8 as the
operating system.
Most software on the cluster is installed through [EasyBuild](https://easybuilders.github.io/).
Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
[Go to the directory](CalcUA-vaughan-rome)
## CalcUA-leibniz/broadwell
Leibniz is a cluster intalled at the University of Antwerpen
as one of the VSC (Vlaams Supercomputer Centrum) Tier-2 clusters.
The cluster has 153 regular CPU nodes with dual Intel Xeon E5-2680v4
"broadwell" CPUs with 128 or 256 GB of RAM memory per node.
Nodes are interconnected through an Infiniband EDR network.
The system has two nodes with dual NVIDIA P100 GPUs and a node
with dual NEC Aurora TSUBASA first generation vector boards.
During the testing period the cluster used CentOS 7 as the
operating system.
Most software on the cluster is installed through [EasyBuild](https://easybuilders.github.io/).
Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
## davide @ Cineca
Build instructions for GPU and CPU on the DAVIDE OpenPower GPU cluster @ Cineca.
These instructions haven't been tested recently. They are still for a version
based on distutils rather then setuptools that was dropped towards the end of
the development of this version of the benchmark suite.
## juwels @ Jülich Supercomputing Centre
Build instructions for the CPU nodes of JUWELS.
These instructions haven't been tested recently. They are still for a version
based on distutils rather then setuptools that was dropped towards the end of
the development of this version of the benchmark suite.
## piz-daint @ CSCS
Build instructions for the Piz Daint (GPU cluster).
These instructions haven't been tested recently. They are still for a version
based on distutils rather then setuptools that was dropped towards the end of
the development of this version of the benchmark suite.