Skip to content
Commits on Source (6)
Expand all Show whitespace changes
Inline Side-by-side
gpaw/README.md
View file @ b57ca544
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
## Summary version ## Summary version
0.1 0.9
## Purpose of the benchmark ## Purpose of the benchmark
...@@ -37,16 +37,23 @@ There is currently no active support for non-CUDA accelerator platforms. ...@@ -37,16 +37,23 @@ There is currently no active support for non-CUDA accelerator platforms.
For the UEABS benchmark version 2.2, the following versions of GPAW were tested: For the UEABS benchmark version 2.2, the following versions of GPAW were tested:
* CPU-based: * CPU-based:
* Version 1.5.2 as this one is the last of the 1.5 branch and since the GPU version * Version 20.1.0, as this one is the version on which the most recent GPU commits
is derived from this version. are based.
* Version 20.1.0, the most recent version during the development of the UEABS * Version 20.10.0, as it was the most recent version during the development of
2.2 benchmark suite. the IEABS 2.2. benchmark suite.
* GPU-based: There is no official release or version number. The UEABS 2.2 benchmark * GPU-based: As there is no official release of the GPU version and as it is
suite was tested using commit TODO of at the moment of the release of the UEABS version 2.2 under heavy development
[the cuda branch of the GitLab for CUDA development](https://gitlab.com/mlouhivu/gpaw/tree/cuda). to also support AMD GPUs, there is no official support for the GPU version
([the cuda branch of the GitLab for CUDA development](https://gitlab.com/mlouhivu/gpaw/tree/cuda))
in UEABS version 2.2.
Versions 1.5.2 and 19.8.1 were also considered but are not compatible with the regular
input files provided here. Hence support for those versions of GPAW was dropped in
this version of the UEABS.
There are three benchmark cases, denotes S, M and L. There are three benchmark cases, denotes S, M and L.
### Case S: Carbon nanotube ### Case S: Carbon nanotube
A ground state calculation for a carbon nanotube in vacuum. By default uses a A ground state calculation for a carbon nanotube in vacuum. By default uses a
...@@ -57,21 +64,25 @@ on 100 cores of a modern x86 cluster. ...@@ -57,21 +64,25 @@ on 100 cores of a modern x86 cluster.
Input file: [benchmark/1_S_carbon-nanotube/input.py](benchmark/1_S_carbon-nanotube/input.py) Input file: [benchmark/1_S_carbon-nanotube/input.py](benchmark/1_S_carbon-nanotube/input.py)
This input file still works with version 1.5.2 and 19.8.1 of GPAW.
### Case M: Copper filament ### Case M: Copper filament
A ground state calculation for a copper filament in vacuum. By default uses a A ground state calculation for a copper filament in vacuum. By default uses a
3x4x4 FCC lattice with 71 atoms (freely adjustable through the variables `x`, 3x4x4 FCC lattice with 71 atoms (freely adjustable through the variables `x`,
`y` and `z` in the input file) and ScaLAPACK for `y` and `z` in the input file) and ScaLAPACK for
parallellisation. Expected to scale up to 100 nodes and/or 1000 MPI tasks. parallelisation. Expected to scale up to 100 nodes and/or 1000 MPI tasks.
Input file: [benchmark/2_M_copper-filament/input.py](benchmark/2_M_copper-filament/input.py) Input file: [benchmark/2_M_copper-filament/input.py](benchmark/2_M_copper-filament/input.py)
The benchmark was tested using 1000 and 1024 cores. For some core configurations, one may This input file does not work with GPAW 1.5.2 and 19.8.1. It requires GPAW
get error messages similar to ``gpaw.grid_descriptor.BadGridError: Grid ... to small 20.1.0 or 20.10.0. Please try older versions of the UEABS if you want to use
for ... cores``. If one really wants to run the benchmark for those number of cores, these versions of GPAW.
one needs to adapt the values of `x`, `y` and `z` in `input.py`. However, this
changes the benchmark so results cannot be compared easily with benchmark runs for The benchmark runs best when using full nodes. Expect a
different values of these variables. performance drop on other configurations.
### Case L: Silicon cluster ### Case L: Silicon cluster
...@@ -82,10 +93,16 @@ and/or 10000 MPI tasks. ...@@ -82,10 +93,16 @@ and/or 10000 MPI tasks.
Input file: [benchmark/3_L_silicon-cluster/input.py](benchmark/3_L_silicon-cluster/input.py) Input file: [benchmark/3_L_silicon-cluster/input.py](benchmark/3_L_silicon-cluster/input.py)
This input file does not work with GPAW 1.5.2 and 19.8.1. It requires GPAW
20.1.0 or 20.10.0. Please try older versions of the UEABS if you want to use
these versions of GPAW.
## Mechanics of building the benchmark ## Mechanics of building the benchmark
Note that GPAW version numbering changed in 2019. Version 1.5.3 is the Installing and running GPAW has changed a lot in the since the previous
versions of the UEABS. GPAW version numbering changed in 2019. Version 1.5.3 is the
last version with the old numbering. In 2019 the development team switched last version with the old numbering. In 2019 the development team switched
to a version numbering scheme based on year, month and patchlevel, e.g., to a version numbering scheme based on year, month and patchlevel, e.g.,
19.8.1 for the second version released in August 2019. 19.8.1 for the second version released in August 2019.
...@@ -94,16 +111,11 @@ Another change is in the Python packages used to install GPAW. Versions up to ...@@ -94,16 +111,11 @@ Another change is in the Python packages used to install GPAW. Versions up to
and including 19.8.1 use the `distutils` package while versions 20.1.0 and later and including 19.8.1 use the `distutils` package while versions 20.1.0 and later
are based on `setuptools`. This does affect the installation process. are based on `setuptools`. This does affect the installation process.
GPAW for a while supports two different ways to run in parallel distributed memory mode: Running GPAW is no longer done via a wrapper executable `gpaw-python` that
* Using a wrapper executable `gpaw-python` that replaces the Python interpreter (it internally replaces the Python interpreter (it internally links to the libpython library)
links to the libpython library) and that provides the MPI functionality. and that provides the MPI functionality. Since version 20.1.0, the standard Python
* Using the standard Python interpreter, including the MPI functionality in the interpreter is used and the MPI functionality is included in the `_gpaw.so` shared library.
`_gpaw.so` shared library.
In the `distutils`-based versions, the wrapper script approach is the default behaviour,
while in the `setuptools`-based versions, the approach using the standard Python interpreter
is the preferred one in the manual. Even though the code in the `setuptools`-based
versions still includes the option to use the wrapper script approach, it does not
work in the tested version 20.1.0.
### Available instructions ### Available instructions
...@@ -128,26 +140,23 @@ Hence GPAW has the following requirements: ...@@ -128,26 +140,23 @@ Hence GPAW has the following requirements:
* BLAS, LAPACK, BLACS and ScaLAPACK. ScaLAPACK is optional for GPAW, but mandatory * BLAS, LAPACK, BLACS and ScaLAPACK. ScaLAPACK is optional for GPAW, but mandatory
for the UEABS benchmarks. It is used by the medium and large cases and optional for the UEABS benchmarks. It is used by the medium and large cases and optional
for the small case. for the small case.
* Python. GPAW 1.5.2 requires * Python. GPAW 20.1.0 requires Python 3.5-3.8 and GPAW 20.10.0 Python 3.6-3.9.
Python 2.7 or 3.4-3.7, GPAW 19.8.1 requires 3.4-3.7, GPAW 20.1.0 Python 3.5-3.8
and GPAW 20.10.0 Python 3.6-3.9.
* Mandatory Python packages: * Mandatory Python packages:
* [NumPY](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) * [NumPY](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 20.1.0/20.10.0)
* [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) GPAW versions before 20.10.0 produce warnings when used with NumPy 1.19.x.
* [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 20.1.0/20.10.0)
* [FFTW](http://www.fftw.org) is highly recommended. As long as the optional libvdwxc * [FFTW](http://www.fftw.org) is highly recommended. As long as the optional libvdwxc
component is not used, the MKL FFTW wrappers can also be used. Recent versions of component is not used, the MKL FFTW wrappers can also be used. Recent versions of
GPAW also show good performance using just the NumPy-provided FFT routines provided GPAW also show good performance using just the NumPy-provided FFT routines provided
that NumPy has been built with a highly optimized FFT library. that NumPy has been built with a highly optimized FFT library.
* [LibXC](https://www.tddft.org/programs/libxc/) 2.X or newer for GPAW 1.5.2, * [LibXC](https://www.tddft.org/programs/libxc/) 3.X or 4.X for GPAW 20.1.0 and 20.10.0.
3.X or 4.X for GPAW 19.8.1, 20.1.0 and 20.10.0. LibXC is a library LibXC is a library of exchange-correlation functions for density-functional theory.
of exchange-correlation functions for density-functional theory. None of the None of the versions currently mentions LibXC 5.X as officially supported.
versions currently mentions LibXC 5.X as officially supported.
* [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package * [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package
from the same group that develops GPAW from the same group that develops GPAW
* Check the release notes of GPAW as the releases of ASE and GPAW should match. * Check the release notes of GPAW as the releases of ASE and GPAW should match.
E.g., during the development of the UEABS version 2.2 benchamark suite, The benchmarks were tested using ASE 3.19.3 with GPAW 20.1.0 and ASE 3.20.1
version 20.1.0 was the most up-to-date release of GPAW with 3.19.1 the matching ASE version with GPAW 20.1.0.
(though 3.18.0 should also work).
* ASE has some optional dependencies that are not needed for the benchmarking: Matplotlib (2.0.0 or newer), * ASE has some optional dependencies that are not needed for the benchmarking: Matplotlib (2.0.0 or newer),
tkinter (Tk interface, part of the Standard Python Library) and Flask. tkinter (Tk interface, part of the Standard Python Library) and Flask.
* Optional components of GPAW that are not used by the UEABS benchmarks: * Optional components of GPAW that are not used by the UEABS benchmarks:
...@@ -194,17 +203,15 @@ git clone -b cuda https://gitlab.com/mlouhivu/gpaw.git ...@@ -194,17 +203,15 @@ git clone -b cuda https://gitlab.com/mlouhivu/gpaw.git
### Install ### Install
Crucial for the configuration of GPAW is a proper `customize.py` (GPAW 19.8.1 and Crucial for the configuration of GPAW is a proper `siteconfig.py` file (GPAW 20.1.0 and later,
earlier) or `siteconfig.py` (GPAW 20.1.0 and later) file. The defaults used by GPAW earlier versions used `customize.py`). The defaults used by GPAW
may not offer optimal performance and the automatic detection of the libraries also may not offer optimal performance and the automatic detection of the libraries also
fails on some systems. fails on some systems.
The UEABS repository contains additional instructions: The UEABS repository contains additional instructions:
* [general instructions](build/build-cpu.md) * [general instructions](build/build-cpu.md)
* [GPGPUs](build/build-cuda.md) - To check
Example [build scripts](build/examples/) are also available for some PRACE and non-PRACE Example [build scripts](build/examples/) are also available.
systems.
## Mechanics of Running the Benchmark ## Mechanics of Running the Benchmark
...@@ -221,26 +228,9 @@ right from this repository. ...@@ -221,26 +228,9 @@ right from this repository.
### Running the benchmarks ### Running the benchmarks
#### Using the `gpaw-python` wrapper script These instructions are exclusively for GPAW 20.1.0 and later.
This is the default approach for versions up to and including 19.8.1 of GPAW
These versions of GPAW come with their own wrapper executable, `gpaw-python`, There are two different ways to start GPAW.
to start a MPI-based GPAW run.
No special command line options or environment variables are needed to run the
benchmarks if your MPI process starter (`mpirun`, Slurm `srun`, ...) communicates
properly with the resource manager. E.g., on Slurm systems, use
```
srun gpaw-python input.py
```
#### Using the regular Python interpreter and parallel GPAW shared library
This is the default method for GPAW 20.1.0 (and likely later).
The wrapper executable `gpaw-python` is no longer available in the default parallel
build of GPAW. There are now two different ways to start GPAW.
One way is through `mpirun`, `srun` or an equivalent process starter and the One way is through `mpirun`, `srun` or an equivalent process starter and the
`gpaw python` command: `gpaw python` command:
...@@ -263,13 +253,6 @@ That option however doesn't do the imports in the same way that the `gpaw` scrip ...@@ -263,13 +253,6 @@ That option however doesn't do the imports in the same way that the `gpaw` scrip
would do. would do.
### Examples
Example [job scripts](scripts/) (`scripts/job-*.sh`) are provided for
different PRACE systems that may offer a helpful starting point.
*TODO: Update the examples as testing on other systems goes on.*
## Verification of Results ## Verification of Results
### Case S: Carbon nanotube ### Case S: Carbon nanotube
......
gpaw/build/build-CPU.md
View file @ b57ca544
...@@ -7,11 +7,11 @@ These instructions are in addition to the brief instructions in [README.md](../R ...@@ -7,11 +7,11 @@ These instructions are in addition to the brief instructions in [README.md](../R
### Libraries and Python interpreter ### Libraries and Python interpreter
GPAW needs (for the UEABS benchmarks) GPAW needs (for the UEABS benchmarks)
* [Python](https://www.python.org/): GPAW 1.5.2 supports Python 2.7 and 3.4-3.7. * [Python](https://www.python.org/): GPAW 20.1.0 requires Python 3.5-3.8, and
GPAW 19.8.1 needs Python 3.4-3.7 and GPAW 20.1.0 requires Python 3.5-3.8. GPAW 20.10.0 and 21.1.0 require Python 3.6-3.9.
* [MPI library](https://www.mpi-forum.org/) * [MPI library](https://www.mpi-forum.org/)
* [LibXC](https://www.tddft.org/programs/libxc/). GPAW 1.5.2 requires LibXC 1.5.2 * [LibXC](https://www.tddft.org/programs/libxc/). GPAW 20.1.0,
or later. GPAW 19.8.1 and 20.1.0 need LibXC 3.x or 4.x. 20.10.0 and 21.1.0 all need LibXC 3.x or 4.x.
* (Optimized) [BLAS](http://www.netlib.org/blas/) and * (Optimized) [BLAS](http://www.netlib.org/blas/) and
[LAPACK](http://www.netlib.org/lapack/) libraries. [LAPACK](http://www.netlib.org/lapack/) libraries.
There are both commercial and free and open source versions of these libraries. There are both commercial and free and open source versions of these libraries.
...@@ -48,15 +48,15 @@ Optional components of GPAW that are not used by the UEABS benchmarks: ...@@ -48,15 +48,15 @@ Optional components of GPAW that are not used by the UEABS benchmarks:
GPAW needs GPAW needs
* [wheel](https://pypi.org/project/wheel/) is needed in most (if not all) ways of * [wheel](https://pypi.org/project/wheel/) is needed in most (if not all) ways of
installing the packages from source. installing the packages from source.
* [NumPy](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) * [NumPy](https://pypi.org/project/numpy/) 1.9 or later (for GPAW 20.1.0/20.10.0/21.1.0)
* Installing NumPy from source will also require * Installing NumPy from source will also require
[Cython](https://pypi.org/project/Cython/) [Cython](https://pypi.org/project/Cython/)
* GPAW 1.5.2 is not fully compatible with NumPy 1.19.x. Warnings about the use * GPAW 20.1.0 is not fully compatible with NumPy 1.19.x or later. Warnings about the use
of deprecated constructs will be shown. of deprecated constructs will be shown.
* [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 1.5.2/19.8.1/20.1.0/20.10.0) * [SciPy](https://pypi.org/project/scipy/) 0.14 or later (for GPAW 20.1.0/20.10.0/21.1.0)
* [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package * [ASE, Atomic Simulation Environment](https://wiki.fysik.dtu.dk/ase/), a Python package
from the same group that develops GPAW. Required versions are 3.17.0 or later for from the same group that develops GPAW. The required versions is 3.18.0 or later for
GPAW 1.5.2 and 3.18.0 or later for GPAW 19.8.1 or 20.1.0. GPAW 20.1.0, 20.10.0 and 21.1.0.
ASE has a couple of dependendencies ASE has a couple of dependendencies
that are not needed for running the UEABS benchmarks. However, several Python that are not needed for running the UEABS benchmarks. However, several Python
package install methods will trigger the installation of those packages, and package install methods will trigger the installation of those packages, and
...@@ -119,11 +119,11 @@ GPAW needs ...@@ -119,11 +119,11 @@ GPAW needs
The table below give the combinations of major packages Python, NumPy, SciPy, ASE and The table below give the combinations of major packages Python, NumPy, SciPy, ASE and
GPAW that were tested: GPAW that were tested:
| Python | NumPy | SciPy | ASE | GPAW | | GPAW | ASE | Python | NumPy | SciPy |
|:-------|:-------|:------|:-------|:--------| |:--------|:--------|:-------|:-------|:------|
| 3.7.9 | 1.18.5 | 1.4.1 | 3.17.0 | 1.5.2 | | 20.1.0 | 3.19.3 | 3.8.7 | 1.18.5 | 1.5.4 |
| 3.7.9 | 1.18.5 | 1.4.1 | 3.18.2 | 19.8.1 | | 20.10.0 | 3.20.1 | 3.9.1 | 1.19.5 | 1.5.4 |
| 3.8.6 | 1.18.5 | 1.4.1 | 3.19.3 | 20.1.0 | | 21.1.0 | 3.21.1 | 3.9.1 | 1.19.5 | 1.5.4 |
## Installing all prerequisites ## Installing all prerequisites
...@@ -213,43 +213,10 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -213,43 +213,10 @@ of NumPy, SciPy and GPAW itself proves much more important.
## Configuring and installing GPAW ## Configuring and installing GPAW
### GPAW 1.5.2 * GPAW 20.1.0 and later use `setuptools`. Customization of the installation process is possible
* GPAW 1.5.2 uses `distutils`. Customization of the installation process is possible
through the `customize.py` file.
* The FFT library: According to the documentation, the following strategy is used
* The compile process searches (in this order) for ``libmkl_rt.so``,
``libmkl_intel_lp64.so`` and ``libfftw3.so`. First one found will be
loaded.
* If none is found, the built-in FFT from NumPy will be used. This does not need
to be a problem if NumPy provides a properly optimized FFT library.
* The choice can also be overwritten using the GPAW_FFTWSO environment variable.
* With certain compilers, the GPAW test suite produced crashes in `xc/xc.py`. The
patch for GPAW 1.5.2 included in the [pathces](patches) subdirectory solved these
problems on the systems tested.
### GPAW 19.8.1
* GPAW 19.8.1 uses `distutils`. Customization of the installation process is possible
through a `customize.py` file.
* The selection process of the FFT library has changed from version 1.5.2. It is
now possible to specify the FFT library in `customize.py` or to simply select to
use the NumPy FFT routines.
### GPAW 20.1.0 and 20.10.0
* GPAW 20.1.0 uses `setuptools`. Customization of the installation process is possible
through the `siteconfig.py` file. through the `siteconfig.py` file.
* The selection process of the FFT library is the same as in version 19.8.1, except * It is possible to specify the FFT library in `siteconfrig.py or to simply select to
that the settings are now in `siteconfrig.py` rather than `customize.py`. use the NumPy FFT routines.
### All versions
* GPAW also needs a number of so-called "Atomic PAW Setup" files. The latest files * GPAW also needs a number of so-called "Atomic PAW Setup" files. The latest files
can be found on the [GPAW website, Atomic PAW Setups page](https://wiki.fysik.dtu.dk/gpaw/setups/setups.html). can be found on the [GPAW website, Atomic PAW Setups page](https://wiki.fysik.dtu.dk/gpaw/setups/setups.html).
For the testing we used []`gpaw-setups-0.9.20000.tar.gz`](https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz) For the testing we used []`gpaw-setups-0.9.20000.tar.gz`](https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz)
...@@ -257,8 +224,7 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -257,8 +224,7 @@ of NumPy, SciPy and GPAW itself proves much more important.
the file and set the environment variable GPAW_SETUP_PATH to point to that directory. the file and set the environment variable GPAW_SETUP_PATH to point to that directory.
In the examples provided we use the `share/gpaw-setups` subdirectory of the install In the examples provided we use the `share/gpaw-setups` subdirectory of the install
directory for this purpose. directory for this purpose.
* Up to and including version 20.1.0, GPAW does comes with a test suite which can be * GPAW 20.1.0 comes with a test suite which can be used after installation.
used after installation.
* Running the sequential tests: * Running the sequential tests:
gpaw test gpaw test
...@@ -288,7 +254,7 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -288,7 +254,7 @@ of NumPy, SciPy and GPAW itself proves much more important.
* Depending on the Python installation, some tests may fail with error messages that point * Depending on the Python installation, some tests may fail with error messages that point
to a package in the Standard Python Library that is not present. Some of these errors have no to a package in the Standard Python Library that is not present. Some of these errors have no
influence on the benchmarks as that part of the code is not triggered by the benchmark. influence on the benchmarks as that part of the code is not triggered by the benchmark.
* The full test suite is missing in GPAW 20.10.0. There is a brief sequential test * The full test suite is missing in GPAW 20.10.0 and later. There is a brief sequential test
that can be run with that can be run with
gpaw test gpaw test
...@@ -298,7 +264,7 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -298,7 +264,7 @@ of NumPy, SciPy and GPAW itself proves much more important.
gpaw -P 4 test gpaw -P 4 test
* Multiple versions of GPAW likely contain a bug in `c/bmgs/fd.c` (around line 44 * Multiple versions of GPAW likely contain a bug in `c/bmgs/fd.c` (around line 44
in GPAW 1.5.2). The code enforces vectorization on OpenMP 4 compilers by using in GPAW 20.1.0). The code enforces vectorization on OpenMP 4 compilers by using
`#pragma omp simd`. However, it turns out that the data is not always correctly `#pragma omp simd`. However, it turns out that the data is not always correctly
aligned, so if the reaction of the compiler to `#pragma omp simd` is to fully vectorize aligned, so if the reaction of the compiler to `#pragma omp simd` is to fully vectorize
and use load/store instructions for aligned data, crashes may occur. It did happen and use load/store instructions for aligned data, crashes may occur. It did happen
...@@ -312,6 +278,6 @@ of NumPy, SciPy and GPAW itself proves much more important. ...@@ -312,6 +278,6 @@ of NumPy, SciPy and GPAW itself proves much more important.
wrappers in the 2020 compilers. Downgrading to the MKL libraries of the 2018 wrappers in the 2020 compilers. Downgrading to the MKL libraries of the 2018
compilers or using the FFTW libraries solves the problem. compilers or using the FFTW libraries solves the problem.
This has been observed not only in GPAW, but also in some other DFT packages. This has been observed not only in GPAW, but also in some other DFT packages.
* The GPAW test code in versions 1.5.2 till 20.1.0 detects that matplotlib is not installed * The GPAW test code in version 20.1.0 detects if matplotlib is not installed
and will skip this test. We did however observe a failed test when Python could not find and will skip this test. We did however observe a failed test when Python could not find
the SQLite package as the Python standard library sqlite3 package is used. the SQLite package as the Python standard library sqlite3 package is used.
gpaw/build/examples/CalcUA-leibniz-broadwell/README.md 0 → 100644
View file @ b57ca544
# Example build scripts for CalcUA-leibniz-broadwell
Cluster characteristics:
* CPU architecture: dual-socket 14-core Intel Xeon E5-2680v4 "broadwell" CPUs
* Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
* Package manager: Most software on the cluster is installed through [
EasyBuild](https://easybuilders.github.io/).
Unless otherwise mentioned below, the following combinations of versions of GPAW, ASE, Python, NumPy
and SciPy were used:
| GPAW | ASE | Python | NumPy | SciPy |
|:--------|:--------|:-------|:-------|:------|
| 20.1.0 | 3.19.3 | 3.8.7 | 1.18.5 | 1.5.4 |
| 20.10.0 | 3.20.1 | 3.9.1 | 1.19.5 | 1.5.4 |
| 21.1.0 | 3.21.1 | 3.9.1 | 1.19.5 | 1.5.4 |
The following example build scripts do a complete build, including a build of Python
and a minimal set of dependencies that is needed to build a large enough part of the
Python standard library to build GPAW and run the benchmarks: zlib,
ncurses, readline, libffi, SQLite (used by one of the GPAW tests but likely
not needed for the benchmarks), FFTW, libxc, Python and the Python packages
NumPy, SciPy, ase and GPAW. The included libraries are not sufficient for a
full Python installation with all standard library packages or a full ASE/GPAW
installation with all optional packages but should be enough to run the benchmarks.
All software is build with the Intel 2020 compilers.
* `build_20.1.0_Python38_FFTW_icc`
* `build_20.10.0_Python39_FFTW_icc`
* `build_21.1.0_Python39_FFTW_icc`
The following build example build scripts install GPAW on top of the Intel Python
distribution. They rely as much as possible on modules already in the Intel Python
distribution or software that should be installed in the OS to run Intel Python.
The configuration accesses the FFT library available through
NumPy rather then linking directly to a FFT library.
The example script does use a module that provides up-to-date build
tools (buildtools/2020a) but on recent OSes it can be omitted.
Rather than loading the Intel and IntelPython3 modules one could as well
run the scripts provided by Intel to initialize the environment variables
for the compilers and Python distribution, and provided the necessary build
tools are also available in the system OS, this build could be done without
relying on any EasyBuild or Spack-generated modules.
* `build_20.1.0_IntelPython3_icc.sh`
* `build_20.10.0_IntelPython3_icc.sh`
* `build_21.1.0_IntelPython3_icc.sh`
gpaw/build/examples/CalcUA-vaughan-rome/build_1.5.2_IntelPython3_icc.sh → gpaw/build/examples/CalcUA-leibniz-broadwell/build_20.1.0_IntelPython3_icc.sh
View file @ b57ca544
#!/bin/bash #!/bin/bash
# #
# Installation script for GPAW 1.5.2: # Installation script for GPAW 20.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized # * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included. # NumPy and SciPy included.
# * Using the matching version of ase, 3.17.0 # * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers # * Compiling with the Intel compilers
# #
# The FFT library is discovered at runtime. With the settings used in this script # The FFT library is discovered at runtime. With the settings used in this script
...@@ -12,40 +12,80 @@ ...@@ -12,40 +12,80 @@
# (link below). # (link below).
# #
# The original installation instructions for GPAW can be found at # The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/1.5.2/doc/install.rst # https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
# #
packageID='1.5.2-IntelPython3-icc' packageID='20.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
download_dir=$install_root/Downloads module purge
install_dir=$install_root/$systemID/Packages/GPAW-manual/$packageID module use $install_root/$systemID/Modules
modules_dir=$install_root/$systemID/Modules/GPAW-manual module load UEABS/$UEABS_version
build_dir="/dev/shm/$USER/GPAW-manual/$packageID"
patch_dir=$VSC_DATA/Projects/PRACE/GPAW-experiments/UEABS/build/patches # Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.02'
libxc_version='4.3.4' libxc_version='4.3.4'
ase_version='3.17.0' ase_version='3.19.3'
GPAW_version='1.5.2' GPAW_version='20.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7' py_maj_min='3.7'
################################################################################ ################################################################################
# #
# Prepare the system # Prepare the system
# #
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
# #
# Load modules # Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
# #
module purge module purge
module load calcua/2020a MODULEPATH=/apps/antwerpen/modules/centos7/calcua-admin-broadwell/
module load intel/2020a module load calcua/admin
module load IntelPython3/2020a module load $compiler_module
module load buildtools/2020a module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
# #
# Create the directories and make sure they are clean if that matters # Create the directories and make sure they are clean if that matters
...@@ -63,7 +103,6 @@ module load buildtools/2020a ...@@ -63,7 +103,6 @@ module load buildtools/2020a
/usr/bin/mkdir -p $build_dir /usr/bin/mkdir -p $build_dir
################################################################################ ################################################################################
# #
# Download components # Download components
...@@ -78,17 +117,14 @@ libxc_file="libxc-$libxc_version.tar.bz2" ...@@ -78,17 +117,14 @@ libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version" libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file" [[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# We do not download ase in this script. As it is pure python and doesn't need # Downloading ase so that we can gather all sources for reproducibility
# customization, we will install it using pip right away. ase_file="ase-$ase_version.tar.gz"
## https://files.pythonhosted.org/packages/d9/08/35969da23b641d3dfca46ba7559f651fcfdca81dbbc00b9058c934e75769/ase-3.17.0.tar.gz ase_url="https://pypi.python.org/packages/source/a/ase"
#ase_file="ase-$ase_version.tar.gz" [[ -f $ase_file ]] || wget "$ase_url/$ase_file"
#ase_url="https://files.pythonhosted.org/packages/d9/08/35969da23b641d3dfca46ba7559f651fcfdca81dbbc00b9058c934e75769"
#[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources. # GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz # https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz" GPAW_file="gpaw-$GPAW_version.tar.gz"
#GPAW_url="https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw" GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file" [[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
...@@ -117,16 +153,15 @@ cd libxc-$libxc_version ...@@ -117,16 +153,15 @@ cd libxc-$libxc_version
autoreconf -i autoreconf -i
export CC=icc export CC=icc
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -fPIC" export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC"
#export CFLAGS="-O0 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -fPIC"
./configure --prefix="$install_dir" \ ./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran --disable-static --enable-shared --disable-fortran
# Build # Build
make -j 16 make -j $parallel
# Install # Install
make -j 16 install make install
# Add bin, lib and include to the PATH variables # Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH PATH=$install_dir/bin:$PATH
...@@ -134,6 +169,37 @@ LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH ...@@ -134,6 +169,37 @@ LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################ ################################################################################
# #
...@@ -142,10 +208,12 @@ CPATH=$install_dir/include:$CPATH ...@@ -142,10 +208,12 @@ CPATH=$install_dir/include:$CPATH
echo -e "\nInstalling ase...\n" echo -e "\nInstalling ase...\n"
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages" # IntelPython3 does not define pip3 or pip3.7.
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages" pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
pip install --prefix=$install_dir --no-deps ase==$ase_version # Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################ ################################################################################
...@@ -172,47 +240,38 @@ cd $build_dir ...@@ -172,47 +240,38 @@ cd $build_dir
# Uncompress # Uncompress
tar -xf $download_dir/$GPAW_file tar -xf $download_dir/$GPAW_file
# Apply patches
patch -p0 <$patch_dir/gpaw-1.5.2.patch
cd gpaw-$GPAW_version cd gpaw-$GPAW_version
# Make the customize.py script # Make the customize.py script
mv customize.py customize.py.orig cat >siteconfig.py <<EOF
cat >customize.py <<EOF print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW build INFO: Starting execution of the customization script' ) print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW build INFO: Variables at the start of the customization script' ) print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW build INFO: libraries = ', libraries ) print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW build INFO: mpi_libaries = ', mpi_libraries ) print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW build INFO: library_dirs = ', library_dirs ) print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW build INFO: mpi_libary_dirs = ', mpi_library_dirs ) print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW build INFO: runtime_library_dirs = ', runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW build INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs ) print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW build INFO: include_dirs = ', include_dirs ) print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW build INFO: mpi_include_dirs = ', mpi_include_dirs ) print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW build INFO: compiler = ', compiler ) print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW build INFO: mpicompiler = ', mpicompiler ) print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW build INFO: mpilinker = ', mpilinker ) print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW build INFO: extra_compile_args = ', extra_compile_args ) print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW build INFO: extra_link_args = ', extra_link_args ) print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW build INFO: define_macros = ', define_macros ) print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW build INFO: mpi_define_macros = ', mpi_define_macros ) print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW build INFO: undef_macros = ', undef_macros ) print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW build INFO: scalapack = ', scalapack ) print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW build INFO: libvdwxc = ', libvdwxc ) print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW build INFO: elpa = ', elpa ) print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
# Reset the lists of libraries as often the wrong BLAS library is picked up. print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
libraries = []
mpi_libraries = [] # LibXC
include_dirs.append('$install_dir/include')
# LibXC. Re-add the library (removed by resetting libraries). #libraries.append('xc')
# There is no need to add the library directory as we do set library_dirs to the
# content of LIBRARY_PATH further down.
# There should be no need to add the include directory as it is in CPATH which is
# set when we install gpaw.
#include_dirs.append('$install_dir/include')
libraries.append('xc')
# libvdwxc # libvdwxc
libvdwxc = False libvdwxc = False
...@@ -220,17 +279,17 @@ libvdwxc = False ...@@ -220,17 +279,17 @@ libvdwxc = False
# ELPA # ELPA
elpa = False elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK # ScaLAPACK
scalapack = True scalapack = True
mpi_libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64'] libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
mpi_define_macros += [('GPAW_NO_UNDERSCORE_CBLACS', '1')]
mpi_define_macros += [('GPAW_NO_UNDERSCORE_CSCALAPACK', '1')]
# Add EasyBuild LAPACK/BLAS libs # MKL BLAS
# This should also enable MKL FFTW according to the documentation of GPAW 1.5.2 libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
libraries += ['mkl_intel_lp64', 'mkl_sequential', 'mkl_core']
# Add other EasyBuild library directories. # Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':') library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers # Set the compilers
...@@ -238,44 +297,49 @@ compiler = os.environ['CC'] ...@@ -238,44 +297,49 @@ compiler = os.environ['CC']
mpicompiler = os.environ['MPICC'] mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC'] mpilinker = os.environ['MPICC']
# We need extra_compile_args to have the right compiler options when re-compiling print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
# files for gpaw-python. It does imply double compiler options for the other print( 'GPAW EasyBuild INFO: libraries = ', libraries )
# compiles though. print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
extra_compile_args = os.environ['CFLAGS'].split(' ') print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW build INFO: Variables at the end of the customization script' ) print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW build INFO: libraries = ', libraries ) print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW build INFO: mpi_libaries = ', mpi_libraries ) print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW build INFO: library_dirs = ', library_dirs ) print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW build INFO: mpi_libary_dirs = ', mpi_library_dirs ) print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW build INFO: runtime_library_dirs = ', runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW build INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs ) print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW build INFO: include_dirs = ', include_dirs ) print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW build INFO: mpi_include_dirs = ', mpi_include_dirs ) print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW build INFO: compiler = ', compiler ) print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW build INFO: mpicompiler = ', mpicompiler ) print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW build INFO: mpilinker = ', mpilinker ) print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW build INFO: extra_compile_args = ', extra_compile_args ) print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW build INFO: extra_link_args = ', extra_link_args ) print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW build INFO: define_macros = ', define_macros ) print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW build INFO: mpi_define_macros = ', mpi_define_macros ) print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW build INFO: undef_macros = ', undef_macros ) print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW build INFO: scalapack = ', scalapack ) print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW build INFO: libvdwxc = ', libvdwxc ) print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
print( 'GPAW build INFO: elpa = ', elpa )
print( 'GPAW build INFO: Ending execution of the customization script' )
EOF EOF
# Now install gpaw # Now install gpaw
export CC=icc export CC=icc
export MPICC=mpiicc export MPICC=mpiicc
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -qno-openmp-simd" export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
export CFLAGS="-O2 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -qno-openmp-simd" python$py_maj_min setup.py build -j $parallel
#export CFLAGS="-O0 -march=core-avx2 -mtune=core-avx2 -ftz -fp-speculation=safe -fp-model source -qno-openmp-simd"
python setup.py build -j 32
# Install GPAW # Install GPAW
python setup.py install --prefix="$install_dir" pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################ ################################################################################
...@@ -283,14 +347,17 @@ python setup.py install --prefix="$install_dir" ...@@ -283,14 +347,17 @@ python setup.py install --prefix="$install_dir"
# Finish the install # Finish the install
# #
echo -e "\nCleaning up and making the LUA-module GPAW-manual/$packageID...\n" echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory # Go to a different directory before cleaning up the build directory
cd $modules_dir cd $modules_dir
#/bin/rm -rf $build_dir /bin/rm -rf $build_dir
# Create a module file # Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF cat >$packageID.lua <<EOF
help([==[ help([==[
...@@ -300,14 +367,23 @@ Description ...@@ -300,14 +367,23 @@ Description
GPAW $GPAW_version for the UEABS benchmark. GPAW $GPAW_version for the UEABS benchmark.
Configuration: Configuration:
* IntelPython3/2020a module, also for NumPy and SciPy
* Parallel GPAW $GPAW_version with ase $ase_version * Parallel GPAW $GPAW_version with ase $ase_version
* FFT library selected at runtime. The default with the path * On top of the system IntelPython3 $IntelPython3_version module, which provides
as set through this module should be MKL but it can be changed Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
by setting GPAW_FFTWSO as indicated in the install instructions at * NumPy is used for FFT, so the performance of the code
https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/install.rst will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers * libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information More information
================ ================
...@@ -316,31 +392,25 @@ More information ...@@ -316,31 +392,25 @@ More information
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/ - GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/ - Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/ - ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
Included extensions
===================
ase-$ase_version, gpaw-$GPAW_version
]==]) ]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version: UEABS benchmark configuration.]==]) whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
conflict("GPAW") family("GPAW")
conflict("GPAW-manual")
if not ( isloaded("calcua/2020a") ) then prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
load("calcua/2020a")
end
if not ( isloaded("intel/2020a") ) then if not ( isloaded("calcua/admin") ) then
load("intel/2020a") load("calcua/admin")
end end
if not ( isloaded("IntelPython3/2020a") ) then if not ( isloaded("$compiler_module") ) then
load("IntelPython3/2020a") load("$compiler_module")
end end
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
prepend_path("PATH", "$install_dir/bin") prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib") prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
......
gpaw/build/examples/CalcUA-leibniz-broadwell/build_20.10.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.10.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.20.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.10.0/doc/install.rst
#
packageID='20.10.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.02'
libxc_version='4.3.4'
ase_version='3.20.1'
GPAW_version='20.10.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos7/calcua-admin-broadwell/
module load calcua/admin
module load $compiler_module
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-leibniz-broadwell/build_21.1.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 21.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.21.1
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/21.1.0/doc/install.rst
#
packageID='21.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-leibniz-broadwell
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.02'
libxc_version='4.3.4'
ase_version='3.21.1'
GPAW_version='21.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos7/calcua-admin-broadwell/
module load calcua/admin
module load $compiler_module
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/CalcUA-vaughan-rome/README.md 0 → 100644
View file @ b57ca544
# Example build scripts for CalcUA-vaughan-rome
Cluster characteristics:
* CPU architecture: dual-socket 32-core AMD EPYC 7452 "Rome" CPUs
* Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
* Package manager: Most software on the cluster is installed through [
EasyBuild](https://easybuilders.github.io/).
Unless otherwise mentioned below, the following combinations of versions of GPAW, ASE, Python, NumPy
and SciPy were used:
| GPAW | ASE | Python | NumPy | SciPy |
|:--------|:--------|:-------|:-------|:------|
| 20.1.0 | 3.19.3 | 3.8.7 | 1.18.5 | 1.5.4 |
| 20.10.0 | 3.20.1 | 3.9.1 | 1.19.5 | 1.5.4 |
| 21.1.0 | 3.21.1 | 3.9.1 | 1.19.5 | 1.5.4 |
The following example build scripts do a complete build, including a build of Python
and a minimal set of dependencies that is needed to build a large enough part of the
Python standard library to build GPAW and run the benchmarks: zlib,
ncurses, readline, libffi, SQLite (used by one of the GPAW tests but likely
not needed for the benchmarks), FFTW, libxc, Python and the Python packages
NumPy, SciPy, ase and GPAW. The included libraries are not sufficient for a
full Python installation with all standard library packages or a full ASE/GPAW
installation with all optional packages but should be enough to run the benchmarks.
All software is build with the Intel 2020 compilers.
* `build_20.1.0_Python38_FFTW_icc`
* `build_20.10.0_Python39_FFTW_icc`
* `build_21.1.0_Python39_FFTW_icc`
The following build example build scripts install GPAW on top of the Intel Python
distribution. They rely as much as possible on modules already in the Intel Python
distribution or software that should be installed in the OS to run Intel Python.
The configuration accesses the FFT library available through
NumPy rather then linking directly to a FFT library.
The example script does use a module that provides up-to-date build
tools (buildtools/2020a) but on recent OSes it can be omitted.
Rather than loading the Intel and IntelPython3 modules one could as well
run the scripts provided by Intel to initialize the environment variables
for the compilers and Python distribution, and provided the necessary build
tools are also available in the system OS, this build could be done without
relying on any EasyBuild or Spack-generated modules.
* `build_20.1.0_IntelPython3_icc.sh`
* `build_20.10.0_IntelPython3_icc.sh`
* `build_21.1.0_IntelPython3_icc.sh`
The following example build scripts depend on other modules (Python and FFTW) that
were installed on the cluster through EasyBuild beyond the compiler modules and a
module providing some up-to-date basic build tools:
* `build_20.1.0_Python39icc_icc.sh`
* `build_20.10.0_Python39icc_icc.sh`
* `build_21.1.0_Python39icc_icc.sh`
gpaw/build/examples/CalcUA-vaughan-rome/build_20.1.0_IntelPython3_icc.sh 0 → 100755
View file @ b57ca544
#!/bin/bash
#
# Installation script for GPAW 20.1.0:
# * Using the existing IntelPython3 module on the system which has an optimized
# NumPy and SciPy included.
# * Using the matching version of ase, 3.19.3
# * Compiling with the Intel compilers
#
# The FFT library is discovered at runtime. With the settings used in this script
# this should be MKL FFT, but it is possible to change this at runtime to either
# MKL, FFTW or the built-in NumPy FFT routines, see the installation instructions
# (link below).
#
# The original installation instructions for GPAW can be found at
# https://gitlab.com/gpaw/gpaw/-/blob/20.1.0/doc/install.rst
#
packageID='20.1.0-IntelPython3-icc'
echo -e "\n### Building GPAW-UEABS/$packageID from $0\n\n"
install_root=$VSC_SCRATCH/UEABS
systemID=CalcUA-vaughan-rome
UEABS_version='2.2'
#
# NOTE: For easy maintanance we load many settings from the UEABS module.
# Since we want to release the module again and only depend on system modules for
# this build, we copy the settings to other variables and unload the module
# again. It is of course possible to those variables directly by hand without
# first creating a module file.
#
module purge
module use $install_root/$systemID/Modules
module load UEABS/$UEABS_version
# Directory in which downloaded files will be stored.
download_dir=$UEABS_DOWNLOADS
# Directory in which the software package will be installed.
install_dir=$UEABS_PACKAGES/GPAW-UEABS/$packageID
# Subdirectory containing the module file.
modules_dir=$UEABS_MODULES/GPAW-UEABS
# Temporary directory for the build.
build_dir="/dev/shm/$USER/GPAW-UEABS/$packageID"
# Subdirectory in the UEABS repo containing the patch file that is needed.
patch_dir=$UEABS_REPOSITORY/gpaw/build/patches
#
# Software versions
#
IntelPython3_version='2020.00'
libxc_version='4.3.4'
ase_version='3.19.3'
GPAW_version='20.1.0'
GPAWsetups_version='0.9.20000' # Check version on https://wiki.fysik.dtu.dk/gpaw/setups/setups.html
# Compiler settings
compiler_module='intel/2020a.00'
opt_level='-O2'
proc_opt_flags='-march=core-avx2 -mtune=core-avx2'
fp_opt_flags='-ftz -fp-speculation=safe -fp-model source'
parallel=32
py_maj_min='3.7'
################################################################################
#
# Prepare the system
#
# We do this through modules, but it could be done by running scripts from the
# Intel compiler suite as well.
#
#
# Load modules:
# - Intel compiler module
# - Intel Python
# - buildtools is a module with a number of build tools that are more up-to-date
# then those on the system. It is mostly used for consistent builds across
# systems.
#
module purge
MODULEPATH=/apps/antwerpen/modules/centos8/calcua-admin-rome/
module load calcua/admin
module load buildtools/2020a
module load IntelPython3/$IntelPython3_version
module load $compiler_module
#
# Create the directories and make sure they are clean if that matters
#
/usr/bin/mkdir -p $download_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/rm -rf $install_dir
/usr/bin/mkdir -p $install_dir
/usr/bin/mkdir -p $modules_dir
/usr/bin/mkdir -p $build_dir
/usr/bin/rm -rf $build_dir
/usr/bin/mkdir -p $build_dir
################################################################################
#
# Download components
#
echo -e "\nDownloading files...\n"
cd $download_dir
# https://gitlab.com/libxc/libxc/-/archive/4.3.4/libxc-4.3.4.tar.bz2
libxc_file="libxc-$libxc_version.tar.bz2"
libxc_url="https://gitlab.com/libxc/libxc/-/archive/$libxc_version"
[[ -f $libxc_file ]] || wget "$libxc_url/$libxc_file"
# Downloading ase so that we can gather all sources for reproducibility
ase_file="ase-$ase_version.tar.gz"
ase_url="https://pypi.python.org/packages/source/a/ase"
[[ -f $ase_file ]] || wget "$ase_url/$ase_file"
# GPAW needs customization, so we need to download and unpack the sources.
# https://files.pythonhosted.org/packages/49/a1/cf54c399f5489cfdda1e8da02cae8bfb4b39d7cb7a895ce86608fcd0e1c9/gpaw-1.5.2.tar.gz
GPAW_file="gpaw-$GPAW_version.tar.gz"
GPAW_url="https://pypi.python.org/packages/source/g/gpaw"
[[ -f $GPAW_file ]] || wget "$GPAW_url/$GPAW_file"
# Download GPAW-setup, a number of setup files for GPAW.
# https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
GPAWsetups_file="gpaw-setups-$GPAWsetups_version.tar.gz"
GPAWsetups_url="https://wiki.fysik.dtu.dk/gpaw-files"
[[ -f $GPAWsetups_file ]] || wget "$GPAWsetups_url/$GPAWsetups_file"
################################################################################
#
# Install libxc
#
echo -e "\nInstalling libxc...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$libxc_file
cd libxc-$libxc_version
# Configure
autoreconf -i
export CC=icc
export CFLAGS="$opt_level $proc_opt_flags $fp_opt_flags -fPIC"
./configure --prefix="$install_dir" \
--disable-static --enable-shared --disable-fortran
# Build
make -j $parallel
# Install
make install
# Add bin, lib and include to the PATH variables
PATH=$install_dir/bin:$PATH
LIBRARY_PATH=$install_dir/lib:$LIBRARY_PATH
LD_LIBRARY_PATH=$install_dir/lib:$LD_LIBRARY_PATH
CPATH=$install_dir/include:$CPATH
# Clean-up
unset CC
unset CFLAGS
################################################################################
#
# Prepare for installing Python packages
#
/usr/bin/mkdir -p "$install_dir/lib/python$py_maj_min/site-packages"
PYTHONPATH="$install_dir/lib/python$py_maj_min/site-packages:$PYTHONPATH"
################################################################################
#
# Optional: Install pytest and its dependencies to test NumPy and SciPy with
# import numpy
# numpy.test()
# import scipy
# scipy.text()
# We don't care about version numbers here as it is not important for the
# reproducibility of the benchmarks.
#
echo -e "\nInstalling pytest...\n"
cd $build_dir
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir pytest
################################################################################
#
# Install ase
#
echo -e "\nInstalling ase...\n"
# IntelPython3 does not define pip3 or pip3.7.
pip install --prefix=$install_dir --no-deps $download_dir/$ase_file
# Brief test
cd $build_dir
python$py_maj_min -c "import ase"
################################################################################
#
# Install GPAW-setups
#
echo -e "\nInstalling gpaw-setups...\n"
mkdir -p $install_dir/share/gpaw-setups
cd $install_dir/share/gpaw-setups
tar -xf $download_dir/$GPAWsetups_file --strip-components=1
################################################################################
#
# Install GPAW
#
echo -e "\nInstalling GPAW...\n"
cd $build_dir
# Uncompress
tar -xf $download_dir/$GPAW_file
cd gpaw-$GPAW_version
# Make the customize.py script
cat >siteconfig.py <<EOF
print( 'GPAW EasyBuild INFO: Starting execution of the customization script' )
print( 'GPAW EasyBuild INFO: Variables at the start of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
# LibXC
include_dirs.append('$install_dir/include')
#libraries.append('xc')
# libvdwxc
libvdwxc = False
# ELPA
elpa = False
# Use NumPy FFTW
fftw = False
# ScaLAPACK
scalapack = True
libraries += ['mkl_scalapack_lp64', 'mkl_blacs_intelmpi_lp64']
# MKL BLAS
libraries += ['mkl_sequential','mkl_core', 'mkl_rt', ]
# Add other EasyBuild library directoryes.
library_dirs = os.environ['LIBRARY_PATH'].split(':')
# Set the compilers
compiler = os.environ['CC']
mpicompiler = os.environ['MPICC']
mpilinker = os.environ['MPICC']
print( 'GPAW EasyBuild INFO: Variables at the end of the customization script' )
print( 'GPAW EasyBuild INFO: libraries = ', libraries )
print( 'GPAW EasyBuild INFO: mpi_libaries = ', mpi_libraries )
print( 'GPAW EasyBuild INFO: library_dirs = ', library_dirs )
print( 'GPAW EasyBuild INFO: mpi_libary_dirs = ', mpi_library_dirs )
print( 'GPAW EasyBuild INFO: runtime_library_dirs = ', runtime_library_dirs )
print( 'GPAW EasyBuild INFO: mpi_runtime_libary_dirs = ', mpi_runtime_library_dirs )
print( 'GPAW EasyBuild INFO: include_dirs = ', include_dirs )
print( 'GPAW EasyBuild INFO: mpi_include_dirs = ', mpi_include_dirs )
print( 'GPAW EasyBuild INFO: compiler = ', compiler )
print( 'GPAW EasyBuild INFO: mpicompiler = ', mpicompiler )
print( 'GPAW EasyBuild INFO: mpilinker = ', mpilinker )
print( 'GPAW EasyBuild INFO: extra_compile_args = ', extra_compile_args )
print( 'GPAW EasyBuild INFO: extra_link_args = ', extra_link_args )
print( 'GPAW EasyBuild INFO: define_macros = ', define_macros )
print( 'GPAW EasyBuild INFO: mpi_define_macros = ', mpi_define_macros )
print( 'GPAW EasyBuild INFO: undef_macros = ', undef_macros )
print( 'GPAW EasyBuild INFO: fftw = ', fftw )
print( 'GPAW EasyBuild INFO: scalapack = ', scalapack )
print( 'GPAW EasyBuild INFO: libvdwxc = ', libvdwxc )
print( 'GPAW EasyBuild INFO: elpa = ', elpa )
print( 'GPAW EasyBuild INFO: Ending execution of the customization script' )
print( 'GPAW EasyBuild INFO: noblas = ', noblas )
print( 'GPAW EasyBuild INFO: parallel_python_interpreter = ', parallel_python_interpreter )
EOF
# Now install gpaw
export CC=icc
export MPICC=mpiicc
export CFLAGS="-std=c99 $opt_level $proc_opt_flags $fp_opt_flags -qno-openmp-simd"
python$py_maj_min setup.py build -j $parallel
# Install GPAW
pip install --prefix=$install_dir --no-deps --ignore-installed --no-build-isolation .
# Brief test
cd $build_dir
python$py_maj_min -c "import gpaw"
# Clean-up
unset CC
unset MPICC
unset CFLAGS
################################################################################
#
# Finish the install
#
echo -e "\nCleaning up and making the LUA-module GPAW-UEABS/$packageID...\n"
# Go to a different directory before cleaning up the build directory
cd $modules_dir
/bin/rm -rf $build_dir
# Create a module file
python_version=$(python -V | cut -d ' ' -f 2)
numpy_version=$(python3 -c "import numpy ; print(numpy.__version__)")
scipy_version=$(python3 -c "import scipy ; print(scipy.__version__)")
cat >$packageID.lua <<EOF
help([==[
Description
===========
GPAW $GPAW_version for the UEABS benchmark.
Configuration:
* Parallel GPAW $GPAW_version with ase $ase_version
* On top of the system IntelPython3 $IntelPython3_version module, which provides
Python $python_version, NumPy $numpy_version and SciPy $scipy_version.
* NumPy is used for FFT, so the performance of the code
will depend a lot on a proper configuration of NumPy for FFT.
* libxc and GPAW compiled with the Intel compilers
Detailed configuration:
* Compiler module: $compiler_module (a module file written
at UAntwerp, not generated via EasyBuild)
* Modules from the system
* Intel Python3 $IntelPython3_version (using a module file written
at UAntwerp)
* libxc $libxc_version
* Python packages: ase-$ase_version, gpaw-$GPAW_version
* GPAW setups $GPAWsetups_version
More information
================
- Homepage: http://wiki.fysik.dtu.dk/gpaw
- Documentation:
- GPAW web-based documentation: https://wiki.fysik.dtu.dk/gpaw/
- Version information at https://gitlab.com/gpaw/gpaw/-/blob/$GPAW_version/doc/
- ASE web-based documentation: https://wiki.fysik.dtu.dk/ase/
]==])
whatis([==[Description: GPAW $GPAW_version with ase $ase_version, Intel Python3 $IntelPython3_version with numpy $numpy_version and scipy $scipy_version: UEABS benchmark configuration.]==])
family("GPAW")
prepend_path('MODULEPATH','/apps/antwerpen/modules/centos8/calcua-admin-rome')
if not ( isloaded("calcua/admin") ) then
load("calcua/admin")
end
--
-- The order of loading is a deliberate choice. It takes some libraries from
-- the compiler directories rather than from the Python directories and has
-- solved some unexplained issues we had with SLURM.
--
if not ( isloaded("IntelPython3/$IntelPython3_version") ) then
load("IntelPython3/$IntelPython3_version")
end
if not ( isloaded("$compiler_module") ) then
load("$compiler_module")
end
prepend_path("PATH", "$install_dir/bin")
prepend_path("LD_LIBRARY_PATH", "$install_dir/lib")
prepend_path("LIBRARY_PATH", "$install_dir/lib")
prepend_path("PYTHONPATH", "$install_dir/lib/python$py_maj_min/site-packages")
setenv("GPAW_SETUP_PATH", "$install_dir/share/gpaw-setups")
EOF
gpaw/build/examples/README.md 0 → 100644
View file @ b57ca544
# Example build scripts
## CalcUA-vaughan-rome
Vaughan is a cluster installed at the University of Antwerpen
as one of the VSC (Vlaams Supercomputer Centrum) Tier-2 clusters.
The cluster has 144 nodes with dual 32-core AMD EPYC 7452 "Rome" CPUs and 256GB of RAM memory
per node. Nodes are interconnected through an InfiniBand HDR100 network. The system
does not contain accelerators. The cluster uses CentOS 8 as the
operating system.
Most software on the cluster is installed through [EasyBuild](https://easybuilders.github.io/).
Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
[Go to the directory](CalcUA-vaughan-rome)
## CalcUA-leibniz/broadwell
Leibniz is a cluster intalled at the University of Antwerpen
as one of the VSC (Vlaams Supercomputer Centrum) Tier-2 clusters.
The cluster has 153 regular CPU nodes with dual Intel Xeon E5-2680v4
"broadwell" CPUs with 128 or 256 GB of RAM memory per node.
Nodes are interconnected through an Infiniband EDR network.
The system has two nodes with dual NVIDIA P100 GPUs and a node
with dual NEC Aurora TSUBASA first generation vector boards.
During the testing period the cluster used CentOS 7 as the
operating system.
Most software on the cluster is installed through [EasyBuild](https://easybuilders.github.io/).
Environment modules are provided through [Lmod](https://lmod.readthedocs.io/en/latest/).
## davide @ Cineca
Build instructions for GPU and CPU on the DAVIDE OpenPower GPU cluster @ Cineca.
These instructions haven't been tested recently. They are still for a version
based on distutils rather then setuptools that was dropped towards the end of
the development of this version of the benchmark suite.
## juwels @ Jülich Supercomputing Centre
Build instructions for the CPU nodes of JUWELS.
These instructions haven't been tested recently. They are still for a version
based on distutils rather then setuptools that was dropped towards the end of
the development of this version of the benchmark suite.
## piz-daint @ CSCS
Build instructions for the Piz Daint (GPU cluster).
These instructions haven't been tested recently. They are still for a version
based on distutils rather then setuptools that was dropped towards the end of
the development of this version of the benchmark suite.