Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
U
ueabs
Manage
Activity
Members
Plan
Wiki
Code
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Analyze
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
UEABS
ueabs
Commits
037f4585
Commit
037f4585
authored
7 years ago
by
Andrew Emerson
Browse files
Options
Downloads
Patches
Plain Diff
README modified
parent
561d4ebf
Branches
Branches containing commit
Tags
Tags containing commit
No related merge requests found
Changes
1
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
quantum_espresso/README.md
+65
-34
65 additions, 34 deletions
quantum_espresso/README.md
with
65 additions
and
34 deletions
quantum_espresso/README.md
+
65
−
34
View file @
037f4585
# Quantum Espresso in the Accelerated Benchmark Suite
# Quantum Espresso in the Accelerated Benchmark Suite
## Document Author: A. Emerson (a.emerson@cineca.it) , Cineca.
## Document Author: A. Emerson (a.emerson@cineca.it) , Cineca.
## Last update: 16th February 2016
## Last update: 16th February 2016
## Contents
## Contents
1.
Introduction
1.
Introduction
...
@@ -12,7 +13,7 @@
...
@@ -12,7 +13,7 @@
7.
References
7.
References
1.
Introduction
##
1. Introduction
The GPU-enabled version of Quantum Espresso (known as QE-GPU) provides
The GPU-enabled version of Quantum Espresso (known as QE-GPU) provides
GPU acceleration for the Plane-Wave Self-Consistent Field (PWscf)
GPU acceleration for the Plane-Wave Self-Consistent Field (PWscf)
code and energy barriers and reaction pathways through the Nudged
code and energy barriers and reaction pathways through the Nudged
...
@@ -25,53 +26,62 @@ QE-GPU is developed by Filippo Spiga and the download and build
...
@@ -25,53 +26,62 @@ QE-GPU is developed by Filippo Spiga and the download and build
instructions for the package are given here [1] if the packages is not
instructions for the package are given here [1] if the packages is not
already available on your system.
already available on your system.
2.
Requirements
##
2. Requirements
Essential
Essential
*
Quantum ESPRESSO 5.4
*
Quantum ESPRESSO 5.4
*
Kepler GPU: (minimum) CUDA SDK 6.5
*
Kepler GPU: (minimum) CUDA SDK 6.5
*
Pascal GPU: (minimum) CUDA SDK 8.0
*
Pascal GPU: (minimum) CUDA SDK 8.0
Optional
Optional
*
A parallel linear algebra library such as Scalapack or Intel MKL. If
*
A parallel linear algebra library such as Scalapack or Intel MKL. If
*
none is available on your system then the installation can use a
none is available
on your system then the installation can use a
*
version supplied with the distribution.
version supplied with the distribution.
3.
Downloading the software
QE distribution
##3. Downloading the software
### QE distribution
Many packages are available from the download page but since you need
Many packages are available from the download page but since you need
only the main base package for the benchmark suite, the
only the main base package for the benchmark suite, the
`expresso-5.4.0.tar.gz`
file will be sufficient. This can be downloaded
`expresso-5.4.0.tar.gz`
file will be sufficient. This can be downloaded
as:
as:
[http://www.quantum-espresso.org/download] (http://www.quantum-espresso.org/download)
[http://www.quantum-espresso.org/download] (http://www.quantum-espresso.org/download)
GPU plugin
###
GPU plugin
The GPU source code can be conveniently downloaded from this link:
The GPU source code can be conveniently downloaded from this link:
[https://github.com/QEF/qe-gpu-plugin] (https://github.com/QEF/qe-gpu-plugin)
[https://github.com/QEF/qe-gpu-plugin] (https://github.com/QEF/qe-gpu-plugin)
4.
Compiling the application
##
4. Compiling the application
The QE-GPU gives more details but for the benchmark suite we followed
The QE-GPU gives more details but for the benchmark suite we followed
this general procedure:
this general procedure:
1.
Uncompress the main QE distribution and copy the GPU source distribution inside:
1.
Uncompress the main QE distribution and copy the GPU source distribution inside:
`tar zxvf espresso-5.4.0.tar.gz
```
shell
tar
zxvf espresso-5.4.0.tar.gz
cp
5.4.0.tar.gz espresso-5.4.0
`
cp
5.4.0.tar.gz espresso-5.4.0
`
```
2.
Uncompress the GPU source inside main distribution and create a symbolic link:
2.
Uncompress the GPU source inside main distribution and create a symbolic link:
`cd espresso-5.4.0
```
shell
cd
espresso-5.4.0
tar
zxvf 5.4.0.tar.gz
tar
zxvf 5.4.0.tar.gz
ln
-s
QE-GPU-5.4.0 GPU
`
ln
-s
QE-GPU-5.4.0 GPU
`
```
3.
Run QE-GPU configure and make:
3.
Run QE-GPU configure and make:
`
cd GPU
```
shell
cd
GPU
./configure
--enable-parallel
--enable-openmp
--with-scalapack
=
intel
\
./configure
--enable-parallel
--enable-openmp
--with-scalapack
=
intel
\
--enable-cuda
--with-gpu-arch
=
Kepler
\
--enable-cuda
--with-gpu-arch
=
Kepler
\
--with-cuda-dir
=
/usr/local/cuda/7.0.1
\
--with-cuda-dir
=
/usr/local/cuda/7.0.1
\
--without-magma
--with-phigemm
--without-magma
--with-phigemm
cd
..
cd
..
make -f Makefile.gpu pw-gpu
`
make
-f
Makefile.gpu pw-gpu
```
In this example we are compiling with the Intel FORTRAN compiler so we
In this example we are compiling with the Intel FORTRAN compiler so we
can use the Intel MKL version of Scalapack. Note also that in the
can use the Intel MKL version of Scalapack. Note also that in the
...
@@ -80,44 +90,55 @@ directory `/usr/local/cuda/7.0.1`.
...
@@ -80,44 +90,55 @@ directory `/usr/local/cuda/7.0.1`.
The QE-GPU executable will appear in the directory
`GPU/PW`
and is called
`pw-gpu.x`
.
The QE-GPU executable will appear in the directory
`GPU/PW`
and is called
`pw-gpu.x`
.
5.
Running the program
##5. Running the program
Of course you need some input before you can run calculations. The
Of course you need some input before you can run calculations. The
input files are of two types:
input files are of two types:
1.
A control file usually called pw.in
1.
A control file usually called
`
pw.in
`
2.
One or more pseudopotential files with extension .UPF
2.
One or more pseudopotential files with extension
`
.UPF
`
The pseudopotential files are placed in a directory specified in the
The pseudopotential files are placed in a directory specified in the
control file with the tag pseudo_dir. Thus if we have
control file with the tag pseudo_dir. Thus if we have
```
shell
pseudo_dir
=
./
pseudo_dir
=
./
```
then QE-GPU will look for the pseudopotential
then QE-GPU will look for the pseudopotential
files in the current directory. The data files themselves can be
files in the current directory. The data files themselves can be
downloaded from the QE website or the PRACE respository. For example,
downloaded from the QE website or the PRACE respository. For example,
wget http://www.prace-ri.eu/UEABS/Quantum_Espresso/QuantumEspresso_TestCaseA.tar.gz
wget http://www.prace-ri.eu/UEABS/Quantum_Espresso/QuantumEspresso_TestCaseA.tar.gz
Once uncompressed you can then run the program like this (e.g. using MPI over 16 cores):
Once uncompressed you can then run the program like this (e.g. using
MPI over 16 cores):
```
shell
mpirun
-n
16 pw-gpu.x
-input
pw.in
mpirun
-n
16 pw-gpu.x
-input
pw.in
```
but check your system documentation since mpirun may be replaced by
but check your system documentation since mpirun may be replaced by
mpiexec, runjob, aprun, srun, etc. Note also that normally you are not
`
mpiexec, runjob, aprun, srun,
`
etc. Note also that normally you are not
allowed to run MPI programs interactively but must instead use the
allowed to run MPI programs interactively but must instead use the
batch system.
batch system.
A couple of examples for PRACE systems are given in the next section.
A couple of examples for PRACE systems are given in the next section.
6.
Example
##
6. Example
We now give a build and run example.
We now give a build and run example.
Cartesius GPU partition, SURFSARA.
Build
Computer System: Cartesius GPU partition, SURFSARA.
# Download and unpack sources
###Build
#### Download and unpack sources
```
shell
wget http://www.qe-forge.org/gf/download/frsrelease/204/912/espresso-5.4.0.tar.gz
wget http://www.qe-forge.org/gf/download/frsrelease/204/912/espresso-5.4.0.tar.gz
tar
zxvf espresso-5.4.0.tar.gz
tar
zxvf espresso-5.4.0.tar.gz
cd
espresso-5.4.0
cd
espresso-5.4.0
wget https://github.com/fspiga/QE-GPU/archive/5.4.0.tar.gz
wget https://github.com/fspiga/QE-GPU/archive/5.4.0.tar.gz
tar
zxvf 5.4.0.tar.gz
tar
zxvf 5.4.0.tar.gz
ln s QE-GPU-5.4.0 GPU
ln
-s
QE-GPU-5.4.0 GPU
# load compiler modules and compile
```
#### load compiler modules and compile
```
shell
cd
GPU
cd
GPU
module load mpi
module load mpi
module load mkl
module load mkl
...
@@ -128,9 +149,12 @@ module load cuda
...
@@ -128,9 +149,12 @@ module load cuda
--without-magma
--with-phigemm
--without-magma
--with-phigemm
cd
..
cd
..
make
-f
Makefile.gpu pw-gpu
make
-f
Makefile.gpu pw-gpu
```
Running
####
Running
Cartesius uses the SLURM scheduler. An example batch script is given below,
Cartesius uses the SLURM scheduler. An example batch script is given below,
```
shell
#!/bin/bash
#!/bin/bash
#SBATCH -N 6 --ntasks-per-node=16
#SBATCH -N 6 --ntasks-per-node=16
#SBATCH -p gpu
#SBATCH -p gpu
...
@@ -140,11 +164,18 @@ module load fortran mkl mpi/impi cuda
...
@@ -140,11 +164,18 @@ module load fortran mkl mpi/impi cuda
export
LD_LIBRARY_PATH
=
${
LD_LIBRARY_PATH
}
:
${
SURFSARA_MKL_LIB
}
export
LD_LIBRARY_PATH
=
${
LD_LIBRARY_PATH
}
:
${
SURFSARA_MKL_LIB
}
srun pw-gpu.x
-input
pw.in
>
job.out
srun pw-gpu.x
-input
pw.in
>
job.out
```
You should create a file containing the above commands (e.g. myjob.sub) and then submit to the batch system, e.g.
You should create a file containing the above commands
(e.g. myjob.sub) and then submit to the batch system, e.g.
```
sbatch myjob.sub
sbatch myjob.sub
Please check the SURFSara documentation for more information on how to use the batch system.
```
7.
References
Please check the SURFSara documentation for more information on how to
use the batch system.
##7. References
1.
QE-GPU build and download instructions, https://github.com/QEF/qe-gpu-plugin.
1.
QE-GPU build and download instructions, https://github.com/QEF/qe-gpu-plugin.
This diff is collapsed.
Click to expand it.
Preview
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment