Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
UEABS
ueabs
Commits
b6bd1751
Commit
b6bd1751
authored
Mar 28, 2017
by
Victor
Browse files
IMPROVE gpaw formating
parent
d0960e4d
Changes
1
Hide whitespace changes
Inline
Side-by-side
gpaw/README.md
View file @
b6bd1751
...
...
@@ -89,23 +89,23 @@ are two additional steps compared to a standard installation:
2.
Edit your GPAW setup script (customize.py) to add correct link and compile
options for CUDA. The relevant lines are e.g.:
```
python
define_macros
+=
[(
'GPAW_CUDA'
,
'1'
)]
libraries
+=
[
'gpaw-cuda'
,
'cublas'
,
'cudart'
,
'stdc++'
]
library_dirs
+=
[
'./c/cuda'
,
'/path/to/cuda/lib64'
]
include_dirs
+=
[
'/path/to/cuda/include'
]
```
```python
define_macros += [('GPAW_CUDA', '1')]
libraries += [
'gpaw-cuda',
'cublas',
'cudart',
'stdc++'
]
library_dirs += [
'./c/cuda',
'/path/to/cuda/lib64'
]
include_dirs += [
'/path/to/cuda/include'
]
```
## Xeon Phi MICs
...
...
@@ -158,31 +158,31 @@ In addition, pyMIC requires:
In addition to using Intel compilers, there are three additional steps apart
from standard installation:
1
)
Compile and install Numpy with a suitable site.cfg to use MKL, e.g.
1
.
Compile and install Numpy with a suitable site.cfg to use MKL, e.g.
```
python
[
mkl
]
library_dirs
=
/
path
/
to
/
mkl
/
lib
/
intel64
include_dirs
=
/
path
/
to
/
mkl
/
include
lapack_libs
=
mkl_libs
=
mkl_rt
```
```python
[mkl]
library_dirs = /path/to/mkl/lib/intel64
include_dirs = /path/to/mkl/include
lapack_libs =
mkl_libs = mkl_rt
```
2
)
Compile and install pyMIC before GPAW.
2
.
Compile and install pyMIC before GPAW.
3
)
Edit your GPAW setup script (customize.py) to add correct link and compile
options for offloading. The relevant lines are e.g.:
3
.
Edit your GPAW setup script (customize.py) to add correct link and compile
options for offloading. The relevant lines are e.g.:
```
python
# offload to KNC
extra_compile_args
+=
[
'-qoffload-option,mic,compiler,"-qopenmp"'
]
extra_compile_args
+=
[
'-qopt-report-phase=offload'
]
```python
# offload to KNC
extra_compile_args += ['-qoffload-option,mic,compiler,"-qopenmp"']
extra_compile_args += ['-qopt-report-phase=offload']
# linker settings for MKL on KNC
mic_mkl_lib
=
'/path/to/mkl/lib/mic/'
extra_link_args
+=
[
'-offload-option,mic,link,"-L'
+
mic_mkl_lib
\
+
' -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread"'
]
```
# linker settings for MKL on KNC
mic_mkl_lib = '/path/to/mkl/lib/mic/'
extra_link_args += ['-offload-option,mic,link,"-L' + mic_mkl_lib \
+ ' -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lpthread"']
```
...
...
@@ -211,11 +211,14 @@ To improve performance, one may also link to Intel TBB to benefit from an
optimised memory allocator (tbbmalloc). This can be done during installation
or at run-time by setting environment variable LD_PRELOAD to point to the
correct libraries, i.e. for example:
export LD_PRELOAD=$TBBROOT/lib/intel64/gcc4.7/libtbbmalloc_proxy.so.2
export LD_PRELOAD=$LD_PRELOAD:$TBBROOT/lib/intel64/gcc4.7/libtbbmalloc.so.2
```
shell
export
LD_PRELOAD
=
$TBBROOT
/lib/intel64/gcc4.7/libtbbmalloc_proxy.so.2
export
LD_PRELOAD
=
$LD_PRELOAD
:
$TBBROOT
/lib/intel64/gcc4.7/libtbbmalloc.so.2
```
It may also be beneficial to use hugepages together with tbbmalloc
(export TBB_MALLOC_USE_HUGE_PAGES=1).
(export TBB_MALLOC_USE_HUGE_PAGES=1
`
).
## Run instructions for PRACE Accelerator Benchmark for GPAW
...
...
@@ -257,19 +260,22 @@ Input file: copper-filament/input.py
No special command line options or environment variables are needed to run the
benchmarks on GPGPUs or KNL (Xeon Phi Knights Landing) MICs. One can simply
say e.g.
mpirun -np 256 gpaw-python input.py
`
mpirun -np 256 gpaw-python input.py
`
For KNCs (Xeon Phi Knights Corner), one needs to use a wrapper script to set
correct affinities for pyMIC (see setup/affinity-wrapper.sh for an example)
and to set two environment variables for GPAW:
GPAW_OFFLOAD=1 (to turn on offloading)
GPAW_PPN=
<no.
of
MPI
tasks
per
node
>
```shel
GPAW_OFFLOAD=1 (to turn on offloading)
GPAW_PPN=<no. of MPI tasks per node>
```
For example, in a SLURM system, this could be:
GPAW_PPN=12 GPAW_OFFLOAD=1 mpirun -np 256 -bootstrap slurm
\
`
``
shell
GPAW_PPN=12 GPAW_OFFLOAD=1 mpirun -np 256 -bootstrap slurm
\
./affinity-wrapper.sh gpaw-python input.py
```
Example job scripts (setup/job-*.sh) for different accelerator architectures
are provided together with related machine specifications (setup/specs.*) that
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment