Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
************************************************************************************
Code_Saturne 4.2.2 is linked to PETSC developer's version, in order to benefit from
its GPU implementation. Note that the normal release of PETSC does not support GPU.
************************************************************************************
Installation
************************************************************************************
The version has been tested for K80s, and with the following settings:-
-OPENMPI 2.0.2
-GCC 4.8.5
-CUDA 7.5
To install Code_Saturne 4.2.2, 4 libraries are required, BLAS, LAPACK, SOWING and CUSP.
The tests have been carried out with lapack-3.6.1 for BLAS and LAPACK, sowing-1.1.23-p1
for SOWING and cusplibrary-0.5.1 for CUSP.
PETSC is first installed, and PATH_TO_PETSC, PATH_TO_CUSP, PATH_TO_SOWING, PATH_TO_LAPACK
have to be updated in INSTALL_PETSC_GPU_sm37 under petsc-petsc-a31f61e8abd0
PETSC is configured for K80s, ./INSTALL_PETSC_GPU_sm37 is used from petsc-petsc-a31f61e8abd0
It is finally compiled and installed, by typing make and make install.
Before installing Code_Saturne, adapt PATH_TO_PETSC in InstallHPC.sh under SATURNE_4.2.2
and type ./InstallHPC.sh
The code should be installed and code_saturne be found under:
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne, which should return:
Usage: ./code_saturne <topic>
Topics:
help
autovnv
bdiff
bdump
compile
config
create
gui
info
run
salome
submit
Options:
-h, --help show this help message and exit
************************************************************************************
Test case - Cavity 13M
************************************************************************************
In CAVITY_13M.tar.gz are found the mesh+partitions and 2 sets of subroutines, one for CPU and the
second one for GPU, i.e.:
CAVITY_13M/PETSC_CPU/SRC/*
CAVITY_13M/PETSC_GPU/SRC/*
CAVITY_13M/MESH/mesh_input_13M
CAVITY_13M/MESH/partition_METIS_5.1.0/*
To prepare a run, it is required to set up a "study" with 2 directories, one for CPU and the other one for GPU
as, for instance:
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --study NEW_CAVITY_13M PETSC_CPU
cd NEW_CAVITY_13M
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --case PETSC_GPU
The mesh has to be copied from CAVITY_13M/MESH/mesh_input_13M into NEW_CAVITY_13M/MESH/.
And the same has to be done for partition_METIS_5.1.0.
The subroutines contained in CAVITY_13M/PETSC_CPU/SRC should be copied into NEW_CAVITY_13M/PETSC_CPU/SRC and
the subroutines contained in CAVITY_13M/PETSC_GPU/SRC should be copied into NEW_CAVITY_13M/PETSC_GPU/SRC.
In each DATA subdirectory of NEW_CAVITY_13M/PETSC_CPU and NEW_CAVITY_13M/PETSC_GPU, the path
to the mesh+partition has to be set as:
cd DATA
cp REFERENCE/cs_user_scripts.py .
edit cs_user_scripts.py
At line 138, change None to "../MESH/mesh_input_13M"
At line 139, change None to "../MESH/partition_METIS_5.1.0"
At this stage, everything is set to run both simulations, one for the CPU and the other one for the GPU.
cd NEW_CAVITY_13M/PETSC_CPU
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
submit the job
cd NEW_CAVITY_13M/PETSC_GPU
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
submit the job