Commit 0c897302 authored by Charles Moulinec's avatar Charles Moulinec
Browse files

Deleted code_saturne/CS_4.2.2_FOR_KNLs/Code_Saturne_on_KNLs.txt,...

Deleted code_saturne/CS_4.2.2_FOR_KNLs/Code_Saturne_on_KNLs.txt, code_saturne/Code_Saturne_Build_Run_4.0.6.pdf, code_saturne/README_ACC_4.2.2.md files
parent 3fc44a91
************************************************************************************
Code_Saturne 4.2.2 is installed for KNLs. It is also linked to PETSC, but the
default linear solvers are the native ones.
************************************************************************************
Installation
************************************************************************************
The installation script is:
SATURNE_4.2.2/InstallHPC_with_PETSc.sh
The path to PETSC (official released version, and therefore assumed to be installed
on the machine) should be added to the aforementioned script.
After typing ./InstallHPC_with_PETSc.sh the code should be installed and code_saturne be found under:
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne, which should return:
Usage: ./code_saturne <topic>
Topics:
help
autovnv
bdiff
bdump
compile
config
create
gui
info
run
salome
submit
Options:
-h, --help show this help message and exit
************************************************************************************
Two cases are dealt with, TGV_256_CS_OPENMP.tar.gz to test the native solvers, and
CAVITY_13M_FOR_KNLs_WITH_PETSC.tar.gz to test Code_Saturne and PETSC on KNLs
************************************************************************************
First test case: TGV_256_CS_OPENMP.tar.gz
************************************************************************************
In TGV_256_CS_OPENMP.tar.gz are found the mesh and the set of subroutines for Code_Saturne on KNLs, i.e.:
TGV_256_CS_OPENMP/MESH/mesh_input_256by256by256
TGV_256_CS_OPENMP/ARCHER_KNL/SRC/*
To prepare a run, it is required to set up a "study" as, for instance:
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --study NEW_TGV_256_CS_OPENMP KNL
The mesh has to be copied from TGV_256_CS_OPENMP/MESH/mesh_input_256by256by256 into NEW_TGV_256_CS_OPENMP/MESH/.
The subroutines contained in TGV_256_CS_OPENMP/KNL/SRC should be copied into NEW_TGV_256_CS_OPENMP/KNL/SRC
In the DATA subdirectory of NEW_TGV_256_CS_OPENMP/KNL the path to the mesh has to be set as:
cd DATA
cp REFERENCE/cs_user_scripts.py .
edit cs_user_scripts.py
At line 138, change None to "../MESH/mesh_input_256by256by256"
At this stage, everything is set to run the simulation:
cd TGV_256_CS_OPENMP/KNL
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
submit the job
************************************************************************************
Second test case: CAVITY_13M_FOR_KNLs_WITH_PETSC.tar.gz
************************************************************************************
In CAVITY_13M.tar.gz are found the mesh and the set of subroutines for PETSC and KNLs, i.e.:
CAVITY_13M/MESH/mesh_input
CAVITY_13M/KNL/SRC/*
To prepare a run, it is required to set up a "study" as, for instance:
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --study NEW_CAVITY_13M PETSC_KNL
The mesh has to be copied from CAVITY_13M/MESH/mesh_input into NEW_CAVITY_13M/MESH/.
The subroutines contained in CAVITY_13M/KNL/SRC should be copied into NEW_CAVITY_13M/PETSC_KNL/SRC
In the DATA subdirectory of NEW_CAVITY_13M/PETSC_KNL the path to the mesh has to be set as:
cd DATA
cp REFERENCE/cs_user_scripts.py .
edit cs_user_scripts.py
At line 138, change None to "../MESH/mesh_input"
At this stage, everything is set to run the simulation:
cd NEW_CAVITY_13M/PETSC_KNL
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
submit the job
# Code_Saturn
## GPU Version
Code_Saturne 4.2.2 is linked to PETSC developer's version, in order to benefit from
its GPU implementation. Note that the normal release of PETSC does not support GPU.
### Installation
The version has been tested for K80s, and with the following settings:
* OPENMPI 2.0.2
* GCC 4.8.5
* CUDA 7.5
To install Code_Saturne 4.2.2, 4 libraries are required, BLAS, LAPACK, SOWING and CUSP.
The tests have been carried out with lapack-3.6.1 for BLAS and LAPACK, sowing-1.1.23-p1
for SOWING and cusplibrary-0.5.1 for CUSP.
PETSC is first installed, and `PATH_TO_PETSC`, `PATH_TO_CUSP`, `PATH_TO_SOWING`, `PATH_TO_LAPACK`
have to be updated in `INSTALL_PETSC_GPU_sm37` under `petsc-petsc-a31f61e8abd0`
PETSC is configured for K80s, `./INSTALL_PETSC_GPU_sm37` is used from `petsc-petsc-a31f61e8abd0`
It is finally compiled and installed, by typing `make` and `make install`.
Before installing Code\_Saturne, adapt `PATH_TO_PETSC` in `InstallHPC.sh` under `SATURNE_4.2.2`
and type `./InstallHPC.sh`
The code should be installed and code_saturne be found under:
```
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne
```
And should return:
```
Usage: ./code_saturne <topic>
Topics:
help
autovnv
bdiff
bdump
compile
config
create
gui
info
run
salome
submit
Options:
-h, --help show this help message and exit
```
### Test case - Cavity 13M
In `CAVITY_13M.tar.gz` are found the mesh+partitions and 2 sets of subroutines, one for CPU and the
second one for GPU, i.e.:
```
CAVITY_13M/PETSC_CPU/SRC/*
CAVITY_13M/PETSC_GPU/SRC/*
CAVITY_13M/MESH/mesh_input_13M
CAVITY_13M/MESH/partition_METIS_5.1.0/*
```
To prepare a run, it is required to set up a "study" with 2 directories, one for CPU and the other one for GPU
as, for instance:
```
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --study NEW_CAVITY_13M PETSC_CPU
cd NEW_CAVITY_13M
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --case PETSC_GPU
```
The mesh has to be copied from `CAVITY_13M/MESH/mesh_input_13M` into `NEW_CAVITY_13M/MESH/`.
And the same has to be done for `partition_METIS_5.1.0`.
The subroutines contained in `CAVITY_13M/PETSC_CPU/SRC` should be copied into `NEW_CAVITY_13M/PETSC_CPU/SRC` and
the subroutines contained in `CAVITY_13M/PETSC_GPU/SRC` should be copied into `NEW_CAVITY_13M/PETSC_GPU/SRC`.
In each DATA subdirectory of `NEW_CAVITY_13M/PETSC_CPU` and `NEW_CAVITY_13M/PETSC_GPU`, the path
to the mesh+partition has to be set as:
```
cd DATA
cp REFERENCE/cs_user_scripts.py .
```
edit cs_user_scripts.py:
* At line 138, change `None` to `../MESH/mesh_input_13M`
* At line 139, change `None` to `../MESH/partition_METIS_5.1.0`
At this stage, everything is set to run both simulations, one for the CPU and the other one for the GPU.
```
cd NEW_CAVITY_13M/PETSC_CPU
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
```
Then submit the job.
```
cd NEW_CAVITY_13M/PETSC_GPU
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
```
submit the job
## KNL Version
Code_Saturne 4.2.2 is installed for KNLs. It is also linked to PETSC, but the
default linear solvers are the native ones.
### Installation
The installation script is:
```
SATURNE_4.2.2/InstallHPC_with_PETSc.sh
```
The path to PETSC (official released version, and therefore assumed to be installed
on the machine) should be added to the aforementioned script.
After typing `./InstallHPC_with_PETSc.sh` the code should be installed and code_saturne be found under:
```
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne
```
And should return:
```
Usage: ./code_saturne <topic>
Topics:
help
autovnv
bdiff
bdump
compile
config
create
gui
info
run
salome
submit
Options:
-h, --help show this help message and exit
```
### Running the code
Two cases are dealt with, `TGV_256_CS_OPENMP.tar.gz` to test the native solvers, and
`CAVITY_13M_FOR_KNLs_WITH_PETSC.tar.gz` to test Code_Saturne and PETSC on KNLs
#### First test case: TGV_256_CS_OPENMP
In `TGV_256_CS_OPENMP.tar.gz` are found the mesh and the set of subroutines for Code_Saturne on KNLs, i.e.:
```
TGV_256_CS_OPENMP/MESH/mesh_input_256by256by256
TGV_256_CS_OPENMP/ARCHER_KNL/SRC/*
```
To prepare a run, it is required to set up a "study" as, for instance:
```
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --study NEW_TGV_256_CS_OPENMP KNL
```
The mesh has to be copied from `TGV_256_CS_OPENMP/MESH/mesh_input_256by256by256` into `NEW_TGV_256_CS_OPENMP/MESH/`.
The subroutines contained in `TGV_256_CS_OPENMP/KNL/SRC` should be copied into `NEW_TGV_256_CS_OPENMP/KNL/SRC`.
In the `DATA` subdirectory of `NEW_TGV_256_CS_OPENMP/KNL` the path to the mesh has to be set as:
```
cd DATA
cp REFERENCE/cs_user_scripts.py .
```
Then edit `cs_user_scripts.py`:
At line 138, change `None` to `../MESH/mesh_input_256by256by256`
At this stage, everything is set to run the simulation:
```
cd TGV_256_CS_OPENMP/KNL
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
```
And then submit the job.
#### Second test case: CAVITY_13M_FOR_KNLs_WITH_PETSC
In `CAVITY_13M.tar.gz` are found the mesh and the set of subroutines for PETSC and KNLs, i.e.:
```
CAVITY_13M/MESH/mesh_input
CAVITY_13M/KNL/SRC/*
```
To prepare a run, it is required to set up a "study" as, for instance:
```
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne create --study NEW_CAVITY_13M PETSC_KNL
```
The mesh has to be copied from `CAVITY_13M/MESH/mesh_input` into `NEW_CAVITY_13M/MESH/`.
The subroutines contained in `CAVITY_13M/KNL/SRC` should be copied into `NEW_CAVITY_13M/PETSC_KNL/SRC`.
In the `DATA` subdirectory of `NEW_CAVITY_13M/PETSC_KNL` the path to the mesh has to be set as:
```
cd DATA
cp REFERENCE/cs_user_scripts.py .
```
Then edit `cs_user_scripts.py`
At line 138, change `None` to `../MESH/mesh_input`
At this stage, everything is set to run the simulation:
```
cd NEW_CAVITY_13M/PETSC_KNL
PATH_TO_CODE_SATURNE/SATURNE_4.2.2/code_saturne-4.2.2/arch/Linux/bin/code_saturne run --initialize
cd RESU/YYYYMMDD-HHMM
```
Finaly submit the job
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment