Commit 5d92ea10 authored by Ozan Karsavuran's avatar Ozan Karsavuran
Browse files

update readme for nemo

parent a54fc041
# NEMO
## Summary Version
1.0
1.1
## Purpose of Benchmark
......@@ -35,90 +36,90 @@ The model is implemented in Fortran 90, with pre-processing (C-pre-processor). I
```
./make_xios --arch local
```
Files for the PRACE Tier-0 systems are available under [architecture_files](architecture_files) folder.
Note that XIOS requires `Netcdf4`. Please load the appropriate `HDF5` and `NetCDF4` modules. You might have to change the path in the configuration file.
Note that XIOS requires `Netcdf4`. Please load the appropriate `HDF5` and `NetCDF4` modules. If path to these models are not loaded, you might have to change the path in the configuration file.
### Building NEMO
1. Download the XIOS source code:
```
svn co https://forge.ipsl.jussieu.fr/nemo/svn/NEMO/releases/release-4.0
```
2. Copy and setup the appropriate architecture file in the arch folder. The following changes are recommended:
2. Copy and setup the appropriate architecture file in the arch folder. Files for the PRACE Tier-0 systems are available under [architecture_files](architecture_files) folder. The following changes are recommended for the GNU compilers:
```
a. add the `-lnetcdff` and `-lstdc++` flags to NetCDF flags
b. using `mpif90` which is a MPI binding of `gfortran-4.9`
c. add `-cpp` and `-ffree-line-length-none` to Fortran flags
d. swap out `gmake` with `make`
```
3. Then build the executable with the following command
```
./makenemo -m MY_CONFIG -r GYRE_XIOS -n MY_GYRE add_key "key_nosignedzero"
```
4. Apply the patch as described here to measure step time :
3. Apply the patch as described here to measure step time :
```
https://software.intel.com/en-us/articles/building-and-running-nemo-on-xeon-processors
```
You may also use [nemogcm.F90](nemogcm.F90) by replacing it with `src/OCE/nemogcm.F90`
4. Add `GYRE_testing OCE TOP` line to `refs_cfg.txt` file.
Then go to cfgs folder and:
``` mkdir GYRE_testing ```
``` rsync -arv GYRE_PISCES/* GYRE_testing/ ```
``` mv GYRE_testing/cpp_GYRE_PISCES.fcm GYRE_testing/cpp_GYRE_testing.fcm ```
``` In GYRE_testing/cpp_GYRE_testing.fcm file replace key_top with key_nosignedzero
```
5. Then build the executable with the following command
```
./makenemo -m MY_CONFIG -r GYRE_testing
```
## Mechanics of Running Benchmark
### Prepare input files
cd MY_GYRE/EXP00
cd GYRE_testing/EXP00
sed -i '/using_server/s/false/true/' iodef.xml
sed -i '/&nameos/a ln_useCT = .false.' namelist_cfg
sed -i '/&namctl/a nn_bench = 1' namelist_cfg
sed -i '/ln_bench/s/false/true/' namelist_cfg
### Run the experiment interactively
mpirun -n 4 ../BLD/bin/nemo.exe -n 2 $PATH_TO_XIOS/bin/xios_server.exe
mpirun -n 4 nemo : -n 2 $PATH_TO_XIOS/bin/xios_server.exe
### GYRE configuration with higher resolution
Modify configuration (for example for the test case A):
```
rm -f time.step solver.stat output.namelist.dyn ocean.output slurm-* GYRE_* mesh_mask_00*
jp_cfg=4
rm -f time.step solver.stat output.namelist.dyn ocean.output slurm-* GYRE_*
sed -i -r \
-e 's/^( *nn_itend *=).*/\1 21600/' \
-e 's/^( *nn_stock *=).*/\1 21600/' \
-e 's/^( *nn_write *=).*/\1 1000/' \
-e 's/^( *jp_cfg *=).*/\1 '"$jp_cfg"'/' \
-e 's/^( *jpidta *=).*/\1 '"$(( 30 * jp_cfg +2))"'/' \
-e 's/^( *jpjdta *=).*/\1 '"$(( 20 * jp_cfg +2))"'/' \
-e 's/^( *jpiglo *=).*/\1 '"$(( 30 * jp_cfg +2))"'/' \
-e 's/^( *jpjglo *=).*/\1 '"$(( 20 * jp_cfg +2))"'/' \
-e 's/^( *nn_itend *=).*/\1 101/' \
-e 's/^( *nn_write *=).*/\1 4320/' \
-e 's/^( *nn_GYRE *=).*/\1 48/' \
-e 's/^( *rn_rdt *=).*/\1 1200/' \
namelist_cfg
```
## Verification of Results
The GYRE configuration is set through the `namelist_cfg` file. The horizontal resolution is determined by setting `jp_cfg` as follows:
The GYRE configuration is set through the `namelist_cfg` file. The horizontal resolution is determined by setting `nn_GYRE` as follows:
```
Jpiglo = 30 × jp_cfg + 2
Jpjglo = 20 × jp_cfg + 2
Jpiglo = 30 × nn_GYRE + 2
Jpjglo = 20 × nn_GYRE + 2
```
In this configuration, we use a default value of 30 ocean levels, depicted by `jpk=31`. The GYRE configuration is an ideal case for benchmark tests as it is very simple to increase the resolution and perform both weak and strong scalability experiment using the same input files. We use two configurations as follows:
In this configuration, we use a default value of 30 ocean levels, depicted by `jpkglo=31`. The GYRE configuration is an ideal case for benchmark tests as it is very simple to increase the resolution and perform both weak and strong scalability experiment using the same input files. We use two configurations as follows:
Test Case A:
```
jp_cfg = 128 suitable up to 1000 cores
Number of Days: 20
Number of Time steps: 1440
nn_GYRE = 48 suitable up to 1000 cores
Number of Time steps: 101
Time step size: 20 mins
Number of seconds per time step: 1200
```
Test Case B:
```
jp_cfg = 256 suitable up to 20,000 cores.
Number of Days (real): 80
Number of time step: 4320
nn_GYRE = 192 suitable up to 20,000 cores.
Number of time step: 101
Time step size(real): 20 mins
Number of seconds per time step: 1200
```
We performed scalability test on 512 cores and 1024 cores for test case A. We performed scalability test for 4096 cores, 8192 cores and 16384 cores for test case B.
Both these test cases can give us quite good understanding of node performance and interconnect behavior.
We switch off the generation of mesh files by setting the `flag nn_mesh = 0` in the `namelist_ref` file. Also `using_server = false` is defined in `io_server` file.
<!--We switch off the generation of mesh files by setting the `flag nn_mesh = 0` in the `namelist_ref` file.
Also `using_server = false` is defined in `io_server` file.-->
We report the performance in step time which is the total computational time averaged over the number of time steps for different test cases.
This helps us to compare systems in a standard manner across all combinations of system architectures.
......
This diff is collapsed.
#!/usr/bin/env bash
# A tool to modify XML files used by FCM
# This is just a regexp search and replace, not a proper XML
# parser. Use at own risk.
fixfcm() {
local name value prog=""
for arg in "$@"; do
name="${arg%%=*}"
value=$(printf %q "${arg#*=}")
value="${value//\//\/}"
prog="s/(^%${name} )(.*)/\\1 ${value}/"$'\n'"$prog"
done
sed -r -e "$prog"
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment