Commit d8f5f124 authored by Sagar Dolas's avatar Sagar Dolas
Browse files

NEMO guidelines

parent 1884420e
...@@ -178,24 +178,51 @@ NAMD is written in C++ and parallelised using Charm++ parallel objects, which ar ...@@ -178,24 +178,51 @@ NAMD is written in C++ and parallelised using Charm++ parallel objects, which ar
# NEMO <a name="nemo"></a> # NEMO <a name="nemo"></a>
NEMO (Nucleus for European Modeling of the Ocean) is a state-of-the-art modeling framework for oceanographic research, operational oceanography seasonal forecast and climate studies. Prognostic variables are the three-dimensional velocity field, a linear or non-linear sea surface height, the temperature and the salinity. In the horizontal direction, the model uses a curvilinear orthogonal grid and in the vertical direction, a full or partial step z-coordinate, or s-coordinate, or a mixture of the two. The distribution of variables is a three-dimensional Arakawa C-type grid. Within NEMO, the ocean is interfaced with a sea-ice model (LIM v2 and v3), passive tracer and biogeochemical models (TOP) and, via the OASIS coupler, with several atmospheric general circulation models. It also supports two-way grid embedding via the AGRIF software. NEMO (Nucleus for European Modelling of the Ocean) [22] is mathematical modelling framework for research activities and prediction services in ocean and climate sciences developed by European consortium. It is intended to be tool for studying the ocean and its interaction with the other components of the earth climate system over a large number of space and time scales. It comprises of the core engines namely OPA (ocean dynamics and thermodynamics), SI3 (sea ice dynamics and thermodynamics), TOP (oceanic tracers) and PISCES (biogeochemical process).
Prognostic variables in NEMO are the three-dimensional velocity field, a linear or non-linear sea surface height, the temperature and the salinity.
The framework includes five major components: In the horizontal direction, the model uses a curvilinear orthogonal grid and in the vertical direction, a full or partial step z-coordinate, or s-coordinate, or a mixture of the two. The distribution of variables is a three-dimensional Arakawa C-type grid for most of the cases.
- the blue ocean (ocean dynamics, NEMO-OPA) The model is implemented in Fortran 90, with preprocessing (C-pre-processor). It is optimized for vector computers and parallelized by domain decomposition with MPI. It supports modern C/C++ and Fortran compilers. All input and output is done with third party software called XIOS with dependency on NetCDF (Network Common Data Format) and HDF5. It is highly scalable and perfect application for measuring supercomputing performances in terms of compute capacity, memory subsystem, I/O and interconnect performance.
- the white ocean (sea-ice, NEMO-LIM)
- the green ocean (biogeochemistry, NEMO-TOP)
- the adaptive mesh refinement software (AGRIF)
- the assimilation component (NEMO_TAM)
NEMO is used by a large community: 240 projects in 27 countries (14 in Europe, 13 elsewhere) and 350 registered users (numbers for year 2008). The code is available under the CeCILL license (public license). The latest stable version is 3.6. NEMO is written in Fortran90 and parallelized with MPI. ### Test Case Description
- Web site: http://www.nemo-ocean.eu/ The GYRE configuration has been built to model seasonal cycle of double gyre box model. It consists of idealized domain over which seasonal forcing is applied. This allows for studying large number of interactions and their combined contribution to large scale circulation.
- Code download: http://www.prace-ri.eu/UEABS/NEMO/NEMO_Source.tar.gz The domain geometry is rectangular bounded by vertical walls and flat bottom. The configuration is meant to represent idealized north Atlantic or north pacific basin. The circulation is forced by analytical profiles of wind and buoyancy fluxes.
- Disclaimer: please note that by downloading the code from this website, you agree to be bound by the terms of the GPL license.
- Build instructions: https://repository.prace-ri.eu/git/UEABS/ueabs/blob/r1.3/nemo/NEMO_Build_README.txt The wind stress is zonal and its curl changes sign at 22 and 36. It forces a subpolar gyre in the north, a subtropical gyre in the wider part of the domain and a small recirculation gyre in the southern corner. The net heat flux takes the form of a restoring toward a zonal apparent air temperature profile.
- Test Case A: http://www.prace-ri.eu/UEABS/NEMO/NEMO_TestCaseA.tar.gz
- Run instructions: https://repository.prace-ri.eu/git/UEABS/ueabs/blob/r1.3/nemo/NEMO_Run_README.txt A portion of the net heat flux which comes from the solar radiation is allowed to penetrate within the water column. The fresh water flux is also prescribed and varies zonally. It is determined such as, at each time step, the basin-integrated flux is zero.
The basin is initialized at rest with vertical profiles of temperature and salinity uniformity applied to the whole domain. The GYRE configuration is set through the namelist_cfg file.
The horizontal resolution is determined by setting jp_cfg as follows:
`Jpiglo = 30 x jp_cfg + 2`
`Jpjglo = 20 x jp_cfg + 2`
In this configuration, we use default value of 30 ocean levels depicted by jpk=31. The GYRE configuration is an ideal case for benchmark test as it is very simple to increase the resolution and perform both weak and strong scalability experiment using the same input files. We use two configurations as follows:
**Test Case A**:
* jp_cfg = 128 suitable up to 1000 cores
* Number of Days: 20
* Number of Time steps: 1440
* Time step size: 20 mins
* Number of seconds per time step: 1200
**Test Case B**
* jp_cfg = 256 suitable up to 20,000 cores.
* Number of Days (real): 80
* Number of time step: 4320
* Time step size(real): 20 mins
* Number of seconds per time step: 1200
* Web site: <http://www.nemo-ocean.eu/>
* Download, Build and Run Instructions : <https://repository.prace-ri.eu/git/UEABS/ueabs/tree/master/nemo>
# QCD <a name="qcd"></a> # QCD <a name="qcd"></a>
......
NEMO 3.6, GYRE configuration NEMO 3.6, GYRE configuration
============================ ============================
juha.lento@csc.fi, 2016-05-16 created by juha.lento@csc.fi, 2016-05-16
Build and test documentation for NEMO 3.6 in GYRE Build and test documentation for NEMO 3.6 in GYRE
configuration. Example commands are tested in CSC's Cray XC40, configuration. Example commands are tested in CSC's Cray XC40,
`sisu.csc.fi`. `sisu.csc.fi`.
updated by sagar.dolas@surfsara.nl in context of benchmarking efforts on Tier-0 supercomputing systems.
Download NEMO and XIOS sources Most of the following instructions should work as it is. The key part is to install XIOS because NEMO needs XIOS libraries to compile. The GYRE configuration is pretty easy to compile and run. Please also refer to following guide for reference. There is not one standard rule for compiling NEMO on all plateforms. Each plateform needs a different and unique approach to make NEMO work. The standard workflow is like this :
------------------------------
### Register
http://www.nemo-ocean.eu 1. Install XIOS.
2. Install NEMO.
3. Apply the patch as described here to measure step time : <https://software.intel.com/en-us/articles/building-and-running-nemo-on-xeon-processors>
4. Run NEMO in GYRE configuration.
Download NEMO and XIOS sources
------------------------------
### Check out NEMO sources ### Check out NEMO sources
`
``` svn co -r6542 http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE/NEMOGCM
svn --username USERNAME --password PASSWORD --no-auth-cache co http://forge.ipsl.jussieu.fr/nemo/svn/branches/2015/nemo_v3_6_STABLE/NEMOGCM `
...
Checked out revision 6542.
```
### Check out XIOS2 sources ### Check out XIOS2 sources
http://www.nemo-ocean.eu/Using-NEMO/User-Guides/Basics/XIOS-IO-server-installation-and-use <http://www.nemo-ocean.eu/Using-NEMO/User-Guides/Basics/XIOS-IO-server-installation-and-use>
``` ```
svn co -r819 http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk xios-2.0 svn co -r819 http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/trunk xios-2.0
...@@ -40,7 +39,7 @@ Build XIOS ...@@ -40,7 +39,7 @@ Build XIOS
### Build environment ### Build environment
Xios requires Netcdf4. Xios requires Netcdf4. Please load the appropriate HDF5 and NetCDF4 modules. You might have to change the path in the configuration file.
``` ```
module load cray-hdf5-parallel cray-netcdf-hdf5parallel module load cray-hdf5-parallel cray-netcdf-hdf5parallel
...@@ -49,16 +48,13 @@ module load cray-hdf5-parallel cray-netcdf-hdf5parallel ...@@ -49,16 +48,13 @@ module load cray-hdf5-parallel cray-netcdf-hdf5parallel
### Build command ### Build command
http://forge.ipsl.jussieu.fr/ioserver/wiki/documentation <http://forge.ipsl.jussieu.fr/ioserver/wiki/documentation>
``` ```
cd xios-2.0 cd xios-2.0
./make_xios --job 8 --arch XC30_Cray ./make_xios --arch XC30_Cray
``` ```
...need to be rerun without `--job 8` and test suite is broken, but library got built?
Build NEMO 3.6 in GYRE configuration Build NEMO 3.6 in GYRE configuration
------------------------------------ ------------------------------------
...@@ -92,10 +88,13 @@ fixfcm < ../ARCH/arch-XC40_METO.fcm > ../ARCH/arch-MY_CONFIG.fcm \ ...@@ -92,10 +88,13 @@ fixfcm < ../ARCH/arch-XC40_METO.fcm > ../ARCH/arch-MY_CONFIG.fcm \
./makenemo -m MY_CONFIG -r GYRE_XIOS -n MY_GYRE add_key "key_nosignedzero" ./makenemo -m MY_CONFIG -r GYRE_XIOS -n MY_GYRE add_key "key_nosignedzero"
``` ```
Apply the patch as described here to measure step time : <https://software.intel.com/en-us/articles/building-and-running-nemo-on-xeon-processors>, otherwise it becomes extermely difficult to measure computational time.
Run first GYRE test Run first GYRE test
------------------- -------------------
Adjust the following configuration or settings as per your HPC site's preferences.
### Preapare input files ### Preapare input files
``` ```
...@@ -134,13 +133,3 @@ sed -i -r \ ...@@ -134,13 +133,3 @@ sed -i -r \
namelist_cfg namelist_cfg
``` ```
### Run the experiment as a SLURM batch job
```
sbatch -N 3 -p test -t 30 << EOF
#!/bin/bash
aprun -n 48 ../BLD/bin/nemo.exe : -n 8 ../../../../xios-2.0/bin/xios_server.exe
EOF
```
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment