Commit 45190c63 authored by Cedric Jourdain's avatar Cedric Jourdain 🐵
Browse files

Fix merge confilcts

parents 186cefd6 83ad2e36
This diff is collapsed.
Alya builds the makefile from the compilation options defined in config.in. In order to build ALYA (Alya.x), please follow these steps:
- Goto to directory: Executables/unix
- Edit config.in (some default config.in files can be found in directory configure.in):
- Select your own MPI wrappers and paths
- Select size of integers. Default is 4 bytes, For 8 bytes, select -DI8
- Choose your metis version, metis-4.0 or metis-5.1.0_i8 for 8-bytes integers
- Configure Alya: ./configure -x nastin parall
- Compile metis: make metis4 or make metis5
- Compile Alya: make
Data sets
---------
The parameters used in the datasets try to represent at best typical industrial runs in order to obtain representative speedups. For example, the iterative solvers are never converged to machine accuracy, but only as a percentage of the initial residual.
The different datasets are:
SPHERE_16.7M ... 16.7M sphere mesh
SPHERE_132M .... 132M sphere mesh
How to execute Alya with a given dataset
----------------------------------------
In order to run ALYA, you need at least the following input files per execution:
X.dom.dat
X.ker.dat
X.nsi.dat
X.dat
In our case X=sphere
To execute a simulation, you must be inside the input directory and you should submit a job like:
mpirun Alya.x sphere
How to measure the speedup
--------------------------
There are many ways to compute the scalability of Nastin module.
1. For the complete cycle including: element assembly + boundary assembly + subgrid scale assembly + solvers, etc.
2. For single kernels: element assembly, boundary assembly, subgrid scale assembly, solvers
3. Using overall times
1. In *.nsi.cvg file, column "30. Elapsed CPU time"
2. Single kernels. Here, average and maximum times are indicated in *.nsi.cvg at each iteration of each time step:
Element assembly: 19. Ass. ave cpu time 20. Ass. max cpu time
Boundary assembly: 33. Bou. ave cpu time 34. Bou. max cpu time
Subgrid scale assembly: 31. SGS ave cpu time 32. SGS max cpu time
Iterative solvers: 21. Sol. ave cpu time 22. Sol. max cpu time
Note that in the case of using Runge-Kutta time integration (the case of the sphere), the element and boundary assembly times are this of the last assembly of current time step (out of three for third order).
3. At the end of *.log file, total timings are shown for all modules. In this case we use the first value of the NASTIN MODULE.
Contact
-------
If you have any question regarding the runs, please feel free to contact Guillaume Houzeaux: guillaume.houzeaux@bsc.es
......@@ -20,9 +20,9 @@ The Alya System is a Computational Mechanics code capable of solving different p
## Mechanics of Building Benchmark
Alya builds the makefile from the compilation options defined in config.in. In order to build ALYA (Alya.x), please follow these steps:
Alya builds the makefile from the compilation options defined in config.in. In order to build ALYA (Alya.x), please follow these steps after unpack the tar.gz:
Goto to directory: Executables/unix
Go to to directory: Executables/unix
```
cd Executables/unix
......@@ -46,6 +46,10 @@ or
make metis5
Finally, compile Alya:
make -j 8
## Mechanics of Running Benchmark
......@@ -55,8 +59,8 @@ The parameters used in the datasets try to represent at best typical industrial
The different datasets are:
SPHERE_16.7M ... 16.7M sphere mesh
SPHERE_132M .... 132M sphere mesh
Test Case A: SPHERE_16.7M ... 16.7M sphere mesh
Test Case B: SPHERE_132M .... 132M sphere mesh
### How to execute Alya with a given dataset
......@@ -73,7 +77,7 @@ To execute a simulation, you must be inside the input directory and you should s
mpirun Alya.x sphere
How to measure the speedup
How to measure the performance
--------------------------
There are many ways to compute the scalability of Nastin module.
......
# Alya - Large Scale Computational Mechanics
Alya is a simulation code for high performance computational mechanics. Alya solves coupled multiphysics problems using high performance computing techniques for distributed and shared memory supercomputers, together with vectorization and optimization at the node level.
Homepage: https://www.bsc.es/research-development/research-areas/engineering-simulations/alya-high-performance-computational
Alya is avaialble to collaboratoring projects and a specific version is being distributed as part of the PRACE Unified European Applications Benchmark Suite (http://www.prace-ri.eu/ueabs/#ALYA)
## Building Alya for GPU accelerators
The library currently supports four solvers:GMRES, Deflated Conjugate Gradient, Conjugate Gradient, and Pipelined Conjugate Gradient.
The only pre-conditioner supported at the moment is 'diagonal'.
Keywords to use the solvers:
```shell
NINJA GMRES : GGMR
NINJA Deflated CG : GDECG
NINJA CG : GCG
NINJA Pipelined CG : GPCG
PRECONDITIONER : DIAGONAL
```
Other options are same a CPU based solver.
### GPGPU Building
This version was tested with the Intel Compilers 2017.1, bullxmpi-1.2.9.1 and NVIDIA CUDA 7.5. Ensure that the wrappers `mpif90` and `mpicc` point to the correct binaries and that `$CUDA_HOME` is set.
Alya can be used with just MPI or hybrid MPI-OpenMP parallelism. Standard execution mode is to rely on MPI only.
- Uncompress the source and configure the depending Metis library and Alya build options:
```shell
tar xvf alya-prace-acc.tar.bz2
```
- Edit the file `Alya/Thirdparties/metis-4.0/Makefile.in` to select the compiler and target platform. Uncomment the specific lines and add optimization parameters, e.g.
```shell
OPTFLAGS = -O3 -xCORE-AVX2
```
- Then build Metis4
```shell
$ cd Alya/Executables/unix
$ make metis4
```
- For Alya there are several example configurations, copy one, e.g. for Intel Compilers:
```shell
$ cp configure.in/config_ifort.in config.in
```
- Edit the config.in:
Add the corresponding platform optimization flags to `FCFLAGS`, e.g.
```shell
FCFLAGS = -module $O -c -xCORE-AVX2
```
- MPI: No changes in the configure file are necessary. By default you use metis4 and 4 byte integers.
- MPI-hybrid (with OpenMP) : Uncomment the following lines for OpenMP version:
```shell
CSALYA := $(CSALYA) -qopenmp (-fopenmp for GCC Compilers)
EXTRALIB := $(EXTRALIB) -qopenmp (-fopenmp for gcc Compilers)
```
- Configure and build Alya (-x Release version; -g Debug version, plus uncommenting debug and checking flags in config.in)
```shell
./configure -x nastin parall
make NINJA=1 -j num_processors
```
### GPGPU Usage
Each problem needs a `GPUconfig.dat`. A sample is available at `Alya/Thirdparties/ninja` and needs to be copied to the work directory. A README file in the same location provides further information.
- Extract the small one node test case and configure to use GPU solvers:
```shell
$ tar xvf cavity1_hexa_med.tar.bz2 && cd cavity1_hexa_med
$ cp ../Alya/Thirdparties/ninja/GPUconfig.dat .
```
- To use the GPU, you have to replace `GMRES` with `GGMR` and `DEFLATED_CG` with `GDECG`, both in `cavity1_hexa.nsi.dat`
- Edit the job script to submit the calculation to the batch system.
```shell
job.sh: Modify the path where you have your Alya.x (compiled with MPI options)
sbatch job.sh
```
Alternatively execute directly:
```shell
OMP_NUM_THREADS=4 mpirun -np 16 Alya.x cavity1_hexa
```
<!-- Runtime on 16-core Xeon E5-2630 v3 @ 2.40GHz with 2 NVIDIA K80: ~1:30 min -->
<!-- Runtime on 16-core Xeon E5-2630 v3 @ 2.40GHz no GPU: ~2:00 min -->
## Building Alya for Intel Xeon Phi Knights Landing (KNL)
The Xeon Phi processor version of Alya is currently relying on compiler assisted optimization for AVX-512. Porting of performance critical kernels to the new assembly instructions is underway. There will not be a version for first generation Xeon Phi Knights Corner coprocessors.
### KNL Building
This version was tested with the Intel Compilers 2017.1, Intel MPI 2017.1. Ensure that the wrappers `mpif90` and `mpicc` point to the correct binaries.
Alya can be used with just MPI or hybrid MPI-OpenMP parallelism. Standard execution mode is to rely on MPI only.
- Uncompress the source and configure the depending Metis library and Alya build options:
```shell
tar xvf alya-prace-acc.tar.bz2
```
- Edit the file `Alya/Thirdparties/metis-4.0/Makefile.in` to select the compiler and target platform. Uncomment the specific lines and add optimization parameters, e.g.
```shell
OPTFLAGS = -O3 -xMIC-AVX512
```
- Then build Metis4
```shell
$ cd Alya/Executables/unix
$ make metis4
```
- For Alya there are several example configurations, copy one, e.g. for Intel Compilers:
```shell
$ cp configure.in/config_ifort.in config.in
```
- Edit the config.in:
Add the corresponding platform optimization flags to `FCFLAGS`, e.g.
```shell
FCFLAGS = -module $O -c -xMIC-AVX512
```
- MPI: No changes in the configure file are necessary. By default you use metis4 and 4 byte integers.
- MPI-hybrid (with OpenMP) : Uncomment the following lines for OpenMP version:
```shell
CSALYA := $(CSALYA) -qopenmp (-fopenmp for GCC Compilers)
EXTRALIB := $(EXTRALIB) -qopenmp (-fopenmp for gcc Compilers)
```
- Configure and build Alya (-x Release version; -g Debug version, plus uncommenting debug and checking flags in config.in)
```shell
./configure -x nastin parall
make -j num_processors
```
## Remarks
If the number of elements is too low for a scalability analysis, Alya includes a mesh multiplication technique. This tool can be used by selecting an input option in the ker.dat file. This option is the number of mesh multiplication levels one wants to apply (0 meaning no mesh multiplication). At each multiplication level, the number of elements is multiplied by 8, so one can obtain a huge mesh automatically in order to study the scalability of the code on different architectures. Note that the mesh multiplication is carried out in parallel and thus should not impact the duration of the simulation process.
This diff is collapsed.
###################################################################
# PGI CONFIGURE #
#POWER9 RECOMENDED MODULE: #
#module load ompi/3.0.0 pgi/18.4 #
###################################################################
F77 = OMPI_FC=pgfortran mpif90
F90 = OMPI_FC=pgfortran mpif90
FCOCC = cc -c
FCFLAGS = -c -fast -Minfo=all -acc -ta=tesla:cuda10.1 -Mpreprocess -I./Objects_x/ -Mbackslash -Mextend -Mnoopenmp -Munroll -Mnoidiom -module $O
FPPFLAGS =
EXTRALIB = -lc
EXTRAINC =
fa2p = pgfortran -c -x f95-cpp-input -DMPI_OFF -J../../Utils/user/alya2pos -I../../Utils/user/alya2pos
fa2plk = pgfortran
###################################################################
# PERFORMANCE FLAGS #
###################################################################
#MINUM
#FOPT = -O1
#MAXIMUM (I have elimated -xHost due to observations by Yacine)
FOPT = -O3
#Compilation flags applied only to the Source/modules folder
#MODULEFLAGS = -ipo
# Uncomment the following line to enable NDIME as a parameter (OPTIMIZATION FOR 3D PROBLEMS)
CSALYA := $(CSALYA) -DNDIMEPAR -DOPENACCHHH -DSUPER_FAST -DDETAILED_TIMES
# Uncomment the following line for DEBUG AND CHECKING FLAGS
#CSALYA := $(CSALYA) -C -Ktrap=fp -Minform=inform
# Vectorization: put vector size (in principle=4 for MN)
CSALYA := $(CSALYA) -DVECTOR_SIZE=32768
###################################################################
# USER SPECIFIC FLAGS #
###################################################################
# HERBERT
#CSALYA := $(CSALYA) -DDETAILS_ORTHOMIN
###################################################################
# PROFILING FLAGS #
###################################################################
# Uncomment the following line to generate profiling info files
#CSALYA := $(CSALYA) -profile-loops=all -profile-loops-report=2
###################################################################
# EXTRAE FLAGS #
###################################################################
# Uncomment the following line to compile Alya using extrae
# Compiler used to compile extrae module (make extrae)
EXTRAE_FC=pgfortran
# Extrae installation directory (for linking) (not necessary if loading extrae using module load extrae)
#EXTRAE_HOME=
#@Linking with Extrae
#EXTRALIB := $(EXTRALIB) -L${EXTRAE_HOME}/lib/ -lmpitracef extrae_module.o
#@Enabling Extrae user calls (normal)
#CSALYA := $(CSALYA) -DEXTRAE
###################################################################
# METIS LIBRARY #
###################################################################
# Uncomment the following lines for using metis 4.0 (default)
EXTRALIB := $(EXTRALIB) -L../../Thirdparties/metis-4.0 -lmetis -acc -ta=tesla:cuda10.1
CSALYA := $(CSALYA) -DMETIS
# Uncomment the following lines for using metis 5.0
#CSALYA := $(CSALYA) -DV5METIS
# uncoment FOR MAC
#EXTRALIB := $(EXTRALIB) -L../../Thirdparties/metis-5.0.2_i8/build/Darwin-i386/libmetis -lmetis
# uncoment FOR LINUX
#EXTRALIB := $(EXTRALIB) -L../../Thirdparties/metis-5.0.2_i8/build/Linux-x86_64/libmetis -lmetis
# Uncomment the following lines for using parmetis
#CSALYA := $(CSALYA) -DPARMETIS
#EXTRALIB := $(EXTRALIB) -L../../Thirdparties/
###################################################################
# BLAS LIBRARY #
###################################################################
# Uncomment the following lines for using blas
#EXTRALIB := $(EXTRALIB) -lblas
#CSALYA := $(CSALYA) -DBLAS
###################################################################
# MPI/THREADS/TASKS #
###################################################################
# Uncomment the following lines for sequential (NOMPI) version
#VERSIONMPI = nompi
#CSALYA := $(CSALYA) -DMPI_OFF
# Uncomment the following lines for OPENMP version
#CSALYA := $(CSALYA) -openmp
#EXTRALIB := $(EXTRALIB) -openmp
# Uncomment the following line to disable OPENMP coloring
#CSALYA := $(CSALYA) -DNO_COLORING
# Uncomment the following line to enable OMPSS in order to activate
# loops with multidependencies whenever we have a race condition
#CSALYA := $(CSALYA) -DALYA_OMPSS
###################################################################
# DLB #
# To use DLB with OMPSS: Define -DALYA_DLB -DALYA_OMPSS #
# To use DLB barrier: Define -DALYA_DLB_BARRIER #
# If we compile with mercurium, there is no need to link #
# with DLB #
# #
###################################################################
#CSALYA := $(CSALYA) -DALYA_DLB
#FCFLAGS := $(FCFLAGS) --dlb
#FPPFLAGS := $(FPPFLAGS) --dlb
###################################################################
# INTEGER TYPE #
###################################################################
# Uncomment the following lines for 8 byte integers
#CSALYA := $(CSALYA) -DI8 -m64
# Uncomment the following line for 8 byte integers ONLY in Metis
# In this mode Alya can work in I4 and Metis in I8 (mixed mode)
# For this option the use of Metis v5 library is mandatory
#CSALYA := $(CSALYA) -DMETISI8
###################################################################
# HDF5 #
# #
#MANDATORY MODULES TO LOAD ON MN3 FOR 4 BYTE INTEGERS VERSION #
#module load HDF5/1.8.14 SZIP #
# #
#MANDATORY MODULES TO LOAD ON MN3 FOR 8 BYTE INTEGERS VERSION #
#module load SZIP/2.1 HDF5/1.8.15p1_static #
# #
#MANDATORY SERVICES TO COMPILE #
#hdfpos #
#MORE INFO:bsccase02.bsc.es/alya/tutorial/hdf5_output.html #
###################################################################
# Uncomment the following lines for using HDF5 IN 4 BYTE INTEGER VERSION
#CSALYA := $(CSALYA) -I/apps/HDF5/1.8.14/INTEL/IMPI/include
#EXTRALIB := $(EXTRALIB) /apps/HDF5/1.8.14/INTEL/IMPI/lib/libhdf5_fortran.a /apps/HDF5/1.8.14/INTEL/IMPI/lib/libhdf5.a
#EXTRALIB := $(EXTRALIB) /apps/HDF5/1.8.14/INTEL/IMPI/lib/libhdf5hl_fortran.a /apps/HDF5/1.8.14/INTEL/IMPI/lib/libhdf5_hl.a
#EXTRALIB := $(EXTRALIB) -L/apps/SZIP/2.1/lib -lsz -L/usr/lib64 -lz -lgpfs
# Uncomment the following lines for using HDF5 IN 8 BYTE INTEGER VERSION
#CSALYA := $(CSALYA) -I/apps/HDF5/1.8.15p1/static/include
#EXTRALIB := $(EXTRALIB) /gpfs/apps/MN3/HDF5/1.8.15p1/static/lib/libhdf5_fortran.a /gpfs/apps/MN3/HDF5/1.8.15p1/static/lib/libhdf5_f90cstub.a
#EXTRALIB := $(EXTRALIB) /gpfs/apps/MN3/HDF5/1.8.15p1/static/lib/libhdf5.a /gpfs/apps/MN3/HDF5/1.8.15p1/static/lib/libhdf5_tools.a
#EXTRALIB := $(EXTRALIB) -L/apps/SZIP/2.1/lib -lsz -L/usr/lib64 -lz -lgpfs
# latest Hdf5 (1.8.16) to be used with latest impi (5.1.2.150) and intel(16.0.1)
# there is module load HDF5/1.8.16-mpi for integers 4 but I have seen it is not necesary to call it
# Toni should clean up this - my reconmendation is to only leave this last version
# moreover there seems to be no need for module load hdf5 (in the i8 case there exists no module load)
# Uncomment the following lines for using HDF5 IN 4 BYTE INTEGER VERSION
#CSALYA := $(CSALYA) -I/apps/HDF5/1.8.16/INTEL/IMPI/include
#EXTRALIB := $(EXTRALIB) /apps/HDF5/1.8.16/INTEL/IMPI/lib/libhdf5_fortran.a /apps/HDF5/1.8.16/INTEL/IMPI/lib/libhdf5hl_fortran.a
#EXTRALIB := $(EXTRALIB) /apps/HDF5/1.8.16/INTEL/IMPI/lib/libhdf5.a /apps/HDF5/1.8.16/INTEL/IMPI/lib/libhdf5_hl.a
#EXTRALIB := $(EXTRALIB) -L/apps/SZIP/2.1/lib -lsz -L/usr/lib64 -lz -lgpfs
# Uncomment the following lines for using HDF5 IN 8 BYTE INTEGER VERSION
#CSALYA := $(CSALYA) -I/apps/HDF5/1.8.16/INTEL/IMPI/int8/include
#EXTRALIB := $(EXTRALIB) /apps/HDF5/1.8.16/INTEL/IMPI/int8/lib/libhdf5_fortran.a /apps/HDF5/1.8.16/INTEL/IMPI/int8/lib/libhdf5hl_fortran.a
#EXTRALIB := $(EXTRALIB) /apps/HDF5/1.8.16/INTEL/IMPI/int8/lib/libhdf5.a /apps/HDF5/1.8.16/INTEL/IMPI/int8/lib/libhdf5_hl.a
#EXTRALIB := $(EXTRALIB) -L/apps/SZIP/2.1/lib -lsz -L/usr/lib64 -lz -lgpfs
###################################################################
# VTK #
# #
#MANDATORY THIRDPARTY COMPILATION #
#Go to Alya/Thirdparties/VTK #
# #
#MORE INFO:bsccase02.bsc.es/alya/tutorial/vtk_output.html #
###################################################################
# Uncomment the following lines for using VTK as output
#CSALYA := $(CSALYA) -DVTK
#EXTRALIB := $(EXTRALIB) ../../Thirdparties/VTK/vtkXMLWriterF.o -L/apps/VTK/6.1.0_patched/lib -lvtkIOXML-6.1 -lvtkIOGeometry-6.1
#EXTRALIB := $(EXTRALIB) -lvtkIOXMLParser-6.1 -lvtksys-6.1 -lvtkIOCore-6.1 -lvtkCommonExecutionModel-6.1 -lvtkCommonDataModel-6.1 -lvtkCommonMisc-6.1
#EXTRALIB := $(EXTRALIB) -lvtkCommonSystem-6.1 -lvtkCommonTransforms-6.1 -lvtkCommonMath-6.1 -lvtkCommonCore-6.1 -lvtkzlib-6.1
#EXTRALIB := $(EXTRALIB) -lvtkjsoncpp-6.1 -lvtkexpat-6.1 -L/gpfs/apps/MN3/INTEL/tbb/lib/intel64/ -ltbb
###################################################################
# NINJA #
#GPU based solvers : GMRES,DEFLATED_CG,CG #
#Specify solver in configuration as: #
#GMRES -------------> GGMR #
#CG ----------------> GCG #
#DEFLATED_CG -------> GDECG #
#GPU Multi Grid-----> GAMGX(Requires CONFIGURATION_FILE in solver)#
#export CUDA_HOME to CUDA version to be used #
###################################################################
# Uncomment the following lines to enable NINJA
#GPU_HOME := ${CUDA_HOME}/
#CSALYA := $(CSALYA) -DNINJA
#EXTRALIB := $(EXTRALIB) -L../../Thirdparties/ninja -L${GPU_HOME}lib64/ -L/lib64 -lninja -lcublas_static -lcublasLt -lcusparse_static -lculibos -lcudart -lpthread -lstdc++ -ldl -lcusolver
# NINJA also support AMGX. Uncomment the following lines and set AMGX_HOME
#AMGX_HOME :=
#CSALYA := $(CSALYA) -DAMGX
#EXTRALIB := $(EXTRALIB) -L${AMGX_HOME}/lib -lamgxsh
###################################################################
# CATALYST #
# #
#MANDATORY THIRDPARTY COMPILATION #
#Go to Alya/Thirdparties/Catalyst #
# #
#MORE INFO:hadrien.calmet at bsc.es #
###################################################################
# Uncomment the following lines for using CATALYST as output
#CSALYA := $(CSALYA) -DVTK -DCATA -mt_mpi -I../../Thirdparties/Catalyst
#EXTRALIB := $(EXTRALIB) ../../Thirdparties/VTK/vtkXMLWriterF.o ../../Thirdparties/Catalyst/FEFortranAdaptor.o ../../Thirdparties/Catalyst/FECxxAdaptor.o
#EXTRALIB := $(EXTRALIB) -L/apps/PARAVIEW/4.2.0/lib/paraview-4.2/ -lvtkIOXML-pv4.2 -lvtkIOGeometry-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkIOXMLParser-pv4.2 -lvtksys-pv4.2 -lvtkIOCore-pv4.2 -lvtkCommonExecutionModel-pv4.2 -lvtkCommonDataModel-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkCommonMisc-pv4.2 -lvtkCommonSystem-pv4.2 -lvtkCommonTransforms-pv4.2 -lvtkCommonMath-pv4.2 -lvtkCommonCore-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkzlib-pv4.2 -lvtkjsoncpp-pv4.2 -lvtkexpat-pv4.2 -lvtkPVPythonCatalyst-pv4.2 -lvtkPVCatalyst-pv4.2 -lvtkPythonInterpreter-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkUtilitiesPythonInitializer-pv4.2 -lvtkPVServerManagerCore-pv4.2 -lvtkPVServerImplementationCore-pv4.2 -lvtkPVClientServerCoreCore-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkFiltersParallel-pv4.2 -lvtkFiltersModeling-pv4.2 -lvtkRenderingCore-pv4.2 -lvtkFiltersExtraction-pv4.2 -lvtkFiltersStatistics-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkImagingFourier-pv4.2 -lvtkImagingCore-pv4.2 -lvtkalglib-pv4.2 -lvtkFiltersGeometry-pv4.2 -lvtkFiltersSources-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkFiltersGeneral-pv4.2 -lvtkCommonComputationalGeometry-pv4.2 -lvtkFiltersProgrammable-pv4.2 -lvtkPVVTKExtensionsCore-pv4.2
#EXTRALIB := $(EXTRALIB) -lvtkPVCommon-pv4.2 -lvtkClientServer-pv4.2 -lvtkFiltersCore-pv4.2 -lvtkParallelMPI-pv4.2 -lvtkParallelCore-pv4.2 -lvtkIOLegacy-pv4.2
#EXTRALIB := $(EXTRALIB) -lprotobuf -lvtkWrappingPython27Core-pv4.2 -lvtkpugixml-pv4.2 -lvtkPVServerManagerApplication-pv4.2 -L/gpfs/apps/MN3/INTEL/tbb/lib/intel64/ -ltbb
###################################################################
# EoCoE Flags #
###################################################################
# Uncomment the following lines to output matrices #
#CSALYA := $(CSALYA) -Doutmateocoe
# Uncomment the following lines to solve with AGMG #
#MANDATORY MODULES TO LOAD ON MN3
#module load MKL/11.3
# serial version -- this is obsolete actually for it to work you need to touch the ifdef in dagmg.f90
#CSALYA := $(CSALYA) -Dsolve_w_agmg -I${MKLROOT}/include
#EXTRALIB := $(EXTRALIB) -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential
#
#
# parallel version academic
#CSALYA := $(CSALYA) -DPARAL_AGMG -DPARAL_AGMG_ACAD -I${MKLROOT}/include -I/gpfs/projects/bsc21/WORK-HERBERT/svnmn3/MUMPS_5.0.1/include
#For Mumps
#EXTRALIB := $(EXTRALIB) -L/gpfs/projects/bsc21/WORK-HERBERT/svnmn3/MUMPS_5.0.1/lib -ldmumps -lmumps_common -lpord
#For agmg
#EXTRALIB := $(EXTRALIB) -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lm -lmkl_blacs_intelmpi_ilp64 -lmkl_blacs_intelmpi_lp64 -lmkl_scalapack_ilp64 -lmkl_scalapack_lp64
#
# parallel version prof (copy dagmg_par.o from Thirdparties/agmg)
#CSALYA := $(CSALYA) -DPARAL_AGMG -Duse_dagmg_mumps -I${MKLROOT}/include
#For agmg
#EXTRALIB := $(EXTRALIB) -L${MKLROOT}/lib/intel64 -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lm -lmkl_blacs_intelmpi_ilp64 -lmkl_blacs_intelmpi_lp64 -lmkl_scalapack_ilp64 -lmkl_scalapack_lp64 -L../../Thirdparties/agmg/ -ldagmg_par
###################################################################
# SUNDIALS CVODE #
###################################################################
# Uncomment the following lines for using Sundials CVode
#EXTRALIB := $(EXTRALIB) ../../Thirdparties/sundials/sundials-install/lib/*.a ../../Thirdparties/sundials/sundials-install/lib/libsundials_cvode.a
###################################################################
# DBParicles LIBRARY #
###################################################################
# Uncomment the following lines for using DBParticles library
#CSALYA := $(CSALYA) -DDBPARTICLES
#--FOR THRIFT connector--
#EXTRALIB := $(EXTRALIB) -L../../Thirdparties/dbparticles -ldbparticles -I/usr/local/include/thrift/ -I/usr/local/include -L/usr/local/lib -lm -lstdc++
#EXTRALIB := $(EXTRALIB) -lthrift -lthriftnb -lthriftqt -lthriftz
#--FOR CASSANDRA connector--
#EXTRALIB := $(EXTRALIB) ../../Thirdparties/dbparticles/MurmurHash3.o -L../../Thirdparties/dbparticles -ldbparticles -I/usr/lib/x86_64-linux-gnu/include -L/usr/lib/x86_64-linux-gnu -luv -lcassandra -I/usr/local/include -L/usr/local/lib -lm -lstdc++
###################################################################
# WSMP SOLVER ILLINOIS #
###################################################################
# Uncomment the following lines to use WSMP library
#CSALYA := $(CSALYA) -DWSMP
#EXTRALIB := $(EXTRALIB) /gpfs/projects/bsc21/bsc21103/wsmpint/wsmp-Linux64-Intel/lib/ -lwsmp64 -lpthread /apps/LAPACK/3.5.0/lib -mkl
###################################################################
# DO NOT TOUCH #
###################################################################
FCFLAGS := $(FCFLAGS) $(CSALYA) $(EXTRAINC)
This diff is collapsed.
#!/bin/bash
#
# Read file timer_stats.csv
#
#
export FILE_LENGTH=`wc -l < timer_stats.csv`
#
## echo "Number of lines $FILE_LENGTH"
#
export TAIL_LINE_NUMBER="$(($FILE_LENGTH-4))"
#
## echo $TAIL_LINE_NUMBER
#
tail -$TAIL_LINE_NUMBER timer_stats.csv > timer_1st.tmp
#
##more timer_1st.tmp
#
awk '{print $2}' timer_1st.tmp > timer_2nd.tmp
#
sed 's/,//g' timer_2nd.tmp > timer_1st.tmp
#
export FILE_LENGTH=`wc -l < timer_1st.tmp`
#
## echo "Number of lines $FILE_LENGTH"
#
export FILE_LENGTH=$(($FILE_LENGTH-1))
#
export HEAD_LINE_NUMBER="-$FILE_LENGTH"
#
head $HEAD_LINE_NUMBER timer_1st.tmp > timer_2nd.tmp
#
export sum_of_lines=`awk '{s+=$1}END{print s}' timer_2nd.tmp`
## echo "Sum of the lines of the file: $sum_of_lines"
#
##more timer_2nd.tmp
#
export average_timing=`echo "$sum_of_lines / $FILE_LENGTH" | bc -l`
echo "Averaged timing for the $FILE_LENGTH entries: $average_timing"
#
rm -rf *.tmp
#!/bin/sh
#################################
## Which version of the code ? ##
#################################
CODE_VERSION=7.0.0
KER_VERSION=${CODE_VERSION}
KERNAME=code_saturne-${KER_VERSION}
################################################
## Installation PATH in the current directory ##
################################################
INSTALLPATH=`pwd`
echo $INSTALLPATH