README.md 17.5 KB
Newer Older
1
# Unified European Applications Benchmark Suite
2

Walter Lioen's avatar
Walter Lioen committed
3
<img alt="PRACE" src="PRACE_logo.png" width="150px">
Walter Lioen's avatar
Walter Lioen committed
4

Walter Lioen's avatar
Walter Lioen committed
5
The Unified European Application Benchmark Suite (UEABS) is a set of currently 13 application codes taken from the pre-existing PRACE and DEISA application benchmark suites, and extended with the PRACE Accelerator Benchmark Suite. The objective is providing a single benchmark suite of scalable, currently relevant and publicly available application codes and datasets, of a size which can realistically be run on large systems, and maintained into the future.
6

7
8
9
The UEABS activity was started during the PRACE-PP project and was publicly released by the PRACE-2IP project.
The PRACE "Accelerator Benchmark Suite" was a PRACE-4IP activity.
The UEABS has been and will be actively updated and maintained by the subsequent PRACE-IP projects.
10

11
Each application code has either one, or two input datasets. If there are two datasets, Test Case A is designed to run on Tier-1 sized systems (up to around 1,000 x86 cores, or equivalent) and Test Case B is designed to run on Tier-0 sized systems (up to around 10,000 x86 cores, or equivalent). If there is only one dataset (Test Case A), it is suitable for both sizes of system.
Victor's avatar
Victor committed
12

Walter Lioen's avatar
Walter Lioen committed
13
Contact: Walter Lioen <mailto:walter.lioen@surf.nl>
Victor's avatar
Victor committed
14

15
16
Current Release
---------------
Victor's avatar
Victor committed
17

Walter Lioen's avatar
Walter Lioen committed
18
The current release is Version 2.2 (December 20, 2022).
Walter Lioen's avatar
Walter Lioen committed
19
See also the [release notes and history](RELEASES.md).
Victor's avatar
Victor committed
20
21
22
23

Running the suite
-----------------

24
Instructions to run each test cases of each codes can be found in the subdirectories of this repository.
Victor's avatar
Victor committed
25

Walter Lioen's avatar
Walter Lioen committed
26
For more details of the codes and datasets, and sample results, please see the PRACE-6IP benchmarking deliverable [D7.5 "Evaluation of Benchmark Performance"](https://prace-ri.eu/wp-content/uploads/PRACE6IP-D7.4.pdf) (November 30, 2021).
27

28
29
The application codes that constitute the UEABS are:
---------------------------------------------------
Walter Lioen's avatar
Walter Lioen committed
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
<table>
  <thead>
    <tr>
      <th rowspan="2">Application</th>
      <th rowspan="2">Lines of<br/>Code</th>
      <th colspan="3">Parallelism</th>
      <th colspan="4">Language</th>
      <th rowspan="2">Code Description/Notes</th>
    </tr>
    <tr>
      <th>MPI</th>
      <th>OpenMP/<br/>Pthreads</th>
      <th>GPU</th>
      <th>Fortran</th>
      <th>Python</th>
      <th>C</th>
      <th>C++</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Alya
        <ul>
          <li><a href="https://www.bsc.es/computer-applications/alya-system">website</a></li>
          <li><a href="https://gitlab.com/bsc-alya/open-alya">source</a></li>
          <li><a href="alya/README.md">instructions</a></li>
          <li><a href="https://gitlab.com/bsc-alya/benchmarks/sphere-16M">Test Case A</a></li>
          <li><a href="https://gitlab.com/bsc-alya/benchmarks/sphere-132M">Test Case B</a></li>
        </ul>
      </td>
      <td>600,000</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td></td>
      <td>The Alya System is a Computational Mechanics code capable of solving different physics, each one with its own modelization characteristics, in a coupled way. Among the problems it solves are: convection-diffusion reactions, incompressible flows, compressible flows, turbulence, bi-phasic flows and free surface, excitable media, acoustics, thermal flow, quantum mechanics (DFT) and solid mechanics (large strain).</td>
    </tr>
    <tr>
Charles Moulinec's avatar
Charles Moulinec committed
71
72
73
74
75
76
77
78
79
      <td>Code_Saturne
        <ul>
         <li><a href="https://www.code-saturne.org/cms/web">Code_Saturne website</a></li>
         <li><a href="https://www.code-saturne.org/cms/sites/default/files/releases/code_saturne-7.0.0.tar.gz">Source code</a></li>
         <li><a href="https://repository.prace-ri.eu/git/UEABS/ueabs/-/blob/r2.2-dev/code_saturne/README.md">Build instuctions</a></li>
         <li><a href="https://repository.prace-ri.eu/ueabs/Code_Saturne/2.2/CS_7.0.0_PRACE_UEABS_CAVITY_13M.tar.gz">Testcase A</a></li>
         <li><a href="https://repository.prace-ri.eu/ueabs/Code_Saturne/2.2/CS_7.0.0_PRACE_UEABS_CAVITY_XXXM.tar.gz">Testcase B</a></li>
        </ul>
      </td>
Walter Lioen's avatar
Walter Lioen committed
80
81
82
83
84
85
86
87
      <td>~350,000</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
Charles Moulinec's avatar
Charles Moulinec committed
88
      <td>The code solves the Navier-Stokes equations for incompressible/compressible flows using a predictor-corrector technique. The Poisson pressure equation is solved by a Conjugate Gradient preconditioned by a multi-grid algorithm, and the transport equations by Conjugate Gradient-like methods. Advanced gradient reconstruction is also available to account for distorted meshes.</td>
Walter Lioen's avatar
Walter Lioen committed
89
90
    </tr>
    <tr>
Holly Judge's avatar
Holly Judge committed
91
92
93
94
      <td>CP2K
        <ul>
          <li><a href="https://www.cp2k.org/">CP2K website</a></li>
          <li><a href="https://github.com/cp2k/cp2k/releases">Source code</a></li>
Holly Judge's avatar
Holly Judge committed
95
          <li><a href="./cp2k/README.md">Build instructions</a></li>
Holly Judge's avatar
Holly Judge committed
96
97
98
99
100
          <li><a href="./cp2k/benchmarks/TestCaseA_H2O-512">Testcase A</a></li>
          <li><a href="./cp2k/benchmarks/TestCaseB_LiH-HFX">Testcase B</a></li>
          <li><a href="./cp2k/benchmarks/TestCaseC_H2O-DFT-LS">Testcase C</a></li>
        </ul>
      </td>
Walter Lioen's avatar
Walter Lioen committed
101
102
103
104
105
106
107
108
109
110
111
      <td>~1,150,000</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td></td>
      <td>CP2K is a freely available quantum chemistry and solid-state physics software package for performing atomistic simulations. It can be run with MPI, OpenMP and CUDA. All of CP2K is MPI parallelised, with some routines making use of OpenMP, which can be used to reduce the memory footprint. In addition some linear algebra operations may be offloaded to GPUs using CUDA.</td>
    </tr>
    <tr>
Miguel Avillez's avatar
Miguel Avillez committed
112
113
      <td>GADGET
       <ul>
Miguel Avillez's avatar
Miguel Avillez committed
114
115
116
117
      		<li><a href="https://wwwmpa.mpa-garching.mpg.de/gadget4">GADET Website</a></li>
          <li><a href="https://gitlab.mpcdf.mpg.de/vrs/gadget4">GADET GitLab</a></li>
        	<li><a href="gadget/README.md#mechanics-of-building-benchmark">Build instructions</a>
          <li><a href="gadget/README.md#mechanics-of-running-benchmark">Run instructions</a>
Miguel Avillez's avatar
Miguel Avillez committed
118
119
          <li><a href="./gadget/gadget4-case-A.tar.gz">Test Case A</a></li>
          <li><a href="./gadget/gadget4-case-B.tar.gz">Test Case B</a></li>
Miguel Avillez's avatar
Miguel Avillez committed
120
121
122
123
124
       </ul>
       </td>
      <td>100,000+</td>
      <td>X</td>
      <td>X</td>
Walter Lioen's avatar
Walter Lioen committed
125
126
127
      <td></td>
      <td></td>
      <td></td>
Miguel Avillez's avatar
Miguel Avillez committed
128
129
130
      <td>X</td>
      <td>X</td>
      <td>GADGET-4 (GAlaxies with Dark matter and Gas intEracT), an evolved and improved version of GADGET-3, is a freely available code for cosmological N-body/SPH simulations on massively parallel computers with distributed memory. GADGET-4 supports collisionless simulations and smoothed particle hydrodynamics on massively parallel computers. All communication between concurrent execution processes is done either explicitly by means of the message passing interface (MPI), or implicitly through shared-memory accesses on processes on multi-core nodes. The code is mostly written in ISO C++ (assuming the C++11 standard), and should run on all parallel platforms that support at least MPI-3.</td>
Walter Lioen's avatar
Walter Lioen committed
131
132
    </tr>
    <tr>
133
134
135
136
137
      <td>GPAW
        <ul>
          <li><a href="https://wiki.fysik.dtu.dk/gpaw/">website</a></li>
          <li><a href="https://gitlab.com/gpaw/gpaw">GPAW GitLab</a></li>
          <li><a href="https://gitlab.com/mlouhivu/gpaw/tree/cuda">GPAW GPU development (cuda branch)</a></li>
138
139
140
141
142
          <li><a href="gpaw/README.md#mechanics-of-building-the-benchmark">Build instructions</a>
          <li><a href="gpaw/README.md#mechanics-of-running-the-benchmark">Run instructions</a>
          <li><a href="gpaw/benchmark/A_carbon-nanotube/input.py">Test Case A</a>
          <li><a href="gpaw/benchmark/B_copper-filament/input.py">Test Case B</a>
          <li><a href="gpaw/benchmark/C_silicon-cluster/input.py">Test Case C</a>
143
144
        </ul>
      </td>
Walter Lioen's avatar
Walter Lioen committed
145
146
147
      <td>132,000</td>
      <td>X</td>
      <td></td>
148
      <td>(X)</td>
Walter Lioen's avatar
Walter Lioen committed
149
150
151
152
      <td></td>
      <td>X</td>
      <td>X</td>
      <td></td>
153
154
155
156
157
158
159
160
161
162
163
      <td>
        GPAW is a density-functional theory (DFT)
        program for ab initio electronic structure calculations using the projector
        augmented wave method. It uses a uniform real-space grid representation of the
        electronic wavefunctions that allows for excellent computational scalability
        and systematic converge properties.
        The GPAW benchmark tests MPI parallelization and the quality of the provided mathematical
        libraries, including BLAS, LAPACK, ScaLAPACK, and FFTW-compatible library. There is
        also an experimental CUDA-based implementation for GPU systems, but it is not covered
        by this UEABS release.
      </td>
Walter Lioen's avatar
Walter Lioen committed
164
165
    </tr>
    <tr>
166
167
      <td>GROMACS
      <ul>
168
      <li><a href="http://www.gromacs.org">website</a></li>
169
170
171
172
      <li><a href="http://www.gromacs.org/Downloads">Source code</a></li>
      <li><a href="https://repository.prace-ri.eu/git/UEABS/ueabs/-/tree/r2.2-dev/gromacs">Build and Run Instructions</a></li>
      <li><a href="https://repository.prace-ri.eu/ueabs/GROMACS/2.2/GROMACS_TestCaseA.tar.xz">Test Case A</a></li>
      <li><a href="https://repository.prace-ri.eu/ueabs/GROMACS/2.2/GROMACS_TestCaseB.tar.xz">Test Case B</a></li>
Dimitris Dellis's avatar
Dimitris Dellis committed
173
      <li><a href="https://repository.prace-ri.eu/ueabs/GROMACS/2.2/GROMACS_TestCaseC.tar.xz">Test Case C</a></li>
174
175
176
      </ul>
      </td>
      <td>862,079</td>
Walter Lioen's avatar
Walter Lioen committed
177
178
179
180
181
182
183
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td>X</td>
      <td>X</td>
Dimitris Dellis's avatar
Dimitris Dellis committed
184
      <td>GROMACS is a versatile package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles.</td>
Walter Lioen's avatar
Walter Lioen committed
185
186
    </tr>
    <tr>
187
188
189
190
191
      <td>NAMD
        <ul>
      <li><a href="http://www.ks.uiuc.edu/Research/namd/">website</a></li>
      <li><a href="http://www.ks.uiuc.edu/Development/Download/download.cgi?PackageName=NAMD">Source code</a></li>
      <li><a href="https://repository.prace-ri.eu/git/UEABS/ueabs/-/tree/r2.2-dev/namd">Build and Run Instructions</a></li>
Dimitris Dellis's avatar
Dimitris Dellis committed
192
193
194
      <li><a href="https://repository.prace-ri.eu/ueabs/NAMD/2.2/NAMD_TestCaseA.tar.gz">Test Case A</a></li>
      <li><a href="https://repository.prace-ri.eu/ueabs/NAMD/2.2/NAMD_TestCaseB.tar.gz">Test Case B</a></li>
      <li><a href="https://repository.prace-ri.eu/ueabs/NAMD/2.2/NAMD_TestCaseC.tar.gz">Test Case C</a></li>
195
196
197
      </ul>
      </td>
      <td>887,547</td>
Walter Lioen's avatar
Walter Lioen committed
198
199
200
201
202
203
204
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td></td>
      <td>X</td>
Dimitris Dellis's avatar
Dimitris Dellis committed
205
      <td>NAMD is a widely used molecular dynamics application designed to simulate bio-molecular systems on a wide variety of compute platforms.
206
      </td>
Walter Lioen's avatar
Walter Lioen committed
207
208
    </tr>
    <tr>
Ozan Karsavuran's avatar
Ozan Karsavuran committed
209
210
211
212
213
      <td>NEMO
      <ul>
          <li><a href="https://www.nemo-ocean.eu/">website</a></li>
          <li><a href="https://forge.ipsl.jussieu.fr/nemo/chrome/site/doc/NEMO/guide/html/install.html#download-and-install-the-nemo-code">source</a></li>
          <li><a href="nemo/README.md">instructions</a></li>
Ozan Karsavuran's avatar
Ozan Karsavuran committed
214
215
          <li><a href="nemo/README.md#verification-of-results">Test Case A</a></li>
          <li><a href="nemo/README.md#verification-of-results">Test Case B</a></li>
Ozan Karsavuran's avatar
Ozan Karsavuran committed
216
217
        </ul>
      </td>
Walter Lioen's avatar
Walter Lioen committed
218
219
220
221
222
223
224
225
226
227
228
      <td>154,240</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td>X</td>
      <td></td>
      <td></td>
      <td>X</td>
      <td>NEMO (Nucleus for European Modelling of the Ocean) is a mathematical modelling framework for research activities and prediction services in ocean and climate sciences developed by a European consortium. It is intended to be a tool for studying the ocean and its interaction with the other components of the earth climate system over a large number of space and time scales. It comprises of the core engines namely OPA (ocean dynamics and thermodynamics), SI3 (sea ice dynamics and thermodynamics), TOP (oceanic tracers) and PISCES (biogeochemical process).</td>
    </tr>
    <tr>
Andrew Sunderland's avatar
Andrew Sunderland committed
229
230
231
232
233
      <td>PFARM
	<ul>
	  <li><a href="https://www.ccpq.ac.uk/node/4">website</a></li>
          <li><a href="https://repository.prace-ri.eu/git/UEABS/ueabs/-/tree/r2.2-dev/pfarm">source</a></li>
          <li><a href="pfarm/PFARM_Build_Run_README.txt">instructions</a></li>
234
235
          <li><a href="https://repository.prace-ri.eu/ueabs/PFARM/2.2/test_case_1_atom.tar.gz">Test Case 1</a></li>
          <li><a href="https://repository.prace-ri.eu/ueabs/PFARM/2.2/test_case_2_mol.tar.gz">Test Case 2</a></li>
Andrew Sunderland's avatar
Andrew Sunderland committed
236
237
	</ul>
      </td>
Walter Lioen's avatar
Walter Lioen committed
238
239
240
241
242
243
244
245
246
247
248
      <td>21,434</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td></td>
      <td>PFARM uses an R-matrix ab-initio approach to calculate electron-atom and electron-molecule collisions data for a wide range of applications including atrophysics and nuclear fusion. It is written in modern Fortran/MPI/OpenMP and exploits highly-optimised dense linear algebra numerical library routines.</td>
    </tr>
    <tr>
249
      <td>QCD
Jacob Finkenrath's avatar
Jacob Finkenrath committed
250
        <ul>
251
        <li><a href='qcd/README.md'>see for more details</a></li>
Jacob Finkenrath's avatar
Jacob Finkenrath committed
252
        </ul>
253
254
255
256
257
      </td>
      <td>100,000+</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
Walter Lioen's avatar
Walter Lioen committed
258
259
      <td></td>
      <td></td>
260
261
262
      <td>X</td>
      <td>X</td>
      <td>The QCD benchmark is, unlike the other benchmarks in the PRACE application benchmark suite, not a full application but a set of 3 parts which are representative of some of the most compute-intensive parts of QCD calculations. The major application of the different parts consists of a Conjugate Gradient solver involving Wilson Dirac stencil in 4 dimension. Keywords of the QCD bencharmks kernels are: Domain Decomposition, Memory bandwidth, strong scaling, MPI latency.</td>
Walter Lioen's avatar
Walter Lioen committed
263
264
    </tr>
    <tr>
Andrew Emerson's avatar
Andrew Emerson committed
265
266
      <td>Quantum&nbsp;ESPRESSO
      <ul>
Andrew Emerson's avatar
Andrew Emerson committed
267
268
269
      <li><a href='https://www.quantum-espresso.org/'>Website</a></li>
      <li><a href='https://www.quantum-espresso.org/download-page/'>Source</a></li>
      <li><a href='quantum_espresso/README.md#installation-and-requirements'>Build and Run instructions</a></li>
Andrew Emerson's avatar
Andrew Emerson committed
270
271
      <li><a href='https://repository.prace-ri.eu/ueabs/Quantum_Espresso/QuantumEspresso_TestCaseA.tar.gz'>Test Case A</a></li>
      <li><a href='https://repository.prace-ri.eu/ueabs/Quantum_Espresso/QuantumEspresso_TestCaseB.tar.gz'>Test Case B</a></li>
Andrew Emerson's avatar
Andrew Emerson committed
272
      </ul>
Andrew Emerson's avatar
Andrew Emerson committed
273
      </td>
Walter Lioen's avatar
Walter Lioen committed
274
275
276
277
278
279
280
281
      <td>92,996</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td></td>
      <td></td>
      <td></td>
282
283
      <td>Quantum Espresso is an integrated suite of Open-Source computer codes for electronic-structure calculations and materials modeling at the nanoscale. It is based on density-functional theory, plane waves, and pseudopotentials. It is written in MPI and OpenMP with a CUDA Fortran version
      available for Nvidia GPUs. In the benchmark suite we consider only the most used program, PWscf. </td>
Walter Lioen's avatar
Walter Lioen committed
284
285
    </tr>
    <tr>
Cedric Jourdain's avatar
Cedric Jourdain committed
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
      <td>SPECFEM3D
        <ul>
          <li><a href="https://geodynamics.org/cig/software/specfem3d_globe/">Website</a></li>
          <li><a href="https://github.com/geodynamics/specfem3d_globe.git">Source</a></li>
          <li><a href="specfem3D/README.md">Run and build instructions</a></li>
          <li><a href="specfem3D/test_case/SPECFEM3D_TestCase_A">Test Case A</a></li>
          <li><a href="specfem3D/test_case/SPECFEM3D_TestCase_B">Test Case B</a></li>
          <li><a href="https://github.com/geodynamics/specfem3d_globe/tree/master/EXAMPLES/small_benchmark_run_to_test_more_complex_Earth">Test Case C</a></li>
        </ul>
      </td>
      <td> ~120,000 (100k Fortran & 20k C)</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>X</td>
Walter Lioen's avatar
Walter Lioen committed
301
      <td></td>
Cedric Jourdain's avatar
Cedric Jourdain committed
302
      <td>X</td>
Walter Lioen's avatar
Walter Lioen committed
303
      <td></td>
Cedric Jourdain's avatar
Cedric Jourdain committed
304
      <td>The software package SPECFEM3D simulates three-dimensional global and regional seismic wave propagation based upon the spectral-element method (SEM).</td>
Walter Lioen's avatar
Walter Lioen committed
305
306
    </tr>
    <tr>
Maxwell Cai's avatar
Maxwell Cai committed
307
308
309
310
311
312
313
314
315
316
317
      <td>TensorFlow
        <ul>
          <li><a href="https://www.tensorflow.org/">website</a></li>
          <li><a href="https://github.com/maxwelltsai/DeepGalaxy">source</a></li>
          <li><a href="tensorflow/README.md">instructions</a></li>
          <li><a href="tensorflow/Testcase_A">Test Case A</a></li>
          <li><a href="tensorflow/Testcase_B">Test Case B</a></li>
          <li><a href="tensorflow/Testcase_C">Test Case C</a></li>
        </ul>
      </td>
      <td>~3,000,000</td>
Maxwell Cai's avatar
Maxwell Cai committed
318
319
320
      <td>X</td>
      <td>X</td>
      <td>X</td>
Walter Lioen's avatar
Walter Lioen committed
321
      <td></td>
Maxwell Cai's avatar
Maxwell Cai committed
322
323
324
325
      <td>X</td>
      <td>X</td>
      <td>X</td>
      <td>TensorFlow is a popular open-source library for symbolic math and linear algebra, with particular optimisation for neural-networks-based machine learning workflow. Maintained by Google, it is widely used for research and production in both the academia and the industry.</td>
Walter Lioen's avatar
Walter Lioen committed
326
327
328
    </tr>
  </tbody>
</table>
Walter Lioen's avatar
Walter Lioen committed
329

Walter Lioen's avatar
Walter Lioen committed
330
331
332
333
334
335
License
-------
All UEABS application codes are covered by their own respective licenses. Code modificiations required by the UEABS might inherit the originating application's license.

<a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />Unless stated otherwise, all work is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.

Walter Lioen's avatar
Walter Lioen committed
336
337
Acknowledgements
----------------
Walter Lioen's avatar
Walter Lioen committed
338
339
<img alt="Co-Funded by the European Union" src="EN_Co-Funded_by_the_EU_POS.png" width="300px">

Walter Lioen's avatar
Walter Lioen committed
340
341
342
343
344
345
This project has received funding from the European Union’s
Seventh Framework Programme (FP7/2007-2013) under grant agreements
n° 211528 (PRACE-PP), n° 261557 (PRACE-1IP), n° 283493 (PRACE-2IP), n° 312763 (PRACE-3IP);
and from the
Horizon 2020 research and innovation programme under grant agreements
No 653838 (PRACE-4IP), No 730913 (PRACE-5IP), No 823767 (PRACE-6IP).