4IP-extention: Run on PCP machines
==================================

This document will describe the work done under the PRACE-4IP extention. This task is dedicated to run the accelerated UEABS on PCP systems to obtain energy metrics on OpenPower+GPU, Xeon Phi and FPGA.

Organisation is publicly available to anyone interrested in the project and rely on the following tools:

 * A slack channel for chat purpose: `PRACE PCP slack channel`_ , ask `Victor Cameo Ponz`_ registration
 * A mailing list: ``TODO, adress & registration process``
 * This documentation.

The following methodology will be followed to lead this task:
 #. define which codes of UEABS will be run by who on which machines (this also meats the PRACE milestone ``TODO, name and PRACE calendar due`` due by August 2017)
 #. grant access to machines (cut off September 2017 )
 #. run benchmarks (cut off October/November)
 #. gather results and report (this adresses the PRACE deliverable ``TODO, name and PRACE calendar due`` due by December 2017)

Code definition
---------------

The table :ref:`table-code-definition` shows all codes availabe with UEABS (regular and accelerated). It states for each codes, tageted architechures and BCOs.

.. _table-code-definition:
.. table:: Code definition
   :widths: auto

   +------------------------+--------------------------+--------------------------------+-----------------------------+
   |                        |    UEABS availability    |           Will run on          |                             |
   |   Code name            +------------+-------------+--------------+----------+------+   4IP-extention BCO         +
   |                        |  Regular   | Accelerated | Power8 + GPU | Xeon Phi | FPGA |                             |
   +========================+============+=============+==============+==========+======+=============================+
   | ALYA                   | ✓          | ✓           |              |          |      |                             |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | Code_Saturne           | ✓          | ✓           |              |          |      | ???Charles Moulinec???      |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | CP2K                   | ✓          | ✓           |              |          |      | Arno Proeme                 |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | GADGET                 | ✓          |             |              |          |      |                             |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | GENE                   | ✓          |             |              |          |      |                             |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | GPAW                   | ✓          | ✓           |              |          |      | Martti Louhivuori (CINCA)   |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | GROMACS                | ✓          | ✓           |              |          |      | Dimitris Dellis (GRNET)     |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | NAMD                   | ✓          | ✓           |              |          |      | Dimitris Dellis (GRNET)     |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | NEMO                   | ✓          |             |              |          |      | ???EPCC???                  |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | PFARM                  |            | ✓           |              |          |      |                             |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | QCD                    | ✓          | ✓           |              |          |      | Arno Proeme (EPCC)          |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | Quantum Espresso       | ✓          | ✓           |              |          |      | Andrew Emerson (CINECA)     |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | SHOC                   |            | ✓           |              |          |      | Valeriu Codreanu (SurfSARA) |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+
   | Specfem3D_Globe        | ✓          | ✓           |    ✓         | ✓        |      | Victor Cameo Ponz (CINES)   |
   +------------------------+------------+-------------+--------------+----------+------+-----------------------------+


PCP systems description and registration process
------------------------------------------------

This section describes the systems where BCOs have been granted access. **The registration process included whithin is aimed at BCOs only.**

The table :ref:`table-system-availability` sums up the when registration process start:

.. _table-system-availability:
.. table:: System availability
   :widths: auto

   +--------------+----------------------------+---------------+------------------------------------------------------+
   |  Technology  | Manufacturer               | Host          | Availability                                         |
   +==============+============================+===============+======================================================+
   | Power8 + GPU | `E4 computer engineering`_ | CINECA_ (It)  | Start migration of nodes to CINECA in June/July 2017 |
   +--------------+----------------------------+---------------+------------------------------------------------------+
   | Xeon Phi     | `Atos/Bull`_               | CINES_ (Fr)   | June 2017                                            |
   +--------------+----------------------------+---------------+------------------------------------------------------+
   | FPGA         | MAXELER_                   | JSC_ (De)     | August 2017                                          |
   +--------------+----------------------------+---------------+------------------------------------------------------+

Power8 + GPU
^^^^^^^^^^^^

This machine has been designed by `E4 computer engineering`_ and is hosted at CINECA_ in Bologna, Italy. In order to access the machine BCO should register on the `CINECA user datatabase`_ and ask `Victor Cameo Ponz`_ to be added to the 4IP-extention project.

Compute technology
""""""""""""""""""

Hardware features fat-nodes with the following design:

 * IBM POWER8 processors + NVIDIA P100 GPU
 * Tightly integrated using NVLink
 * CPU and GPU liquid cooling based on CoolIT_ solution

Energy sampling technology
""""""""""""""""""""""""""

Xeon Phi
^^^^^^^^

This machine has been designed by `Atos/Bull`_ and is hosted at CINES_ in Montpellier, France. In order to access the machine BCO should fill the request to open login form (`odt <cines-login-form-odt>`_ or `rtf <cines-login-form-rtf>`_) and send it back to `Victor Cameo Ponz`_.

Compute technology
""""""""""""""""""
Hardware features the following nodes:

 * Intel Xeon Phi KNL
 * Liquid cooled PSU

Energy sampling technology
""""""""""""""""""""""""""

FPGA
^^^^

This machine has been designed by MAXELER_ and is hosted at JSC_ in Julich, Germany.

Compute technology
""""""""""""""""""

Energy sampling technology
""""""""""""""""""""""""""



.. _Victor Cameo Ponz: cameo+4ip-extention@cines.fr

.. _PRACE PCP slack channel: https://prace-pcp.slack.com

.. _E4 computer engineering: https://www.e4company.com
.. _Atos/Bull: https://bull.com/
.. _MAXELER: http://maxeler.com/

.. _CINECA: http://hpc.cineca.it/
.. _CINES: https://www.cines.fr/
.. _JSC: http://www.fz-juelich.de/ias/jsc/EN/Home/home_node.html

.. _CINECA user datatabase: https://userdb.hpc.cineca.it/
.. _cines-login-form-odt: https://www.cines.fr/wp-content/uploads/2014/01/opening_renewal_login_2017.odt
.. _cines-login-form-rtf: https://www.cines.fr/wp-content/uploads/2014/01/opening_renewal_login_2017.rtf