\chapter{Execution model}\label{sec:exmodel}

The terminology ``host'' is used to refer to the CPU that is hosting
the execution of the application, and ``target'' refers to the
device targeted for execution of data parallel operations. The target
could be the same CPU as the host, or it could be a separate device such
as an accelerator (depending on the hardware available). 

The targetDP API follows the fork-join model of parallel
execution. When the application initiates, a single thread executes
sequentially on the host, until it encounters a function to be
launched on the target. This function will be executed by a team of
threads on the target cooperating in a data-parallel manner (e.g. for
a structured grid problem each thread is responsible for a subset of
the grid). Each thread in the team will have a unique index. For some
architectures, it is important to expose not just thread-level
parallelism (TLP), but also instruction-level parallelism (ILP). The
targetDP model facilitates this by allowing striding at the thread
level, such that each thread can operate on a chunk of data
(e.g. multiple grid points) at the instruction level. The size of the
chunk, which we call the ``Virtual Vector Length'' (VVL), can be tuned
to the hardware.

To ensure that the target function has completed, a
\verb+targetSynchronize+ statement should follow the code location
where target function is launched. When the initial thread encounters
this statement, it will wait until the target region has completed. It
is possible, in principle, for the initial thread to execute other instructions
(which do not depend on the results of the target function), after the
function launch but before the synchronisation call. This may result in
overlapping of host and target execution, and hence optimisation, in
some implementations. Once the target function has completed, the
initial thread will continue sequentially until another target
function launch is encountered.

Within each target function, each thread is given a unique index which
it uses to work in a data-parallel manner. Each thread works
independently from all others, but usually operating on a shared data
structure where the index is used to determine the portion of data
to process.

In applications, it is sometimes necessary to perform reductions,
where multiple data values are combined in a certain way. For example,
values existing on each grid point may be summed into a single total
value. The targetDP model supports such operations in a simplistic
way. It is the responsibility of the application to create the array
of values (using standard targetDP functionality) to act as the input
to the reduction operation. The application can then pass this array
to the API function corresponding to the desired reduction operation
(e.g. \verb+targetDoubleSum+ for the summation of an array of
double-precision Values). If the required reduction operation does not yet
exist, the user can simply extend the targetDP API using existing
functionality as a template.
