\newtheorem{thm}{Theorem}
\newtheorem{lem}{Lemma}

\newcommand{\oton}[1]{{#1}_1,\ldots,{#1}_n}
\newcommand{\otok}[2]{{#2}_1,\ldots,{#2}_{#1}}
\newcommand{\dplus}{\text{+\!+}}
\newcommand{\llbracket}{[\![}
\newcommand{\rrbracket}{]\!]}
\newcommand{\tuple}[2]{\langle #1, #2 \rangle}
\makeatletter
% macros for consistency flavors
\newcommand{\simH}{\sim_{\!H}}
% usually: \mapconsist{\Omega}
\newcommand{\mapconsist}[2][M]{#1 \sim #2}
% usually: \localconsist{L}{\Gamma}
\newcommand{\localconsist}[3][M]{#2 \simH \@ifmtarg{#1}{#3}{#1 \llbracket #3 \rrbracket}}
% usually: \storeconsist{S}{H}
\newcommand{\storeconsist}[2]{#1 \sim #2}
% usually: \valueconsist{v}{T}
\newcommand{\valueconsist}[3][M]{#2 \simH \@ifmtarg{#1}{#3}{#1 \llbracket #3 \rrbracket}}
% usually: \traceconsist{E}
\newcommand{\traceconsist}[2][H]{#2 \sim #1}
% usually: \privconsist{E}{\Phi}
\newcommand{\privconsist}[3][M]{#2 \@ifmtarg{#1}{:}{:_{\!#1}} #3}

\newcommand{\nonint}[1][]{\#\@ifmtarg{#1}{}{_{\!\!#1}}}
\makeatother

% override BCP's typesetrule function to left-justify multiple lines of hypotheses
\renewcommand{\typesetrule}[2]{%
   \setrulebody{%
      \frac{\begin{array}{@{}l@{}}#1\end{array}}%
           {\begin{array}{@{}l@{}}#2\end{array}}}}

% fun latex tricks to make typeenv and opsenv more friendly
\makeatletter
\define@key{typeenv}{G}{\def\typeenv@G{#1}}
\define@key{typeenv}{P}{\def\typeenv@P{#1}}
\define@key{typeenv}{O}{\def\typeenv@O{#1}}
\newcommand{\typeenvx}[1][]{
{
% default values
\def\typeenv@G{\Gamma}
\def\typeenv@P{\Phi}
\def\typeenv@O{\Omega}
\setkeys{typeenv}{#1}
\typeenv@G, \typeenv@P, \typeenv@O \vdash \,
}}
\newcommand{\typeenv}[3][]{\typeenvx[#1] {#2} : {#3}}
\define@key{opsenv}{M}{\def\opsenv@M{#1}}
\define@key{opsenv}{L}{\def\opsenv@L{#1}}
\define@key{opsenv}{H}{\def\opsenv@H{#1}}
\define@key{opsenv}{S}{\def\opsenv@S{#1}}
\define@key{opsenv}{C}{\def\opsenv@C{#1}}
\newcommand{\opsenvx}[1][]{
{
% default values
\def\opsenv@M{M}
\def\opsenv@L{L}
\def\opsenv@H{H}
\def\opsenv@S{S}
\def\opsenv@C{C}
\setkeys{opsenv}{#1}
\opsenv@M, \opsenv@L, \opsenv@H, \opsenv@S, \opsenv@C \vdash \,
}}
\newcommand{\opsenv}[4][]{\opsenvx[#1] {#2} \mapsto {#3}, {#4}}
\makeatother

\begin{figure*}
\centering
{\small
\begin{tabular}{cclr|cclr}

$T$ & ::= &  & types & $bv$ & ::= & false $\;\;\;\mid\;\;\;$ true & \\
  &$\mid$& bool $\;\;\;\mid\;\;\;$ int & base types & & & & \\
  &$\mid$& $\langle T_1, T_2 \rangle$ & tuple & $iv$ & ::= & 0 $\;\;\;\mid\;\;\;$ 1 $\ldots$ & \\
  &$\mid$& $T@(\oton{r})$ & pointer & & & & \\
  &$\mid$& $\text{coloring}(r)$ & region coloring & $e$ & ::= & & expressions \\
  &$\mid$& $\exists \oton{r}. T\text{ where }\Omega$ & region relationship &   &$\mid$& $bv$ $\;\;\;\mid\;\;\;$ $iv$ & constants \\
  &$\mid$& $\forall \oton{r}. (\oton{T}), \Phi, Q \rightarrow T_r$ & functions &   &$\mid$& $\langle e_1, e_2 \rangle$ $\;\;\;\mid\;\;\;$ $e$.1 $\;\;\;\mid\;\;\;$ $e$.2 & tuple \\
& & & &   &$\mid$& $id$ &  \\
$\Omega$ & ::= & $\{ \oton{\omega} \}$ & region constraints &   &$\mid$& $\text{new}\ T@r$ $\;\;\;\mid\;\;\;$ $\text{null }T@r$ $\;\;\;\mid\;\;\;$ $\text{isnull}(e)$ & \\
$\omega$ & ::= & $r_1 \leq r_2$ & subregion   &
  &$\mid$& $\text{upregion}(e, r_1,\ldots,r_n)$ & \\
  &$\mid$& $r_1 * r_2$ & disjointness &
  &$\mid$ & $\text{downregion}(e, r_1,\ldots,r_n)$ & \\
& & &  &
  &$\mid$& $\text{read}(e_1)$ $\;\mid\;$ $\text{write}(e_1, e_2)$ & memory access \\
$\Phi$ & ::= & $\{ \oton{\phi} \}$ & privileges &
  & $\mid$ & $\text{reduce}(id, e_1, e_2)$ & \\
$\phi$ & ::= & reads$(r)$ $\;\mid\;$ writes$(r)$ $\;\mid\;$ $\text{reduces}_{id}(r)$ &  &
  &$\mid$& $\text{newcolor}\ r$ $\;\mid\;$ $\text{color}(e_1, e_2, e_3)$ & coloring \\
& & & &
  &   $\mid$& $e_1 + e_2$ & integer ops \\
$Q$ & ::= & $\{ \oton{q} \}$ & coherence modes  &
       & $\mid$& $e_1 < e_2$ & comparisons \\
$q$ & ::= & atomic$(r)$ $\;\;\;\mid\;\;\;$ simult$(r)$ & &
  &$\mid$& $\text{let}\ id : T = e_1 \text{in}\ e_2$ &  \\
& & & &
       & $\mid$& $\text{if}\ e_1\ \text{then}\ e_2\ \text{else}\ e_3$ &  \\
$v$ & ::= & & values &
       & $\mid$& $id[r_1, \ldots, r_n](e_1,\ldots,e_n)$ & function calls \\
  &$\mid$& $bv$ $\;\;\;\mid\;\;\;$ $iv$ & base values &
       & $\mid$ & $\text{partition}\ r_p\text{ using }e_1\text{ as }\oton{r}\text{ in }\ e_2$ &  \\
  &$\mid$& $\langle v_1, v_2 \rangle$ & tuple &
       & $\mid$& $\text{pack}\ e_1\ \text{as}\ T[r_1,\ldots,r_n]$ &  \\
  &$\mid$& null $\;\;\;\mid\;\;\;$ $a$ & address &
       & $\mid$& $\text{unpack}\ e_1\ \text{as}\ id : T[r_1,\ldots,r_n]\ \text{in}\ e_2$ &  \\
  &$\mid$& $\{ (a_i, iv), \ldots \}$ & coloring &
    & & & \\
  &$\mid$& $\langle \langle \oton{\rho}, v\rangle \rangle$ & reg. relation instance &
   & & & \\
%$bv$ & ::= & false $\;\;\;\mid\;\;\;$ true \\
%\\
%$iv$ & ::= & 0 $\;\;\;\mid\;\;\;$ 1 $\ldots$ \\
%\\
%$e$ & ::= & & expressions \\
%  &$\mid$& $bv$ $\;\;\;\mid\;\;\;$ $iv$ & constants \\
%  &$\mid$& $\langle e_1, e_2 \rangle$ $\;\;\;\mid\;\;\;$ $e$.1 $\;\;\;\mid\;\;\;$ $e$.2 & tuple \\
%  &$\mid$& $id$ &  \\
%  &$\mid$& $\text{new}\ T@r$ $\;\;\;\mid\;\;\;$ $\text{null }T@r$ $\;\;\;\mid\;\;\;$ $\text{isnull}(e)$ & \\
%  &$\mid$& $\text{upregion}(e, r_1,\ldots,r_n)$ $\;\;\;\mid\;\;\;$ $\text{downregion}(e, r_1,\ldots,r_n)$ & \\
%  &$\mid$& $\text{read}(e_1)$ $\;\;\;\mid\;\;\;$ $\text{write}(e_1, e_2)$ $\;\;\;\mid\;\;\;$ $\text{reduce}(id, e_1, e_2)$ & memory access \\
%  &$\mid$& $\text{newcolor}\ r$ $\;\;\;\mid\;\;\;$ $\text{color}(e_1, e_2, e_3)$ & coloring operations \\
%  &$\mid$& $e_1 + e_2$ & integer operations \\
%  &$\mid$& $e_1 < e_2$ & comparison operations \\
%  &$\mid$& $\text{let}\ id : T = e_1 \text{in}\ e_2$ &  \\
%  &$\mid$& $\text{if}\ e_1\ \text{then}\ e_2\ \text{else}\ e_3$ &  \\
%  &$\mid$& $id[r_1, \ldots, r_n](e_1,\ldots,e_n)$ & function calls \\
%  &$\mid$& $\text{partition}\ r_p\text{ using }e_1\text{ as }\oton{r}\text{ in }\ e_2$ &  \\
%  &$\mid$& $\text{pack}\ e_1\ \text{as}\ T[r_1,\ldots,r_n]$ &  \\
%  &$\mid$& $\text{unpack}\ e_1\ \text{as}\ id : T[r_1,\ldots,r_n]\ \text{in}\ e_2$ &  \\
\end{tabular}
}
\caption{Core Legion}
\label{fig:langdef}
\vspace{-5mm}
\end{figure*}

\newcommand{\placeat}[3]{%
\makebox[0pt]{%
\hspace*{#1in} \raisebox{#2in}{#3}%
}%
}

\newcommand{\infrulez}[2]{\displaystyle\frac{\displaystyle\strut{#1}}{\displaystyle\strut {#2}}}
\newcommand{\cinfrulez}[3]{\parbox{14cm}{\hfil$\infrulez{#1}{#2}$\hfil}\parbox{4cm}{$\,#3$\hfil}}
\newcommand{\finfrulez}[2]{\framebox{$\infrulez{#1}{#2}$}}

\newcommand{\ruleat}[5]{
\node[below right] at (#1) {$\infrulez{#4}{#5}$};
\node[below] at (#2) {[#3]};
}

\newcommand{\ruleatx}[5]{
\node[below right,fill=white!50] at (#1) {$\infrulez{\begin{array}{l}#4\end{array}}{#5}$};
\node[below] at (#2) {[#3]};
}

\newcommand{\axiomat}[4]{
\node[below right] at (#1) {$#4$};
\node[below] at (#2) {[#3]};
}

%\newcommand{\infrule}[2]{\displaystyle\frac{\displaystyle\strut{#1}}{\displaystyle\strut {#2}}}
%\newcommand{\cinfrule}[3]{\parbox{14cm}{\hfil$\infrule{#1}{#2}$\hfil}\parbox{4cm}{$\,#3$\hfil}}
%\newcommand{\finfrule}[2]{\framebox{$\infrule{#1}{#2}$}}
%\newcommand{\oldfinfrule}[2]{\vspace{10pt}\framebox{$\infrule{#1}{#2}$}\vspace{10pt}}

%\newcommand{\infx}[2]{\infrule{\begin{array}{l}{#1}\end{array}}{#2}}

\section{Legion}
\label{sec:legioncore}

Figure~\ref{fig:langdef} defines Core Legion, a subset of the full Legion language
that illustrates the important issues.
%Types include booleans, integers, tuples, and pointers.
Pointer types are annotated with a list of logical regions---a non-null pointer value must point to a
location that is contained in at least one of the regions. There is a special
{\em coloring} type, used to specify partitions of regions into subregions.
%Functions in Legion are named, accept one or more arguments and
%return a value of some type.
Functions declare the region
access privileges and coherence they require.  Some function instances are {\em tasks}; a function may be executed
within the current task or run in parallel as a separate (sub)task---this decision is made
by the Legion task scheduler.  Function types are
universally quantified over all region names appearing in the type.
The other important type is a {\em region relationship}, an instance of which captures a value and
one or more regions satisfying associated constraints.  A region relation instance
can be written into the heap and later read and
unpacked, giving new local names to the regions contained in
the instance.

To achieve high performance in the presence of aliasing, the Legion type system
leverages both static and dynamic checks, resulting in interesting extensions beyond
a basic expression language to both the type system and the operational semantics.
The following two subsections illustrate these decisions and how they have been
chosen to allow a straightforward proof of type soundness.

\begin{figure}
\centering{
$\begin{array}{l}
\Omega \subseteq \Omega^* \\
r_i \leq r_j \in \Omega^*  \Rightarrow r_i \leq r_i \in \Omega^*\wedge r_j \leq r_j \in \Omega^* \\
r_i \leq r_j \in \Omega^* \wedge r_j \leq r_k \in \Omega^* \Rightarrow r_i \leq r_k \in \Omega^* \\
r_i \leq r_j \in \Omega^* \wedge r_j * r_k \in \Omega^* \Rightarrow r_i * r_k \in \Omega^* \\
r_i * r_j \in \Omega^* \Rightarrow r_j * r_i \in \Omega^* \\
\\
\Phi \subseteq \Phi^* \\
r_i \leq r_j \in \Omega^* \wedge \text{reads}(r_j) \in \Phi^* \Rightarrow \text{reads}(r_i) \in \Phi^* \\
r_i \leq r_j \in \Omega^* \wedge \text{writes}(r_j) \in \Phi^* \Rightarrow \text{writes}(r_i) \in \Phi^* \\
r_i \leq r_j \in \Omega^* \wedge \text{reduces}_{id}(r_j) \in \Phi^* \Rightarrow \text{reduces}_{id}(r_i) \in \Phi^* \\
\text{reads}(r) \in \Phi^* \wedge \text{writes}(r) \in \Phi^* \Rightarrow \text{reduces}_{id}(r) \in \Phi^* \\
\ \ \ \mbox{for every function identifier $id$}
\end{array}$
}
\caption{Privilege and Constraint Closure}
\vspace{-5mm}
\label{fig:closure}
\end{figure}

\subsection{Core Type System}
\label{subsec:coretypes}

Core Legion is explicitly typed using judgments of the form
$$\typeenv{e}{T}$$
Besides a type environment $\Gamma$, type judgments include the
access privileges $\Phi$ for the logical regions in the expression $e$ as well as
constraints $\Omega$ that must hold between those logical regions.

A representative selection of the type rules is given in Figure~\ref{fig:langrules}.  Both $\Phi$ and $\Omega$ are used in the heap access expressions {\em read}, {\em write}, and {\em reduce}.
A valid heap access has the needed permission for logical region(s) in the
pointer's type.  Note the exact region need not be named in $\Phi$ if permissions
exist for logical regions that provably contain the pointer's region(s).  To simplify this check, Figure~\ref{fig:closure}
defines closure operations $\Omega^*$ (all constraints implied by $\Omega$) and
$\Phi^*$ (all privileges implied by $\Phi$ and $\Omega^*$).

Core Legion provides no automatic pointer type conversions.  Valid pointers are created via the {\em new}
expression in the specified region.
% and invalid (i.e. {\em null}) pointers must also be typed.
Pointers can be
``upcast'' via the {\em upregion} expression; the type rule verifies
every possible region the pointer might point into is covered by at least one of the regions in the target
pointer type.  A ``downcast'' using the {\em downregion} expression performs a dynamic
test to verify the region a run-time pointer value points into.
%A static test similar to the one used
%for {\em upregion} could be used to catch cases in which the downcast can never succeed at runtime, but we
%have left it out for simplicity.

Region constraints are introduced by the {\em partition} expression,
which makes use of {\em colorings}, a new type specific to Legion.  A coloring is an opaque mapping from
pointers (which must point into the region being colored) to integer {\em color} values.
%(For simplicity,
%Core Legion allows only an iterative construction of a coloring via a {\em newcolor} expression that generates
%an empty map, and a {\em color} expression which adds a new entry to an existing map. The opacity of the
%coloring allows for more efficient generation (and storage) of colorings as needed.)
Partitioning introduces names for the subregions corresponding to each color in the coloring, guaranteeing that each
subregion is disjoint from the others and all are subregions of the original region.

The {\em pack} and {\em unpack} expressions allow regions and pointers into those regions to be stored in the heap.
%  Logical region names are lexically scoped and if stored in the heap as-is, can only be read within the same scope.
A {\em region relationship} captures statically known constraints between regions;
because the {\em pack} rule verifies these constraints hold when the
region relationship value is created, {\em unpack} can reintroduce these constraints on
the (fresh) logical region names when the value is unpacked from the heap.
Crucially for soundness, region relationships do not capture privileges.
%
% FIXME  It would be nice to add some comments about unpack here, and how the constraints introduced by unpack
% must be sufficient to allow access to the new regions if they are to be used, which generally requires the type
% have a free region variable capturing subregioning relationships between the packed regions and some existing region
% in scope for which privileges are held.
%
Task calls verify the requirement that the privileges held by a called task are a subset of those held
by the caller.  In addition to being necessary for the proof of soundness, this property is critical for
hierarchical and distributed scheduling of tasks.

\begin{comment}
\begin{figure*}
\centering{
\framebox{$\typeenv{bv}{bool}$}
\framebox{$\typeenv{iv}{int}$}
\finfrule
{\begin{array}{l}
\typeenvx e_1 : T_1 \\
\typeenvx e_2 : T_2
\end{array}}
{\typeenvx \langle e_1, e_2 \rangle : \langle T_1, T_2 \rangle}
\finfrule{\typeenvx e : \langle T_1,T_2 \rangle}{\typeenvx e\text{.1}\ : T_1}
\finfrule{\typeenvx e : \langle T_1,T_2 \rangle}{\typeenvx e\text{.2}\ : T_2}
\finfrule{\Gamma(id) = T}{\typeenvx id : T}
\framebox{$\typeenvx \text{null }T@r : T@r$}
\framebox{$\typeenvx \text{new }T@r : T@r$}
\finfrule{\typeenvx e : T@(\oton{r})}{\typeenvx \text{isnull}(e) : bool}
\finfrule
{\begin{array}{l}
\typeenvx e : T@(r'_1, \ldots r'_k) \\
\forall i. \exists j, r'_i \leq r_j \in \Omega^* \\
\end{array}}
{\typeenvx upregion(e,\oton{r}) : T@(\oton{r})}
\finfrule
{\begin{array}{l}
\typeenvx e : T@(r'_1, \ldots r'_k) \\
\forall j. \exists i, r_j \leq r'_i \in \Omega^* \\
\end{array}}
{\typeenvx downregion(e,\oton{r}) : T@(\oton{r})}
\finfrule
{\begin{array}{l}
\typeenvx e_1 : T@(\oton{r}) \\
\forall i, reads(r_i) \in \Phi^*\end{array}}
{\typeenvx \text{read}(e_1) : T}
\finfrule
{\begin{array}{l}
\typeenvx e_1 : T@(\oton{r}) \\
\typeenvx e_2 : T \\
\forall i, writes(r_i) \in \Phi^*
\end{array}}
{\typeenvx \text{write}(e_1, e_2) : T@(\oton{r})}
\finfrule
{\begin{array}{l}
\Gamma(id) = (T_1, T_2), \emptyset, \emptyset \rightarrow T_1 \\
\typeenvx e_1 : T_1@(\oton{r}) \\
\typeenvx e_2 : T_2 \\
\forall i, reduces_{id}(r_i) \in \Phi^*
\end{array}}
{\typeenvx \text{reduce}(id, e_1, e_2) : T_1@(\oton{r})}
\framebox{$\typeenvx \text{newcolor }r : \text{coloring}(r)$}
\finfrule{\begin{array}{l}
\typeenvx e_1 : \text{coloring}(r) \\
\typeenvx e_2 : T@r \\
\typeenvx e_3 : int
\end{array}}
{\typeenvx \text{color}(e_1, e_2, e_3) : \text{coloring}(r)}
\finfrule{\begin{array}{l}\typeenvx e_1 : int \\ \typeenvx e_2 : int\end{array}}{\typeenvx e_1 + e_2 : int}
\finfrule{\begin{array}{l}\typeenvx e_1 : int \\ \typeenvx e_2 : int\end{array}}{\typeenvx e_1 < e_2 : bool}
\finfrule{\begin{array}{l}
\typeenvx e_1 : T_1 \\
\typeenvx[G={\Gamma[id/T_1]}] e_2 : T_2
\end{array}}
{\typeenvx : \text{let}\ id : T_1 \text{in}\ e_2 : T_2}
\finfrule{\begin{array}{l}\typeenvx e_1 : bool \\ \typeenvx e_2 : T \\ \typeenvx e_3 : T\end{array}}{\typeenvx \text{if}\ e_1\ \text{then}\ e_2\ \text{else}\ e_3 : T}
\finfrule{
\begin{array}{l}
\Gamma(id) = \forall r'_1, \ldots r'_k.(\oton{T}),\Phi', Q' \rightarrow T_r \\
\typeenvx e_i : T_i[r_1/r'_1,\ldots,r_k/r'_k] \\
\Phi'[r_1/r'_1,\ldots,r_k/r'_k] \subseteq \Phi^*
\end{array}}
{\typeenvx id[\otok{k}{r}](\oton{e}) : T_r[r_1/r'_1,\ldots,r_k/r'_k]}
\finfrule{
\begin{array}{l}
\typeenvx e_1 : \text{coloring}(r_p) \\
\Omega' = \Omega \wedge \bigwedge_{i \in [1,k]} r_i \leq r_p \wedge \bigwedge_{1 \leq i < j \leq k} r_i * r_j \\
\typeenvx[O=\Omega'] e_2 : T \\
\{ \oton{r} \} \cap \textit{regions\_of}(\Gamma, T) = \emptyset
\end{array}}
{\typeenvx \text{partition}\ r_p\text{ using }e_1\text{ as }\otok{k}{r}\text{ in }e_2 : T}
\finfrule{
\begin{array}{l}
T_1 = \exists r'_1, \ldots r'_n.\ T_2\text{ where }\Omega_1 \\
\Omega_1[r_1/r'_1,\ldots,r_k/r'_k] \subseteq \Omega^* \\
\typeenvx e_1 : T_2[r_1/r'_1,\ldots,r_k/r'_k]
\end{array}}
{\typeenvx \text{pack}\ e_1\ \text{as}\ T_1[\otok{k}{r}] : T_1}
\finfrule{
\begin{array}{l}
T_1 = \exists r'_1, \ldots, r'_n.\ T_2\text{ where }\Omega_1 \\
\typeenvx e_1 : T_1 \\
\Gamma' = \Gamma[T_2[r_1/r'_1,\ldots,r_k/r'_k] / id] \\
\Omega' = \Omega \cup \Omega_1[r_1/r'_1,\ldots,r_k/r'_k] \\
\typeenvx[G=\Gamma',O=\Omega'] e_2 : T_3 \\
\{ \oton{r} \} \cap \textit{regions\_of}(\Gamma, T_1, T_3) = \emptyset
\end{array}}
{\typeenvx \text{unpack}\ e_1\ \text{as}\ id : T_1[\otok{k}{r}] \text{in}\ e_2 : T_3}
\finfrule{
\begin{array}{l}
\text{for }1 \leq i \leq p, \\
\Gamma(id_i) = \forall r_{i1}, \ldots r_{ik_i}. (T_{i1}, \ldots, T_{in_i}), \Phi_i, Q_i \rightarrow T_{ir} \\
\Gamma_i = \Gamma[a_{i1}/T_{i1}, \ldots, a_{in_i}/T_{in_i}] \\
\typeenv[G={\Gamma_i},P={\Phi_i},O=\emptyset]{e_i}{T_{ir}}
\end{array}}
{
\begin{array}{l@{ }l}
\vdash \{ & \text{function }id_1[r_{11}, \ldots, r_{1k_1}]( a_{11} : T_{11}, \ldots a_{1n_1} : T_{1n_1} ), \Phi_1, Q_1 : T_{1r} : e_1, \\
          & \ldots \\
          & \text{function }id_p[r_{p1}, \ldots, r_{pk_1}]( a_{p1} : T_{p1}, \ldots a_{pn_p} : T_{pn_p} ), \Phi_p, Q_p : T_{pr} : e_p \} : \bullet
\end{array} }
}
\caption{Legion Core Type System}
\label{fig:types}
\end{figure*}
\end{comment}

\texcomment{
\begin{figure*}
\begin{adjustwidth}{-4em}{-4em}
\begin{center}
% Row 1: bool, int, tuple1, tuple2
\begin{minipage}{\linewidth}
%\centering
\begin{tabular}{m{4cm}m{3.5cm}m{5cm}m{5cm}}
\infax[T-Bool]{\typeenv{bv}{bool}} &
\infax[T-Int]{\typeenv{iv}{int}} &
\infrule[T-Tuple1]{\typeenvx e : \langle T_1,T_2 \rangle}{\typeenvx e\text{.1}\ : T_1} &
\infrule[T-Tuple2]{\typeenvx e : \langle T_1,T_2 \rangle}{\typeenvx e\text{.2}\ : T_2}
\end{tabular}
\end{minipage}
% Row 2:  var, null, isnull, maketuple
\begin{minipage}{\linewidth}
\vspace{-0.4cm}
%\centering
\begin{tabular}{m{3cm}m{3cm}m{5cm}m{6cm}}
\infrule[T-Var]{\Gamma(id) = T}{\typeenvx id : T} &
\infax[T-Null]{\typeenvx \text{null }T@r : T@r} &
\infrule[T-IsNull]{\typeenvx e : T@(\oton{r})}{\typeenvx \text{isnull}(e) : bool} &
\infrule[T-MakeTuple]{\typeenvx e_1 : T_1 \andalso \typeenvx e_2 : T_2}{\typeenvx \langle e_1, e_2 \rangle : \langle T_1, T_2 \rangle}
\end{tabular}
\end{minipage}
% Row 3: new, newcolor, coloring
\begin{minipage}{\linewidth}
\vspace{-0.6cm}
%\centering
\begin{tabular}{m{4cm}m{5cm}m{9cm}}
\infax[T-New]{\typeenvx \text{new }T@r : T@r} &
\infax[T-Newcolor]{\typeenvx \text{newcolor }r : \text{coloring}(r)} &
\infrule[T-Color]{\typeenvx e_1 : \text{coloring}(r) \andalso \typeenvx e_2 : T@r \andalso \typeenvx e_3 : int}{\typeenvx \text{color}(e_1, e_2, e_3) : \text{coloring}(r)}
\end{tabular}
\end{minipage}
% Row 4: let, add, compare
\begin{minipage}{\linewidth}
\vspace{-0.7cm}
%\centering
\begin{tabular}{m{6cm}m{5cm}m{6cm}}
\infrule[T-Let]{\typeenvx e_1 : T_1 \\ \typeenvx[G={\Gamma[id/T_1]}] e_2 : T_2}{\typeenvx : \text{let}\ id : T_1 \text{in}\ e_2 : T_2} &
\infrule[T-Add]{\typeenvx e_1 : int \\ \typeenvx e_2 : int}{\typeenvx e_1 + e_2 : int} &
\infrule[T-Compare]{\typeenvx e_1 : int \\ \typeenvx e_2 : int}{\typeenvx e_1 < e_2 : bool}
\end{tabular}
\end{minipage}
% Row 5: upregion, downregion
\begin{minipage}{\linewidth}
\vspace{-0.4cm}
\begin{tabular}{m{9cm}m{10cm}}
\infrule[T-UpRgn]{\typeenvx e : T@(r'_1, \ldots r'_k) \andalso \forall i. \exists j, r'_i \leq r_j \in \Omega^*}{\typeenvx upregion(e,\oton{r}) : T@(\oton{r})} &
\infrule[T-DownRgn]{\typeenvx e : T@(r'_1, \ldots r'_k)
% \andalso \forall j. \exists i, r_j \leq r'_i \in \Omega^*
}{\typeenvx downregion(e,\oton{r}) : T@(\oton{r})}
\end{tabular}
\end{minipage}
% Row 6: read, write, reduce
\begin{minipage}{\linewidth}
\vspace{-0.3cm}
\begin{tabular}{m{5cm}m{5cm}m{8cm}}
\infrule[T-Read]{\typeenvx e_1 : T@(\oton{r}) \\ \forall i, reads(r_i) \in \Phi^*}{\typeenvx \text{read}(e_1) : T} &
\infrule[T-Write]{\typeenvx e_1 : T@(\oton{r}) \\ \typeenvx e_2 : T \\ \forall i, writes(r_i) \in \Phi^*}{\typeenvx \text{write}(e_1, e_2) : T@(\oton{r})} &
\infrule[T-Reduce]{\Gamma(id) = (T_1, T_2), \emptyset, \emptyset \rightarrow T_1 \andalso \typeenvx e_2 : T_2 \\
 \typeenvx e_1 : T_1@(\oton{r}) \andalso \forall i, reduces_{id}(r_i) \in \Phi^*}{\typeenvx \text{reduce}(id, e_1, e_2) : T_1@(\oton{r})}
\end{tabular}
\end{minipage}
% Row 7: pack, if-else, task call
\begin{minipage}{\linewidth}
\vspace{-0.6cm}
\begin{tabular}{m{5cm}m{5cm}m{8cm}}
\infrule[T-Pack]{T_1 = \exists r'_1, \ldots r'_k.\ T_2\text{ where }\Omega_1 \\ \Omega_1[r_1/r'_1,\ldots,r_k/r'_k] \subseteq \Omega^* \\ \typeenvx e_1 : T_2[r_1/r'_1,\ldots,r_k/r'_k]}{\typeenvx \text{pack}\ e_1\ \text{as}\ T_1[\otok{k}{r}] : T_1} &
\infrule[T-IfElse]{\typeenvx e_1 : bool \\ \typeenvx e_2 : T \\ \typeenvx e_3 : T}{\typeenvx \text{if}\ e_1\ \text{then}\ e_2\ \text{else}\ e_3 : T} &
\infrule[T-Call]{\Gamma(id) = \forall r'_1, \ldots r'_k.(\oton{T}),\Phi', Q' \rightarrow T_r \\ \typeenvx e_i : T_i[r_1/r'_1,\ldots,r_k/r'_k] \\ \Phi'[r_1/r'_1,\ldots,r_k/r'_k] \subseteq \Phi^*}{\typeenvx id[\otok{k}{r}](\oton{e}) : T_r[r_1/r'_1,\ldots,r_k/r'_k]}
\end{tabular}
\end{minipage}
% Row 8: unpack, partition
\begin{minipage}{\linewidth}
\vspace{-0.7cm}
\begin{tabular}{m{10cm}m{8cm}}
\infrule[T-Unpack]{T_1 = \exists r'_1, \ldots, r'_k.\ T_2\text{ where }\Omega_1 \andalso \typeenvx e_1 : T_1 \\
  \Gamma' = \Gamma[T_2[r_1/r'_1,\ldots,r_k/r'_k] / id] \andalso \Omega' = \Omega \cup \Omega_1[r_1/r'_1,\ldots,r_k/r'_k] \\
  \typeenvx[G=\Gamma',O=\Omega'] e_2 : T_3 \andalso \{ \otok{k}{r} \} \cap \textit{regions\_of}(\Gamma, T_1, T_3) = \emptyset}
  {\typeenvx \text{unpack}\ e_1\ \text{as}\ id : T_1[\otok{k}{r}]\ \text{in}\ e_2 : T_3} &
\infrule[T-Part]{\typeenvx e_1 : \text{coloring}(r_p) \\ \Omega' = \Omega \wedge \bigwedge_{i \in [1,k]} r_i \leq r_p \wedge \bigwedge_{1 \leq i < j \leq k} r_i * r_j \\ \typeenvx[O=\Omega'] e_2 : T \\ \{ \otok{k}{r} \} \cap \textit{regions\_of}(\Gamma, T) = \emptyset}
  {\typeenvx \text{partition}\ r_p\text{ using }e_1\text{ as }\otok{k}{r}\text{ in }e_2 : T}
\end{tabular}
\end{minipage}
% Row 9: program
\begin{minipage}{\linewidth}
\vspace{-0.5cm}
\infrule[T-Program]{\text{for }1 \leq i \leq p, \\
  \Gamma(id_i) = \forall r_{i1}, \ldots r_{ik_i}. (T_{i1}, \ldots, T_{in_i}), \Phi_i, Q_i \rightarrow T_{ir} \andalso
  \Gamma_i = \Gamma[a_{i1}/T_{i1}, \ldots, a_{in_i}/T_{in_i}] \andalso
  \typeenv[G={\Gamma_i},P={\Phi_i},O=\emptyset]{e_i}{T_{ir}}}
  {\begin{array}{@{}l@{}l@{}}
   \vdash \{ & \text{function }id_1[r_{11}, \ldots, r_{1k_1}]( a_{11} : T_{11}, \ldots a_{1n_1} : T_{1n_1} ), \Phi_1, Q_1 : T_{1r} : e_1, \\
           & \ldots \\
           & \text{function }id_p[r_{p1}, \ldots, r_{pk_1}]( a_{p1} : T_{p1}, \ldots a_{pn_p} : T_{pn_p} ), \Phi_p, Q_p : T_{pr} : e_p \} : \bullet
   \end{array}
  }
\end{minipage}
\end{center}
\end{adjustwidth}
\caption{Legion Core Type System}
\vspace{-5mm}
\label{fig:types}
\end{figure*}
}

\subsection{Operational Semantics}
\label{subsec:opsemantics}

Core Legion's operational semantics rules have the form
$$\opsenv{e}{v}{E}$$
and specify the evaluation of expression $e$ yields a value $v$ and a
{\em dynamic memory trace} $E$, which is a sequence of all heap operations (reads, writes, reductions) $e$ performs.
%during the evaluation, including their location and the value read or written.
In addition to  mappings for local variables $L$ and the store (i.e. heap) $S$, the
environment includes an immutable heap typing $H$ assigning types to heap locations, and a region
mapping $M$ mapping logical regions $r_i$ to physical regions $\rho_i$, which are sets of concrete memory
locations.

The environment also has a {\em clobber set} $C$, which is the key to handling
limited nondeterminism  while still having an operational
semantics that is parallel and hierarchical.  Nondeterminism arises in a Legion
program because of relaxed coherence on
regions shared between parallel tasks and choices of reduction orderings.
Thus, heap accesses performed by one expression may observe side-effects from
concurrent evaluations.  The traditional solution to this issue
is to use a top-down, small-step semantics with a global view of all concurrently executing expressions (e.g., \cite{CML}).
%Rules are specified for which
%expression(s) are valid choices for the next small step of execution.
This approach is incompatible with a hierarchical view in which local
decisions are made with limited information.

Instead we use a bottom-up approach with big-step semantics.
Each expression has a clobber set $C$ of memory locations that might
be modified by concurrent evaluations.  Reads of locations in $C$ may be
chosen to return a value other than what was most recently written by the current
expression, as long as a
valid interleaving of the operations with other concurrent operations can be found within
an enclosing expression.
%% While a read from a location not in $C$ is {\em coherent}
%% (returning the most recently written value), the value read from a location $l$ in $C$ may be any
%% value.  Determining which value is deferred until all potentially-concurrent
%% modifications to $l$ are observable in the evaluation of a parent expression whose clobber set does
%% not include $l$ and a consistent interleaving of the subexpressions' memory traces is determined.
%With this formulation, the criteria for when a hierarchical scheduler may execute subexpressions in parallel can be
%made clear, but they are no longer constructive.  Instead, we will have to show that the scheduling algorithm
%used by the Legion runtime satisfies these constraints, guaranteeing the soundness of the Legion programming model.

%FIXME  need to add much earlier that the complete report has all missing type and operational rules and proofs

We show the most interesting operational rules for Core Legion in
Figure~\ref{fig:langrules}.  The rules for the {\em read}, {\em
write}, and {\em reduce} show how dynamic memory traces are
constructed and how the result of a read expression is relaxed when
the location falls within the clobber set.  The write
and reduce operations, with their two subexpressions, also demonstrate how
the traces of subexpressions are interleaved within the environment
(and clobber set) of the parent expression.  (A formal definition of
the {\em valid\_interleave} predicate can be found in
Figure~\ref{fig:validinterleave}, and will be discussed in
Section~\ref{sec:coherence}.)

%The first use of the region mapping $M$ occurs in the {\em new} and {\em downregion} expressions.
The {\em new} expression selects a location (with the right heap typing) from those assigned to
the logical region argument.
%(Although one would expect the new expression to also return a location that is not
%currently in use, our soundness result does not require this additional constraint, so we have omitted it for
%simplicity.)
Similarly, {\em downregion} checks whether a location is within the set assigned to the logical region.
%logical region(s) have been mapped to, returning {\em null} if not.
As discussed above,
{\em upregion} checks are performed statically.

% FIXME The explanation of newcolor is confusing.
%
%The general form of a coloring being an abitrary mapping of locations to ``colors'' (integers) is shown in the
%operational semantics rules for the {\em newcolor} and {\em color} expressions.
The {\em color} expression updates an existing coloring with a new color for a location, replacing any
existing coloring for that location.  The behavior of {\em newcolor} is subtler.
Legion supports first partitioning regions and then allocating data in the subregions.
%the {\em partition-then-allocate} model (see Section~\ref{sec:example}) in which the {\em new} expression may be called
%on subregions,
Thus, each subregion resulting from a partitioning may need an arbitrary number of locations that are
unknown at partitioning time.  The semantics of newcolor allows extra locations to be placed in every
subregion while
guaranteeing all explicit colorings performed by the program are honored.

%The semantics for the {\em pack} and {\em unpack} expressions are straightforward.
Packing a region relationship captures the physical regions to which the logical regions are mapped;
unpacking simply gives those physical regions fresh logical names.

Finally, calls to subtasks evaluate actual arguments and create the subtasks's
environment (with potentially different names for logical regions).  A key point in Legion task calls is the
handling of relaxed coherence.  Any location that falls within
a logical region whose coherence has been relaxed (via {\em atomic} or {\em simult}) is added to the
subtask's clobber set $C'$, allowing non-deterministic evaluation within the subtask with respect to
those locations.  All actual accesses to those locations within the resulting memory trace ($E_{n+1}$)
are marked with the corresponding coherence mode, allowing the {\em valid\_interleave} check within
the caller's environment to verify the consistency of the overall memory trace.

\texcomment{
\begin{figure*}
\centering{
$
\begin{array}{@{}l@{}}
\begin{array}{@{}l@{\hspace{-0.25in}}l@{}}
\begin{array}{@{}lll}
apply(S, []) & = & S \\
apply(S, E \dplus [ read(l, c, v) ]) & = & apply(S, E) \\
apply(S, E \dplus [ write(l, c, v) ]) & = & apply(S, E)[v/l] \\
apply(S, E \dplus [ reduce_{id}(l, c, v) ]) & = & S'[id(S'(l), v)/l], \\
& & \text{ where } S' = apply(S, E)
\end{array} &
\begin{array}{@{}ll}
mark\_coh([], \hat Q) & = [] \\
mark\_coh([ op(l, c, v) ] \dplus E, \hat Q) & = [ op(l, c', v) ] \dplus mark\_coh(E, \hat Q), \\
\multicolumn{2}{@{}l}{ \hspace{1.0in} \text{ where } c' =
\begin{cases}
simult, & \text{if }\exists \rho. l \in \rho \wedge \text{simult}(\rho) \in \hat Q \\
atomic, & \text{if }\exists \rho. l \in \rho \wedge \text{atomic}(\rho) \in \hat Q \\
excl, & \text{otherwise}
\end{cases}}
\end{array}
\end{array} \\
\begin{array}{@{}lll}
any\_interleave([], [], \ldots, []) & = & true \\
any\_interleave([ \epsilon ] \dplus E', E_1, \ldots, [ \epsilon ] \dplus E_i, \ldots, E_n) & = & any\_interleave(E', E_1, \ldots, E_i, \ldots, E_n) \\
valid\_interleave(S, C, E', \oton{E}) & \Rightarrow & any\_interleave(E', \oton{E}) \\
\end{array}
\end{array}
$}
\caption{Operational Semantics Helper Functions \label{fig:opsemfns}}
\vspace{-5mm}
\end{figure*}
}

\begin{comment}
\begin{figure*}
\centering{\small
\framebox{$\opsenvx bv \mapsto bv, []$}
\framebox{$\opsenvx iv \mapsto iv, []$}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto v_1, E_1 \\
S' = apply(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v_2, E_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx \langle e_1, e_2 \rangle \mapsto \langle v_1, v_2 \rangle, E'}
\finfrule
{\opsenvx e \mapsto \langle v_1, v_2 \rangle, E}
{\opsenvx e\text{.1} \mapsto v_1, E} \hspace{1cm}
\finfrule
{\opsenvx e \mapsto \langle v_1, v_2 \rangle, E}
{\opsenvx e\text{.2} \mapsto v_2, E}
\finfrule
{L(id) = v}
{\opsenvx id \mapsto v, []}
\framebox{$\opsenvx \text{null }T@r \mapsto null, []$}
\finfrule
{\begin{array}{l}
l \in M(r) \\
H(l) = M \llbracket T \rrbracket
\end{array}}
{\opsenvx \text{new }T@r \mapsto l, []}
\finfrule
{\opsenvx e \mapsto l, E}
{\opsenvx \text{isnull}(e) \mapsto \textit{false}, E}
\finfrule
{\opsenvx e \mapsto null, E}
{\opsenvx \text{isnull}(e) \mapsto true, E}
\finfrule
{\opsenvx e \mapsto v, E}
{\opsenvx \text{upregion}(e, \oton{r}) \mapsto v, E}
\finfrule
{\begin{array}{l}
\opsenvx e \mapsto l, E \\
v = \begin{cases}
l, & \text{if $\exists i, l \in M(r_i)$}. \\
null, & \text{otherwise}.
\end{cases}
\end{array}}
{\opsenvx \text{downregion}(e, \oton{r}) \mapsto v, E}
\finfrule
{\begin{array}{l}
\opsenvx e \mapsto l, E \\
S' = \text{apply}(S, E) \\
v = \begin{cases}
S'(l), & \text{if } l \not\in C \\
v' : H(l), & \text{otherwise}
\end{cases}
\end{array}}
{\opsenvx \text{read}(e) \mapsto v, E \dplus [ read(l, excl, v) ]}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto l, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v, E_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx \text{write}(e_1, e_2) \mapsto l, E' \dplus [ write(l, excl, v) ]}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto l, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v, E_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx \text{reduce}(id, e_1, e_2) \mapsto l, E' \dplus [ reduce_{id}(l, excl, v) ]}
\finfrule{
\begin{array}{l@{}l}
K = \{ (l_1, iv_1), & \ldots, (l_p, v_p) \}, \text{ where } \\
& (\forall i \in [1,p]. l_i \in M(r)) \wedge \\
& (\forall i,j \in [1,p]. l_i \not= l_j)
\end{array}
}
{\opsenvx \text{newcolor }r \mapsto K, []}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto K, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto l, E_2 \\
S'' = \text{apply}(S', E_2) \\
\opsenvx[S=S''] e_3 \mapsto v, E_3 \\
K' = \{ (l,v) \} \cup \{ (l_i,v_i) : (l_i,v_i) \in K \wedge l \not= l_i \} \\
valid\_interleave(S, C, E', E_1, E_2, E_3)
\end{array}}
{\opsenvx \text{color}(e_1, e_2, e_3) \mapsto K', E'}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto v_1, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v_2, E_2 \\
v' = v_1 + v_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx e_1 + e_2 \mapsto v', E'}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto v_1, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v_2, E_2 \\
v' = \begin{cases}
true, & \text{if $v_1 < v_2$}. \\
\textit{false}, & \text{otherwise}.
\end{cases} \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx e_1 < e_2 \mapsto v', E'}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto v_1, E_1 \\
L' = L[v_1/id] \\
S' = \text{apply}(S, E_1) \\
\opsenvx[L=L',S=S'] e_2 \mapsto v_2, E_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx \text{let }id : T = e_1\text{ in }e_2 \mapsto v_2, E'}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto true, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v_2, E_2
\end{array}}
{\opsenvx \text{if }e_1\text{ then }e_2\text{ else }e_3 \mapsto v_2, E_1 \dplus E_2}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto \textit{false}, E_1 \\
S' = \text{apply}(S, E_1) \\
\opsenvx[S=S'] e_3 \mapsto v_3, E_3
\end{array}}
{\opsenvx \text{if }e_1\text{ then }e_2\text{ else }e_3 \mapsto v_3, E_1 \dplus E_3}
\finfrule
{\begin{array}{l}
\opsenvx e_1 \mapsto v_1, E_1 \\
S_1 = \text{apply}(S, E_1) \\
\opsenvx e_2 \mapsto v_2, E_2 \\
\ldots \\
S_n = \text{apply}(S_{n-1}, E_n) \\
valid\_interleave(S, C, E', \oton{E})
\vspace{1.5mm} \\
\text{function }id[\otok{k}{r'}](a_1 : T_1, \ldots, a_n : T_n), \Phi', Q' : T_r = e_{n+1} \\
M' = \{ (r'_1, M(r_1)), \ldots (r'_k, M(r_k)) \} \\
L' = \{ (a_1, v_1), \ldots, (a_n, v_n) \} \\
S' = \text{apply}(S, E') \\
C' = C \cup \{ l : \exists \rho. atomic(\rho) \in M' \llbracket Q' \rrbracket \vee simult(\rho) \in M' \llbracket Q' \rrbracket \} \\
\opsenvx[M=M',L=L',S=S'] e_{n+1} \mapsto v_{n+1}, E_{n+1}
\vspace{1.5mm} \\
E'_{n+1} = mark\_coherence(E_{n+1}, M' \llbracket Q' \rrbracket) \\
valid\_interleave(S, C, E'', E', E_{n+1})
\end{array}}
{\opsenvx id[\otok{k}{r}](\oton{e}) \mapsto v_{n+1}, E''}
\finfrule{
\begin{array}{l}
\opsenv{e_1}{K}{E_1} \\
\rho_i = \{ l : (l, i) \in K \}, \text{ for } 1 \leq i \leq k \\
M' = M[\rho_1/r_1, \ldots, \rho_k/r_k] \\
S' = \text{apply}(S, E_1) \\
\opsenvx[M=M',S=S'] e_2 \mapsto v, E_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx \text{partition}\ r_p\text{ using }e_1\text{ as }\otok{k}{r}\text{ in }e_2 \mapsto v, E'}
\finfrule{
\begin{array}{l}
\opsenvx e_1 \mapsto v, E \\
\rho_i = M[r_i], \text{ for } 1 \leq i \leq k \\
v' = \langle \langle \otok{k}{\rho}, v \rangle \rangle
\end{array}}
{\opsenvx \text{pack}\ e_1\ \text{as}\ T_1[\otok{k}{r}] \mapsto v', E}
\finfrule{
\begin{array}{l}
\opsenvx e_1 \mapsto \langle \langle \otok{k}{\rho} , v_1 \rangle \rangle, E_1 \\
M' = M[\rho_1/r_1, \ldots, \rho_k/r_k] \\
L' = L[v_1/id] \\
S' = \text{apply}(S, E_1) \\
\opsenvx[M=M',L=L',S=S'] e_2 \mapsto v_2, E_2 \\
valid\_interleave(S, C, E', E_1, E_2)
\end{array}}
{\opsenvx \text{unpack}\ e_1\ \text{as}\ id : T_1[\otok{k}{r}] \text{in}\ e_2 \mapsto v_2, E' }
}
\caption{Legion Core Operational Semantics}
\label{fig:semantics}
\end{figure*}
\end{comment}

\texcomment{
\begin{figure*}
\begin{adjustwidth}{-4em}{-4em}
\begin{center}
% Row 1: bool, int, val
\begin{minipage}{\linewidth}
\begin{tabular}{m{6cm}m{6cm}m{6cm}}
\infax[E-Bool]{\opsenvx bv \mapsto bv, []} &
\infax[E-Int]{\opsenvx iv \mapsto iv, []} &
\infax[E-Null]{\opsenvx \text{null }T@r \mapsto null, []}
\end{tabular}
\end{minipage}
% Row 2: tuple1, tuple2, make tuple
\begin{minipage}{\linewidth}
\vspace{-0.6cm}
\begin{tabular}{m{9cm}m{4.5cm}m{4.5cm}}
\infrule[E-MakeTuple]{\opsenvx e_1 \mapsto v_1, E_1 \andalso S' = apply(S, E_1) \\
  \opsenvx[S=S'] e_2 \mapsto v_2, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)}
  {\opsenvx \langle e_1, e_2 \rangle \mapsto \langle v_1, v_2 \rangle, E'} &
\infrule[E-Tuple1]{\opsenvx e \mapsto \langle v_1, v_2 \rangle, E}{\opsenvx e\text{.1} \mapsto v_1, E} &
\infrule[E-Tuple2]{\opsenvx e \mapsto \langle v_1, v_2 \rangle, E}{\opsenvx e\text{.2} \mapsto v_2, E}
\end{tabular}
\end{minipage}
% Row 3: new, null, isnull, isnull
\begin{minipage}{\linewidth}
\vspace{-0.6cm}
\begin{tabular}{m{4cm}m{4.5cm}m{5cm}m{5cm}}
\infrule[E-New]{l \in M(r) \\ H(l) = M \llbracket T \rrbracket}{\opsenvx \text{new }T@r \mapsto l, []} &
\infrule[E-Var]{L(id) = v}{\opsenvx id \mapsto v, []} &
\infrule[E-IsNull-F]{\opsenvx e \mapsto l, E}{\opsenvx \text{isnull}(e) \mapsto \textit{false}, E} &
\infrule[E-IsNull-T]{\opsenvx e \mapsto null, E}{\opsenvx \text{isnull}(e) \mapsto true, E}
\end{tabular}
\end{minipage}
% Row 4: let, add, compare
\begin{minipage}{\linewidth}
\vspace{-0.7cm}
\begin{tabular}{m{6cm}m{6cm}m{6cm}}
\infrule[E-Let]{\opsenvx e_1 \mapsto v_1, E_1 \\ L' = L[v_1/id] \\ S' = \text{apply}(S, E_1) \\ \opsenvx[L=L',S=S'] e_2 \mapsto v_2, E_2 \\ valid\_interleave(S, C, E', E_1, E_2)}{\opsenvx \text{let }id : T = e_1\text{ in }e_2 \mapsto v_2, E'} &
\infrule[E-Add]{\opsenvx e_1 \mapsto v_1, E_1 \\ S' = \text{apply}(S, E_1) \\ \opsenvx[S=S'] e_2 \mapsto v_2, E_2 \\ v' = v_1 + v_2 \\ valid\_interleave(S, C, E', E_1, E_2)}{\opsenvx e_1 + e_2 \mapsto v', E'} &
\infrule[E-Compare]{\opsenvx e_1 \mapsto v_1, E_1 \\ S' = \text{apply}(S, E_1) \\ \opsenvx[S=S'] e_2 \mapsto v_2, E_2 \\ v' = \begin{cases} true, & \text{if $v_1 < v_2$}. \\ \textit{false}, & \text{otherwise}. \end{cases} \\ valid\_interleave(S, C, E', E_1, E_2)}{\opsenvx e_1 < e_2 \mapsto v', E'}
\end{tabular}
\end{minipage}
% Row 5: if-else, if-else, new color
\begin{minipage}{\linewidth}
\vspace{-0.7cm}
\begin{tabular}{m{6.5cm}m{6.5cm}m{6cm}}
\infrule[E-IfElse-T]{\opsenvx e_1 \mapsto true, E_1 \\ S' = \text{apply}(S, E_1) \\ \opsenvx[S=S'] e_2 \mapsto v_2, E_2}{\opsenvx \text{if }e_1\text{ then }e_2\text{ else }e_3 \mapsto v_2, E_1 \dplus E_2} &
\infrule[E-IfElse-F]{\opsenvx e_1 \mapsto \textit{false}, E_1 \\ S' = \text{apply}(S, E_1) \\ \opsenvx[S=S'] e_3 \mapsto v_3, E_3}{\opsenvx \text{if }e_1\text{ then }e_2\text{ else }e_3 \mapsto v_3, E_1 \dplus E_3} &
\infrule[E-Newcolor]{K = \{ (l_1, iv_1), \ldots, (l_p, iv_p) \}, \text{ where } \\ (\forall i \in [1,p]. l_i \in M(r)) \wedge (\forall i,j \in [1,p]. l_i \not= l_j)}{\opsenvx \text{newcolor }r \mapsto K, []}
\end{tabular}
\end{minipage}
% Row 6: partition, color
\begin{minipage}{\linewidth}
\vspace{-0.6cm}
\begin{tabular}{m{8.5cm}m{10.5cm}}
\infrule[E-Part]{\opsenv{e_1}{K}{E_1} \andalso \rho_i = \{ l : (l, i) \in K \}, \text{ for } 1 \leq i \leq k \\
  M' = M[\rho_1/r_1, \ldots, \rho_k/r_k] \andalso S' = \text{apply}(S, E_1) \\
  \opsenvx[M=M',S=S'] e_2 \mapsto v, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)}
  {\opsenvx \text{partition}\ r_p\text{ using }e_1\text{ as }\otok{k}{r}\text{ in }e_2 \mapsto v, E'} &
\infrule[E-Color]{\opsenvx e_1 \mapsto K, E_1 \andalso S' = \text{apply}(S, E_1) \\
  \opsenvx[S=S'] e_2 \mapsto l, E_2 \andalso S'' = \text{apply}(S', E_2) \\
  \opsenvx[S=S''] e_3 \mapsto v, E_3 \\
  K' = \{ (l,v) \} \cup \{ (l_i,v_i) : (l_i,v_i) \in K \wedge l \not= l_i \} \\
  valid\_interleave(S, C, E', E_1, E_2, E_3)}
  {\opsenvx \text{color}(e_1, e_2, e_3) \mapsto K', E'}
\end{tabular}
\end{minipage}
% Row 7: upregion, downregion, read
\begin{minipage}{\linewidth}
\vspace{-0.2cm}
\begin{tabular}{m{6cm}m{6.5cm}m{6.5cm}}
\infrule[E-UpRgn]{\opsenvx e \mapsto v, E}{\opsenvx \text{upregion}(e, \oton{r}) \mapsto v, E} &
\infrule[E-DownRgn]{\opsenvx e \mapsto l, E \\ v = \begin{cases} l, & \text{if $\exists i, l \in M(r_i)$}. \\ null, & \text{otherwise}. \end{cases}}{\opsenvx \text{downregion}(e, \oton{r}) \mapsto v, E} &
\infrule[E-Read]{\opsenvx e \mapsto l, E \andalso S' = \text{apply}(S, E) \\ v = \begin{cases} S'(l), & \text{if } l \not\in C \\ v' : H(l), & \text{otherwise} \end{cases}}{\opsenvx \text{read}(e) \mapsto v, E \dplus [ read(l, excl, v) ]}
\end{tabular}
\end{minipage}
% Row 8: write, reduce
\begin{minipage}{\linewidth}
\vspace{-0.5cm}
\begin{tabular}{m{9cm}m{9cm}}
\infrule[E-Write]{\opsenvx e_1 \mapsto l, E_1 \andalso S' = \text{apply}(S, E_1) \\
  \opsenvx[S=S'] e_2 \mapsto v, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)}
  {\opsenvx \text{write}(e_1, e_2) \mapsto l, E' \dplus [ write(l, excl, v) ]} &
\infrule[E-Reduce]{\opsenvx e_1 \mapsto l, E_1 \andalso S' = \text{apply}(S, E_1) \\
  \opsenvx[S=S'] e_2 \mapsto v, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)}
  {\opsenvx \text{reduce}(id, e_1, e_2) \mapsto l, E' \dplus [ reduce_{id}(l, excl, v) ]}
\end{tabular}
\end{minipage}
% Row 9: pack, unpack
\begin{minipage}{\linewidth}
\vspace{-0.5cm}
\begin{tabular}{m{8cm}m{10cm}}
\infrule[E-Pack]{\opsenvx e_1 \mapsto v, E \andalso \rho_i = M[r_i], \text{ for } 1 \leq i \leq k \\
  v' = \langle \langle \otok{k}{\rho}, v \rangle \rangle}
  {\opsenvx \text{pack}\ e_1\ \text{as}\ T_1[\otok{k}{r}] \mapsto v', E} &
\infrule[E-Unpack]{\opsenvx e_1 \mapsto \langle \langle \otok{k}{\rho} , v_1 \rangle \rangle, E_1 \andalso M' = M[\rho_1/r_1, \ldots, \rho_k/r_k] \\
  L' = L[v_1/id] \andalso S' = \text{apply}(S, E_1) \\
  \opsenvx[M=M',L=L',S=S'] e_2 \mapsto v_2, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)}
  {\opsenvx \text{unpack}\ e_1\ \text{as}\ id : T_1[\otok{k}{r}] \text{in}\ e_2 \mapsto v_2, E'}
\end{tabular}
\end{minipage}
% Row 10: task call
\begin{minipage}{\linewidth}
%\vspace{-0.5cm}
\infrule[E-Call]{\opsenvx e_1 \mapsto v_1, E_1 \andalso S_1 = \text{apply}(S, E_1) \andalso \opsenvx[S=S_1] e_2 \mapsto v_2, E_2 \andalso \ldots \andalso S_n = \text{apply}(S_{n-1}, E_n) \\
  valid\_interleave(S, C, E', \oton{E}) \andalso \text{function }id[\otok{k}{r'}](a_1 : T_1, \ldots, a_n : T_n), \Phi', Q' : T_r = e_{n+1} \\
  M' = \{ (r'_1, M(r_1)), \ldots (r'_k, M(r_k)) \} \andalso L' = \{ (a_1, v_1), \ldots, (a_n, v_n) \} \andalso S' = \text{apply}(S, E') \\
  C' = C \cup \{ l : \exists \rho. atomic(\rho) \in M' \llbracket Q' \rrbracket \vee simult(\rho) \in M' \llbracket Q' \rrbracket \} \\
  \opsenvx[M=M',L=L',S=S',C=C'] e_{n+1} \mapsto v_{n+1}, E_{n+1} \andalso E'_{n+1} = mark\_coherence(E_{n+1}, M' \llbracket Q' \rrbracket) \andalso valid\_interleave(S, C, E'', E', E'_{n+1})}
  {\opsenvx id[\otok{k}{r}](\oton{e}) \mapsto v_{n+1}, E''}
\end{minipage}
\end{center}
\end{adjustwidth}
\caption{Legion Core Operational Semantics}
\label{fig:semantics}
\end{figure*}
}

\begin{figure*}
{
\scalebox{0.9}{
\begin{tikzpicture}[x=1in,y={(0,-1in)}]
%\draw[thin] (0,0) to (7,0);
%\draw[thin] (0,0) to (0,6);
\foreach \y in {0,...,80}{
%  \draw[thin] (0,0.1*\y) to (7,0.1*\y);
}
\foreach \y in {0,...,16}{
%  \draw[red,thick] (0,0.5*\y) to (7,0.5*\y);
}
\foreach \y in {0,...,8}{
%  \draw[blue,very thick] (0,\y) to (7,\y);
}

% column positions
% \XA - left edge of type rules
% \XB - center of type rule names
% \XC - vertical line
% \XD - left edge of semantics rules
% \XE - center of semantics rule names

\newcommand{\XA}{0.0}
\newcommand{\XB}{2.8}
\newcommand{\XC}{3.4}
\newcommand{\XD}{3.65}
\newcommand{\XE}{7.3}

\draw[very thin] (\XC,0.1) -- (\XC,7.6);

% start with read/write/reduce rules
\ruleat{\XA,0.0}{\XB,0.2}{T-Read}
{\begin{array}{l}
\typeenvx e_1 : T@(\oton{r}) \\
\forall i.\ \text{reads}(r_i) \in \Phi^*\end{array}}
{\typeenvx \text{read}(e_1) : T}

\ruleat{\XA,0.6}{\XB,0.8}{T-Write}
{\begin{array}{l}
\typeenvx e_1 : T@(\oton{r}) \\
\typeenvx e_2 : T \\
\forall i.\ \text{writes}(r_i) \in \Phi^*
\end{array}}
{\typeenvx \text{write}(e_1, e_2) : T@(\oton{r})}

\ruleat{\XA,1.3}{\XB,1.5}{T-Reduce}
{\begin{array}{l}
\Gamma(id) = (T_1, T_2), \emptyset, \emptyset \rightarrow T_1 \\
\typeenvx e_1 : T_1@(\oton{r}) \\
\typeenvx e_2 : T_2 \\
\forall i.\ \text{reduces}_{id}(r_i) \in \Phi^*
\end{array}}
{\typeenvx \text{reduce}(id, e_1, e_2) : T_1@(\oton{r})}

\axiomat{\XA,2.2}{\XB,2.2}{T-New}
{\typeenvx \text{new }T@r : T@r}

\ruleat{\XA,2.5}{\XB,2.55}{T-UpRgn}
{\begin{array}{l}
\typeenvx e : T@(r'_1, \ldots r'_k) \\
\forall i. \exists j, r'_i \leq r_j \in \Omega^* \\
\end{array}}
{\typeenvx \text{upregion}(e,\oton{r}) : T@(\oton{r})}

\ruleat{\XA,3.1}{\XB,3.05}{T-DnRgn}
{\begin{array}{l}
\typeenvx e : T@(r'_1, \ldots r'_k) \\
%\forall j. \exists i, r_j \leq r'_i \in \Omega^* \\
\end{array}}
{\typeenvx \text{downregion}(e,\oton{r}) : T@(\oton{r})}

\axiomat{\XA,3.6}{\XB,3.6}{T-NewColor}
{\typeenvx \text{newcolor }r : \text{coloring}(r)}

\ruleat{\XA,3.9}{\XB,4.1}{T-Color}
{\begin{array}{l}
\typeenvx e_1 : \text{coloring}(r) \\
\typeenvx e_2 : T@r \\
\typeenvx e_3 : \text{int}
\end{array}}
{\typeenvx \text{color}(e_1, e_2, e_3) : \text{coloring}(r)}

\ruleat{\XA,4.6}{\XB,4.55}{T-Partition}
{\begin{array}{l}
\typeenvx e_1 : \text{coloring}(r_p) \\
\Omega' = \Omega \wedge \bigwedge_{i \in [1,k]} r_i \leq r_p \wedge \bigwedge_{1 \leq i < j \leq k} r_i * r_j \\
\typeenvx[O=\Omega'] e_2 : T \\
\{ \otok{k}{r} \} \cap \textit{regions\_of}(\Gamma, T) = \emptyset
\end{array}}
{\typeenvx \text{partition}\ r_p\text{ using }e_1\text{ as }\otok{k}{r}\text{ in }e_2 : T}

\ruleat{0,5.5}{\XB,5.7}{T-Pack}
{\begin{array}{l}
T_1 = \exists r'_1, \ldots r'_k.\ T_2\text{ where }\Omega_1 \\
\Omega_1[r_1/r'_1,\ldots,r_k/r'_k] \subseteq \Omega^* \\
\typeenvx e_1 : T_2[r_1/r'_1,\ldots,r_k/r'_k]
\end{array}}
{\typeenvx \text{pack}\ e_1\ \text{as}\ T_1[\otok{k}{r}] : T_1}

\ruleat{\XA,6.2}{\XB,6.4}{T-Unpack}
{\begin{array}{l}
T_1 = \exists r'_1, \ldots, r'_k.\ T_2\text{ where }\Omega_1 \\
\typeenvx e_1 : T_1 \\
\Gamma' = \Gamma[T_2[r_1/r'_1,\ldots,r_k/r'_k] / id] \\
\Omega' = \Omega \cup \Omega_1[r_1/r'_1,\ldots,r_k/r'_k] \\
\typeenvx[G=\Gamma',O=\Omega'] e_2 : T_3 \\
\{ \otok{k}{r} \} \cap \textit{regions\_of}(\Gamma, T_1, T_3) = \emptyset
\end{array}}
{\typeenvx \text{unpack}\ e_1\ \text{as}\ id : T_1[\otok{k}{r}]\ \text{in}\ e_2 : T_3}

\ruleat{\XA,7.4}{\XB,7.6}{T-Call}
{\begin{array}{l}
\Gamma(id) = \forall r'_1, \ldots r'_k.(\oton{T}),\Phi', Q' \rightarrow T_r \\
\typeenvx e_i : T_i[r_1/r'_1,\ldots,r_k/r'_k] \\
\Phi'[r_1/r'_1,\ldots,r_k/r'_k] \subseteq \Phi^*
\end{array}}
{\typeenvx id[\otok{k}{r}](\oton{e}) : T_r[r_1/r'_1,\ldots,r_k/r'_k]}

% now the operational semantics versions

\ruleatx{\XD,0.0}{\XE,0.2}{E-Read}{
\opsenvx e \mapsto l, E \andalso S' = apply(S, E) \\
v = \begin{cases} S'(l), & \text{if } l \not\in C \\
v' : H(l), & \text{otherwise} \end{cases}
}{\opsenvx \text{read}(e) \mapsto v, E \dplus [ read(l, excl, v) ]}

\ruleatx{\XD,0.9}{\XE,0.85}{E-Write}{
\opsenvx e_1 \mapsto l, E_1 \andalso S' = apply(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)
}{\opsenvx \text{write}(e_1, e_2) \mapsto l, E' \dplus [ write(l, excl, v) ]}

\ruleatx{\XD,1.5}{\XE,1.45}{E-Reduce}{
\opsenvx e_1 \mapsto l, E_1 \andalso S' = apply(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto v, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)
}{\opsenvx \text{reduce}(id, e_1, e_2) \mapsto l, E' \dplus [ reduce_{id}(l, excl, v) ]}

\ruleatx{\XD,2.1}{\XE,2.2}{E-New}{
l \in M(r) \andalso
H(l) = M \llbracket T \rrbracket
}{\opsenvx \text{new }T@r \mapsto l, []}

\ruleat{\XD,2.6}{\XE,2.7}{E-UpRgn}{
\opsenvx e \mapsto v, E
}{\opsenvx \text{upregion}(e, \oton{r}) \mapsto v, E}

\ruleatx{\XD,3.1}{\XE,3.3}{E-DnRgn}{
\opsenvx e \mapsto l, E \\
v = \begin{cases} l, & \text{if $\exists i, l \in M(r_i)$}. \\ null, & \text{otherwise}. \end{cases}
}{\opsenvx \text{downregion}(e, \oton{r}) \mapsto v, E}

\ruleatx{\XD,4.0}{\XE,4.2}{E-NewColor}{
K = \{ (l_1, iv_1), \ldots, (l_p, iv_p) \}, \text{ where } \\
(\forall i \in [1,p]. l_i \in M(r)) \wedge (\forall i,j \in [1,p]. l_i \not= l_j)
}{\opsenvx \text{newcolor }r \mapsto K, []}

\ruleatx{\XD,4.7}{\XE,4.9}{E-Color}{
\opsenvx e_1 \mapsto K, E_1 \andalso S' = apply(S, E_1) \\
\opsenvx[S=S'] e_2 \mapsto l, E_2 \andalso S'' = apply(S', E_2) \\
\opsenvx[S=S''] e_3 \mapsto v, E_3 \\
K' = \{ (l,v) \} \cup \{ (l_i,v_i) : (l_i,v_i) \in K \wedge l \not= l_i \} \\
valid\_interleave(S, C, E', E_1, E_2, E_3)
}{\opsenvx \text{color}(e_1, e_2, e_3) \mapsto K', E'}

\ruleatx{\XD,5.7}{\XE,5.55}{E-Partition}{
\opsenv{e_1}{K}{E_1} \andalso \rho_i = \{ l : (l, i) \in K \}, \text{ for } 1 \leq i \leq k \\
M' = M[\rho_1/r_1, \ldots, \rho_k/r_k] \andalso S' = apply(S, E_1) \\
\opsenvx[M=M',S=S'] e_2 \mapsto v, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)
}{\opsenvx \text{partition}\ r_p\text{ using }e_1\text{ as }\otok{k}{r}\text{ in }e_2 \mapsto v, E'}

\ruleatx{\XD,6.5}{\XE,6.6}{E-Pack}{
\opsenvx e_1 \mapsto v, E \andalso \rho_i = M[r_i], \text{ for } 1 \leq i \leq k \\
v' = \langle \langle \otok{k}{\rho}, v \rangle \rangle
}{\opsenvx \text{pack}\ e_1\ \text{as}\ T_1[\otok{k}{r}] \mapsto v', E}

\ruleatx{\XD,7.1}{\XE,6.95}{E-Unpack}{
\opsenvx e_1 \mapsto \langle \langle \otok{k}{\rho} , v_1 \rangle \rangle, E_1 \andalso M' = M[\rho_1/r_1, \ldots, \rho_k/r_k] \\
L' = L[v_1/id] \andalso S' = apply(S, E_1) \\
\opsenvx[M=M',L=L',S=S'] e_2 \mapsto v_2, E_2 \andalso valid\_interleave(S, C, E', E_1, E_2)
}{\opsenvx \text{unpack}\ e_1\ \text{as}\ id : T_1[\otok{k}{r}] \text{in}\ e_2 \mapsto v_2, E'}

\ruleatx{\XA,8.1}{\XE,8.4}{E-Call}{
\opsenvx e_1 \mapsto v_1, E_1 \andalso S_1 = apply(S, E_1) \andalso \opsenvx[S=S_1] e_2 \mapsto v_2, E_2 \andalso \ldots \andalso S_n = apply(S_{n-1}, E_n) \\
valid\_interleave(S, C, E', \oton{E}) \andalso \text{function }id[\otok{k}{r'}](a_1 : T_1, \ldots, a_n : T_n), \Phi', Q' : T_r = e_{n+1} \\
M' = \{ (r'_1, M(r_1)), \ldots (r'_k, M(r_k)) \} \andalso L' = \{ (a_1, v_1), \ldots, (a_n, v_n) \} \andalso S' = apply(S, E') \\
C' = C \cup \{ l : \exists \rho. atomic(\rho) \in M' \llbracket Q' \rrbracket \vee simult(\rho) \in M' \llbracket Q' \rrbracket \} \\
\opsenvx[M=M',L=L',S=S',C=C'] e_{n+1} \mapsto v_{n+1}, E_{n+1} \andalso E'_{n+1} = mark\_coherence(E_{n+1}, M' \llbracket Q' \rrbracket) \andalso valid\_interleave(S, C, E'', E', E'_{n+1})
}{\opsenvx id[\otok{k}{r}](\oton{e}) \mapsto v_{n+1}, E''}

% try to fit helper rules at the bottom instead of in a separate figure

\draw (0.05,9.12) -- (7.6,9.12);

\node[below right] at (\XA,9.15) {
$
\begin{array}{@{}l@{}}
\begin{array}{@{}l@{}l@{}}
\begin{array}{@{}lll}
apply(S, []) & = & S \\
apply(S, E \dplus [ read(l, c, v) ]) & = & apply(S, E) \\
apply(S, E \dplus [ write(l, c, v) ]) & = & apply(S, E)[v/l] \\
apply(S, E \dplus [ reduce_{id}(l, c, v) ]) & = & S'[id(S'(l), v)/l], \\
& & \text{ where } S' = apply(S, E)
\end{array} &
\begin{array}{@{}ll}
mark\_coherence([], \hat Q) & = [] \\
mark\_coherence([ op(l, c, v) ] \dplus E, \hat Q) & = [ op(l, c', v) ] \dplus mark\_coherence(E, \hat Q), \\
\multicolumn{2}{@{}l}{ \hspace{1.0in} \text{ where } c' =
\begin{cases}
simult, & \text{if }\exists \rho. l \in \rho \wedge \text{simult}(\rho) \in \hat Q \\
atomic, & \text{if }\exists \rho. l \in \rho \wedge \text{atomic}(\rho) \in \hat Q \\
excl, & \text{otherwise}
\end{cases}}
\end{array}
\end{array}
\end{array}
$};

\end{tikzpicture}
}
\vspace*{-0.5cm}
\texcomment{
\framebox[7in]{
\placeat{2}{3}{bar}%
\placeat{3}{1}{3,1}%
\placeat{-3}{1}{-3,1} %
\makebox[0pt]{%
\hspace*{2in} \raisebox{3in}{---}%
} %
\makebox[0pt]{%
\hspace*{1in} \raisebox{1in}{foo}%
} %
\makebox[0pt]{%
\hspace*{2in} \raisebox{3in}{bar}%
}%
\rule{0.1pt}{6in}
}}
}
\caption{Type System and Operational Semantics}
\label{fig:langrules}
\end{figure*}

\section{Soundness of Privileges}
\label{sec:soundness}

Our first result shows a well-typed expression accesses the heap in ways consistent with its static privileges.
A judgment $\privconsist{E}{\Phi}$ holds if memory operations in memory trace $E$ have types and
locations covered by privileges $\Phi$:
\[
\begin{array}{l}
\privconsist{E}{\Phi} \Leftrightarrow \forall \epsilon \in E. \\
\quad\quad (\epsilon = read(l, c, v) \Rightarrow \exists r, l \in M(r) \wedge \text{reads}(r) \in \Phi)\ \wedge \\
\quad\quad (\epsilon = write(l, c, v) \Rightarrow \exists r, l \in M(r) \wedge \text{writes}(r) \in \Phi)\ \wedge \\
\quad\quad (\epsilon = reduce_{id}(l, c, v) \Rightarrow \exists r, l \in M(r) \wedge \text{reduces}_{id}(r) \in \Phi)
\end{array}
\]

\noindent
%We must show $\privconsist{E}{\Phi}$ holds for any execution of an expression $e$ with static privileges $\Phi$.
As usual, the soundness claim is proven assuming the initial type and execution environments are consistent.
For our results, three consistency properties are needed:
\begin{itemize}
\item mapping consistency, written $\mapconsist{\Omega}$, guarantees a region mapping $M$ satisfies the region constraints $\Omega$% under which an expression was typed
\item local value consistency, written $\localconsist{L}{\Gamma}$, guarantees local values in $L$ have types consistent with the environment $\Gamma$ (using $M$ to map logical regions in $\Gamma$ to physical regions)
%under which the expression was typed
\item store consistency, written $\storeconsist{S}{H}$, guarantees locations in $S$ have values consistent with heap typing $H$
\end{itemize}

\noindent Two additional properties are proven for each subexpression:
\begin{itemize}
\item result value consistency, written $\valueconsist{v}{T}$, guarantees any evaluation of an expression yields a value of the right type
\item memory trace consistency, written $\traceconsist{E}$, guarantees that all writes and reductions use values of the right types
\end{itemize}
Figure~\ref{fig:constprop} defines these properties.

\begin{figure*}
\centering{\footnotesize
$
\begin{array}{l}
\begin{array}{ll}
\begin{array}{lll}
\mapconsist{\Omega} & \Leftrightarrow & (\forall r_i, r_j. r_i \leq r_j \in \Omega \Rightarrow M(r_i) \subseteq M(r_j))\ \wedge \\
              &                 & (\forall r_i, r_j. r_i * r_j \in \Omega \Rightarrow M(r_i) \cap M(r_j) = \emptyset) \\
\localconsist{L}{\Gamma} & \Leftrightarrow & \forall (id,v) \in L. \valueconsist[]{v}{M \llbracket \Gamma \rrbracket (id)} \\
\storeconsist{S}{H} & \Leftrightarrow & \forall (l,v) \in S. \valueconsist[]{v}{H(l)} \\
\end{array} &
\begin{array}{lll}
\traceconsist{E} & \Leftrightarrow & (\forall l, c, v. write(l,c, v) \in E \Rightarrow \valueconsist[]{v}{H(l)})\ \wedge \\
& & (\forall id, l, v. reduce_{id}(l, v) \in E \Rightarrow \\
& & (M \llbracket \Gamma \rrbracket (id) = (\hat T_1, \hat T_2), \emptyset, \emptyset \rightarrow \hat T_1) \wedge H(l) = \hat T_1 \wedge \valueconsist[]{v}{\hat T_2}) \\
\end{array}
\end{array} \vspace{3mm} \\
\begin{array}{l@{\hspace{1cm}}l}
\begin{array}{l}
\valueconsist[]{bv}{bool} \\
\valueconsist[]{iv}{int} \\
\valueconsist[]{null}{\hat T@\rho} \\
\end{array} &
\begin{array}{lll}
\valueconsist[]{l}{\hat T@\rho} & \Leftrightarrow & l \in \rho \wedge H(l) = \hat T \\
\valueconsist[]{\tuple{v_1}{v_2}}{\tuple{\hat T_1}{\hat T_2}} & \Leftrightarrow & (\valueconsist[]{v_1}{\hat T_1}) \wedge (\valueconsist[]{v_2}{\hat T_2}) \\
\valueconsist[]{\langle \langle \oton{\rho}, v \rangle \rangle}{\text{rr}[\oton{r}]\ \hat T\text{ where }\hat \Omega} & \Leftrightarrow & (\valueconsist[]{v}{T[\rho_1/r_1, \ldots \rho_n/r_n]}) \wedge (\mapconsist[\{ (r_i, \rho_i) \}]{\hat \Omega}) \\
\valueconsist[]{K}{coloring(\rho)} & \Leftrightarrow & \forall l_1, v_1. (l_1, v_1) \in K \Rightarrow (l_1 \in \rho\ \wedge \\
& & \forall l_2, v_2. (l_2, v_2) \in K \Rightarrow (l_1 \not= l_2) \vee (v_1 = v_2)) \\
\end{array}
\end{array}
\end{array}
$
}
\caption{Consistency Properties}
\label{fig:constprop}
\vspace{-5mm}
\end{figure*}

\begin{thm}
\label{thm:effects}
\rm
If $\typeenv{e}{T}$ and $\opsenv{e}{v}{E}$  and $\mapconsist{\Omega}$, $\localconsist{L}{\Gamma}$ and $\storeconsist{S}{H}$,
then
$\valueconsist{v}{T}$, $\traceconsist{E}$ and $\privconsist{E}{\Phi}$.
\end{thm}

The proof is made straightforward by the compositional operational
semantics.  The proof, along with proofs of all theorems and lemmas
in this paper, can be found in the submitted appendix.
%The proof, which can be found in the submitted appendix, is made
%straightforward by the compositional operational semantics.
%It proceeds by standard induction on the structure of the
%derivation in the operational semantics, at each step showing that
%subexpressions have consistent starting environments and that the
%value and trace consistency of the subexpressions is sufficient to prove
%the consistency of the parent expression.

\texcomment{
We spare the reader the lengthy proof, which can be found in
\cite{LegionTypes12}, and merely outline the general strategy.  The
proof is a standard induction on the structure of the derivation in
the operational semantics.  The proof is organized into six pieces,
one for each of the three consistency properties that must be shown
for the execution environment of subexpressions and one for each of
the three properties of the execution's results we wish to show:

\texcomment{
For each of
the 23 different forms a Legion expression can have, we assume the soundness of the subexpressions'
evaluations (provided we can show the consistency of the mappings and store used by that
subexpression) and prove that the result of the expression's evaluation is sound.  Although there
are a multitude of cases to be proven, many of them are similar, and benefit from
the use of the following lemmas:

\begin{lem}
\label{lemma:heapconst:apply}
If $\storeconsist{S}{H}$ and $\traceconsist{E}$, then $\storeconsist{\text{apply}(S, E)}{H}$.
\end{lem}

\begin{lem}
\label{lemma:heapconst:effects1}
If $\traceconsist{E_1}$ and $\traceconsist{E_2}$, then $\traceconsist{E_1 \dplus E_2}$.
\end{lem}

\begin{lem}
\label{lemma:heapconst:effects2}
If $\traceconsist{E_1}$ and $\traceconsist{E_2}$ and $valid\_interleave(S, C, E', E_1, E_2)$, then $\traceconsist{E'}$.
\end{lem}

\begin{lem}
\label{lemma:effsound:effects1}
If $\privconsist{E_1}{\Phi}$ and $\privconsist{E_2}{\Phi}$, then $\privconsist{E_1 \dplus E_2}{\Phi}$.
\end{lem}

\begin{lem}
\label{lemma:effsound:effects2}
If $\privconsist{E_1}{\Phi}$ and $\privconsist{E_2}{\Phi}$ and $valid\_interleave(S, C, E', E_1, E_2)$, then $\privconsist{E'}{\Phi}$.
\end{lem}

\begin{lem}
\label{lemma:omegaclosure}
$\mapconsist{\Omega^*}$ if and only if $\mapconsist{\Omega}$.
\end{lem}

\begin{lem}
\label{lemma:phiclosure}
$\privconsist{E}{\Phi^*}$ if and only if $\privconsist{E}{\Phi}$.
\end{lem}

The proofs of these lemmas can also be found here\cite{LegionTypes12}.
}

\vspace{4pt}
\hangindent=2\parindent \hangafter=1 \noindent %
$\mapconsist{\Omega}$ - Three expressions have subexpressions that modify $M$ or $\Omega$ and therefore do not
trivially satisfy region mapping consistency.  For {\tt partition},
the consistency of the coloring preserves region mapping consistency
with respect to the constraints.  For {\tt unpack}, the consistency of
a region relation instance guarantees consistency of region mapping.
Finally, the body of a called function uses an initially-empty set of
constraints, which are trivially satisfied.

\vspace{4pt}
\hangindent=2\parindent \hangafter=1 \noindent %
$\localconsist{L}{\Gamma}$ -  Four expressions have subexpressions that modify $L$, $\Gamma$, or $M$.  For
{\tt partition}, which only modifies $M$, the requirement that it not reuse existing
names ensures that $M \llbracket \Gamma \rrbracket$ does not change.  For {\tt let},
the value and type of the binding is obviously consistent, while the binding created in
an {\tt unpack} is less obviously so, requiring an induction over the type of the
unpacked value to show equivalence under the new mapping.  The last case is the body of a
called function, which requires the same style of proof as for {\tt unpack}
for each formal parameter.

\vspace{4pt}
\hangindent=2\parindent \hangafter=1 \noindent %
$\storeconsist{S}{H}$ -  The heap typing consistency of all stores used in subexpressions follows directly from the following
lemma: If $\storeconsist{S}{H}$ and $\traceconsist{E}$, then $\storeconsist{\text{apply}(S, E)}{H}$.

\vspace{4pt}
With the necessary consistency conditions proven for a subexpression, the conclusion of the theorem follows for
the subexpression by the inductive hypothesis.  The remaining obligation is to prove the the three parts of the
conclusion for an entire expression:

\vspace{4pt}
\hangindent=2\parindent \hangafter=1 \noindent %
$\valueconsist{v}{T}$ - The consistency of
{\tt upregion} is guaranteed by the type checking requirement of appropriate subregion
constraints and the mapping's consistency with those constraints, and {\tt downregion}'s
result is consistent because of the run time check.
%(The requirement
%for appropriate constraints in the {\em downregion} typing rule is primarily for performance
%(i.e. knowing which part of the region tree to examine) and to catch downcasts that will never
%work at runtime.)
The consistency of a {\tt read}'s result is trivial for an address in the
clobber set and uses the consistency of the store otherwise.  The consistency of a
{\tt color}'s result depends on the pointer subexpression's consistency and the
removal of any previous coloring of that location from the coloring set.

\hangindent=2\parindent \hangafter=1 \indent\indent\indent %
The remaining interesting cases arise from changes to the mapping $M$ rather than transformations on the
value $v$.  In the case of {\tt partition} and {\tt unpack}, the type system
guarantees that the subexpression's result cannot use the regions that were added to the
mapping, allowing the changes to the mapping to be ignored.  The last case is again the body
of a called function, and the same strategy that was used for the type consistency of the
formal parameters works in reverse for the function's result.

\vspace{4pt}
\hangindent=2\parindent \hangafter=1 \noindent %
$\traceconsist{E}$ -  The type consistency of the values in an expression's memory
trace follows from the following lemma applied to the traces of subexpressions:
if $\traceconsist{E_1}$ and $\traceconsist{E_2}$ and $any\_interleave(E', E_1, E_2)$, then $\traceconsist{E'}$.
New memory operations are added by {\tt write} and {\tt reduce} expressions, but consistency follows
directly from the induction hypothesis.  Finally, the consistency of the
values in a called function's memory trace is addressed in the same way as the return value.

\vspace{4pt}
\hangindent=2\parindent \hangafter=1 \noindent %
$\privconsist{E}{\Phi}$ -  The proof of the crucial property of containment of heap accesses within the available privileges
is similar in outline to the previous step.  The easy cases are covered by a lemma combining the corresponding results
for subexpressions: if $\privconsist{E_1}{\Phi}$ and $\privconsist{E_2}{\Phi}$ and $any\_interleave(E', E_1, E_2)$, then $\privconsist{E'}{\Phi}$.
Straightforward proofs cover {\tt read}, {\tt write}, {\tt reduce}, and one final special case for function calls.
}

\section{Coherence}
\label{sec:coherence}

An important insight from the proof of Theorem~\ref{thm:effects} is that
it does not rely on the full definition of {\em valid\_interleave}.  Weakening
{\em valid\_interleave} to {\em any\_interleave} (Figure~\ref{fig:validinterleave}),
which  allows arbitrary interleavings of memory traces from subexpressions,
is sufficient for the proof of soundness.
The stronger constraints address the coherence of heap accesses;
{\em valid\_interleave} specifies permitted interleavings of memory operations for
the particular coherence modes on logical regions.

To determine whether an interleaving of two or more memory traces is valid, we consider three sets of addresses:
\begin{itemize}
\item exclusive locations ($l \in L_{excl}$) are those which have at least one access in exclusive mode in the traces
and are not in the clobber set.  For these locations, we require sequential execution semantics - all
reads to these locations see the effect of previous writes and reductions, and the resulting state of the store is
as if all writes and reductions were applied from each trace in order.
\item atomic locations ($l \in L_{atomic}$) are those which have at least one access in atomic mode in the traces and
are in neither $L_{excl}$ nor the clobber set.  For these locations, we allow permutations of the original
subexpression trace order.
\item for locations with only access in {\em simult} mode or in the clobber set, no constraints are
enforced.  The validity of the interleaving of these accesses is determined within the context of some enclosing expression.
\end{itemize}

\begin{figure*}
$$
\begin{array}{@{}l@{}}
\begin{array}{@{}lll@{}}
any\_interleave([], [], \ldots, []) & = & true \\
any\_interleave([ \epsilon ] \dplus E', E_1, \ldots, [ \epsilon ] \dplus E_i, \ldots, E_n) & = & any\_interleave(E', E_1, \ldots, E_i, \ldots, E_n) \\
\\
\multicolumn{3}{@{}l}{
\begin{array}{@{}l|l@{}}
\begin{array}{@{}l@{}}
valid\_interleave(S, C, E', \oton{E}) = \vspace{0.03in} \\
\hspace{0.2in} any\_interleave(E', \oton{E})\ \wedge \vspace{0.03in} \\
\hspace{0.2in} coherent(S, L_{excl}(E', C), L_{excl}(E', C), E')\ \wedge \vspace{0.03in} \\
\hspace{0.2in} seq\_equiv(S, L_{excl}(E', C), L_{excl}(E', C), E', \oton{E})\ \wedge \vspace{0.03in} \\
\hspace{0.2in} coherent(S, L_{atomic}(E', C), \emptyset, E')\ \wedge \vspace{0.03in} \\
\hspace{0.2in} \exists \oton{\pi}, (\oton{\pi}) \text{ is a permutation of } (1,\ldots,n)\ \wedge \\
\hspace{0.3in} seq\_equiv(S, L_{atomic}(E', C), \emptyset, E', E_{\pi_1}, \ldots, E_{\pi_n})
\end{array} &
\begin{array}{@{}l@{}}
coherent(S, L_1, L_2, []) = \text{true} \\
coherent(S, L_1, L_2, [ \epsilon ] \dplus E) = \\
\hspace{0.2in}
\begin{cases}
(l \in L_2 \Rightarrow S(l) = v)\ \wedge \\
\ \ \ coherent(S, L_1, L_2, E), & \text{if $\epsilon = read(l, c, v)$} \\
coherent(apply(S, \epsilon), L_1, L_2 \cup \{ l \}, E), & \text{if $\epsilon = write(l, c, v)$} \\
& \hspace{0.25in} \text{and $l \in L_1$} \\
coherent(apply(S, \epsilon), L_1, L_2, E), & \text{otherwise}
\end{cases}
\end{array}
\end{array}
} \\
\\
\end{array} \\
\begin{array}{@{}ll@{}}
\begin{array}{@{}l@{}}
seq\_equiv(S, L_1, L_2, E', \oton{E}) = \\
\hspace{0.2in} coherent(S, L_1, L_2, E_1 \dplus \ldots \dplus E_2)\ \wedge \\
\hspace{0.2in} \forall l \in L_1. apply(S, E')(l) = apply(S, E_1 \dplus \ldots \dplus E_n)(l)
\end{array} &
\begin{array}{@{\hspace{0.2in}}lll@{}}
L_{excl}(E, C) & = & \{ l : op(l, excl, v) \text{ in } E  \} \setminus C \\
L_{atomic}(E, C) & = & \{ l : op(l, atomic, v) \text{ in } E \} \setminus (C \cup L_{excl}(E, C)) \\
\end{array}
\end{array}
\end{array}
$$
\caption{Valid Interleaving Test}
\label{fig:validinterleave}
\vspace{-5mm}
\end{figure*}

\subsection{Sequential Execution}
\label{subsec:coherenceseq}

%Although it is our goal to enable parallel execution whenever possible, we briefly
%consider sequential execution of a Legion program.
We now show that a sequential execution trivially
satisfies the interleaving criteria required by the operational semantics.  Our proof
of the soundness of parallel scheduling depends on this result.
%In addition to
%sequential execution being the default behavior of the hierarchical scheduler, our proof of the
%soundness of the parallel scheduling will depend on this result.

Sequential execution ignores the coherence mode $Q$ in all function calls, using $Q' = \emptyset$
instead, and interleaves traces by concatenating the subexpressions' traces in program order.  By ignoring
the coherence modes, the clobber set remains empty and the result of all {\em read} expressions is fully
determined.  The following lemma and theorem show that the
value and memory trace that result from a sequential execution are always valid executions.

\begin{lem}
\label{lem:clobberreduce}
\rm
Let $\typeenv{e}{T}$ and $\opsenv{e}{v}{E}$ and $\storeconsist{S}{H}$.  If $C \subseteq C'$, then
$\opsenv[C=C']{e}{v}{E}$.
\end{lem}

\begin{thm}
\label{thm:sequential}
\rm
Let $\oton{e}$ be expressions such that
%$\opsenv[S={S_{i-1}}]{e_i}{v_i}{E_i}$,
$M, L, H, S_{i-1},\linebreak C \vdash e_i \mapsto v_i, E_i$,
where $S_i = apply(S_{i-1}, E_i)$.  If $E' = E_1 \dplus \ldots \dplus E_n$, then
$valid\_interleave(S_0, C, E', \oton{E})$.
\end{thm}

\subsection{Parallel Execution}
\label{subsec:coherenceparallel}

To determine when parallel execution is safe, we start from the sequential execution trace and allow the reordering of
adjacent heap operations that do not change the behavior of the application.  If we can show that it is safe to
reorder any pair of operations that come from two different constituent traces, then any interleaving of the constituent
traces will be equivalent to a sequential execution and parallel execution is safe.  To
efficiently discover these cases at runtime, we require a test that can determine this property prior
to the actual execution of the tasks that create the traces.  We show that a test based on the subtask privileges and
the current region mapping can soundly predict when this property will hold.
%If we can show that reordering is allowed
%between any two operations that come from different constituent traces, any interleaving of the constituent traces will
%be equivalent to the serialized execution and parallel execution was safe.  Finally, to allow the safety of parallel
%execution to be determined {\em before} the execution is actually performed, we show that a test based on the subtask
%privileges and the current region mapping can soundly predict when this property will hold.
We begin by defining a {\em non-interference} operator on two memory operations
$\epsilon_1 = op_1(l_1, c_1, v_1)$ and $\epsilon_2 = op_2(l_2, c_2, v_2)$:
$$
\begin{array}{l@{ }l}
\epsilon_1\ \nonint\ \epsilon_2 \Leftrightarrow & (op_1 = read \wedge op_2 = read)\ \vee \\
& (op_1 = reduce_{id_1} \wedge op_2 = reduce_{id_2} \wedge id_1 = id_2) \vee \\
& l_1 \not= l_2
\end{array}
$$
Reads have no side effects, and cannot change what another read returns.  The safety of the second case
follows from the requirement that reduction operations be commutative.
Finally, accesses to different locations cannot affect each other.
Therefore, an adjacent pair of non-interfering memory operations in a memory trace can be reordered while
preserving the validity of an interleaving.
%We next show that this non-interference operator preserves the validity of an interleaving
%of two or more traces when reordering a pair of memory operations.
\begin{lem}
\label{lem:nonintswap}
\rm
Let $S$ be a store, $C$ a clobber set, $\oton{E}, E'_a, E'_b$ memory traces, and
$\epsilon_1, \epsilon_2$ be two memory operations from $E_i$ and $E_j$ ($i \not= j$).  Then,
%%$$\begin{array}{l}
\begin{align*}
valid&\_interleave(S, C, E'_a \dplus [ \epsilon_1, \epsilon_2 ] \dplus E'_b, \oton{E}) \wedge \epsilon_1 \nonint \epsilon_2 \\
&\Rightarrow valid\_interleave(S, C, E'_a \dplus [ \epsilon_2, \epsilon_1 ] \dplus E'_b, \oton{E}).
\end{align*}
%%\end{array}$$
\end{lem}
%The proof follows from the definitions of {\em coherent} and {\em seq\_equiv} in Figure~\ref{fig:validinterleave}.

\noindent Two whole memory traces are non-interfering if no operation from one
trace interferes with any operation from the other:
$$E_1 \nonint E_2 \Leftrightarrow \displaystyle\bigwedge_{\epsilon_1 \text{ in } E_1, \epsilon_2 \text{ in } E_2} \epsilon_1 \nonint \epsilon_2$$
If whole memory traces are non-interfering, any interleaving can be sorted via
pairwise swaps to match the sequential memory
trace. This gives us a result permitting safe parallel execution:

\begin{lem}
\label{lem:noninteffects}
\rm
Let $S$ be an initial store, $C$ be a clobber set, $\oton{E}$ be memory traces such that $E_i \nonint E_j$
for every $1 \leq i < j \leq n$.  Then,
$any\_interleave(E', \oton{E}) \Rightarrow$ \\
\hspace*{0pt} \hfill $valid\_interleave(S, C, E', \oton{E})$.
\end{lem}

\noindent
We now use the bounds that static privileges place on
runtime accesses to give an efficient runtime test for non-interference.  We first extend the non-interference operator
to work on privileges:

$\begin{array}{@{}l@{}l@{}}
\multicolumn{2}{@{}l}{priv_1(r_1) \nonint[M] priv_2(r_2) \Leftrightarrow} \\
\hspace*{0.5cm} & (priv_1 = \text{reads} \wedge priv_2 = \text{reads})\ \vee \\
& (priv_1 = \text{reduces}_{id_1} \wedge priv_2 = \text{reduces}_{id_2} \wedge id_1 = id_2)\ \vee \\
& (r_1 * r_2)\ \vee \\
& (M(r_1) \cap M(r_2) = \emptyset) \\
\\
\multicolumn{2}{@{}l}{\Phi_1 \nonint[M] \Phi_2 \Leftrightarrow \displaystyle\bigwedge_{\phi_1 \in \Phi_1, \phi_2 \in \Phi_2} \phi_1 \nonint[M] \phi_2}
\end{array}$

\noindent The cases where both subtasks have read-only privileges or both subtasks have reduce-only privileges have equivalents
for regions, which can be tested statically.  Detecting the case where the two sets of memory addresses are disjoint is
approximated by two tests.  The first uses (logical) region disjointness constraints from the type system to
statically infer non-interference.  The second uses the region mapping $M$ to dynamically determine the
disjointness of the two regions.  Although a dynamic test, it is performed
once per pair of regions rather than for every pair of memory operations.  An algorithm
to perform the dynamic test efficiently is given in \cite{Legion12}.
%and often need not be performed at all
%(if one of the other three conditions is shown to hold at compile-time).
As region non-interference is an approximation of memory trace non-interference, we must show that it is sound.
%We state the Lemma here, and give the proof in \cite{LegionTypes12}.

\begin{lem}
\label{lem:nonintpriv}
\rm
Let $M$ be a region mapping and $E_1$ and $E_2$ two memory traces such that $\privconsist{E_1}{\Phi_1}$ and $\privconsist{E_2}{\Phi_2}$.  If $\Phi_1$ and $\Phi_2$ are non-interfering under $M$, then $E_1$ and $E_2$ must be non-interfering.
\end{lem}

%We are now able to put all the pieces together and
We now state the theorem that allows the Legion runtime to perform hierarchical
and parallel scheduling of non-interfering tasks.

\begin{thm}
\label{thm:parallelexec}
\rm
Let $\oton{e}$ be well-typed Legion expressions, each with its own privileges $\Phi_i$.
Let $M$ be a region mapping, $L$ a local value mapping,
$H$ a heap typing, and $S$ be an initial store satisfying
$\mapconsist{\Omega}$, $\localconsist{L}{\Gamma}$, and $\storeconsist{S}{H}$.
If $\Phi_i \nonint[M] \Phi_j$ for $1 \leq i < j \leq n$, then any parallel execution of expressions
$\oton{e}$ results in a valid interleaving of memory operations.
\end{thm}

%The proof follows directly from Lemmas~\ref{lem:noninteffects} and \ref{lem:nonintpriv}.
\noindent This result holds even if the
clobber set $C$ is non-empty, allowing locally independent
subtasks to run in parallel even if they interact
(in a programmer-permitted way) with another  subtask.

We highlight an important aspect of a Legion
implementation that is different from other systems and relies on the
soundness of privileges.  Dynamic non-interference of
memory operations can only be determined after
evaluation of an expression is completed, and only at great expense,
as illustrated by work on transactional
memory \cite{Harris05}.
%However, many systems are willing to pay this cost to
%extract parallelism from operations that access aliased data.
At the other extreme are systems like
Jade \cite{Rinard98} and DPJ \cite{Bocchino11} that check non-interference
statically, but must disallow aliasing to do so.  In contrast, Legion
can verify non-interference of privileges
at runtime, which is much simpler and more efficient than checking
non-interference of dynamic memory traces.  Even though the privileges
themselves are static, the region mapping $M$ is dynamic.  Dynamically testing non-interference
on the privileges of physical regions allows parallel execution in many more cases than a purely
static analysis can achieve in the presence of aliasing.

\subsection{Atomic Coherence}
\label{subsec:coherenceatomic}

In cases where Legion cannot safely infer non-interference of
privileges (perhaps because two tasks actually access the same data in aliased regions),
relaxation of the constraints on execution order can
still be requested by the programmer through the use of coherence
annotations on individual regions passed to a task.
%the default behavior of sequential execution can
%be avoided through the use of relaxed coherence.
The {\em atomic} coherence mode specifies that although two
tasks interfere due to accessing aliased regions, they may execute in either order, allowing
the task issued later in program order to possibly run before the task issued earlier
in program order.  This relaxation only applies if all aliased regions are annotated with atomic coherence.
%or a more relaxed coherence mode (see Section~\ref{subsec:coherencesimult}).
To show this is safe, we define a relaxed version of non-interference for atomic coherence:
%$$
%\begin{array}{l@{}l}
\begin{align*}
op_1 &(l_1, c_1, v_1)\ \nonint^A\ op_2(l_2, c_2, v_2) \Leftrightarrow  \\
& op_1(l_1, c_1, v_1)\ \nonint\ op_2(l_2, c_2, v_2) \vee (c_1 = atomic \wedge c_2 = atomic)
\end{align*}
%\end{array}
%$$

We repeat the steps in Section~\ref{subsec:coherenceparallel} using the $\nonint^A$ operator and reach another
result used by the Legion runtime scheduler:

\begin{thm}
\label{thm:atomicexec}
\rm
Let $\oton{e}$ be well-typed Legion expressions, each with its own privileges $\Phi_i$.
Let $M$ be a region mapping, $L$ a local value mapping,
$H$ a heap typing, and $S$ be an initial store satisfying
$\mapconsist{\Omega}$, $\localconsist{L}{\Gamma}$, and $\storeconsist{S}{H}$.
If $\Phi_i \nonint[M]^A \Phi_j$ for $1 \leq i < j \leq n$, then for any permutation $(\oton{\pi})$ of
$(1, \ldots, n)$, $E_{\pi_1} \dplus \ldots \dplus E_{\pi_n}$ is a valid interleaving.
\end{thm}

\subsection{Simultaneous Coherence}
\label{subsec:coherencesimult}

Coherence also can be relaxed using the {\em simult} mode, which allows
multiple tasks to access the same region concurrently.  The simult coherence mode is appropriate in two
important cases:
\begin{enumerate}
\item When subtasks are accessing disjoint data, but the disjointness is difficult to describe (e.g. walking
separate linked lists that have been allocated in the same region).
\item When the algorithm is tolerant of non-determinism (e.g. in a breadth-first search, setting the parent
pointer of a node with multiple equally-short paths to the root).
\end{enumerate}

\noindent To support the {\em simult} coherence, the non-interference test is extended
%When one or more regions have been marked as using the simult coherence mode,
%the non-interference test is relaxed analagously to the atomic coherence case, yielding a $\nonint^S$ operator.
with a $\nonint^S$ operator, analogous to $\nonint^A$ for atomic coherence.
Because the rules for valid interleavings exclude locations that are only accessed in simult mode, it is straightforward
to extend Theorem~\ref{thm:parallelexec} to show that parallel execution is safe as long as $\Phi_i \nonint[M]^S \Phi_j$.

It is also possible to have both atomic and simult coherence modes at the same time for different regions in a task call.
In this case the non-interference test $\nonint^{AS}$ uses both the atomic and simult relaxations, and
Theorem~\ref{thm:atomicexec} is extended to allow arbitrary reordering (but not simultaneous execution) of subtasks when
$\Phi_i \nonint[M]^{AS} \Phi_j$.

\section{Hierarchical Scheduling}
\label{sec:scheduling}

Because testing non-interference of tasks is a pairwise operation, scheduling $n$ tasks
can require ${\cal O}(n^2)$ tests.  Thus, a scheduler that must globally consider all
pairs of tasks will be impractical for large machines and large numbers of tasks.  The
following theorem, however, shows that Legion programs enjoy a locality property that
limits the scope of needed non-interference tests.

%The enforcement of a (locally) valid interleaving at every step in the operational semantics
%captures the hierarchical nature of the scheduler used by the Legion runtime.  As the cost of
%a scheduling decision is quadratic in the number of function calls (more precisely, the number of
%privileges for those calls), it should be clear that a hierarchical scheduling algorithm will
%be vastly more efficient than a global algorithm that must consider the privileges of every
%concurrently executing expression.
%However, a scheduling algorithm that makes decisions hierarchically must do so with limited
%information, and must respect all required scheduling dependencies while ideally imposing as
%few extra dependencies as possible.  The following theorem forms the basis of the
%Legion scheduler.

\texcomment{
Let $e_1$ and $e_2$ be well-typed Legion expressions using privileges $\Phi_1$ and
$\Phi_2$.  Now let $e_1'$ be some subexpression of $e_1$ and $e_2'$ be some
subexpression of $e_2$, with corresponding privileges $\Phi_1'$ and $\Phi_2'$.
By Theorem~\ref{thm:parallelexec}, if the runtime can verify $\Phi_1 \nonint[M] \Phi_2$, then
expressions $e_1$ and $e_2$ may be evaluated in parallel.  The evaluation of the subexpressions
$e_1'$ and $e_2'$ will therefore be allowed to run in parallel, and we must show that this is
permissible.  We do so by proving that any memory traces that can result from the evaluation
of the subexpression must be non-interfering (allowing parallel execution by
Lemma~\ref{lem:noninteffects}).
}
\begin{thm}
\label{thm:hiersched}
\rm
Let $e_1$ and $e_2$ be well-typed expressions using privileges $\Phi_1$ and
$\Phi_2$ respectively, where $\Phi_1 \nonint[M] \Phi_2$.  Let $e_1'$ be a subexpression of $e_1$ and $e_2'$ be a
subexpression of $e_2$.  Any memory traces $E_1'$ of $e_1'$ and $E_2'$ of $e_2'$ resulting from evaluation of
$e_1$ and $e_2$ (with the usual consistent $M$, $L$, $H$, and $S$) are
non-interfering.
\end{thm}
\texcomment{
\begin{proof}
Let $E_1$ be the result of an evaluation of $e_1$, with $E_1'$ being the memory trace from the
evaluation of $e_1'$ within the expression tree.  $E_1$ will be formed from a tree of valid
interleavings, each of which must include all the memory operations of each constituent trace.
By induction over the number of intermediate subexpressions between $e_1$ and $e_1'$, we
can show that any memory operation $\epsilon_1'$ that is in $E_1'$ must also be in $E_1$.
Similarly, any memory operation $\epsilon_2'$ that is in $E_2'$ must also be in $E_2$.  We have
$E_1 \nonint E_2$ from Lemma~\ref{lem:nonintpriv}, so and therefore $\epsilon_1' \nonint \epsilon_2'$.  Since
this holds for all $\epsilon_1'$ in $E_1'$ and all $\epsilon_2'$ in $E_2'$, we have $E_1' \nonint E_2'$.
\end{proof}
}

The Legion task scheduler uses Theorem~\ref{thm:hiersched} as follows:
sibling function calls (those invoked within the same
function body) $e_1$ and $e_2$ are checked for non-interference of their (dynamic)
privileges.  Since $e_1$ and $e_2$ are called on the parent task's node, no communication
is required to perform the non-interference
test.  If they interfere they are executed in program order or serialized
depending on their coherence specifications; otherwise they are
considered for execution as parallel subtasks.    If $e_1$ and $e_2$ are
determined to be non-interfering and are scheduled in parallel on different
remote processors then Theorem~\ref{thm:hiersched}
guarantees that there is no communication required between $e_1$ and $e_2$
to perform non-interference tests between their sub-tasks.  Therefore,
the runtime requires no communication for scheduling.

%
% FIXME
% This is a nice and useful discussion, but also not on the main track and good candidate for space savings.
% Marking this with a FIXME so we can reconsider if we like.
%
%
%Although determining that $e_1$
%and $e_2$ are non-interfering guarantees $e_3$ and $e_4$ are
%non-interfering, the converse is not true.  It is possible for $e_1$
%and $e_2$ to be judged to be interfering and be executed sequentially
%even when the subexpressions are non-interfering.  This loss of
%precision can impact performance, and did in a few instances in our
%experiments.
%When the performance impact is significant, two options
%exist to improve matters.
%In cases where $e_1$ and $e_2$ are in fact
%non-interfering, region partitions can be further refined to make the
%disjointness visible in the privileges.  Alternately, the code can be
%refactored to lift the subexpressions up to the same level, allowing
%the dynamic test based on the region mapping to infer non-interference
%in more cases.
%Although this can increase the overhead of scheduling
%decisions, judicious use of this technique in practice has not shown
%any measurable performance degradation.


The simulation of eight different selfish mining strategies with five distinct distributions of computational power provides a reasonable outcome to argue the two research questions of the thesis.

\section{RQ1}

\textit{Do the simulations of selfish mining with the proposed software solutions show an increase of the total and relative gain for the selfish miner compared to the normal, honest mining behaviour?}

The simulation showed that the relative share of mining rewards obtained with selfish mining is higher compared to the honest mining if the miner has over 40\% of mining power and conducts a satisfactory selfish mining strategy.
To the satisfactory strategies, all strategies expect the combination lead stubborn and equal-fork stubborn (\textit{L, F}) and selfish mining with all three modifications (\textit{L, F T\textsubscript{1}}) can be counted.
The curves of the six good-performing strategies depicted in figure \ref{fig:accepted_blocks_selfish} show all a similar behaviour where their relative gain increases non-linearly with a higher computational share.
Out of these six strategies, the most promising strategies are normal selfish mining (\textit{S}) and equal-fork stubbornness (\textit{F}).
When these two strategies are applied, the selfish miner can create 49.1\%/50\% of the blocks of the longest chain even though its share of the mining power is only 45\%.

In all cases where the selfish miner has a low share of computational power, the results of the simulation show that the selfish mining does not increase the relative gain.
For example, when the selfish miner has 15\% of the mining power in the network only about 5\% of its blocks end up in the longest chain.
Even if the selfish miner has 37.5\% share of the computational power the most advantageously strategy (\textit{S}) only creates 32.4\% blocks of the longest chain.
The worst performing mining strategy in the scenario where the selfish miner has 37.5\% is the strategy with all three modifications (\textit{L, F, T\textsubscript{1}}) generating only 22.7\% of the accepted blocks.

Concluding it can be said that with the defined simulation scenario and the proposed software solutions only for a very high computational share an increase of the relative gain could be observed.
For all scenarios where the selfish miner has less than 40\% percent, the miner would earn relatively more by behaving honestly.

\section{RQ2}

\textit{How do the obtained results of the simulation match the outcome of previous research in the area of selfish mining?}

When comparing the outcome from the simulations with recent studies, it first needs to be considered that with the introduced simulation framework the outcome of the block races cannot be defined directly, as it is the case in previously used types of simulation.
Instead, the block races happen naturally during the execution of a scenario.
With the used simulation scenario and implementation of the selfish proxy, it is likely that the parameter $\gamma$, denoting if an honest node extends the private chain during a block race, is almost zero.
This assumption is based on the configured network topology and the implementation of the selfish proxy.
In the scenario, a fully connected network topology is used where each connection has the same latency.
Hence, when a node finds a block, it sends the block to all nodes directly.
Then it is unlikely that the selfish node can advertise its block faster to other nodes using the same connections before the honest nodes have already adopted to the new public tip.
The implementation of the selfish proxy further worsens the position of the selfish miner when trying to match a competing block during a block race.
The proxy which eclipses the private node does not implement the fast block propagation mechanism \textit{compact blocks}.
Instead, the proxy uses the standard but slower block propagation technique to receive and send blocks.
Thus, the proxy receives and sends blocks slower than the rest of the network.

Assuming that the probability of the selfish miner to win a block race is very low, the attained results match the outcome of previous research.
In figure \ref{fig:eyal_results} the red line shows the relative revenue of the selfish miner conducting normal selfish mining when it is not able to win any block race \cite{eyal2014majority}.
The curve shows the same concave course as in figure \ref{fig:accepted_blocks_selfish} underlining that with an increase of the computational share the efficiency of the selfish mining is amplified.
Moreover, the selfish miner achieved more relative revenue compare to the honest miner with about 36\% of the mining power, similar to the slightly worse results of 40\% showed in chapter \ref{chap:results}.
Comparable outcomes for the selfish mining without any modification were produced in other recent research \cite{nayak2016stubborn, sapirshtein2016optimal}.

\begin{figure}[t]
\includegraphics[width=8cm]{eyal_results}
\centering
\caption{Results obtained by \cite{eyal2014majority} with normal selfish mining}
\label{fig:eyal_results}
\end{figure}

In figure \ref{fig:nayak_results_1} optimal selfish mining strategies including stubborn modifications from the research of \cite{nayak2016stubborn} are shown.
In the case, where the selfish miner loses all block races, the honest behaviour is the most profitable strategy until a computational share of 34\% is reached.
Afterwards, the most advantageous mining strategy is trail stubbornness (\textit{T\textsubscript{1}}) up to a computational share of 45\%.
From 45\% to 50\% the dominant strategy is trail stubborn in combination with equal-fork stubborn (\textit{T\textsubscript{1}, F}), but in many cases, the best strategy is not clear signalised by the black dots in the graphic.
The results of the optimal stubborn mining strategy by \cite{nayak2016stubborn} slightly differ to the outcome of the simulation from chapter \ref{chap:results}.
In the proposed simulation the best performing strategies were the normal selfish mining (\textit{S}) and equal-fork stubborn mining (\textit{F}) with trail stubbornness combined with equal-fork stubbornness (\textit{T\textsubscript{1}, F}) and trail stubbornness (\textit{T\textsubscript{1}}) being only the third and fourth most satisfactory strategy.

\cite{nayak2016stubborn} does not propose the actual ranking of different selfish strategies regarding their performance for a specific $\gamma$ but provides a comparison of the relative gain obtained by selfish mining and selfish mining with stubborn modifications.
On the assumption that $\gamma$ is zero, the relative gain achieved by using stubborn modifications over selfish mining is very low as shown in figure \ref{fig:nayak_results_2}.
Thus, also in the research of \cite{nayak2016stubborn}, the normal selfish mining forms a viable strategy comparable to the results obtained in the previous chapters.
Furthermore, figure \ref{fig:nayak_results_1} showed that between 45\% and 50\% the best strategy is not always known denoted by the black dots.
From that concludes, that in the case where all block races are won by the honest network, the optimal strategy is not identifiable, similar to the outcome from the proposed simulations where the four best strategies were 3\% apart.

\begin{figure}[t]
\includegraphics[width=8cm]{nayak_results_1}
\centering
\caption{Optimal stubborn mining strategies retrieved by \cite{nayak2016stubborn}}
\label{fig:nayak_results_1}
\end{figure}

Summarising the obtained simulations results are comparable to recent research even though the outcome of the simulation was slightly worse than results from previous research.
Regarding the best performing selfish mining strategies also the research of \cite{nayak2016stubborn} showed that differences in the case where $\gamma$ equals zero are subtle.
Nevertheless, both studies showed similar good performing selfish mining strategies.

\begin{figure}[t]
\includegraphics[width=8cm]{nayak_results_2}
\centering
\caption{Selfish mining compared to optimal stubborn mining strategies by \cite{nayak2016stubborn}}
\label{fig:nayak_results_2}
\end{figure}

\section{Profitability}

As previously evaluated selfish mining can increase the relative gain of the misbehaving miner, but the results of the simulations show further that the absolute amount of accepted blocks is lower as shown in figure \ref{fig:accepted_blocks_selfish}.
Hence, the selfish miner is earning less mining rewards even though its relative revenue is increasing.
The loss of profit is possible because the stale block rate increases during selfish mining, and thus, fewer blocks end up in the longest chain.
To still profit from the attack the miner would need to wait for the difficulty adjustment.
After the difficulty adjustment, the nodes in the network can find more blocks and hence, also the selfish miner can create more blocks.
Since selfish mining was never observed for such a long time and no research was conducted in this area, it is not clear if such a long attack would be successful \cite{nayak2016stubborn}.