\subsection{Models for CDF Instances}~\label{sec:model}
\begin{definition} \label{def:ont-th}
Let $\cO$ be a CDF instance.  We define as $\cL_{\cO}$, the ontology
language whose functions and constant symbols are restricted to the
identifiers in $\cO$.  $TH(\cO)$ is the ontology theory over
$\cL_{\cO}$ whose non-logical axioms consist of the core axioms and
the instance axiom for each fact in $\cO$.
% while $\cO^{\cI}$ is the theory over $\cL_{\cO}$ containing only the
% instance axioms.
We say that an ontology structure $\cM$ is a model of $\cO$ if it is a
model of $TH(\cO)$.

If $\cO$ is Type-0 it is {\em well-typed} if each instance axiom is
consistent with Core Axioms~\ref{ax:distinct} (Distinct Identifiers)
and Axiom Schema~\ref{ax:downcl} (Downward Closure)
\end{definition}

\begin{theorem} \label{thm:equality}
Let $\cO$ be a well-typed CDF instance containing no {\tt minAttr/4}
or {\tt maxAttr/4} facts.  Then $\cO$ has a model.
\end{theorem}
\input{eqconstproof}
%---------------------------------------------------------
\mycomment{
From a practical point of view, the Core
Axioms~\ref{ax:distinct}-\ref{ax:nonnull}, and \ref{ax:downcl} can be
checked in a simple manner directly from a CDF instance, while Core
Axiom~\ref{ax:contained} is consistent with any CDF instance.
Determining that Core Axiom~\ref{ax:implsc} holds in a CDF instance is
less straightforward.  We now examine when a finite CDF instance
translates into a consistent theory that has a finite model.  The
notion of {\em conformability} is a syntactic property of a CDF
instance $\cO$ that is used to ensure that the translation of $\cO$
will satisfy Axiom~\ref{ax:implsc}.

Let $\cO$ be a CDF instance.  Then an identifier $C$ is a {\em class
identifier} if ${\tt class(Atom,C)} \in \cO$ for some $Atom$; an {\em
object identifier} if ${\tt object(Atom,C)} \in \cO$ for some $Atom$;
and a {\em relation identifier} if ${\tt relation(Atom,C)} \in \cO$
for some $Atom$.
}
%-------------------------------------------------------------

Theorem~\ref{thm:equality} indicates that any well-typed CDF instance
has a model, but in order to satisfy the various axioms, the model may
need to map a number of distinct identifiers into the same individual
in the universe of the model.  Such a model may be an undesirable
semantics for a CDF instance since using it for computation would
require unification over various equality theories.  We therefore
consider {\em freeness axioms}, which contains an axiom schema that
specifies that equality does not hold for any pair of distinct
constants or functions; and an axiom schema stating that no term can
be equal to a subterm it contains.  These axioms are commonly used in
logic programming (e.g. \cite{Lloy84}) and we do not include them
here.

\begin{definition}
Let $\cO$ be a CDF instance.  A {\em free ontology theory} for $\cO$,
$TH_F(\cO)$ is an ontology theory for $\cO$ extended with the freeness
axioms.  An ontology structure for a free ontology theory is called a
free model of $\cO$.
\end{definition}

Example~\ref{ex:equality} shows that CDF instances in which product
identifiers occur may not give rise to free models.  We derive a
condition for the existence of such models.

\begin{definition} \label{def:conformable}
Let $I$ and $I'$ be two identifiers occurring in a CDF instance $\cO$.
Then $I'$ is reachable from $I$ in $\cO$ if $I = I'$ or if $isa(I',I)$
is in $\cO$ or if $isa(I',I_n)$ is in $\cO$ and $I_n$ is reachable
from $I$.  If either $I$ is reachable from $I'$ or $I'$ is reachable
from $I$, $I$ and $I'$ are termed {\em connected}.

An identifier $I$ {\em locally conforms to} an identifier $I'$ if
\begin{itemize}
\item $I'$ is an atomic identifier; or
\item The argument of $I'$ is $f(I'_1,...,I'_n)$; the argument of  $I$
is $f(I_1,...,I_n)$ and $I_i$ {\em locally conforms to} $I'_i$, for $1
\leq i \leq n$.
\end{itemize}
$I$ and $I'$ are {\em conformable} if $I$ locally conforms to all
identifiers from which $I'$ is reachable in $\cO$, and $I'$ locally
conforms to identifiers from which $I$ is reachable in $\cO$.  An
identifier $I$ is self-conforming if it is conformable with itself.
\end{definition}
%---------------------------------------------------------

As will be shown, acceptability is necessary for a CDF instance to
have a finite free model.

%---------------------------------------------------------
\begin{definition} \label{def:acceptable}
A CDF instance $\cO$ is {\em acceptable} if
\begin{enumerate}
\item Each class or object identifier $I$ in $\cO$ is self-conforming;

%\item There is no product idenifier $C_P$ and identifier $C$ such that
%there is a path from $C$ to $C_P$ in $\cO$ and $C$ occurs in $C_P$.

\item Let $allAttr(I_1,R_1,I_3)$ or $maxAttr(I_1,R_1,I_3) \in \cO$,
and $R_2$ reachable from $R_1$.  Then
\begin{enumerate}
\item if $I_1$ and $I_2$ are connected, and $hasAttr(I_2,R_2,I_4),
allAttr(I_2,R_2,I_4),$ $minAttr(I_2,R_2,I_4,N)$ or
$maxAttr(I_2,R_2,I_4,N) \in \cO$, then $I_3$ and $I_4$ are
conformable.
%
\item if $classHasAttr(I_1,R_2,I_4) \in \cO$, then $I_3$ and $I_4$ are
conformable.
\end{enumerate}
\end{enumerate}
\end{definition}

%---------------------------------------------------------
\begin{theorem} \label{thm:acceptable}
Let $\cO$ be a finite well-typed CDF instance.  Then $\cO$ is
acceptable and has a model iff $\cO$ has a finite free model.
\end{theorem}
\begin{proof}
The proof is contained in the appendix.
\end{proof}

\begin{corollary} \label{thm:atomic}
Let $\cO$ be a finite CDF instance in which no product identifiers
occur.  Then $\cO$ has a finite free model iff $\cO$ has a model.
\end{corollary}

\begin{theorem} \label{thm:poly}
Let $\cO$ be a finite CDF instance.  Then it can be determined in
polynomial time whether there exists a finite free model for $\cO$.
\end{theorem}


\documentclass[12pt,letterpaper]{article}

% to-do list
% ----------

% style notes
% -----------
% - This file generates by Makefile; don't be typing ``pdflatex'' or some
%   bullshit.
% - Line break between sentences to make the git diffs readable.
% - Use \, as a multiply operator.
% - Reserve () for function arguments; use [] or {} for outer shit.
% - Use \sectionname not Section, \figname not Figure, \documentname not Article
%   or Paper or paper.

\include{preamble}
\include{gitstuff}

% packages
\graphicspath{{figures/}}

\newcommand{\gaia}{\project{Gaia}}
\newcommand{\tgas}{\acronym{TGAS}}

\begin{document}\sloppy\sloppypar\raggedbottom\frenchspacing % trust Hogg

\title{OMG who cares if they are bound}

\author{
  Adrian M. Price-Whelan
}

\maketitle

\section{The model}
\label{sec:model}

The \tgas\ catalog provides measurements of sky position, $(\alpha, \delta)$,
parallax, $\pi$, and proper motions, $(\mu_{\alpha^*}, \mu_\delta)$, along with
the covariance matrix, $\tilde{\mat{C}}$, associated with these data for each
star in the pair.
Each star additionally has a precise radial-velocity measurement, $v_r$, with
uncertainty $\sigma_{v_r}$ derive from high-resolution, high signal-to-noise
spectroscopy.
For brevity in defining probability distributions below, we pack these 6D
kinematic data into vectors
\begin{eqnarray}
  \vec{y} &=&
      \transpose{\left(
        \begin{array}{c c c c c c}
          \alpha &
          \delta &
          \pi &
          \mu_{\alpha^*} &
          \mu_\delta &
          v_r
        \end{array}
      \right)}
\end{eqnarray}
and define a new $6\times6$ covariance matrix $\mat{C}$ for each star that
contains the astrometric covariance matrix in the upper-left block, the
radial-velocity variance $\sigma_{v_r}^2$ in the bottom-right diagonal element,
and all other matrix elements set to zero.
We additionally have measurements of the masses of the two stars, $\hat{M}_1$
and $\hat{M}_1$ with uncertainties $\sigma_{M_1}$ and $\sigma_{M_2}$,
respectively.

Our model parameters include, amongst others, the true 6D Cartesian phase-space
coordinates for each star, $\vec{w}_1=(\vec{x}_1, \vec{v}_1)$ and
$\vec{w}_2=(\vec{x}_2, \vec{v}_2)$.
These vectors $\vec{w}$ can be transformed to the data-space---heliocentric
spherical coordinates, where $\vec{w}$ becomes $\vec{w}^*$---where the
likelihood evaluation for the kinematic data is a multivariate Gaussian:
\begin{eqnarray}
    p(\vec{y}_1, \vec{y}_2 \given \vec{w}_1, \vec{w}_2, \mat{C}_1, \mat{C}_2)
        &=&
        \norm\left(\vec{y}_1 \given \vec{w}_1^*, \mat{C}_1 \right) \,
        \norm\left(\vec{y}_2 \given \vec{w}_2^*, \mat{C}_2 \right)
\end{eqnarray}
with
\begin{eqnarray}
    \norm\left(\vec{x} \given \vec{\mu}, \mat{\Sigma}\right) &=&
        \left[\det\left(\frac{\mat{\Sigma}^{-1}}{2\pi}\right)\right]^{1/2} \,
          \exp \left[ -\frac{1}{2} \transpose{\left(\vec{x} - \vec{\mu} \right)} \,
          \mat{\Sigma}^{-1} \,
          \left(\vec{x} - \vec{\mu} \right) \right] \quad .
\end{eqnarray}
The likelihood for the mass measurements given the true masses $M_1$ and $M_2$
are also Gaussian:
\begin{eqnarray}
    p(\hat{M}_1 \given M_1, \sigma_{M_1}) &=&
        \norm\left(\hat{M}_1 \given M_1, \sigma_{M_1}^2\right) \\
    p(\hat{M}_2 \given M_2, \sigma_{M_2}) &=&
        \norm\left(\hat{M}_2 \given M_2, \sigma_{M_2}^2\right)
\end{eqnarray}

Our prior over the true 6D phase-space coordinates for each star, $p(\vec{w}_1,
\vec{w}_2)$, contains three components corresponding to the three scenarios:
\begin{enumerate}
    \item The stars are a bound, wide-binary system on a circular orbit;
    \item The stars are co-moving within some tolerance (velocity dispersion)
    but unbound;
    \item The stars are independently sampled from the field population.
\end{enumerate}
In case 1 (wide-binary), the prior can be expressed in terms of the 6D
barycentric phase-space coordinates of the pair, $\vec{w}_{\rm bary}$, and the
difference in 6D coordinates, $\Delta\vec{w}$, and is conditional on the true
masses of the stars and a (\emph{hack}) softening velocity dispersion, $V_1$,
introduced to approximately account for eccentricity:
\begin{eqnarray}
    p_1(\vec{w}_1, \vec{w}_2 \given M_1, M_2) &=&
        p(\vec{x}_{\rm bary} \given M_1, M_2) \,
        p(\Delta\vec{x} \given M_1, M_2) \, \\
    && \times \,
        p(\vec{v}_{\rm bary} \given M_1, M_2) \,
        p(\Delta\vec{v} \given M_1, M_2, V_1) \\
    p(\vec{x}_{\rm bary} \given M_1, M_2) &=& \mathcal{U}(-1,1)~[{\rm kpc}] \\
    p(\Delta\vec{x} \given M_1, M_2) &\propto& \left|\Delta\vec{x}\right|^{-3} \\
    p(\vec{v}_{\rm bary} \given M_1, M_2) &=&
      \mathcal{N}(\vec{v}_{\rm bary} \given 0, V_3) \\
    p(\Delta\vec{v} \given M_1, M_2, V_1) &=&
      \mathcal{N}\left(\Delta\vec{v} \given
        2\,\left[G \, (M_1+M_2)\right]/\left|\Delta\vec{x}\right|, V_1\right) \\
\end{eqnarray}
The peculiarity here is that for this component, the prior is only over the
magnitude of the coordinate separation and the magnitude of the velocity
difference (2 of 6 dimensions), the other 4 dimensions correspond to angles,
which we can assume to be isotropic.

Cases 2 and 3 are simple to write down.

%\bibliography{ref}

\end{document}


\section{Implementation}
\label{sec:implementation}

In this section, we describe the implementation of the tool.
The SAT-encoding we use is similar to the one introduced in \newcite{listenmaa_claessen2015}, with one key difference: in this paper, we operate on {\em symbolic sentences} instead of concrete sentences from a corpus. The idea is that the SAT-solver is going to find the concrete sentence for us.

%\paragraph
\subsection{Preliminaries}

Our analysis operates on a rule $r$, which is preceded by a list of rules $R$, and is concerned with answering the following question: ``Does there exist an input sentence $S$ that can trigger rule $r$, even after passing all rules $R$ that came before $r$?''

Before we can do any analysis any of the rules, we need to find out what the set of all possible readings of a word is. We can do this by extracting this information from a lexicon, but there are other ways too. In our experiments, the number of readings has ranged from about 300 to about 9000.

Furthermore, when we analyse a rule $r$, we need to decide the {\em width} $w(r)$ of the rule $r$: How many different words should there be in a sentence that can trigger $r$? Most often, $w(r)$ can be easily determined by looking at how far away the rule context indexes in the sentence relative to the target. For example, in the rule mentioned in the introduction, the width is 2.

If the context contains a \verb!*! (context word can be anywhere),
we may need to make an approximation of $w(r)$, which may result in false positives or negatives later on in the analysis.
%we create sentences that are up to 3 words wider than the context without \verb!*!, and try them all.

%\paragraph
\subsection{Symbolic sentences}

We start each analysis by creating a so-called {\em symbolic sentence}, which is our representation of the sentence $S$ we are looking for. A symbolic sentence is a sequence of {\em symbolic words}; a symbolic word is a table of all possible readings that a word can have, where each reading is paired up with a SAT-variable.

The number of words in the symbolic sentence we create when we analyse a rule $r$ is $w(r)$. For the rule in the introduction, we have $w(r)=2$ and a symbolic sentence may look as follows:
\begin{center}
\begin{tabular}{c|c|c}
word1 & word2 & reading \\
\hline
$v_1$ & $w_1$ & det def \\
$v_2$ & $w_2$ & noun sg \\
$v_3$ & $w_3$ & noun pl \\
$v_4$ & $w_4$ & verb sg \\
$v_5$ & $w_5$ & verb pl \\
\end{tabular}
\end{center}
Here, $v_i$ and $w_j$ are SAT-variables belonging to word1 and word2, respectively. We can also see that the possible number of readings here was 5.

The SAT-solver contains extra constraints about the variables. Input sentences should have at least one reading per word, so we add the following two constraints:
\begin{center}
\begin{tabular}{c}
$v_1 \vee v_2 \vee v_3 \vee v_4 \vee v_5$, \\
$w_1 \vee w_2 \vee w_3 \vee w_4 \vee w_5$ \\
\end{tabular}
\end{center}
Any solution to the constraints found by the SAT-solver can be interpreted as a concrete sentence with $w(r)$ words that each have a set of readings.

%\paragraph
\subsection{Applying a rule}

Next, we need to be able to apply any given rule $r'$ to a symbolic sentence, resulting in a new symbolic sentence.

For example, if we apply the rule from the introduction to the symbolic sentence above, the result is the following symbolic sentence:
\begin{center}
\begin{tabular}{c|c|c}
word1 & word2 & reading \\
\hline
$v_1$ & $w_1$ & det def \\
$v_2$ & $w_2$ & noun sg \\
$v_3$ & $w_3$ & noun pl \\
$v_4$ & $w_4'$ & verb sg \\
$v_5$ & $w_5'$ & verb pl \\
\end{tabular}
\end{center}
The example rule can only affect readings of word2 that have a ``verb'' tag, so we create only two new variables $w_4'$ and $w_5'$ for the result, and reuse the other variables. We must also add the following constraint for $w_4'$:
\begin{center}
\begin{tabular}{c}
$w_4' \Leftrightarrow [ w_4 \wedge \neg{}(v_1 \wedge (w_1 \vee w_2 \vee w_3)) ]$ \\
\end{tabular}
\end{center}
In other words, after applying the rule, the reading ``verb sg'' (represented by the variable $w_4'$) can only be in the resulting sentence exactly when (1) ``verb sg'' was a reading of the input sentence (so $w_4$ is true) and (2) the rule has not been triggered (the rule triggers when $v_1$ is true and at least one of the non-verb readings $w_1 \dots w_3$ is true). We add a similar constraint for the new variable $w_5'$:
\begin{center}
\begin{tabular}{c}
$w_5' \Leftrightarrow [ w_5 \wedge \neg{}(v_1 \wedge (w_1 \vee w_2 \vee w_3)) ]$ \\
\end{tabular}
\end{center}

%\paragraph
\subsection{Putting it all together}

Once we know how to apply any rule $r'$ to a symbolic sentence, resulting in a new symbolic sentence, we can apply all rules preceding the rule $r$ that is under analysis. We simply apply each rule to the result of applying the previous rule. In this way, we end up with a symbolic sentence that represents all sentences that could be the result of applying all those rules.

Finally, we can take a look at the rule $r$ we want to analyse. Here is an example:
\begin{itemize}
\item[] \texttt{REMOVE det IF (1 verb) ;}
\end{itemize}
If we take the symbolic sentence above as input, we want to ask whether or not it can trigger the rule $r$. We do this by adding some more constraints to the SAT-solver.

First, the context of the rule should be applicable, meaning that the second word should have a reading with a ``verb'' tag:
\begin{center}
\begin{tabular}{c}
$w_4' \vee w_5'$
\end{tabular}
\end{center}
Second, the rule should be able to remove the ``det'' tag, meaning that the first word should have a reading with a ``det'' tag, and there should be at least one other reading:
\begin{center}
\begin{tabular}{c}
$v_1 \wedge (v_2 \vee v_3 \vee v_4 \vee v_5)$
\end{tabular}
\end{center}
If the SAT-solver can find a solution to all constraints generated so far, we have found a concrete sentence that satisfies our goal. If the SAT-solver cannot find a solution, it means that there are no sentences that can ever trigger rule $r$. (This means that there is something wrong with the grammar.)

%\paragraph{Creating realistic readings}
\subsection{Creating realistic readings}
\label{sec:realistic_readings}

Earlier we have shown an example with 5 readings (``det def'', ``noun sg'', ...). In a realistic case, we operate between hundreds and thousands of possible readings.
%This is very much dependent on language: the simplest language we tested was Dutch, with 336 readings. The most complex was
In order to find the set of readings, we expand a morphological lexicon\footnote{We used the lexica from Apertium, found in \url{https://svn.code.sf.net/p/apertium/svn/languages/}.}, ignore the word forms and lemmas, and take all distinct analyses.
However, many grammar rules target a specific lemma or word form.
A simple solution is to retain the lemmas and word forms only for those entries where it is specified in the grammar, and otherwise leave them out. For example, the Dutch grammar contains the following rule:

\begin{itemize}
 \item[] \texttt{REMOVE ("zijn" vbser) IF (-1 Prep) (1 Noun) ;}
\end{itemize}

This hints that there is something special about the verb \emph{zijn}, compared to the other verbs. Looking at the lexicon, we find \emph{zijn} in the following entries:

\begin{itemize}
 \item[]
\begin{verbatim}zijn:zijn<det><pos><mfn><pl>
zijn:zijn<det><pos><mfn><sg>
zijn:zijn<vbser><inf>
zijn:zijn<vbser><pres><pl>
\end{verbatim}
\end{itemize}

Thus we add special entries for these: in addition to the anonymous
``det pos mfn pl'' reading, we add ``\emph{zijn} det pos mfn pl''.
The lemma is treated as just another tag.

 However, for languages with more readings, this may not be feasible. For instance, Spanish has a high number of readings, not only because of many inflectional forms, but because it is possible to add 1--2 clitics to the verb forms.
The number of verb readings without clitics is 213, and with clitics 1572.
With the previously mentioned approach, we would have to duplicate 1572 entries for each verb lemma. Even ignoring the clitics, each verb lemma still adds 213 new readings.

The readings in a grammar can be underspecified: for example, the rule
\texttt{REMOVE (verb sg) IF (-1 det)} gives us ``verb sg'' and ``det''.
In contrast, the lexicon only gives us fully specified readings, such
as ``verb pres p2 sg''. We implemented a version where we took
the tag combinations specified in the grammar directly as our
readings, and we could insert them into the symbolic sentences as well.
The shortcut works most of the time, but if we only take the readings
from the grammar and ignore the lexicon, it is possible to
miss some cases: e.g. the rule \texttt{SELECT Pron + Rel IF (0 Nom)}
may require ``pron rel nom'' in one reading, but this method only gives
``pron rel'' and ``nom'' separately.

In addition, we found that the tag lists in the grammars sometimes
contain errors, such as using a nonexistent tag or using a wrong level
in a subreading. If we accept those lists as readings, we will
generate symbolic sentences that are impossible, and not discover
the bug in the grammar.
However, if we are primarily interested in rule interaction, then using
the underspecified readings from the grammar may be an adequate solution.

%In fact, it may even catch interaction conflicts between rules which have an internal conflict

\subsection{Creating realistic ambiguities}

In the previous section, we have created realistic \emph{readings}, by simply hardcoding legal tag combinations into variables. The next step in creating realistic \emph{ambiguities} is to constrain which readings can go together. For instance, the case of \emph{zijn} shows us that ``determiner or verb'' is a possible ambiguity. In contrast, there is no word form in the lexicon that would be ambiguous between an adjective and a comma, hence we do not want to generate such ambiguity in our symbolic sentences.

\begin{center}
\begin{tabular}{c|c|c|c|c}

            & n nt sg  & n f pl  & vblex sep inf & det pos mfn  \\ \hline
uitgaven    & 0        & 1       & 1             & 0    \\
toespraken  & 0        & 1       & 1             & 0    \\
haar        & 1        & 0       & 0             & 1    \\

\end{tabular}
\end{center}

We solve the problem by creating \emph{ambiguity classes}: groups of readings that can be ambiguous with each other.
We represent the expanded morphological lexicon as a matrix, as seen
above: word forms on the rows and analyses on the columns. Each
distinct row forms an ambiguity class. For example, one class may
contain words that are ambiguous between plural feminine nouns and
separable verb infinitives; another contains masculine plural adjectives
and masculine plural past participles.
Then we form SAT-clauses that allow or prohibit certain combinations. These clauses will interact with the constraints created from the rules, and the end result will be closer to real-life sentences.

Our approach is similar to \newcite{cutting_etal92}, who use ambiguity classes instead of distinct word forms, in order to reduce the number of parameters in a Hidden Markov Model. They take advantage of the fact that they don't have to model ``bear'' and ``wish'' as separate entries, but they can just reduce it to ``word that can be ambiguous between noun and verb'', and use that as a parameter in their HMM.
%We can do a similar thing by saving a list of words with each ambiguity class. For example, we map the ambiguity class ``feminine plural noun or a separable verb infinitive'' to the list of word forms \{``uitgaven'', ``toespraken''\}, and then, if we generate such reading for our symbolic word, we can give one of these words as an example word.

There are two advantages of restricting the ambiguity within words.
Firstly, we can create more realistic example sentences, which should help the grammar writer.
Secondly, we can possibly detect some more conflicts. Assume that the grammar contains the following rules:

 \begin{itemize}
 \item[]
\begin{verbatim}
 REMOVE adj IF (-1 aux) ;
 REMOVE pp  IF (-1 aux) ;
 \end{verbatim}
 \end{itemize}

 With our symbolic sentence, these rules will be no problem; to apply the latter, we only need to construct a target that has a realistic ambiguity with a past participle; the adjective will be gone already.
However, it could be that past participles (pp) only ever get confused with adjectives---in that case, the above rules would contradict each other.
 By removing the adjective reading, the first rule selects the past participle reading, making it an instance of ``$r$ selects something in a context, $r'$ removes~it''.
The additional constraints will prevent the SAT-solver from creating an ambiguity outside the allowed classes, and such a case would be caught as a conflict.

% \paragraph{Other questions we can ask}

% Apart from finding out what rules prevent others from applying, we can also find out if there is a conflict that is rule-internal, such as nonexisting tag set or contradicting requirements of a context word (e.g. a word must be unambiguously two different POS).

% In addition to analysing a whole grammar,
% we can construct all kinds of symbolic sentences to test out individual rules.
% We can set the length, restrict individual words (e.g. ``3rd word must be a noun''), require that it triggers some rule but not other.
% This functionality can aid the grammar writer in the process, to see if they have missed a case or defined the tagset correctly.


\newcommand{\coreivory}{Core Ivory}

\section{Ivory Semantics}
\label{sec:semantics}

In the previous section, we described our embedding of Ivory into the
GHC type system and made the claim that this guarantees memory safety.
We modeled a simplified version of the Ivory language inside
Isabelle/HOL\cite{isabelle}, henceforth \emph{\coreivory{}}, to support
this claim.  In this section we present a semantics based upon the
Isabelle/HOL development, and outline the proof of type safety.

Developing this model provides a number of benefits for a modest
investment---we developed the model in under a person month, albeit
one of the authors has significant experience with Isabelle.  In
addition to the basic benefits formalisation provides, we can
experiment with extensions to Ivory.

In one such experiment, we extended the model to allow references in
the heap, a feature we avoided in the development of Ivory due to
soundness concerns.  While a simple extension to the syntax and
semantics of Ivory, the effort involved in extending the soundness
proofs was almost as much as developing the initial model.

Due to space constraints we discuss only those particulars of
\coreivory{} which differ significantly from a standard imperative
\trevor{Remember to fix this reference in the final version}
language; see the supplemental material for more details.

\newcommand{\sep}{\ |\ }
\newcommand{\syntaxtitle}[1]{\multicolumn{3}{l}{\textit{#1}}}

\begin{figure}[t]

\[
\begin{array}{crl}
\syntaxtitle{pure expressions}\\
e & ::= & 0 \sep{} 1 \sep{} \ldots{} \sep{} \texttt{true} \sep{} \texttt{false} \sep{} () \sep{} x \sep{}
          e_1 \mathbin{\mathit{op}} e_2 \\
\syntaxtitle{impure expressions}\\
i & ::= & \texttt{pure}(e) \sep{} \texttt{alloc}(e) \sep{}
          \texttt{read}(e) \sep{} \texttt{write}(e_1, e_2)\\
\syntaxtitle{statements}\\
s & ::= & \texttt{skip} \sep{} \texttt{return}(e) \sep{} s_1; s_2 \\
  & |   & \texttt{if}(e)\texttt{\;then\;} s_1 \texttt{\;else\;} s_2 \\
  & |   & \texttt{for}(x = e_1; e_2; e_3) \{ s \} \\
  & |   & \texttt{let\;} x = i \texttt{\;in\;} s \\
  & |   & \texttt{let\;} x = f(e_1, \ldots{}, e_n) \texttt{\;in\;} s \\
\syntaxtitle{values}\\
v & ::= & 0 \sep{} 1 \sep{} \ldots{} \sep{} \texttt{true} \sep{} \texttt{false} \sep{} () \\
w & ::= & \texttt{stored}(v) \sep{} \texttt{ref}(r, n) \\
\syntaxtitle{Stores}\\
E & \in{} & x \to w \\
\syntaxtitle{regions and heaps}\\
R & \in{} & \mathbb{N} \to w\\
H & ::=   & H, R \sep{} \emptyset{}\\
\syntaxtitle{stacks}\\
F & ::= & \texttt{rframe}(x, E, s) \sep{} \texttt{sframe}(E, s) \\
S & ::= & F, S \sep{} \emptyset{} \\
\syntaxtitle{configurations}\\
C & \in & H \times S \times E \times s \sep{} \texttt{finished}(v) \\
\syntaxtitle{types}\\
\rho   & \in & \textit{region variables}\\
\alpha & ::= & \texttt{nat} \sep{} \texttt{bool} \sep{} \texttt{unit} \\
\tau   & ::= & \texttt{storedt}(\alpha) \sep{} \texttt{reft}(\rho, \alpha) \\
\syntaxtitle{procedure definitions}\\
P & ::= & \texttt{proc\;} f(\tau_1\;x_1, \ldots{}, \tau_n\;x_n) : \tau\; \{ s \}
\end{array}
\]

\sjw{values (in or out?), $\in$ or $\subseteq$?}

\caption{Concrete syntax of \coreivory{}}
\label{fig:syntax}
\end{figure}

\subsection{Syntax}

The syntax for \coreivory{} is given in \autoref{fig:syntax}.
\coreivory{} is based upon a typical typed imperative language with
function calls, references, and memory allocation (but not memory
deallocation).  \coreivory{} attempts to stay faithful to Ivory
wherever possible, and so variables are let-bound with forms for
binding the result of expression evaluation and function
calls. Furthermore, \coreivory{} expressions are stratified into
\emph{pure} and \emph{impure}, the latter allowing operations on the
heap: allocation, reading, and writing references.

Ivory uses regions to manage memory.  Thus, the heap is modeled as a
list of \emph{regions}, each region a finite map from \emph{offsets},
modeled as natural numbers, to \emph{stored values}; Ivory does not
allow references in heap allocated values, and so a stored value is
any value which is not a reference. A \emph{reference} contains both a
\emph{region index} into the list of regions, and an offset with the
region.  To simplify the presentation, we will use $H(r, n)$ to denote
the value at offset $n$ in the $r$th region of $H$, and similarly with
updates.

As with values, types classifying values in \coreivory{} are
stratified into storable and reference types; a reference type
$\texttt{reft}(\rho, \alpha)$ is a reference to an object of type
$\alpha$ in region $\rho$, where $\alpha$ is not a reference.

\sjw{talk about structs and arrays?}

\subsection{Operational Semantics}

% These are here so we can talk about them in the text, but they are
% mainly used in the appendix

\newcommand{\stepsX}[2]{\models #1 \longmapsto{} #2}
\newcommand{\stepsXX}[3]{\models #2 \longmapsto^{#1} #3}
\newcommand{\steps}[4]{\stepsX{#1; #2}{#3; #4}}

\newcommand{\denoteexp}[2]{\llbracket{}#1\rrbracket{}#2}

\newcommand{\stepsH}[2]{#1 \longmapsto_I #2}
\newcommand{\hsteps}[5]{#1 \models \stepsH{#2; #3}{#4; #5}}

% We ignore the procedure environment for clarity
\newcommand{\wfstmt}[5]{#1; #3 \vdash_s #4 : #5}
\newcommand{\wfexp}[3]{#1 \vdash_e #2 : #3}
\newcommand{\wfimp}[4]{#1; #2 \vdash_i #3 : #4}

\newcommand{\rulestitle}[1]{\textnormal{#1}\hspace*{\fill}}

\coreivory{}'s semantics are modeled as an abstract machine
over configurations.  The judgement
\[
\stepsX{C}{C'}
\]
states that configuration $C$ transitions to configuration $C'$.  A
configuration consists of a heap, a stack, a store, and the current
statement. The stack contains continuations for both function calls
and statement sequences while the store maps variables to values. The
semantics of sequencing is slightly non-standard as variables are
let-bound rather than assigned, and so statement sequencing preserves
the store across execution of the first statement.

For example, the semantics of dereferencing is given by the rule
\begin{mathpar}
\inferrule{ \denoteexp{e}{E} = \texttt{ref}(r, n)  \\ (r, n) \in \texttt{dom}(H) }%
          {\steps{(H; S; E)}{\texttt{let\;} x = \texttt{read}(e) \texttt{\;in\;} s}{(H; S; E[x \mapsto{} w])}{s}}
\end{mathpar}
where \( \denoteexp{e}{E} \)
evaluates the pure expression $e$ under the store $E$.  The premise
\( (r, n) \in \texttt{dom}(H) \)
requires the existence of the region and offset pointed to by the
reference resulting from evaluating $e$.

Operationally, the heap is extended on a function call, an empty
region being added to the end of the list, and shrunk on function
return, removing the last region.  Allocating an object extends the
current (last) region.

A configuration is stuck if there is no available transition.  For
instance, an attempted heap access or update where the region index
does not exist or at an offset which has not been allocated will
result in a stuck configuration.  In particular, accessing a region
after it has been removed will result in a stuck state.

\subsection{Typing Ivory}

\coreivory{}'s typing judgements extend standard statement typing
judgements with a current region variable.  The typing judgement
\[
\wfstmt{\Gamma}{\Psi}{\rho}{s}{\tau}
\]
holds when the statement $s$ is well-formed under the store
environment $\Gamma$, current region $\rho$, with any return
statements returning values of type $\tau$.

The region variable $\rho$ represents the current region and is used
when checking memory allocation.  The typing rule for allocation is then
\[
\inferrule{
\wfexp{\Gamma}{e}{\alpha} \\
\wfstmt{\Gamma[x \mapsto \texttt{reft}(\rho, \alpha)]}{\Psi}{\rho}{s}{\tau} }%
{ \wfstmt{\Gamma}{\Psi}{\rho}{\texttt{let\;} x = \texttt{alloc}(e) \texttt{\;in\;} s}{\tau} }
\]
where the body of the let statement is checked under the additional
assumption that the variable $x$ has reference type
$\texttt{reft}(\rho, \alpha)$, noting the region variable on the
reference type comes from the current region variable.  The judgement
\( \wfexp{\Gamma}{e}{\alpha} \) holds when pure expression $e$ has type $\alpha$.

We fix the set of procedures as \texttt{Procs}. The typing rule for procedure bodies
\begin{mathpar}
\inferrule{\forall\;\texttt{proc\;} f(\tau_1\;x_1, \ldots, \tau_n\;x_n) : \tau\;\{ \mathit{body} \} \in \texttt{Procs}\\\\
            \rho \textrm{ fresh} \\
            \texttt{frees}(\tau) \subseteq \texttt{frees}(\tau_1) \cup \ldots \cup \texttt{frees}(\tau_n) \\
            \wfstmt{[x_1 \mapsto \tau_1, \ldots, x_n \mapsto \tau_n]}{\Psi}{\rho}{s}{\tau} }%
{ \vdash \texttt{Procs} }
\end{mathpar}
ensures that this region variable
is fresh; this constraint, together with the constraint that region
variables in the procedure's return type must occur in an argument
type, ensures that references cannot escape the scope in which they
were allocated.  These constraints are then fundamental to the type
safety of \coreivory{} programs.

\subsection{Type Safety}

We prove type safety, that is, well-typed programs do not get stuck,
by proving the usual progress and preservation lemmas.  As is
common  with type safety for imperative languages, we
define auxiliary well-formedness invariants on configurations.  The
progress lemma then states that well-formed configurations are not
stuck, and preservation states that well-formed configurations
transition to well-formed configurations.

\begin{theorem}[Type Safety]
Given \( \vdash \texttt{Procs} \) and
\[
\texttt{proc\;} \texttt{main}() : \texttt{nat}\;\{ \mathit{s} \} \in \texttt{Procs}\\
\]
there either exists some number of steps $n$ and value $v \in \mathbb{N}$ such that
\[
\stepsXX{n}{(\emptyset; \emptyset; \emptyset); \texttt{let\;} x = \texttt{main}() \texttt{\;in\;} \texttt{return}(x)}{\texttt{finished}(v)}
\]
or, for all $n$ there is a well-formed configuration $C$ such that
\[
\stepsXX{n}{(\emptyset; \emptyset; \emptyset); \texttt{let\;} x = \texttt{main}() \texttt{\;in\;} \texttt{return}(x)}{C}
\]
\end{theorem}
Informally, a well-formed program, when called via the \texttt{main}
procedure, will either terminate in a finite number of steps, or will
diverge through well-formed configurations.

The well-formedness invariants are typical, and are based upon a
well-formed value judgement.  For our purposes, the interesting rule
here is for references
\[
\inferrule{\Delta(\rho) = r \\ \Theta(r, n) = \alpha }%
{\Theta; \Delta \vdash \texttt{ref}(r, n) : \texttt{reft}(\rho, \alpha)}
\]
which links reference types to reference values through the region
environment $\Delta$, mapping region variables to region indices, and
heap type $\Theta$, mapping region indices and offsets to types.
Well-formed heaps and stores then follow point-wise from this
judgement.  Well-formedness of stacks follows, ensuring that each
continuation on the stack is well-formed.

A well-formed configuration, in addition to the well-formedness of the
heap, stack, store, and current statement, constrains
the region environment ($\Delta$).  In particular, the variable
representing the current region must be mapped to the length of the
current heap, every region index in the range of $\Delta$ must be
below this length, and the type variables occurring in the store
environment $\Gamma$ must be mapped by $\Delta$.

The progress lemma follows from well-formedness.  The preservation
proof, as usual, is the trickier of the two proofs.  In particular,
the case for return involves showing that the various configuration
members are well-formed under a heap where the last region has been
removed from the heap.  This involves showing that references are
well-formed under this smaller heap which follows from the stack and
region environment well-formedness invariants.  The case for function
calls is also involved, although this is primarily due to the
instantiation of the type variables in the type of the called
function.

\subsection{Discussion}
\label{sec:sem-discuss}

The type safety proofs are greatly simplified by the restriction of
heap values to non-references: it is trivially true, for example, that
the heap is well-formed after a return as the well-formedness of
non-reference values does not depend on the type of other heap
elements.

We extended the Isabelle model to remove the stratification of values,
allowing references to appear in the heap.  Syntactically, this
simplifies the language, at the expense of more complicated proofs.
For example, we extend the well-formedness of heaps to require that
heaps are downward closed with respect to region references: that is,
references in the heap refer only to regions up to and including the
containing region.  This work extended the proof development from
approximately 2500 lines of proof to 3300 lines of proof, and took
approximately 2 person weeks to finish.

\sjw{move to end?}
Developing this model uncovered a bug in the Ivory embedding into
Haskell, namely that the Haskell erroneously allowed functions to
end without encountering a return statement.  Thus, a program could
claim to return, say, a valid index but in reality return an
out-of-bounds index, breaking memory safety.  In the model this
check is part of the type checking rules (although elided for clarity
in this paper) but is implemented as an explicit type-check over the Ivory AST
in the
Haskell implementation.

The existence of the formal model is not a guarantee that the Ivory
implementation is sound, however: there is no formal link between the
implementation and the model.  For example, we discovered a number of
bugs due to the way in which references are initialized: these bugs
are impossible in the model, but allowed in the implementation.

In addition, \coreivory{} covers a subset of the Ivory language,
currently missing data structures and arrays.  We are working to
reduce this gap; furthermore, in future, we plan to investigate making
the core of the implementation more closely resemble \coreivory{}.
Doing so would have the additional benefit of allowing us to verify
the correctness of operations performed over the core AST.

We plan on investigating first-class regions as a further extension to
the model.  This feature would allow programmers to name and pass
around allocation contexts, allowing helper functions to, for example,
allocate and return objects in a parent functions region.  While we
are confident that this extension is sound, \emph{proving} it sound
allows a much greater confidence in the implementation: memory
deallocation is notoriously easy to get wrong, so having a formal
proof of soundness would be greatly comforting.


\section{Modeling the macromolecular system}

Consider a biomacromolecule in a solvated environment under typical biological or experimental conditions.
Our interest lies in the conformational dynamics of this macromolecule.
We presume the system can be treated entirely within the realm of classical statistical mechanics for the following reasons:
\begin{enumerate}
  \item Because atomic nuclei are at least three orders of magnitude more massive than the electrons, the electronic degrees of freedom relax much more rapidly than the nuclei move, and can be considered as always occupying their ground state in the field of the nuclei.
  This is the celebrated Born-Oppenheimer approximation \cite{born:1927a}.
  \item Quantized vibrations and zero-point energy effects are minimal due to the thermal kinetic energies under biological conditions\footnote{Comparison with experiment will occasionally still require correction terms to account for quantum effects that cannot be treated classicaly, such as quantized vibration effects, as in \cite{horn:jcp:2004}}.
  \item In restricting our consideration to conformational dynamics and not chemical reactions, we require that chemical bonds are neither broken nor formed.
  This ensures that we do not need to treat nuclear wavefunctions, as might be the case in enzyme-catalyzed hydrogen transfer reactions \cite{hammes-schiffer:acc-chem-res:2006}.
\end{enumerate}
The interatomic interactions can be completely described by some potential function $V(\q)$ that depends only on the Cartesian coordinates\footnote{We note that this exposition could be extended to allow the use of generalized coordinates and momenta, but this would necessitate the use of a Jacobian correction factor in the resulting phase space densities and partition functions.  Therefore, for simplicity, we leave this extension as an exercise to the reader, should it be necessary.} of the atomic nuclei $\q \in \Re^{3N}$.
This potential is sometimes termed a \emph{forcefield} due to the fact that it is the field whose negative gradient yields the instantaneous interatomic forces: \index{forcefield} \label{notation:force} \index{force}
\begin{eqnarray}
\bfm{F}(\q) &=& - \grad_{\q} V(\q) \label{equation:force-definition}
\end{eqnarray}
The exact form of the potential is not of interest here, but there are several popular \emph{molecular mechanics forcefields} in common use today, such as AMBER \cite{AMBER-parm94,AMBER-parm96,AMBER-gaff,duan:2003a}, CHARMM \cite{mackerell:jpcb:1998}, and GROMOS \cite{oostenbrink:j-comput-chem:2004}. \index{forcefield!molecular mechanics}

Although macromolecular crowding is known to have an effect on thermodynamics and conformational dynamics \cite{elcock:pnas:2003:crowding,chebotareva:biochem-moscow:2004:crowding,zhou:j-mol-recognit:2004:crowding,zhou:jbc:2004:crowding,volker:annu-rev-biophys-biomol-struct:2005,minton:j-cell-sci:2006}, and many macromolecules primarily exist in complexes or assemblies \cite{sali:nature:2003}, we restrict our consideration here to a single independent macromolecule\footnote{While the methods presented here may be extended to more complex systems of multiple interacting macromolecules and ligands, we restrict ourselves to consideration of a single macromolecule here, for simplicity.}. \index{crowding}
This situation corresponds to a solution of macromolecules sufficiently dilute such that each can be considered as independent or noninteracting, which may be the case in some, but not all, experiments.
Typically, the macromolecule is surrounded by a large amount of solvent (generally water) under near-atmospheric external pressures.
In biological systems, as well as many experiments, the salt concentration ensures that some mobile counterions will surround the macromolecule\footnote{Standard practice in molecular simulations is to add only the minimal number of individual counterions to bring the system to net neutrality.  This practice may result in an atypical effective salt concentration that is not representative of either \emph{in vitro} or \emph{in vivo} conditions.  As an example, a recent simulation of the trpzip2 hairpin \cite{pitera:2006a} employed a cubic simulation box (48 \AA)$^3$ and a single chloride ion, for an effective salt concentration of 7--15 mM.  The corresponding experiments \cite{yang:2004a} employed a 50 mM potassium phosphate buffer.  A larger system, such as the 80-residue $\lambda_{6-85}$, might require an (88 \AA)$^3$ simulation box, where the addition of a single neutralizing chloride counterion would give an effective salt concentration of 1--2 mM, while experiments employing 50 mM phosphate buffer \cite{gruebele:pnas:2005} would require, on average, 21 pairs of counterions. In this system, addition or subtraction of a pair of neutralizing counterions only changes the effective salt concentration by 2 mM, so we might expect a broad distribution of counterion numbers if we examine comparable volumes in the experimental system.  It should be noted that even this large simulation volume gives an unphysically large protein concentration --- 15 mM for trpzip2 and 2 mM for $\lambda_{6-85}$, while the corresponding experiments employ concentrations of 1--5 $\mu$M \cite{yang:2003a} or 50--360 $\mu$M \cite{gruebele:pnas:2005}.}.
Many macromolecules, especially nucleic acids, carry a net charge, so the nearby presence counterions is almost ensured so as to preserve net macroscopic neutrality.
The environment in biological and experimental contexts is generally buffered to maintain near-constant pH, and the protonation states of titratable groups may change in response to conformational changes, ligand binding, or post-translational modifications, a switching behavior that is relevant to the biological function of several systems \cite{srivastava:2006a}.
Additionally, the system might include the presence of small ligands or cosolvent molecules whose association with the macromolecule may have significant impact on its conformational dynamics.

Here, we only consider Hamiltonian systems \emph{without} holonomic constraints.
In molecular simulations, it is common practice to constrain bond lengths to hydrogen in the macromolecule \cite{SHAKE,RATTLE} or the water model \cite{jorgensen:1983a}.
These fast vibrations otherwise constrain the longest timestep that can be used to achieve stable dynamics to approximately 1 fs at 300 K.
Bond constraints introduce a coupling between the coordinates and momenta, resulting in non-Hamiltonian dynamics with correspondingly more complex statistical mechanical properties\footnote{In particular, the canonical partition function is no longer separable into a product of independent partition functions for coordinates and momenta.  There is also some disagreement about whether molecular dynamics and Metropolis Monte Carlo simulations conducted in Cartesian coordinates require expectations to be estimated by including a weighting term for each snapshot based on the metric tensor factor.  Simulation codes that conduct dynamics in torsion space, however, have employed such a correction for a number of years.  This term can be computed from the determinant of the metric tensor which can be computed by an expression originally derived by Fixman \cite{fixman:1974a} or from the constraint forces in constraint schemes based upon Lagrange multiplier methods \cite{tuckerman:1992a}.  This factor is universally ignored in standard simulation practice, though there is insufficient evidence to suggest whether this is, in fact, reasonable. An early study by Chandler and Berne  demonstrated a significant discrepancy between the uncorrected and Fixman-corrected potential of mean force of the torsion angle of $n$-butane in a simulation where bond lengths and angles were constrained by SHAKE \cite{chandler:jcp:1979}.} \cite{fixman:1974a,tuckerman:2001a,hartmann:2004b}.
We will not consider constrained systems in the theoretical framework developed here\footnote{In subsequent sections, simulation data will be presented where these constraints were employed out of necessity due to the lack of adequate implementations of multiple timestep integrators in common molecular mechanics packages, as well as water models that have been parameterized for fully flexible use.  It is hoped this situation can be rectified in the near future.  If performance is a concern in unconstrained systems, multiple timestep integrators derived over a decade ago \cite{tuckerman:1992a} allow for unconstrained simulations to be run without incurring the large computational penalty otherwise encountered.}.

We define the \emph{Hamiltonian} of the system $H(\z) = H(\q,\p) = T(\q) + V(\p)$ to denote the sum of the kinetic energy $T(\p) = \frac{1}{2} \p^\T \M^{-1} \p$, with $\M$ the diagonal mass matrix, and the potential energy $V(\q)$. \index{Hamiltonian} \label{notation:Hamiltonian}
The value of the Hamiltonian at any phase space point $\z$ therefore gives the \emph{total energy} of the system, $E = T + V$. \index{total energy} \label{notation:total-energy}
In the absence of any thermostat or energy exchange, the system evolves according \emph{Hamilton's equations of motion} which can be expressed succinctly in terms of the Hamiltonian as  \index{Hamilton's equations of motion}
\begin{eqnarray}
\dot{\q} &=& \grad_\p H(\q,\p) \nonumber \\
\dot{\p} &=& - \grad_\q H(\q,\p)
\end{eqnarray}
where the dot denote differentiation with respect to time.

It can easily be seen that Hamilton's equations of motion reduce to an expression of Newton's Second Law:
\begin{eqnarray}
\dot{\q} &=& \grad_\p [T(\p) + V(\q)] = \M^{-1} \p \nonumber \\
\dot{\p} &=& - \grad_\q [T(\p) + V(\q)] = - \grad_\q V(\q) = \bfm{F}(\q)
\end{eqnarray}
Because $\p = \M \bfm{v}$, where $\bfm{v}$ denotes the velocity, and $\dot{\p} = \M \dot{\bfm{v}} = \M \bfm{a}$, where $\bfm{a}$ denotes the acceleration, we recover the familiar equation $\bfm{F} = \M \bfm{a}$. \index{Hamilton's equations of motion} \index{Newton's Second Law}

\section{Equilibrium statistical ensembles}
\index{equilibrium statistical ensembles} \index{ensemble|see{equilibrium statistical ensembles}} \index{equilibrium thermodynamic ensembles|see{equilibrium statistical ensembles}}

The equilibrium distribution of the macromolecule and its environment can be modeled by several statistical mechanical ensembles.
We shall give an overview here.

\subsection{Equivalence of ensembles}
\label{section:equivalence-of-ensembles}
\index{equilibrium statistical ensembles!equivalence}

[General comments on the equivalence of all thermodynamic ensembles in the limit of large systems.]

\subsection{Microcanonical (NVE)}
\label{section:nve}
\index{equilibrium statistical ensembles!NVE} \index{NVE|see{equilibrium statistical ensembles}}

In an isolated, confined system of constant volume that does not exchange energy with its environment, the total system energy $E$ will be constant.
For sufficiently complex systems, the ergodic theorem will hold, and the macromolecule will be able to access all relevant regions of phase space \cite{kinchin:statmech:1949}.

The equilibrium phase space distribution function is given by
\begin{eqnarray}
\rho(\z) &=& [\Omega(E)]^{-1} \, \delta(H(\z) - E) \nonumber \\
\Omega(E) &\equiv& \int d\z \, \delta(H(\z) - E)
\end{eqnarray}
Here, $\Omega(E)$ denotes the \emph{total energy density of states}, which is a measure of the volume of phase space with total energy $E$, and $\delta(x)$ is the Dirac delta function. \index{density of states!energy density of states} \label{notation:total-energy-density-of-states}

The temperature can be determined by
\begin{eqnarray}
\beta^{-1} &=& \left( \frac{\partial \Omega}{\partial E} \right)
\end{eqnarray}

In principle, we should write $\Omega(E,V,N)$, as the density of states also depends on the system volume and number of particles, but we suppress these variables in general unless we consider their variation, as in the calculation of pressure:
\begin{eqnarray}
p &=& \beta \left( \frac{\partial \Omega}{\partial V} \right)
\end{eqnarray}

% JDC: Is this definition of the density of states kosher, as long as we use Cartesian coordinates?  Double-check with Kinchin.

Because the Hamiltonian can be written $H(\q,\p) = T(\p) + V(\q)$, the total energy density of states can be written in terms of the product of individual kinetic and potential energy densities of states $\Omega_T(T)$ and $\Omega_V(V)$, respectively.
Because $T(\p)$ has a simple functional form, we can further write
\begin{eqnarray}
\Omega_T(T) &=& \int d\p \, \delta( T(\p )- T ) \nonumber \\
&=& \int d\p \, \delta(\frac{1}{2} \p^\T \M^{-1} \p - T) \nonumber \\
&=& \left[ \prod_{n=1}^N (2 m_n)^{3/2} \right] \, \int d\x \, \delta(\x^T \x - T) \nonumber \\
&=& \left[ \prod_{n=1}^N (2 m_n)^{3/2} \right] \, S_N(T)
\end{eqnarray}
where we have made use of the transformation of variables $\x = (2 \M)^{-1/2} \p$ to simplify the integral of the delta function of all of Cartesian momenta space to a surface integral over a hypersphere.
Here, $m_n$ denotes the mass of particle $n$.
$S_N(r)$ denotes the surface area of the $N$-sphere of radius $r$, and is given by
\begin{eqnarray}
S_N(r) &=& \frac{2 \pi^{N/2} r^{N-1}}{\Gamma(N/2)}
\end{eqnarray}
where $\Gamma(x)$ is the well-known Gamma function.
Note that the kinetic energy is always non-negative, \emph{i.e.} $T \ge 0$,
Because the potential function $V(\q)$ is often much more complicated, we can generally only write
\begin{eqnarray}
\Omega_V(V) &=& \int d\q \, \delta( V(\q) - V )
\end{eqnarray}
The total energy density of states $\Omega(E)$ can therefore be computed from $\Omega_T(T)$ and $\Omega_V(V)$ by integrating over all combinations of $T$ and $V$ that yield $T + V = E$:
\begin{eqnarray}
\Omega(E) &=& \int_0^E dT \, \Omega_T(T) \, \Omega_V(E - T)
\end{eqnarray}

If the system size is large, and hence contains a large number of solvent molecules, it can be shown that the solvated macromolecule will experience a distribution of kinetic and potential energies and volumes consistent with the temperature and pressure desired.  % JDC: Cite proof of this -- often used as derivation of canonical ensemble.
The effective temperature and pressure the macromolecule experiences will be a complicated function of the total system energy $E$ and volume $V$ cannot generally be easily determined \emph{a priori} and depends on the density of states of the system.
If a big enough box can be afforded, it may be therefore appropriate to model the macromolecule in a solvated bath of constant volume and total system energy, both chosen to reproduce the desired average temperature and pressure by some equilibration process.

%In principle, if the system is sufficiently large, the macromolecule will experience appropriate volume and energy fluctuations.
% How big would this need to be?
% In the limit of infinite number of particles, all ensembles are equivalent to this.
% Subsystem will experience appropriate particle number fluctuations.
% Proof of canonical ensemble distribution of subsystem?

% JDC: How big does the system have to be for the subsystem to experience the appropriate thermal fluctuations?

\subsection{Canonical (NVT)}
\label{section:nvt}
\index{equilibrium statistical ensembles!NVT} \index{NVT|see{equilibrium statistical ensembles}}

Because the systems typically treated in molecular mechanics simulations are finite in extent, the macromolecule may not experience a sufficiently large range of kinetic and potential energy distribution indicative of the canonical ensemble when the entire system is constrained to have constant total energy.
For this reason, the NVT ensemble, where the distribution is given by
\begin{eqnarray}
\rho(\z) &=& [Z(E)]^{-1} \, e^{-\beta H(\bfm{\z})} \nonumber \\
Z(\beta) &\equiv& \int d\z \, e^{-\beta H(\bfm{\z})}
\end{eqnarray}
where the inverse temperature $\beta = (\kB T)^{-1}$, where $T$ is the absolute temperature. \label{notation:canonical-partition-function}

The result is a system in which the temperature is fixed to some value $T$.
The temperature is defined through the expectation of the kinetic energy
\begin{eqnarray}
\frac{3}{2} N \beta^{-1} &=& \expect{ \frac{1}{2} \p^\T \bfm{M} \p }_\beta \label{equation:canonical-temperature}
\end{eqnarray}

Because the phase space weight, or \emph{Boltzmann factor}, $e^{-\beta H(\p,\q)}$ \index{Boltzmann factor} can be decomposed into a product of individual factors factors $e^{-\beta T(\p)}$ and $e^{-\beta V(\p)}$ (something that is not true of non-Hamiltonian systems that employ constraints), we can deco,pose partition function $Z(\beta)$ can be decomposed into a product of the kinetic and potential energy partition functions:
\begin{eqnarray}
Z(\beta) &=& \int d\z \, e^{-\beta H(\bfm{\z})} \nonumber \\
&=& \int d\p \, d\q \, e^{-\beta (T(\bfm{\p}) + U(\bfm{\q}))} \nonumber \\
&=& \left[ \int d\p \, e^{-\beta T(\bfm{\p})} \right] \, \left[ \int d\q \, e^{-\beta U(\bfm{\q}))} \right] \nonumber \\
&=& P(\beta) \, Q(\beta)
\end{eqnarray}
where the kinetic partition function $P(\beta)$ is given by
\begin{eqnarray}
P(\beta) &\equiv& \int d\p \, e^{-\beta T(\bfm{\p})} \nonumber \\
&=& \label{equation:kinetic-energy-partition-function}
\end{eqnarray}
and the potential partition function $Q(\beta)$ by
\begin{eqnarray}
Q(\beta) &\equiv& \int d\q \, e^{-\beta U(\bfm{\q})}
\end{eqnarray}
% JDC: Show analytical form of P(\beta), computed from Gaussian Integral.
Both the kinetic and potential energies are therefore individually canonically distributed\footnote{Because of this decomposition, and because the form of the kinetic energy function $T(\p)$ is known, we can write the probability distribution of kinetic energies $p(T)$ in the canonical ensemble as
\begin{eqnarray}
%p(T) &=& [P(\beta)]^{-1} \, \int d\p \,  \exp[-\frac{1}{2} \p^\T \M^{-1} \p] \, \delta(T - \frac{1}{2} \p^\T \M^{-1} \p) \nonumber \\
%&=& \frac{\left[ \prod_{n=1}^N m_{n}^{3/2} \right] \, \int d\p \,  \exp[-\frac{1}{2} \x^T \x] \, \delta(T - \frac{1}{2} \x^T \x)}{\left[ \prod_{n=1}^N m_{n}^{3/2} \right] \, \int d\p \,  \exp[-\frac{1}{2} \x^T \x]} \nonumber \\
p(T ; \beta) &=& [P(\beta)]^{-1} \, \Omega_T(T) \, e^{-\beta T} \\
&=& \frac{\left[ \prod_{n=1}^N (2 m_n)^{3/2} \right] \, S_N(T) \, e^{-\beta T}}{\left[ \prod_{n=1}^N (2 m_n)^{3/2} \right] \int_0^\infty dT\, S_N(T) \, e^{-\beta T}} \\
&=& (2 \pi)^{-3 N /2} \, S_N(T) \, e^{-\beta T}
\end{eqnarray}
%%% fix here.

Comparison of observed kinetic energy distributions with this distribution function provides a good way to validate thermostat implementations in molecular dynamics simulation codes.}.

The volume, since it is fixed, should be chosen appropriately for the conditions of interest.
Because the volume is fixed, the pressure fluctuations that result, especially if the system is small and the macromolecule experiences a large volume change as a result of some conformational change (such as folding), the pressure fluctuations that result may be artificially larger than the corresponding experimental system.

In the limit of an infinitely large system, the NVT ensemble has identical distribution function to the NVE ensemble.

\subsection{Isothermal-isobaric (NPT)}
\index{equilibrium statistical ensembles!NPT} \index{NPT|see{equilibrium statistical ensembles}}

In order

In the limit of an infinitely large system, the NPT ensemble has identical distribution function to the NVE ensemble.

\subsection{Semi-grand canonical ($\mu$VT, $\mu$PT)}
\index{equilibrium statistical ensembles!$\mu$VT} \index{mVT@$\mu$VT|see{equilibrium statistical ensembles}}
\index{equilibrium statistical ensembles!$\mu$PT} \index{mVT@$\mu$PT|see{equilibrium statistical ensembles}}

\section{Evolution of phase space densities}

Let $\z$ denote a point in the phase space of the system, consisting of Cartesian coordinates $\q$ and momenta $\p$.
A mechanical property or \emph{observable} $A$ will in general be a function of the phase space of the system, and will be denoted $A(\z)$.
The expectation of $A$ with respect to some probability distribution $\rho(\z)$ is simply
\begin{eqnarray}
\expect{A}_\rho &=& \int d\z \, \rho(\z) \, A(\z).
\end{eqnarray}
The evolution of an ensemble of systems or, equivalently, our state of uncertainty about the microscopic state of a system, can be described by writing the density as a function of time as well as space, $\rho(\z,t)$.  Given an initial distribution $\rho(\z,0)$ that has been allowed to evolve for some time $t$, the expectation is given by
\begin{eqnarray}
\expect{A(t)}_\rho &=& \int d\z \, \rho(\z,t) \, A(\z). \label{equation:schrodinger-picture}
\end{eqnarray}
Instead of expressing the time dependence in the density, we can move the time-dependence to the observable $A$ and write this expectation as a time-evolved observable $A(\z,t)$ integrated against the initial density $\rho(\z,0)$:
\begin{eqnarray}
\expect{A(t)}_\rho &=& \int d\z \, \rho(\z,0) \, A(\z, t). \label{equation:heisenberg-picture}
\end{eqnarray}
This is a similar story as in quantum mechanics: Eq. \ref{equation:schrodinger-picture} represents the Schr\"{o}dinger picture, where the distribution $\rho$ evolves \emph{forward} in time and is integrated against an observable that is time-independent, whereas Eq. \ref{equation:heisenberg-picture} corresponds to the Heisenberg picture, where the phase function $A$ evolves \emph{backward} in time and is integrated against the initial distribution.
For the purposes of this exposition, it is convenient to adopt the Heisenberg picture.

We presume that evolution of the phase function $A$ is governed by the equation
\begin{eqnarray}
\frac{dA}{dt} &=& i \op{L} A \label{equation:liouville-equation}
\end{eqnarray}
where $i \op{L}$ is termed the \emph{Liouvillean}, and is the instantaneous generator of dynamics. \index{Liouvillean} \label{notation:liouvillean}
$\op{L}$ has been defined this way so that we may also require it to be Hermitian, conferring upon it many of the same useful properties as familiar operators in quantum mechanics also enjoy.
The time evolution of a phase space density $\rho(\bfm{z},t)$ can be shown by the chain rule to be governed by
\begin{eqnarray}
\frac{d\rho}{dt} &=& \frac{\partial \rho}{\partial \z} \cdot \frac{\partial \z}{\partial t} \nonumber \\
&=& \nabla \rho \cdot \dot \z \nonumber \\
&=& \nabla \rho \cdot iL \z \nonumber \\
&=& - i \op{L} \rho \label{equation:liouville-equation-density}
\end{eqnarray}
where $\nabla$ represents the gradient operator with respect to $\z$.
The first step is simple application of the chain rule, while the last step is due to integration by parts\footnote{To see how the last step proceeds, note that integration of $\nabla \rho_0(\z) \cdot \z$ by parts yields
\begin{eqnarray}
\int d\z \, \nabla \rho_0(\z) \cdot \z &=& [\rho_0(\z) \cdot \z]_S - \int d\z \, \rho_0(\z) \cdot 1
\end{eqnarray}
where the surface term $[\rho_0(\z) \cdot \z]_S$ is zero since we require $\rho_0(\z)$ to vanish at large $\z$ in order to be normalizable.
Evans and Morriss \cite{evans:statistical-mechanics-of-nonequilibrium-liquids} Section 3.3 contains an extended discussion on the relation of the Liouvillean acting on phase functions like $A$ to acting on distribution functions like $\rho$.}.
We require that the Liouvillean $i \op{L}$ preserves an equilibrium density $\rho_0(\z)$, such that $\partial \rho_0 / \partial t = 0$.

With these requirements, the Liouvillean $iL$ can represent a variety of dynamical schemes, including field-free Newtonian (Hamiltonian) mechanics, as well as thermostated (e.g. Nos\'{e}-Hoover or Andersen) or barostated (e.g. Andersen piston) dynamics.
% JDC: Give examples of dynamical schemes.
The equilibrium density $\rho_0(\z)$ can therefore be microcanonical, canonical, or even isothermal-isobaric.
Expectations over this equilibrium density are written
\begin{eqnarray}
\expect{A} &=& \int d\z \, \rho_0(\z) \, A(\z).
\end{eqnarray}
A \emph{scalar product} (or \emph{inner product}) can be defined with respect to the equilibrium density $\rho_0$ in the space of phase functions.  \index{scalar product} \index{inner product|see{scalar product}} \label{notation:inner-product}
The inner product of two phase functions $A$ and $B^*$ is given by
\begin{eqnarray}
(A,B^*) &\equiv& \expect{A B^*} \nonumber \\
&=& \int d\z \, \rho_0(\z) \, A(\z) \, B^*(\z)
\end{eqnarray}
where $^*$ denotes the complex conjugate.

We will find occasion to use the Dirac bracket notation when convenient.
In this notation, the phase functions $A(\z)$ and $B(\z)$ are denoted by their corresponding \emph{kets} $\ket{A}$ and $\ket{B}$, with the corresponding \emph{bra} (e.g. $\bra{A}$) corresponding to complex conjugation and the scalar product denoted
\begin{eqnarray}
\braket{A}{B} &\equiv& \expect{A^* B} \nonumber \\
&=& \int d\z \, \rho_0(\z) \, A(\z) \, B^*(\z).
\end{eqnarray}
\index{bra} \index{ket} \index{bra-ket notation}
Note that $\braket{A}{B} = (A,B^*)$.

With the time evolution of $A$ governed by Eq. \ref{equation:liouville-equation}, the explicit time dependence of $A$ in the Heisenberg picture can be written as
\begin{eqnarray}
A(\z,t) &=& e^{i \op{L} t} A(\z,0) \nonumber \\
&=& \op{P}_t A(\z,0)
\end{eqnarray}
where the operator $\op{P}_t \equiv e^{i \op{L} t}$ is termed the \emph{propagator}, since it evolves\footnote{The sign difference between the time evolution operator for an observable (Eq. \ref{equation:liouville-equation-density}) and a density (Eq. \ref{equation:liouville-equation-density}) is sometimes a source of confusion.  One way to think of this is that, in order for the time-evolved phase function to give the proper expectation when integrated against the initial density, the value of the phase function must be evolved \emph{backward} in time at every point in phase space for the two pictures to coincide.} the phase function $A$ by time $t$.  \index{propagator} \label{notation:propagator}
Since $L$ is Hermitian, the propagator $e^{iLt}$ is unitary, and time evolution corresponds to a rotation in the space in which the phase functions live.
The exponential is simply a formal shorthand for the infinite power series expansion
\begin{eqnarray}
\op{P}_t \equiv e^{i \op{L} t} \equiv \sum_{n=0}^\infty \frac{(i \op{L} t)^n}{n!}.
\end{eqnarray}

\section{Dynamics in the canonical ensemble}

For the majority of the remainder of this dissertation, we consider the canonical ensemble almost exclusively due to its widespread popularity in molecular simulations.
Many of the results that follow can easily be adapted to other ensembles, but this may be nontrivial in some cases.

Because the canonical ensemble is purely an equilibrium construct, there is no definitive specification of how equilibrium dynamics is to be modeled.
A number of choices are possible, each with advantages (theoretical, computational, or conceptual) that make it difficult to decide which is most appropriate.
Here, we enumerate a number of these models, all of which produce the canonical \emph{equilibrium} distribution (though perhaps in the limit of long sampling times), but treat the issue of dynamics in different manners.

\subsection{Canonical average over constant-energy trajectories}

In the simplest model of dynamics in the canonical ensemble, the system evolves according to Hamilton's equations of motion (that is, motion is purely Newtonian).
Because the system is confined to a single energy shell $E$, we must therefore consider a canonical distribution of energy shells, weighting the dynamical trajectories from each energy shell appropriately:
\begin{eqnarray}
p(E) &=& [Z(\beta)]^{-1} \, \Omega(E) \, e^{-\beta E}
\end{eqnarray}
where $\Omega(E)$ is the total energy density of states defined above.

\subsection{Stochastic thermostats}

In this section, we briefly consider some of the more popular stochastic thermostats which have been employed.

\subsubsection{Smoluchowski dynamics}

\subsubsection{Langevin dynamics}

\subsubsection{The Andersen thermostat}

Andersen demonstrated that a canonical ensemble can be recovered by having molecules of the system undergo stochastic collisions with fictitious bath particles, where upon collision, the momenta of the atoms in the molecule undergoing collision would be reassigned from the Maxwell distribution \cite{andersen:1980a}.
A constant collision rate $\nu$ per unit time can be employed, or collisions could occur at predetermined intervals.  % check latter part
Alternatively, each atom may undergo collisions independently\footnote{It should be note that the Andersen thermostat with each atom independently undergoing collision only produces correct temperatures and distributions when no constraints are employed.  It has been observed anecdotally that, due to the removal of velocities in the direction of constraints, the kinetic energy distribution is disrupted.}, or even the entire system could undergo ``massive collision'' at random (or regular) intervals, though such massive collisions are obviously less suitable for modeling realistic kinetics.
Andersen demonstrated that this procedure preserves the canonical distribution, as it consists of two components:

\subsection{Deterministic thermostats}

\subsubsection{The Berendsen weak-coupling algorithm}

SImulations sometimes employ the Berendsen weak-coupling algorithm for thermal control \cite{berendsen:1984a}, in which a the \emph{instantaneous kinetic temperature}, $K(\z) \equiv \p^\T \M \p / 3 N \kB$, derived by analogy with Eq. \ref{equation:canonical-temperature} above, is restrained to fluctuate the desired temperature.
However, it is well known that while this method may produce correct average temperatures computed by Eq. \ref{equation:canonical-temperature} above, the distribution of kinetic energies differs from that in the canonical ensemble, which leads to an incorrect phase space distribution\cite{morishita:2000a}.
In very large solvated systems, however, the effect of the thermostat may be minimal, and a near-correct ensemble may be obtained.

\subsubsection{The Nos\'e-Hoover thermostat}