\chapter{Learning from examples}
\begin{singlespace}
\setlength{\epigraphwidth}{0.6\textwidth}
\epigraph{\textit{Now the learning process may be regarded as a search for a form of
behaviour which will satisfy the teacher...}}{\textsc{Alan Turing \\  Computing Machinery and Intelligence (1950)}}
\end{singlespace}

% 4.1 Examples aid communication
% 4.2 Learning, a relation
% 4.3 Metatheory
% 4.4 Learning identity, not, and successor

% 5.1 Humans learn languages
% 5.2 Computers learn languages
% '''SICP quote, every program is a programming language
% 5.3 Learning a compiler
% 5.4 Learning an interpreter

\section{Examples aid communication}

Remember, programs communicate. But the burden lay entirely on the human. They must be precise to say something---every instruction a meticulous curation. Last chapter we saw how types helped shift the burden. Instead of writing out a full program, we can just state a type and let the machine learn a program that fits the type. We're closer to a language which permits this sort of imprecise yet productive use:
\begin{displayquote}
\centering
\textit{A glork smashed my car.}
\end{displayquote}
This sentence has useful information, even if you don't know what a glork is. But what if  you wanted to know? Surely you can't avoid glorks if you don't even know what they are. Maybe glorks are elephants. Maybe they're hurricanes.

Enter examples. They aid communication. For teachers they're a weapon of choice, of necessity. Types combined with examples offer precision.
\begin{displayquote}
\centering
\textit{A glork smashed my car. This morning it was fine. \\But now there's a dent in the shape of a humungous footprint.}
\end{displayquote}

With the help of an example (of what happened), it's clearer what a glork is. It's likely not a hurricane, but an elephant or some other animal with huge feet. So we still don't get complete precision, but we often get enough from just a few examples.

We can harness examples to build languages which allow for this sort of productive and precise enough kind of communication. We provide a type and examples, then let the machine learn what we mean.

\section{Learning, a relation}

As before, I describe learning as a relation. This relation extends the presentation of learning from last chapter, now with support for examples.
\begin{displayquote}
\centering
$\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$\\
\textit{Given a context} $\Gamma$ \textit{, type} $\tau$ \textit{, and examples} $[\chi_1,\dots,\chi_n]$\textit{, I can learn program} $e$.
\end{displayquote}

The main difference is that types can now be further specified by examples---think when I described a glork by its type "smasher" and what it did to my car. Importantly, examples $[\chi_1,\dots,\chi_n]$ are not a new syntactic form in System F. It's only a notational convenience. In actuality, examples are tuples of inputs and outputs, e.g. examples for the identity function on natural numbers could be $\langle 1,1\rangle$. And last chapter, we saw that tuples are learnable in System F.

\subsection{What are examples?}

Examples $[\chi_1,\dots,\chi_n]$ are lists of tuples, containing the inputs and output to a program. For instance $[\langle1,1\rangle]$ describes an example whose input is $1$ and output is $1$. If I want to specify more than one example, I can package examples $\chi$ into the list: $[\langle 1,1\rangle, \langle2,2\rangle]$. Here I have two examples, each with one input and one output. In general, examples take the form \vspace{-1.0em}
\begin{singlespace}
$$\chi ::= \langle e, \chi\rangle \mid \langle e,Nil\rangle$$
\end{singlespace}
where $e$ is an arbitrary program in System F and $\chi$ is an example. This syntax for examples lets us construct examples with arbitrary numbers of inputs, e.g. $\langle10,10,20\rangle \equiv \langle 10, \langle 10, \langle 20,Nil\rangle\rangle\rangle.$ Note that when examples have multiple inputs I use the short-hand notation for describing an $n-$tuple, $\langle10,10,20\rangle$ in lieu of full notation $ \langle 10, \langle 10, \langle 20,Nil\rangle\rangle\rangle$. Likewise when an example is merely $\langle e,Nil\rangle$, I use the short-hand $\langle e\rangle$---as $Nil$ can be interpreted as the empty element.

For an example $\chi \equiv \langle e_1,\dots,e_n\rangle$, an ordered list of inputs is given by $e_1,\dots,e_{n-1}$. The last index always denotes an output. An example satisfies or describes a program, if when the ordered list of inputs $e_1,\dots,e_{n-1}$ is applied to a program $e$, it is equivalent to the output $e_n$. That is,\vspace{-1.0em}
\begin{singlespace}
$$(((e\,e_1)e_2)\dots e_{n-1}) =_\beta e_n$$
\end{singlespace}
For instance, $\chi \equiv \langle 1,1\rangle$ satisfies the identity program $\lam x{:}nat.x$ because \vspace{-1.0em}
\begin{singlespace}
$$(\lam x{:}nat.x)1 =_\beta 1$$
\end{singlespace}
Similarly, a list of examples $[\chi_1,\dots,\chi_n]$ satisfies some program if each example in the list satisfies the program. Note that with this notion of satisfaction, we can construct examples which satisfy any program $e$, that is $\langle e\rangle$. It's an example with no input, and whose output is $e$. Because no inputs can be applied, and that $e =_\beta e$, $\langle e\rangle$ satisfies $e$.
\subsection{Learning, informally}

With the learning relation, we can ask whether \textsc{identity} is learnable given a context, type, and examples. \textsc{identity} is a program which takes a natural number and returns it. \vspace{-1.0em}
\begin{singlespace}
$$\cdot \vdash nat \!\to\! nat \rhd \langle\langle1,1\rangle,\langle2,2\rangle\rangle \rightsquigarrow \blacksquare$$
\end{singlespace}
Examples are stored as tuples. They describe possible worlds, one where our program's input is 1 and the other where our program's input is 2. Throughout learning we need a way to keep track of these distinct worlds. So our first step is always to duplicate $\blacksquare$, so that there is one per example. \vspace{-1.0em}
\begin{singlespace}
$$\cdot \vdash list \,nat \!\to\! nat \rhd \langle\langle1,1\rangle,\langle2,2\rangle\rangle \rightsquigarrow [\blacksquare, \blacksquare]$$
\end{singlespace}
Let's refine these worlds, by applying them to their respective inputs. We extract the inputs from each example tuple.\vspace{-1.0em}
\begin{singlespace}
$$1{:}nat, 2{:}nat \vdash list \,nat \rhd \langle\langle1\rangle,\langle2\rangle\rangle \rightsquigarrow [(\blacksquare)1, (\blacksquare)2]$$
\end{singlespace}
Because $\blacksquare$ is applied to an argument, we know it must be an abstraction. Hence, we can also claim: \vspace{-1.0em}
\begin{singlespace}
$$1{:}nat, 2{:}nat \vdash list \,nat \rhd \langle\langle1\rangle,\langle2\rangle\rangle \rightsquigarrow [(\lam x{:}nat.\blacksquare)1, (\lam x{:}nat.\blacksquare)2]$$
\end{singlespace}
Now that we've ran out of inputs in our examples, the problem becomes how to generate a program which satisfy the outputs left in the example tuples:\vspace{-1.0em}
\begin{singlespace}
$$1{:}nat, 2{:}nat \vdash list \,nat\, \rightsquigarrow [(\lam x{:}nat.\blacksquare)1, (\lam x{:}nat.\blacksquare)2]$$
$$(\lam x{:}nat.\blacksquare)1 =_\beta 1\;\land\;(\lam x{:}nat.\blacksquare)2=_\beta2$$
\end{singlespace}
Given the constraints on well-typed terms, it's easy to find $x$ to fill the body of the abstraction. This will become clear in the formal proof to follow. \vspace{-1.0em}
\begin{singlespace}
$$1{:}nat, 2{:}nat \vdash list \,nat\, \rightsquigarrow [(\lam x{:}nat.x)1, (\lam x{:}nat.x)2]$$
$$(\lam x{:}nat.x)1 =_\beta 1\;\land\;(\lam x{:}nat.x)2=_\beta2$$
\end{singlespace}
Having satisfied the outputs from our examples, we've informally shown $\textsc{identity} \equiv \lam x{:}nat.x$ is learnable in System F. And all the machinery comes from types and operators we can encode in System F: list and product types along with their constructors and deconstructors.

\subsection{Learning, formally}

\begin{figure}[h!]
\centering
\setlength{\tabcolsep}{12pt}
\begin{tabular}{l r}
\specialrule{.1em}{0em}{.2em}
\specialrule{.1em}{0em}{1em}
    \Large \textsc{Learning} &
    \framebox{$\Gamma \vdash \tau \rhd \chi \rightsquigarrow e$}\\
    & \\
    \multicolumn{2}{c}{
    \def\extraVskip{4pt}
    \def\labelSp]acing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma \vdash list \,\tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
        \AxiomC{$\bigwedge_{i=1}^n e_i =_\beta e_n$}
        \RightLabel{\textsc{(L-Wrld)}}
        \BinaryInfC{$\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$}

        \DisplayProof
    } \\\\
    \multicolumn{2}{c}{
    \def\extraVskip{4pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma, \bigcup_{i=1}^n \chi_i{:}\tau \vdash list \,\tau \rightsquigarrow [e_1,\dots,e_n]$}
        \AxiomC{$\bigwedge_{i=1}^{n} e_i =_\beta \chi_i$}
            \RightLabel{\textsc{(L-Base)}}
        \BinaryInfC{$\Gamma \vdash list \,\tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}

        \DisplayProof
    } \\\\
    \multicolumn{2}{c}{
    \def\extraVskip{4pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma, \bigcup_{i=1}^n\pi_1(\chi_i){:}\tau_a \vdash list \,\tau_b \rhd [\pi_2(\chi_1),\dots,\pi_2(\chi_n)] \rightsquigarrow [e_1\pi_1(\chi_1),\dots,e_n\pi_1(\chi_n)]$}
        \RightLabel{\textsc{(L-EAbs)}}
        \UnaryInfC{$\Gamma \vdash list \,\tau_{a} \!\to\! \tau_b \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
        \DisplayProof
    } \\\\
    \multicolumn{2}{c}{
    \def\extraVskip{4pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma, \alpha \vdash list \,\tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1\lceil\alpha\rceil,\dots,e_n\lceil\alpha\rceil]$}
        \RightLabel{\textsc{(L-ETAbs)}}
        \UnaryInfC{$\Gamma \vdash list \,\forall\alpha.\tau \rhd [\Lambda\alpha.\chi_1,\dots,\Lambda\alpha.\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
        \DisplayProof
    } \\\\
    \multicolumn{2}{c}{
    \def\extraVskip{1pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
    \alwaysNoLine
        \AxiomC{$\Gamma\vdash list \,\tau_{a} \!\to\! \tau_c \rhd [\chi_1,\dots,\chi_j] \rightsquigarrow [e_1,\dots,e_j]$}
        \UnaryInfC{$\Gamma\vdash list \,\tau_{b} \!\to\! \tau_c \rhd [\chi_1,\dots,\chi_k] \rightsquigarrow [e_1,\dots,e_k]$}
        \RightLabel{\textsc{(L-Sum)}}
        \alwaysSingleLine
        \def\extraVskip{4pt}
        \UnaryInfC{$\Gamma \vdash list \,(\tau_a\!+\!\tau_b) \!\to\! \tau_c \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
        \DisplayProof
    } \\
    & \\
\specialrule{.1em}{1em}{0em}
\end{tabular}
\caption{Learning from examples in System F}
    \label{fig:learning-examples}
\end{figure}

The informal process of learning described can be made formal via the relation presented in Figure \ref{fig:learning-examples}.

\textsc{(L-Wrld)} says that if you can learn a list of programs $[e_1,\dots,e_n]$, where $e_1,\dots,e_n$ are equivalent, then you can learn the program $e_1$. This rule is used to create $n$ worlds for $n$ examples at the start of learning.
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\Gamma \vdash list\,nat\!\to\!nat \rhd [\langle1,1\rangle,\langle2,2\rangle] \rightsquigarrow [\lam x{:}nat.x,\lam x{:}nat.x]$}
	\RightLabel{\textsc{(L-Wrld)}}
	\UnaryInfC{$\Gamma \vdash nat\!\to\!nat \rhd [\langle1,1\rangle,\langle2,2\rangle] \rightsquigarrow \lam x{:}nat.x$}
\end{prooftree}

\textsc{(L-Base)} says that if you can learn a list of programs from its type and each $e_i$ is equivalent to some $\chi_i$ for $0\leq i\leq n$, then we can use each $\chi_i$ as an example output. For instance, $(\lam x{:}nat.x)1 =_\beta 1$. This means we can use $1$ as an example output for $(\lam x{:}nat.x)$. When learning, this rule is the "base" case. After exhausting the example information, this rule turns the learning process into learning from types.
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\Gamma, 1{:}nat \vdash list\,nat \rightsquigarrow [(\lam x{:}nat.x)1]$}
	\AxiomC{$(\lam x{:}nat.x)1 =_\beta 1$}
	\RightLabel{\textsc{(L-Base)}}
	\BinaryInfC{$\Gamma \vdash list\,nat \rhd [\langle1\rangle] \rightsquigarrow [(\lam x{:}nat.x)1]$}
\end{prooftree}

\textsc{(L-EAbs)} says that if you can learn a list of applications $[e_1\pi_1(\chi_1),\dots,e_n\pi_1(\chi_n)]$, then you can learn a list of abstractions $[e_1,\dots,e_n]$ from examples where each $\pi_1\chi_i$ are inputs for $0 \leq i \leq n$. Note that $\pi_1$ is the first projection of an example tuple.
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\Gamma, 1{:}nat, 2{:}nat \vdash list\,nat \rhd [\langle1\rangle,\langle2\rangle] \rightsquigarrow [(\lam x{:}nat.x)1,(\lam x{:}nat.x)2]$}
	\RightLabel{\textsc{(L-EAbs)}}
	\UnaryInfC{$\Gamma \vdash list\,nat\!\to\!nat \rhd [\langle1,1\rangle,\langle2,2\rangle] \rightsquigarrow [\lam x{:}nat.x,\lam x{:}nat.x]$}
\end{prooftree}

\textsc{(L-ETAbs)} says that if you can learn a list of applications $[e_1\lceil\alpha\rceil,\dots,e_n\lceil\alpha\rceil]$, then you can learn a list of polymorphic abstractions $[e_1,\dots,e_n]$ from examples where each $\Lambda\alpha.\chi_i$ are inputs for $0 \leq i \leq n$.
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\Gamma, \alpha \vdash list \,\alpha\!\to\!\alpha \rhd [\langle z,z\rangle] \rightsquigarrow [(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil]$}
	\RightLabel{\textsc{(L-TAbs)}}
	\UnaryInfC{$\Gamma \vdash list \,\forall\alpha.\alpha\!\to\!\alpha \rhd [\Lambda\alpha.\langle z,z\rangle] \rightsquigarrow [\Lambda\alpha.\lam x{:}\alpha.x]$}
\end{prooftree}

\textsc{(L-Sum)} says that if you can learn a list of programs whose input is type $\tau_a$ and another list of programs whose input is type $\tau_b$, then you can learn a list of program whose input is the sum type $\tau_a\!+\!\tau_b$ and whose examples contain inputs of both type $\tau_a$ and type $\tau_b$. During learning, this rule is perhaps the most useful. It lets you distribute examples when encountering sum types as inputs, which ends up creating two sub-problems, each of which dealing with a smaller set of examples, which are easier to satisfy. In the example, let $e \equiv case(b)\,of\, \iota_1(true) \mapsto false \mid \iota_2(false) \mapsto true$
\begin{prooftree}
    \def\extraVskip{1pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
    \alwaysNoLine
        \AxiomC{$\Gamma\vdash list \,bool \!\to\! bool \rhd [\langle true, false\rangle] \rightsquigarrow [\lam x{:}bool.false]$}
        \UnaryInfC{$\Gamma\vdash list \,bool \!\to\! bool \rhd [\langle false, true\rangle] \rightsquigarrow [\lam y{:}bool.true]$}
        \RightLabel{\textsc{(L-Sum)}}
        \alwaysSingleLine
        \def\extraVskip{4pt}
        \UnaryInfC{$\Gamma \vdash list \,(bool\!+\!bool) \!\to\! bool \rhd [\langle true,false \rangle, \langle false, true\rangle] \rightsquigarrow [e,e]$}
\end{prooftree}

This new relation extends learning from types. Now System F can learn from examples too. So far, our presentation of learning is both sound and complete. You can learn every program in System F from types. We now show that these results hold when learning from examples too.

\section{Metatheory}
Because our aim is to show every program in System F is learnable from examples, we want to show that learning is still equivalent to typing. As with learning from types, we show both completeness and soundness of learning with respect to typing---giving us the equivalence. These proofs entail a bit more work, but are far simpler than similar presentations of metatheory for languages which permit learning from examples, e.g. \textsc{Myth} \cite{osera2015program}. The mathematical convenience is afforded by not introducing any machinery into System F for learning.

\subsection{Typing and Learning are still equivalent}

To show completeness, we need to show that for any program in System F there exists a list of examples from which it can be learned. This turns out to be trivial. Hence, a stronger statement we want is that we can learn any program which is satisfied by a list of examples. It would be problematic if we could learn any program in System F, but only from a particular subset of the examples which describe that program. Showing both of these gives strong guarantees on learning. If a list of examples describes a program, you can learn that program---and this list exists for every program in System F.

\begin{lemma}[\textsc{Completeness$_a$ of Learning}]
If $\,\Gamma \vdash e : \tau$ then there exist some $[\chi_1,\dots,\chi_n]$ such that $\,\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$
\label{completeness-learning-examples-a}
\end{lemma}
\begin{proof}
For an arbitrary program $e$ of type $\tau$, let $[\langle e\rangle]$ constitute its example list. This example list has no input, only an ouput $e$. Hence the statemement $\,\Gamma \vdash \tau \rhd [\langle e\rangle] \rightsquigarrow e$ asks whether we can learn a program $e$ of type $\tau$ whose output is $e$.

Now, note that for any program $e =_\beta e$ by definition of the reflexive, transitive, and symmetric evaluation relation. Additionally, for any program $e$ of type $\tau$, a list $[e]$ can be learned from type $list\,\tau$. Because learning from types is complete, this works for any program.

Knowing this, we can apply the following learning rules:
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\,\Gamma,x{:}\tau \vdash list\,\tau \rightsquigarrow [e]$}
	\AxiomC{$e=_\beta e$}
	\RightLabel{\textsc{(L-Base)}}
	\BinaryInfC{$\,\Gamma \vdash list\,\tau \rhd [\langle e\rangle] \rightsquigarrow [e]$}
	\AxiomC{$e=_\beta e$}
	\RightLabel{\textsc{(L-Wrld)}}
	\BinaryInfC{$\,\Gamma \vdash \tau \rhd [\langle e\rangle] \rightsquigarrow e$}
\end{prooftree}

Hence for any program $e$ in System F, it can be learned from examples when the example is $[\langle e\rangle]$.
\end{proof}

This is nice, but the less interesting completeness result. We want to make sure that a list of examples which describes a program can be used to learn it. An example list satisfies (or describes) a program, where if the program is applied to the inputs, the corresponding outputs are equivalent.

\begin{lemma}[\textsc{Completeness$_b$ of Learning}]
If $\,\Gamma \vdash e : \tau$ and there exist some $[\chi_1,\dots,\chi_n]$ which satisfies $e$, then $\,\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$.
\label{completeness-learning-examples-b}
\end{lemma}
\begin{proof}
There are two general cases to prove, when $e$ has inputs and when $e$ has no inputs. For programs without input, see previous lemma. For programs with input let $\Gamma \vdash e : \tau$, where $[\chi_1,\dots,\chi_n]$ are examples which satisfy $e$. Because of Theorem \ref{equivalence-learning}, we also know $\Gamma \vdash \tau \rightsquigarrow e$. Because lists are learnable, we also know $\Gamma \vdash list\,\tau \rightsquigarrow [e_1,\dots,e_n]$, where $\bigwedge_{i=1}^n e_i =_\beta e_n$.
\vspace{.3em}

Now let's deconstruct each $\chi_i$ into its input and output components: $\langle\chi^{in}_i, \chi^{out}_i\rangle$. Since each $\chi_i$ satisfies $e$, it must be that $ \bigwedge_{i=1}^n e_i\chi_i^{in} =_\beta \chi_i^{out}$. Due to satisfaction, $e_i\chi_i^{in}$ is guaranteed to be well-typed. Hence $\Gamma,\bigcup_{i=1}^n\chi_i^{out}{:}\tau^{out}, \bigcup_{i=1}^n\chi_i^{in}{:}\tau^{in} \vdash list\,\tau^{out} \rightsquigarrow [e_1\chi_i^{in},\dots,e_n\chi_n^{in}]$. With this, we can apply the following rules:

\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\Gamma,\bigcup_{i=1}^n\chi_i^{out}{:}\tau^{out}, \bigcup_{i=1}^n\chi_i^{in}{:}\tau^{in} \vdash list\,\tau^{out} \rightsquigarrow [e_1\chi_i^{in},\dots,e_n\chi_n^{in}]$}
	\AxiomC{$\bigwedge_{i=1}^n e_i\chi_i^{in} =_\beta \chi_i^{out}$}
	\RightLabel{\textsc{(L-Base)}}
	\BinaryInfC{$\Gamma,  \bigcup_{i=1}^n\chi_i^{in}{:}\tau^{in} \vdash list\,\tau^{out} \rhd [\langle\chi_1^{out}\rangle,\dots,\langle\chi_n^{out}\rangle] \rightsquigarrow [e_1\chi_i^{in},\dots,e_n\chi_n^{in}]$}
	\RightLabel{\textsc{(L-EAbs)}}
	\UnaryInfC{$\Gamma \vdash list\,\tau \rhd [\langle\chi_1^{in},\chi_1^{out}\rangle,\dots,\langle\chi_n^{in},\chi_n^{out}\rangle] \rightsquigarrow [e_1,\dots,e_n]$}
\end{prooftree}
Remembering that $\chi_i \equiv \langle\chi_i^{in},\chi_i^{out}\rangle$ and that $\bigwedge_{i=1}^n e_i =_\beta e_n$, we finally prove the necessary result.
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
	\AxiomC{$\Gamma \vdash list\,\tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
	\AxiomC{$\bigwedge_{i=1}^n e_i =_\beta e_n$}
	\RightLabel{\textsc{(L-Wrld)}}
	\BinaryInfC{$\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$}
\end{prooftree}

Note that if there are $n$ inputs to the examples which satisfy $e$, then \textsc{(L-EAbs)} must be applied $n$ times to fully reconstruct the examples.
\end{proof}

\begin{lemma}[\textsc{Soundness of Learning}]
If $\,\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$ then $\Gamma \vdash e : \tau$
\label{soundness-learning-examples}
\end{lemma}
\begin{proof}
Case analysis on learning rules.

\underline{Case 1}: \textsc{(L-Wrld)}\\
We know that we can learn from examples a list of length $n$ where each entry is $e$.
\begin{prooftree}
    \def\extraVskip{4pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma \vdash list \,\tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
        \AxiomC{$\bigwedge_{i=1}^n e_i =_\beta e_n$}
        \RightLabel{\textsc{(L-Wrld)}}
        \BinaryInfC{$\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$}
\end{prooftree}
After applying \textsc{(L-EAbs)} and \textsc{(L-Base)} it must be that $\Gamma \vdash list \,\tau' \rightsquigarrow [e_1\chi_1^{in},\dots,e_n\chi_n^{in}]$. The list is only learnable if each element is learnable, hence $\Gamma \vdash \tau' \rightsquigarrow e_i\chi_i^{in}$. And an application is only learnable if each side of the application is learnable, hence $\Gamma \vdash \tau \rightsquigarrow e$ (noting $e_i =_\beta e$). Because of the equivalence of learning from types and typing, we have $\Gamma \vdash e : \tau$.

\underline{Case 2}: \textsc{(L-Base)}\\
We know we can learn learn from types a list of length $n$ where each entry is $e$.

\begin{prooftree}
    \def\extraVskip{4pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma, \bigcup_{i=1}^n \chi_i{:}\tau \vdash list \,\tau \rightsquigarrow [e_1,\dots,e_n]$}
        \AxiomC{$\bigwedge_{i=1}^{n} e_i =_\beta \chi_i$}
            \RightLabel{\textsc{(L-Base)}}
        \BinaryInfC{$\Gamma \vdash list \,\tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
\end{prooftree}

The list is only learnable if each element is learnable, hence $\Gamma \vdash \tau \rightsquigarrow e$. Because of the equivalence of learning from types and typing, we have $\Gamma \vdash e : \tau$.

\underline{Case 3}: \textsc{(L-EAbs)}\\
We know we can learn learn from types a list of length $n$ and type $\tau_b$ where each entry is $e\pi_1(\chi_i)$.

\begin{prooftree}
    \def\extraVskip{4pt}
    \def\labelSpacing{4pt}
    \def\defaultHypSeparation{\hskip .4in}
        \AxiomC{$\Gamma, \bigcup_{i=1}^n\pi_1(\chi_i){:}\tau_a \vdash list \,\tau_b \rhd [\pi_2(\chi_1),\dots,\pi_2(\chi_n)] \rightsquigarrow [e_1\pi_1(\chi_1),\dots,e_n\pi_1(\chi_n)]$}
        \RightLabel{\textsc{(L-EAbs)}}
        \UnaryInfC{$\Gamma \vdash list \,\tau_{a} \!\to\! \tau_b \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow [e_1,\dots,e_n]$}
\end{prooftree}

After applying \textsc{(L-Base)} it must be that $\Gamma, \bigcup_{i=1}^n\pi_1(\chi_i){:}\tau_a \vdash list \,\tau_b \rightsquigarrow [e_1\pi_1(\chi_1),\dots,e_n\pi_1(\chi_n)]$. The list is only learnable if each element is learnable, hence $\Gamma, \bigcup_{i=1}^n\pi_1(\chi_i){:}\tau_a \vdash \tau_b \rightsquigarrow e_1\pi_1(\chi_1)$. And an application is only learnable if each side of the application is learnable, hence $\Gamma \vdash \tau_a\!\to\!\tau_b \rightsquigarrow e$ (noting $e_i =_\beta e$). Because of the equivalence of learning from types and typing, we have $\Gamma \vdash e : \tau_a\!\to\!\tau_b$.

\underline{Case 4}: \textsc{(L-ETAbs)}\\
Same strategy as Case 3, except using type application.

\underline{Case 5}: \textsc{(L-Sum)}\\
After assuming \textsc{(L-Sum)}, reduces to proof of Case 3.
\end{proof}
\vspace{-1.0em}

\begin{theorem}[\textsc{Equivalence of Typing and Learning}]
If and only if $\,\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$, then $\Gamma \vdash e : \tau$ and $[\chi_1,\dots,\chi_n]$ satisfies $e$.
\label{equivalence-learning-examples}
\end{theorem}
\begin{proof}
Directly from Lemmas \ref{completeness-learning-examples-a}, \ref{completeness-learning-examples-b} and \ref{soundness-learning-examples}.
\end{proof}
\vspace{-.8em}

Because we can only learn a program if and only if it is well typed, it follows that learned programs obey progress, preservation, and normalization. Each proof invokes the equivalence theorem between typing and learning, and then the respective progress, preservation, and normalization theorems for typing.

\subsection{Learned programs still don't get stuck}

\begin{corollary}[\textsc{Progress in Learning}]
If $e$ is a learned program, then either $e$ is a value or else there is some program $e'$ such that $e \to_\beta e'$.
\label{progress-learning}
\end{corollary}
\begin{proof}
Directly from Theorems \ref{equivalence-learning-examples} and \ref{progress-typing}.
\end{proof}
\vspace{-.8em}

We shouldn't be able to learn programs which get stuck during evaluation, same as with typing. If I learn a program, either its a value or it can be evaluated to another program. When learning from examples, learning still obeys progress.

\subsection{Learned programs still don't change type}

\begin{corollary}[\textsc{Preservation in Learning}]
If $\,\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e$ and $e \to_\beta e'$, then $\,\Gamma \vdash \tau \rhd [\chi_1,\dots,\chi_n] \rightsquigarrow e'$.
\label{preservation-learning}
\end{corollary}
\begin{proof}
Directly from Theorems \ref{equivalence-learning-examples} and \ref{preservation-typing}.
\end{proof}
\vspace{-.8em}

We shouldn't be able to learn programs of a different type than the one provided. If I learn a program, and it evaluates to another program, then I should be able to learn that new program from the same type. When learning from examples, learning still obeys preservation.

\subsection{Learned programs still always halt}

\begin{corollary}[\textsc{Normalization in Evaluation}]
Learned programs in System F always evaluate to a value, to a normal form.
\label{normalization-learning}
\end{corollary}
\begin{proof}
Directly from Theorems \ref{equivalence-learning-examples} and \ref{normalization-evaluation}.
\end{proof}
\vspace{-.8em}

We shouldn't be able to learn programs which never finish computing. They must halt. As with learning from types, learning from examples only lets you learn halting programs.

\section{Learning identity, not, and successor}

For examples, any program in System F can serve as input or output. There are no restrictions. In similar works which allow learning from examples, like \textsc{Myth} \cite{osera2015program}, there are restrictions on what examples can look like. Namely, functions cannot appear as output in an example. This makes it impossible to learn many higher-order programs. In fact, the motivation for this work started from observing this limitation in \textsc{Myth}. It made it impossible to learn compilers, higher-order programs which specify programming languages.

Here we show how to learn several programs from examples before discussing further the prospect of learning not only programs, but programming languages.

\subsection{Learning polymorphic identity}

\begin{lemma}[\textsc{Polymorphic identity is learnable}] $\cdot \vdash \forall\alpha.\alpha \!\to\! \alpha \rhd [\Lambda\alpha.\langle z,z\rangle] \rightsquigarrow \Lambda\alpha.\lam x{:}\alpha.x$
\label{learning-poly-identity}
\end{lemma}
\begin{proof}

\begin{enumerate}[label=\textit{(\roman*)}]

\item $\alpha, z{:}\alpha \vdash \alpha\!\to\!\alpha \rightsquigarrow (\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil$
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
    \AxiomC{$x{:}nat \in \alpha, z{:}\alpha, x{:}\alpha$}
    \RightLabel{\textsc{(L-Var)}}
    \UnaryInfC{$\alpha, z{:}\alpha, x{:}\alpha \vdash \alpha \rightsquigarrow x$}
    \RightLabel{\textsc{(L-Abs)}}
    \UnaryInfC{$\alpha, z{:}\alpha \vdash \alpha\!\to\!\alpha \rightsquigarrow \lam x{:}\alpha.x$}
    \RightLabel{\textsc{(L-TAbs)}}
    \UnaryInfC{$\alpha, z{:}\alpha \vdash \forall\alpha.\alpha\!\to\!\alpha \rightsquigarrow \Lambda\alpha.\lam x{:}\alpha.x$}
    \RightLabel{\textsc{(L-Abs)}}
    \AxiomC{$\alpha, z{:}\alpha \vdash \alpha \rightsquigarrow \alpha$}
    \RightLabel{\textsc{(L-App)}}
    \BinaryInfC{$\alpha, z{:}\alpha \vdash \alpha\!\to\!\alpha \rightsquigarrow (\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil$}
    \alwaysNoLine
    \UnaryInfC{}
\end{prooftree}

\item $\alpha, z{:}\alpha \vdash \alpha \rightsquigarrow (\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z$
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
    \AxiomC{\textit{(i)}}
    \UnaryInfC{$\alpha, z{:}\alpha \vdash \alpha\!\to\!\alpha \rightsquigarrow (\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil$}
    \AxiomC{$z{:}\alpha \in \alpha, z{:}\alpha$}
    \RightLabel{\textsc{(L-Var)}}
    \UnaryInfC{$\alpha, z{:}\alpha \vdash \alpha \rightsquigarrow z$}
    \RightLabel{\textsc{(L-App)}}
    \BinaryInfC{$\alpha, z{:}\alpha \vdash \alpha \rightsquigarrow (\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z$}
    \alwaysNoLine
    \UnaryInfC{}
\end{prooftree}

\item $\alpha, z{:}\alpha \vdash list\,\alpha \rightsquigarrow [(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z]$
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
    \AxiomC{\textit{(ii)}}
    \UnaryInfC{$\alpha, z{:}\alpha \vdash \alpha \rightsquigarrow (\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z$}
    \AxiomC{$\alpha, z{:}\alpha \vdash list \,\alpha \rightsquigarrow [\,]$}
    \RightLabel{\textsc{(L-Cons)}}
    \BinaryInfC{$\alpha, z{:}\alpha \vdash list\,\alpha \rightsquigarrow [(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z]$}
    \alwaysNoLine
    \UnaryInfC{}
\end{prooftree}

\item $\cdot \vdash \forall\alpha.\alpha \!\to\! \alpha \rhd [\Lambda\alpha.\langle z,z\rangle] \rightsquigarrow \Lambda\alpha.\lam x{:}\alpha.x$
\begin{prooftree}
\def\extraVskip{4pt}
\def\labelSpacing{4pt}
    \AxiomC{\textit{(iii)}}
    \UnaryInfC{$\alpha, z{:}\alpha \vdash list\,\alpha \rightsquigarrow [(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z]$}
    \AxiomC{$(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z =_\beta z$}
    \RightLabel{\textsc{(L-Base)}}
    \alwaysSingleLine
    \def\extraVskip{4pt}
    \BinaryInfC{$\alpha, z{:}\alpha \vdash list\,\alpha \rhd [\langle z\rangle] \rightsquigarrow [(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil z]$}
    \RightLabel{\textsc{(L-EAbs)}}
    \UnaryInfC{$\alpha \vdash list\,\alpha \!\to\! \alpha \rhd [\langle z,z\rangle] \rightsquigarrow [(\Lambda\alpha.\lam x{:}\alpha.x)\lceil\alpha\rceil]$}
    \RightLabel{\textsc{(L-ETAbs)}}
    \UnaryInfC{$\cdot \vdash list\,\forall\alpha.\alpha \!\to\! \alpha \rhd [\Lambda\alpha.\langle z,z\rangle] \rightsquigarrow [\Lambda\alpha.\lam x{:}\alpha.x]$}
    \RightLabel{\textsc{(L-Wrld)}}
    \UnaryInfC{$\cdot \vdash \forall\alpha.\alpha \!\to\! \alpha \rhd [\Lambda\alpha.\langle z,z\rangle] \rightsquigarrow \Lambda\alpha.\lam x{:}\alpha.x$}
    \alwaysNoLine
    \UnaryInfC{}
\end{prooftree}

\end{enumerate}
\end{proof}

\subsection{Learning boolean not}
\begin{lemma}[\textsc{Not is learnable}] \begin{singlespace}$\cdot \vdash (bool\!+\!bool)\!\to\!bool \rhd [\langle true,false \rangle, \langle false, true\rangle] \rightsquigarrow e$

\textsc{Note:} $e \equiv case(b)\,of\, \iota_1(true) \mapsto false \mid \iota_2(false) \mapsto true$. Additionally, that $true$ and $false$ are learnable from any context. \end{singlespace}
\label{learning-boolean-not}
\end{lemma}
\begin{proof}

\begin{enumerate}[label=\textit{(\roman*)}]

\item $true{:}bool \vdash bool \rightsquigarrow (\lam x{:}bool.false)true$
\begin{prooftree}
    \alwaysSingleLine
    \def\extraVskip{4pt}
    		\AxiomC{$true{:}bool, x{:}bool \vdash bool \rightsquigarrow false$}
    		\RightLabel{\textsc{(L-Abs)}}
        \UnaryInfC{$true{:}bool \vdash bool \rightsquigarrow \lam x{:}bool.false$}
        \AxiomC{$true{:}bool \in true{:}bool$}
        \RightLabel{\textsc{(L-Var)}}
        \UnaryInfC{$true{:}bool \vdash bool \rightsquigarrow true$}
        \RightLabel{\textsc{(L-App)}}
        \BinaryInfC{$true{:}bool \vdash bool \rightsquigarrow (\lam x{:}bool.false)true$}

\end{prooftree}

\item $\cdot\vdash list \,bool \!\to\! bool \rhd [\langle true, false\rangle] \rightsquigarrow [\lam x{:}bool.false]$
\begin{prooftree}
    \alwaysSingleLine
    \def\extraVskip{4pt}
    		\AxiomC{\textit{(i)}}
    		\AxiomC{$true{:}bool \vdash list\,bool \rightsquigarrow []$}
    		\RightLabel{\textsc{(L-Cons)}}
        \BinaryInfC{$true{:}bool \vdash list\,bool \rightsquigarrow [(\lam x{:}bool.false)true]$}
        \AxiomC{$(\lam x{:}bool.false)true =_\beta false$}
        \RightLabel{\textsc{(L-Sum)}}
        \BinaryInfC{$true{:}bool \vdash list bool \rhd [\langle false\rangle] \rightsquigarrow [(\lam x{:}bool.false)true]$}
        \RightLabel{\textsc{(L-EAbs)}}
		\UnaryInfC{$\Gamma\vdash list \,bool \!\to\! bool \rhd [\langle true, false\rangle] \rightsquigarrow [\lam x{:}bool.false]$}
\end{prooftree}

\item $false{:}bool \vdash list\,bool \rightsquigarrow (\lam y{:}bool.true)false$
\begin{prooftree}
    \alwaysSingleLine
    \def\extraVskip{4pt}
    		\AxiomC{$false{:}bool, y{:}bool \vdash bool \rightsquigarrow true$}
    		\RightLabel{\textsc{(L-Abs)}}
        \UnaryInfC{$false{:}bool \vdash bool \rightsquigarrow \lam y{:}bool.true$}
        \AxiomC{$false{:}bool \in false{:}bool$}
        \RightLabel{\textsc{(L-Var)}}
        \UnaryInfC{$false{:}bool \vdash bool \rightsquigarrow false$}
        \RightLabel{\textsc{(L-App)}}
        \BinaryInfC{$false{:}bool \vdash bool \rightsquigarrow (\lam y{:}bool.true)false$}

\end{prooftree}

\item $\cdot\vdash list \,bool \!\to\! bool \rhd [\langle false, true\rangle] \rightsquigarrow [\lam y{:}bool.true]$
\begin{prooftree}
    \alwaysSingleLine
    \def\extraVskip{4pt}
    		\AxiomC{\textit{(iii)}}
    		\AxiomC{$false{:}bool \vdash list\,bool \rightsquigarrow []$}
    		\RightLabel{\textsc{(L-Cons)}}
        \BinaryInfC{$false{:}bool \vdash list\,bool \rightsquigarrow [(\lam y{:}bool.true)false]$}
        \AxiomC{$(\lam y{:}bool.true)false =_\beta true$}
        \RightLabel{\textsc{(L-Sum)}}
        \BinaryInfC{$false{:}bool \vdash list bool \rhd [\langle true\rangle] \rightsquigarrow [(\lam y{:}bool.true)false]$}
        \RightLabel{\textsc{(L-EAbs)}}
		\UnaryInfC{$\cdot\vdash list \,bool \!\to\! bool \rhd [\langle false, true\rangle] \rightsquigarrow [\lam y{:}bool.true]$}
\end{prooftree}

\item $\cdot \vdash (bool\!+\!bool)\!\to\!bool \rhd [\langle true,false \rangle, \langle false, true\rangle] \rightsquigarrow e$
\begin{prooftree}
    \alwaysSingleLine
    \def\extraVskip{4pt}
        \AxiomC{\textit{(ii)}}
        \AxiomC{\textit{(iii)}}
        \RightLabel{\textsc{(L-Sum)}}
        \BinaryInfC{$\cdot \vdash list \,(bool\!+\!bool) \!\to\! bool \rhd [\langle true,false \rangle, \langle false, true\rangle] \rightsquigarrow [e,e]$}
        \RightLabel{\textsc{(L-Wrld)}}
		\UnaryInfC{$\cdot \vdash (bool\!+\!bool)\!\to\!bool \rhd [\langle true,false \rangle, \langle false, true\rangle] \rightsquigarrow e$}
\end{prooftree}
\end{enumerate}
\end{proof}

\subsection{Learning church successor}

\begin{lemma}[\textsc{Successor is learnable}] \begin{singlespace} Let $church \equiv \forall\alpha.(\alpha\!\to\!\alpha)\!\to\!\alpha\!\to\!\alpha$, $\bar{0} \equiv \Lambda\alpha.\lam f{:}\alpha\!\to\!\alpha.\lam x{:}\alpha.x$, $\bar{1} \equiv \Lambda\alpha.\lam f{:}\alpha\!\to\!\alpha.\lam x{:}\alpha.fx$, and $succ \equiv \lam n{:}church.\Lambda\alpha.\lam f{:}\alpha\!\to\!\alpha.\lam x{:}\alpha.f(n \lceil\alpha\rceil f x)$. And assume that $\bar{0}$, $\bar{1}$, and $succ$ can be learned from any context.

Show $\cdot \vdash church\!\to\!church \rhd [\langle\bar{0},\bar{1}\rangle] \rightsquigarrow succ$.\end{singlespace}
\label{learning-succ}
\end{lemma}

\begin{proof}

\begin{enumerate}[label=\textit{(\roman*)}]

\item $\bar{0}{:}church \vdash church\!\to\!church \rightsquigarrow succ$

\textsc{Note:} Can be learned from any context.

\item $\cdot \vdash church\!\to\!church \rhd [\langle\bar{0},\bar{1}\rangle] \rightsquigarrow succ$
\begin{prooftree}
    \alwaysSingleLine
    \def\extraVskip{4pt}
    		\AxiomC{\textit{(i)}}
    		\AxiomC{$\bar{0}{:}church \in \bar{0}{:}church$}
    		\UnaryInfC{$\bar{0}{:}church \vdash church \rightsquigarrow \bar{0}$}
    		\BinaryInfC{$\bar{0}{:}church \vdash church \rightsquigarrow (succ)\bar{0}$}
    		\AxiomC{$\bar{0}{:}church \vdash list\,church \rightsquigarrow []$}
    		\BinaryInfC{$\bar{0}{:}church \vdash list\,church \rightsquigarrow [(succ)\bar{0}]$}
    		\AxiomC{$(succ)\bar{0} =_\beta \bar{1}$}
    		\RightLabel{\textsc{(L-Base)}}
        \BinaryInfC{$\cdot \vdash list\,church \rhd [\langle\bar{1}\rangle] \rightsquigarrow [(succ)\bar{0}]$}
        \RightLabel{\textsc{(L-EAbs)}}
        \UnaryInfC{$\cdot \vdash list\,church\!\to\!church \rhd [\langle\bar{0},\bar{1}\rangle] \rightsquigarrow [succ]$}
        \RightLabel{\textsc{(L-Wrld)}}
        \UnaryInfC{$\cdot \vdash church\!\to\!church \rhd [\langle\bar{0},\bar{1}\rangle] \rightsquigarrow succ$}
\end{prooftree}

\end{enumerate}
\end{proof}


% Hubway Plots

%\begin{figure}
%	\includegraphics{figures/hubway_nodes.pdf}
%	\caption{Hubway node objective evolution.}
%	\label{fig:hubway_nodes}
%\end{figure}
%
%\begin{figure}
%	\includegraphics{figures/hubway_edges.pdf}
%	\caption{Hubway edge objective evolution.}
%	\label{fig:hubway_edges}
%\end{figure}

\begin{table*}[htbp]\small
\centering
\begin{tabular}{@{}llcccc@{}}
\toprule
\multirow{2}{*}{\begin{tabular}[l]{@{}l@{}}Graph\end{tabular}}                                                                         & \multirow{2}{*}{\begin{tabular}[l]{@{}l@{}}Item\\Distribution\ \ \ \ \ \ \end{tabular}} & \multirow{2}{*}{\begin{tabular}[l]{@{}l@{}}$r({\nodegreedy})$\end{tabular}} & \multirow{2}{*}{\begin{tabular}[l]{@{}l@{}}$r({\tt Node-Baseline}^\ast)$\end{tabular}} & \multirow{2}{*}{\begin{tabular}[l]{@{}l@{}}$r({\edgegreedy})$\end{tabular}} & \multirow{2}{*}{\begin{tabular}[l]{@{}l@{}}$r({\tt Edge-Baseline}^\ast)$\end{tabular}}\\ \\ \toprule
\multirow{4}{*}{\begin{tabular}[l]{@{}l@{}}{\autonomoussystems}\end{tabular}}  & {\ego}               & 0.06                                  & 0.24                                   & 0.14                                  & 0.15                                   \\
                                                                               & {\direct}            & 0.66                                  & 0.67                                   & 0.99                                  & 0.99                                   \\
                                                                               & {\uniform}           & 0.38                                  & 0.40                                   & 0.97                                  & 0.99                                   \\
                                                                               & {\inverse}           & 0.38                                  & 0.40                                   & 0.97                                  & 0.99                                   \\ \cmidrule(l){2-6}
\multirow{4}{*}{\begin{tabular}[l]{@{}l@{}}{\geo}\end{tabular}} 			   & {\ego}               & 0.00                                  & 0.06                                   & 0.01                                  & 0.02                                   \\
                                                                               & {\direct}            & 0.00                                  & 0.06                                   & 0.20                                  & 0.65                                   \\
                                                                               & {\uniform}           & 0.00                                  & 0.06                                   & 0.15                                  & 0.65                                   \\
                                                                               & {\inverse}           & 0.00                                  & 0.07                                   & 0.15                                  & 0.65                                   \\ \cmidrule(l){2-6}
\multirow{4}{*}{\begin{tabular}[l]{@{}l@{}}{\grid}\end{tabular}} 	           & {\ego}               & 0.27                                  & 0.27                                   & 0.29                                  & 0.29                                   \\
                                                                               & {\direct}            & 0.92                                  & 0.92                                   & 0.98                                  & 0.98                                   \\
                                                                               & {\uniform}           & 0.92                                  & 0.92                                   & 0.98                                  & 0.98                                   \\
                                                                               & {\inverse}           & 0.92                                  & 0.92                                   & 0.98                                  & 0.98                                   \\ \cmidrule(l){2-6}
\multirow{4}{*}{\begin{tabular}[l]{@{}l@{}}{\ba}\end{tabular}} 	               & {\ego}               & 0.18                                  & 0.56                                   & 0.26                                  & 0.26                                   \\
                                                                               & {\direct}            & 0.71                                  & 0.71                                   & 0.99                                  & 0.99                                   \\
                                                                               & {\uniform}           & 0.63                                  & 0.63                                   & 0.98                                  & 0.98                                   \\
                                                                               & {\inverse}           & 0.63                                  & 0.63                                   & 0.98                                  & 0.98                                   \\ \bottomrule
\end{tabular}
\caption{Comparison of greedy algorithms with the best-performing baseline (${\tt Node-Baseline}^\ast$ and ${\tt Edge-Baseline}^\ast$) for $k=50$.
For a given pair of graph and item-distribution scheme, $r(A)$ expresses the
ratio of the expected uncertainty that algorithm $A$ achieves with $k=50$
monitoring operations over the initial uncertainty $\uncertainty_0$
(for $k=0$). Note that the best-performing baseline is different for different rows of the table.
}
\label{tab:objective-evolution-table}
\end{table*}

%\begin{table}[htbp]
%\centering
%\begin{tabular}{@{}ll@{}}
%\toprule
%ID & Landmark                           \\ \toprule
%33 & Kenmore Sq.                        \\
%36 & Copley Sq./Boston Public Library \\
%41 & Packard's Corner                   \\
%42 & Boston Public Garden               \\
%52 & Newbury St.                        \\ \bottomrule
%\end{tabular}
%\caption{Hubway stations in Boston city minimizing uncertainty (k=5).}
%\label{tab:hubway-table}
%\end{table}

% For both {\nodeproblem} and {\edgeproblem} problems, the number of items allocated to each node during initialization are chosen using three criteria,
% \begin{itemize}
% 	\item Uniformly distributed over nodes.
% 	\item Directly proportional to the out-degree of the node.
% 	\item Inversely proportional to the out-degree of the node.
% \end{itemize}
% For each of the above three item initializations, we compare the following baseline approaches of choosing 'k' nodes,
% \begin{itemize}
% 	\item {\nodeproblem} baselines
% 	\begin{itemize}
% 		\item $k$ randomly chosen nodes.
% 		\item Top-$k$ nodes of highest incoming probabilities.
% 		\item Top-$k$ nodes of highest in-degree centrality scores.
% 		\item Top-$k$ nodes of highest degree-centrality scores.
% 		\item Top-$k$ nodes chosen using {\nodegreedy} algorithm.
% 	\end{itemize}
% 	\item {\edgeproblem} baselines
% 	\begin{itemize}
% 		\item $k$ randomly chosen edges.
% 		\item Top-$k$ edges of highest probabilities.
% 		\item Top-$k$ edges chosen using {\edgegreedy} algorithm.
% 	\end{itemize}
% \end{itemize}
% For each of the configurations described above, we make following two plots,
% \begin{itemize}
% 	\item Evolution of objective function value for $k=0$ over time. (At some time, the items distribution over nodes approaches the stationary distribution. The objective function value for $k=0$ remains constant thereafter.)

% 	\item Evolution of objective function value over different values of $k$. (As the $k$ increases, the uncertainty should decrease ultimately to zero.)

% \end{itemize}

\begin{figure}
\begin{subfigure}[b]{0.23\textwidth}
\includegraphics[width=\textwidth]{figures/hubway_nodes.pdf}
\caption{{\nodeproblem}}
\label{fig:hubway_nodes}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\includegraphics[width=\textwidth]{figures/hubway_edges.pdf}
\caption{{\edgeproblem}}
\label{fig:hubway_edges}
\end{subfigure}
\caption{{\hubway} data; $y$-axis: expected uncertainty, $x$-axis: number of monitored nodes or edges.}
\end{figure}

\subsection{Experimental results}
In this section, we report the
performance of algorithms for the {\mcproblem} problem -
first on the graph datasets, combined with item distribution schemes;
then on the \hubway\ dataset.
As objective we always use the expected uncertainty
achieved for a given budget $k$ of nodes or edges -- the smaller its value,
the better the performance of the algorithm.
Note that we do not report separately the performance of \edgeDP, as it achieves
same performance as \edgegreedy, but is not as efficient.

We provide the results for the graph datasets in Table \ref{tab:objective-evolution-table}.
In all these experiments we use $k=50$.
Moreover, $r(A)$ is the ratio of the achieved objective value (for $k=50$) over
the initial value $\uncertainty_0$ of the measure
(for no monitoring operations, i.e., $k=0$).
The table shows four quantities for every graph-item distribution pair
: $r(A)$ for $A=\{${\nodegreedy}, {\tt Node-Baseline}$^\ast$, {\edgegreedy}, {\tt Edge-Baseline}$^\ast$ $\}$.
Note that {\tt Node-Baseline}$^\ast$ (resp.\ {\tt Edge-Baseline}$^\ast$) refers to the baseline
algorithm with the best performance.
For every algorithm $A$, $r(A)\in[0,1]$ and the smaller the value of $r(A)$ the better the performance of the algorithm.

From the table, we observe that for
the {\autonomoussystems} dataset,  {\nodegreedy}  significantly outperforms the best
baseline for the {\ego} item distribution, while performing marginally better for other item distributions.
The value of  $r({\edgegreedy})$ is only slightly less than
the best baselines across all the configurations. However, we observe that there is no baseline
which performs uniformly the best across different item distributions. For example,  {\edgebetweenness}
is the best baseline for {\direct} item distribution, the {\edgenumitems} for {\ego}, while they both
perform worse than even randomly chosen edges for {\uniform} and {\inverse} item distributions.
Notably, for the {\geo} graphs, the greedy algorithms significantly outperform the baselines

For the {\grid} graphs, the baselines perform exactly
the same as our algorithms. This can be explained by the nature of the
{\grid} graph, where all the nodes except the ones on the boundary are similar to each other,
thereby rendering the {\direct}, {\uniform} and {\inverse} item distributions very similar
to each other. For the {\ego} distribution, the greedy algorithms perform marginally better
than the baselines.
%, depending on the number of nodes within the fixed radius around the randomly chosen node,
%and the percentage of items distributed among these nodes.
Again,
there is no baseline which performs uniformly the best.
Similar is our explanation for the results on {\ba} graphs as in these graphs most of the nodes have
almost the same (small) degrees too.

A more thorough investigation of the performance of the algorithms for different
values of $k$ and for all datasets we consider is shown in
Supplementary Material, Section~\ref{sec:additional_results}.

\spara{Experiments with {\hubway} data:}
In our last experiment,
we explore the performance of our algorithms on the {\hubway} dataset. From Figure \ref{fig:hubway_nodes}
and Figure \ref{fig:hubway_edges} we observe that the {\nodegreedy} and {\edgegreedy} algorithms
are consistently the best at reducing expected uncertainty, although the baselines are competitive on the
relatively smaller graph. In Figure \ref{fig:hubway_stations}, we plot the Hubway stations across Boston
chosen by the {\nodegreedy} algorithm with $k=5$. The nodes chosen by the algorithm are supported by
the anecdotal evidence of being exactly some of the of the most popular landmarks around the city.
From a managerial perspective, tracking the number of trips starting or ending at these Hubway stations
can help the operators better reduce the expected uncertainty around the expected number of bikes
available at its different stations and anticipate future bike ``re-balancing"\footnote{https://www.citylab.com/transportation/2014/08/balancing-bike-share-stations-has-become-a-serious-scientific-endeavor/379188/} operations.

\spara{Running times:} For all our experiments we use a single process
implementation of our algorithms on a 24-core 2.9GHz Intel Xeon E5 processor
with 512GB memory. For the  largest graph in our experiments,
the parallelized version of the {\nodegreedy}\ takes about $5-10$
seconds per selected node, while the parallelized version of {\edgegreedy}\
takes about $1$ minute per selected edge.

\begin{figure}
	\centering
	\includegraphics[width=8cm, height=4cm]{figures/hubway_stations_5.pdf}
	\caption{{\hubway} data. IDs of stations picked as a solution to {\nodeproblem}
	for $k=5$;
	33: Kenmore Sq., 36: Copley Sq./Boston Public Library, 41: Packard's Corner,
	42: Boston Public Garden, 52: Newbury St.                        }
	\label{fig:hubway_stations}
\end{figure}

\spara{Discussion:} Our experiments show that  {\nodegreedy} and  {\edgegreedy}
consistently perform better than or on par with other popular baseline methods. Also, for
graphs with relatively large number of nodes, the solutions to the {\nodeproblem} problem
are more effective at reducing
the expected uncertainty than the solutions to the {\edgeproblem} problem for the same number of
node (resp.\ edge) monitors.
This is
especially important considering our analysis from Section \ref{sec:nodes} and \ref{sec:edges} which show
that the {\nodegreedy} algorithm has a better time complexity compared to the {\edgegreedy} for dense graphs.


%\documentclass[12pt]{article}

%\usepackage{alltt}
%\usepackage{epsfig}
%\usepackage{pstricks}
%\usepackage{xspace}
%\usepackage{url}
%\usepackage{makeidx}
%\usepackage{index}
%\usepackage{multind}

\renewcommand{\t}[1]{\mbox{\small\tt #1}}
%\newcommand{\prev}[1]{#1}

%\newcommand{\varord}[1]{#1}

%\newcommand{\ma}[1]{{{$#1$}}}
%\newcommand{\id}[1]{#1}

\newcommand\Hol{HOL}

%\newcommand{\els}{\mid}
%\newcommand{\Imp}{\Rightarrow}

%\renewcommand{\prod}{\mbox{\tt{*}}}
%\newcommand{\SP}{~}
%\newcommand{\SPP}{~}

%\newcommand{\homedir}{\mbox{$\sim$}}

%\newcommand{\Turn}{\(\turn\)}
%\newcommand{\And}{\(\wedge\)}
%\newcommand{\Or}{\(\vee\)}
%\newcommand{\Not}{\(\neg\)}
%\newcommand{\Forall}{\(\forall\)}
%\newcommand{\Exists}{\(\exists\)}
%\newcommand{\Mapsto}{\(\mapsto\)}

%\input{../LaTeX/commands}

\setcounter{sessioncount}{0}

%\begin{document}
\index{HolSatLib|(}
\index{SAT solvers|see {HolSatLib}}
\index{decision procedures!propositional satisfiability}

The purpose of {\tt{HolSatLib}} is to provide a platform for experimenting with combinations of theorem proving and SAT solvers. Only black box functionality is provided at the moment; an incremental interface is not available.

{\tt{HolSatLib}} provides a function {\tt SAT\_PROVE} \index{HolSatLib!SAT\_PROVE@\ml{SAT\_PROVE}} for propositional satisfiability testing and for proving propositional tautologies. It uses an external SAT solver (currently MiniSat 1.14p) to find an unsatisfiability proof or satisfying assignment, and then reconstructs the proof or checks the assignment deductively in \HOL.

Alternatively, the function {\tt SAT\_ORACLE} \index{HolSatLib!SAT\_ORACLE@\ml{SAT\_ORACLE}} has the same behaviour as {\tt SAT\_PROVE} but asserts the result of the solver without proof. The theorem thus asserted is tagged with ``{\tt{HolSatLib}}'' to indicate that it is unchecked. Since proof reconstruction can be expensive, the oracle facility can be useful during prototyping, or if proof is not required.

The following example illustrates the use of {\tt{HolSatLib}} for proving propositional tautologies:

\begin{session}
\begin{verbatim}
- load "HolSatLib"; open HolSatLib;
(* output omitted *)
> val it = () : unit

- show_tags := true;
> val it = () : unit

- SAT_PROVE ``(a ==> b) /\ (b ==> a) = (a=b)``;
> val it = [oracles: DISK_THM] [axioms: ] []
           |- (a ==> b) /\ (b ==> a) = (a = b) : thm

- SAT_PROVE ``(a ==> b) ==> (a=b)``
   handle HolSatLib.SAT_cex th => th;
> val it = [oracles: DISK_THM] [axioms: ] []
           |- ~a /\ b ==> ~((a ==> b) ==> (a = b)) : thm

- SAT_ORACLE ``(a ==> b) /\ (b ==> a) = (a=b)``;
> val it = [oracles: DISK_THM, HolSatLib] [axioms: ] []
           |- (a ==> b) /\ (b ==> a) = (a = b) : thm
\end{verbatim}
\end{session}

Setting \t{show\_tags} to \t{true} makes the \HOL{} top-level print theorem tags. The \t{DISK\_THM} oracle tag has nothing to do with \t{HolSatLib}. It just indicates the use of theorems from \HOL{} libraries read in from permanent storage.

Note that in the case where the putative tautology has a falsifying interpretation, a counter-model can be obtained by capturing the special exception {\tt{SAT\_cex}}, which contains a theorem asserting the counter-model.

The next example illustrates using {\tt{HolSatLib}} for satisfiability testing. The idea is to negate the target term before passing it to {\tt{HolSatLib}}.

\begin{session}
\begin{verbatim}
- SAT_PROVE ``~((a ==> b) ==> (a=b))``
   handle HolSatLib.SAT_cex => th;
> val it = [oracles: DISK_THM ] [axioms: ] []
           |- a /\ ~b ==> ~~((a ==> b) ==> (a = b)) : thm

- SAT_PROVE ``~(a /\ ~a)``;
> val it = [oracles: DISK_THM ] [axioms: ] []
           |- ~(a /\ ~a) : thm
\end{verbatim}
\end{session}

As expected, if the target term is unsatisfiable we get a theorem saying as much.

{\tt{HolSatLib}} can only handle purely propositional terms (atoms must be propositional variables or constants) involving the usual propositional connectives as well as Boolean-valued conditionals. If you wish to prove tautologies that are instantiations of propositional terms, use {\tt{tautLib}} (see \S\ref{tautlib} below).

If MiniSat failed to build when \HOL{} was built, or proof replay fails for some other reason, \t{SAT\_PROVE} falls back to a DPLL-based propositional tautology prover implemented in SML, due to Michael Norrish (see the \HOL{} Tutorial). {\tt{HolSatLib}} prints out a warning if this happens. On problems with more than a thousand or so clauses (in conjunctive normal form (CNF)), the SML prover will likely take too long to be of any use.

{\tt{HolSatLib}} will delete temporary files generated by the SAT solver, such as the proof file and any statistics. This is to avoid accumulating thousands of possibly large files. Currently {\tt HolSatLib} has only been tested on Linux, and on Windows XP using MinGW.

\subsection{{\tt tautLib}}\label{tautlib}

{\tt tautLib} predates {\tt{HolSatLib}} by over a decade. It used a Boolean case analysis algorithm suggested by Tom Melham and implemented by R.~J.~Boulton. This algorithm has since been superseded and the functions in the {\tt tautLib} signature now act as wrappers around calls to {\tt{HolSatLib}}. However, the wrappers are able to provide the following extra functionality on top of {\tt{HolSatLib}}:

\begin{enumerate}
\item They can handle top level universal quantifiers.
\item They can reason about (the propositional structure of) terms that are instances of purely propositional terms. This is done by a preprocessing step that replaces each unique instantiation with a fresh propositional variable.
\end{enumerate}

For details, see the source file \t{src/taut/tautLib.sml} which contains comprehensive comments. Note however that the extra functionality in {{\tt tautLib}} was not engineered for very large problems and can become a performance bottleneck.

\subsection{Support for other SAT solvers}\label{subsec:hs_zchaff}

The ZChaff SAT solver has a proof production mode and is supported by {\tt{HolSatLib}}. However, the ZChaff end user license is not compatible with the \HOL{} license, so we are unable to distribute it with \HOL{}. If you wish to use ZChaff, download and unpack it in the directory {\tt src/HolSat/sat\_solvers/} under the main \HOL{} directory, and compile it with proof production mode enabled (which is not the default). This should create a binary {\tt zchaff} in the directory {\tt src/HolSat/sat\_solvers/zchaff/}. ZChaff can now be used as the external proof engine instead of MiniSat, by using the HolSatLib functions described above, prefixed with a ``{\tt Z}'', e.g., {\tt ZSAT\_PROVE}.

A file \texttt{resolve\_trace} may be created in the current working directory, when working with ZChaff. This is the proof trace file produced by ZChaff, and is hardwired.

Other SAT solvers are currently not supported. If you would like such support to be added for your favourite solver, please send a feature request via \url{http://github.com/mn200/HOL}.

\subsection{The general interface}

The functions described above are wrappers for the function \texttt{GEN\_SAT}, which is the single entry point for {\tt{HolSatLib}}. \texttt{GEN\_SAT} can be used directly if more flexibility is required. \texttt{GEN\_SAT} takes a single argument, of type \texttt{sat\_config}, defined in \texttt{satConfig.sml}. This is an opaque record type, currently containing the following fields:

\begin{enumerate}
\item{\texttt{term : Term.term}}

The input term.
\item{\texttt{solver : SatSolvers.sat\_solver}}

The external SAT solver to use. The default is \texttt{SatSolvers.minisatp}. If ZChaff is installed (see \S\ref{subsec:hs_zchaff}), then \texttt{SatSolvers.zchaff} may also be used.
\item{\texttt{infile : string option}}

The name of a file in DIMACS format.\footnote{\url{http://www.satlib.org/Benchmarks/SAT/satformat.ps}}. Overrides \texttt{term} if set. The input term is instead read from the file.
\item{\texttt{proof : string option}}

The name of a proof trace file. Overrides \texttt{solver} if set. The file must be in the native format of {\tt{HolSatLib}}, and must correspond to a proof for \texttt{infile}, which must also be set. The included version of MiniSat has been modified to produce proofs in the native format, and ZChaff proofs are translated to this format using the included proof translator \texttt{src/HolSat/sat\_solvers/zc2hs} (type \texttt{zc2hs -h} for usage help). \texttt{zc2hs} is used internally by \texttt{ZSAT\_PROVE} etc.
\item{\texttt{is\_cnf : bool}}

If true then the input term is expected to be a negated CNF term. This is set automatically if \texttt{infile} is set. Typically a user will never need to modify this field directly.
\item{\texttt{is\_proved : bool}}

If true then \HOL{} will prove the SAT solver's results.
\end{enumerate}

A special value \texttt{base\_config~:~sat\_config} is provided for which the term is \texttt{T}, the solver is MiniSat, the options are unset, the CNF flag is false and the proof flag is true. This value can be inspected and modified using getter and setter functions provided in \texttt{src/HolSat/satConfig.sig}. For example, to invoke ZChaff (assuming it is installed), on a file \texttt{unsat.cnf} containing an unsatisfiable problem, we do:

\begin{session}
\begin{verbatim}
- open satConfig;
(* output omitted *)

- val c = (set_infile "unsat.cnf" o set_solver SatSolvers.zchaff) base_config;
> val c = <sat_config> : sat_config

- GEN_SAT c;
> val it = [oracles: DISK_THM ] [axioms: ] []
           |- ~<unsat.cnf term here> : thm
\end{verbatim}
\end{session}

If the problem were satisfiable, the model can be captured via exception, as shown earlier.

Normally, {\tt{HolSatLib}} will delete the files generated by the SAT solver, such as the output proof, counter-model, and result status. However, if \texttt{infile} is set, the files are not deleted, in case they are required elsewhere.

\subsection{Notes}

On Linux and MacOS, \texttt{g++} must be installed on the system for MiniSat and \texttt{zc2hs} to build.

Temporary files are generated using the Moscow ML function \t{FileSys.tmpName}. This usually writes to the standard temporary file space on the operating system. If that file space is full, or if it is inaccessible for some other reason, {\tt{HolSatLib}} calls may fail mysteriously.

The function {\tt dimacsTools.readDimacs}~{\it file}  reads a DIMACS format file and returns
a CNF \HOL{} term corresponding to the SAT problem in the file named by {\it file}. Since DIMACS uses numbers to denote variables, and numbers are not legal identifiers in \HOL{}, each variable number is prefixed with the string ``{\tt v}''. This string is defined in the reference variable {\tt dimacsTools.prefix} and can be changed if required. This function can be used independently of {\tt{HolSatLib}} to read in DIMACS format files.

\index{HolSatLib|)}

%\end{document}
%%% Local Variables:
%%% mode: latex
%%% TeX-master: "description"
%%% End:


\subsection{Monads}
\begin{frame}[fragile]{Monad Definition}
\begin{lstlisting}[frame=htrbl]
class Monad m where
	(>>=) :: m a -> (a -> m b) -> m b
	return :: a -> m a
\end{lstlisting}

Similar to Java's Optional, we have \code{Maybe a}:
\begin{lstlisting}[frame=htrbl]
instance Monad Maybe where
	(Just a) >>= f = f a
	Nothing >>= _ = Nothing
	return a = Just a
\end{lstlisting}

$\Rightarrow$ composable computation descriptions

\end{frame}

\begin{frame}[fragile]{Monad Usage}
With monadic functions like
\begin{lstlisting}[frame=htrbl]
func :: Int -> Maybe Int
func x
	| x < 0 = Nothing
	| otherwise = Just (x * 2)
\end{lstlisting}
we can compose computations:
\begin{lstlisting}[frame=htrbl]
complicatedFunc :: Int -> Maybe Int
complicatedFunc x = (return x) >>= func >>= ...
\end{lstlisting}
\end{frame}


\input{../header_class}

%---------- start document ---------- %
 \section{poly.ratfunc -- rational function}\linkedzero{poly.ratfunc}
 \begin{itemize}
   \item {\bf Classes}
   \begin{itemize}
     \item \linkingone{poly.ratfunc}{RationalFunction}
   \end{itemize}
 \end{itemize}

 A rational function is a ratio of two polynomials.

 Please don't expect this module is useful.
 It just provides an acceptable container for polynomial division.
\C

 \subsection{RationalFunction -- rational function class}\linkedone{poly.ratfunc}{RationalFunction}
 \initialize
  \func{RationalFunction}{\hiki{numerator}{polynomial},\ \hikiopt{denominator}{polynomial}{1}}{\out{RationalFunction}}\\
  \spacing
  % document of basic document
  \quad Make a rational function with the given \param{numerator} and
  \param{denominator}.
  % added document
  %
  % \spacing
  % input, output document
  If the \param{numerator} is a {\tt RationalFunction} instance and
  \param{denominator} is not given, then make a copy.
  If the \param{numerator} is a kind of polynomial, then make a rational
  function whose numerator is the given polynomial.
  Additionally, if \param{denominator} is also given,
  the denominator is set to its values, otherwise the denominator is 1.
  \begin{at}
    \item[numerator]\linkedtwo{poly.ratfunc}{RationalFunction}{numerator}:\\
      polynomial.
    \item[denominator]\linkedtwo{poly.ratfunc}{RationalFunction}{denominator}:\\
      polynomial.
  \end{at}
  \begin{op}
    \verb+A==B+ & Return whether {\tt A} and {\tt B} are equal or not.\\
    \verb+str(A)+ & Return readable string.\\
    \verb+repr(A)+ & Return string representing {\tt A}'s structure.\\
  \end{op}
  \method
  \subsubsection{getRing -- get rational function field}\linkedtwo{poly.ratfunc}{RationalFunction}{getRing}
   \func{getRing}{\param{self}}{\linkingone{poly.ring}{RationalFunctionField}}\\
   \spacing
   % document of basic document
   \quad Return the rational function field to which the rational function belongs.
\C

%---------- end document ---------- %

\input{../footer}


\section{Overview}
\index{metadynamics}

Metadynamics \cite{laio-02a,quigley-09a} is a method for studying the
thermodynamics of activated processes, for which purpose it accelerates the
time scale for structural changes to occur and, at the same time, accumulates
data describing the free energy surface, from which the free energy of the the
structural transition may be obtained. The specific implementation within \D{}
uses this method in the context of transitions to/from crystalline phases.
Phase transitions, for example from the liquid to the solid state, frequently
exhibit {\em hysteresis} such that one state persists after it has become
thermodynamically unstable. This is due to the existence of a free energy
barrier between the states, which inhibits the transition. Metadynamics
provides a means to overcome the free energy barrier and facilitate the phase
transition on a timescale accessible by molecular dynamics simulation.

Metadynamics was originally devised by Laio and Parrinello
\cite{laio-02a} and the implementation in \D{} is based on the methodology
described by Quigley and Rodger \cite{quigley-09a}.
The metadynamics routines in \D{} were originally written by David Quigley and
Mark Rodger at the University of Warwick and incorporated into the package by
W. Smith.

Note that it is intended that this facility be used to study phase transitions
and there is an accompanying expectation that such studies will be undertaken
using either an NVT, NPT or N$\sigma$T ensemble. Note also that when used
together with shell model electrostatics the metadynamics routines revert
to velocity Verlet integration .

\section{Theory of Metadynamics}
\index{metadynamics!theory of}
\label{metadtheory}

In metadynamics the Hamiltonian that defines the dynamics of an $N$-particle
system is augmented by a {\em time dependent} bias potential which is a function
of appropriate order parameters\footnote{The term {\em collective variable}
  may be used as an alternative to {\em order parameter}. } that characterise
the structure of the system:
\begin{equation}
H[\vek{r}^{N},t]=\sum_{i=1}^{N}\frac{p_{i}}{2m_{i}} + U(\vek{r}^{N})+
V[\vek{s}^{M}(\vek{r}^{N}),t].\label{metah}
\end{equation}
In this equation $U(\vek{r}^{N})$ is the usual potential energy function
describing the interactions between, and within, the molecules. $p_{i}$ is the
momentum of the $i'th$ atom and $m_{i}$ its mass. The novel term
$V[\vek{s}(\vek{r}^{N}),t]$ is the time dependent bias potential, which is a
function of a vector $\vek{s}^{M}$ that is an ordered set of $M$ order
parameters, each of which is defined by the instantaneous positions
$\vek{r}^{N}$ of the atoms in system. The bias potential is time dependent in
the sense that it can be `grown' by adding, at periodic intervals of time
$\tau_{G}$, a Gaussian term of weight $w$ and width $\delta h$:
\begin{equation}
\index{metadynamics!Gaussian potential}
V[\vek{s}^{M}(\vek{r}^{N}),t]=w\sum_{k=1}^{N_{G}}
exp \left [\frac{-|\vek{s}^{M}(k\tau_{G})-\vek{s}^{M}(t)|^{2}}
{2\delta h^{2}} \right ], \label{depgauss}
\end{equation}
where $k$ runs over all previously deposited Gaussians and
$N_{G}=int(t/\tau_{G})$.
The force on each atom $\vek{f}_{i}$ derived from the Hamiltonian
(\ref{metah}) is given by
\begin{equation}
\vek{f}_{i}=-\vek{\nabla}_{i} U(\vek{r}^{N}) - \sum_{j=1}^{M}
\frac{\partial V}{\partial s_{j}}\vek{\nabla}_{i} s_{j}(\vek{r}^{N})
\end{equation}
If the deposition rate $w/\tau_{G}$ is slow enough the motion of the order
parameters $\vek{s}^{M}$ is adiabatically separated from the motion of the
atomic system. After a sufficiently long simulation, the bias potential
cancels out or `fills' the free energy landscape of the potential
$U(\vek{r}^{N})$ and permits an accelerated dynamics. Meanwhile the bias
potential becomes a measure of the free energy surface i.e.
\begin{equation}
F_{G}(\vek{s}^{M})=-\frac{lim}{t \rightarrow \infty}
V[\vek{s}^{M}(\vek{r}^{N}),t] \label{metafree}
\end{equation}
The accuracy of this estimation of the free energy surface is dependent on the
deposition rate and the effective diffusion constant of the order
parameters. Typically the error is of order $w$, the Gaussian weight.
A discussion of these issues is given by Laio {\em et al} \cite{laio-05a}.
and Quigley and Rodger \cite{quigley-09a}.

The importance of using order parameters in the Hamiltonian (\ref{metah}) is
that they are a direct measure of the structure of a particular phase.
Increasing the bias potential therefore has the effect of destabilising phases
that are characterised by these parameters, forcing the simulation to move to
alternative structures with lower free energy. In general several different
order parameters can be used at the same time, to improve the control of the
selectivity of the various phases and the pathways between them. However, this
must be weighed against the additional computational cost, which grows
exponentially with the number of order parameters.

Quigley and Rodger have described a protocol for deciding which order
parameters to use in \cite{quigley-09a}. Firstly a set of simulations of the
disordered state and any accessible crystalline polymorph are performed and
the equilibrium distributions for the candidate order parameters obtained (see
section \ref{metadextras}). Any sets of parameters for which the distributions
overlap are discarded until the sets remaining describe the known states with
minimum ambiguity. This ensures that the realisable structures are distinct in
the collective space of the order parameters. However this approach does not
guarantee that pathways between states will match those that occur in the
unbiased system, though it does set an upper bound for the corresponding free
energy barrier. An alternative method devised by Peters and Trout
offers a better description of pathways \cite{peters-06a}.

\section{Order Parameters}
\index{metadynamics!order parameters}
\index{metadynamics!collective variables|see{hyperdynamics, order parameters}}

The order parameters available in \D{} are as follows:
\begin{enumerate}
\item Potential energy \cite{donadio-05a};
\item The Q4 and Q6 parameters of Steinhardt {\em et al} \cite{steinhardt-83a};
\item The $\zeta$ tetrahedral parameter of Chau and Hardwick \cite{chau-98a};
\end{enumerate}
These order parameters are described below.

\subsection{Potential Energy as an Order Parameter}
\index{metadynamics!potential energy}
\index{metadynamics!order parameters}
The use of potential energy as an order parameter was pioneered by Donadio
{\em et al} \cite{donadio-05a}. It is self evident that the configuration
energy is a well behaved function that takes on distinct values for different
structures. It has the additional advantage that it requires no additional
computation time, since it is normally calculated automatically during any
molecular dynamics simulation. It is also straightforward to calculate the
associated biasing forces and stress tensor contributions:
\begin{eqnarray}
\vek{f}_{i} &\rightarrow& \vek{f}_{i} (1+\frac{\partial V}{\partial U}) \\
\vek{\vek{\sigma}} &\rightarrow& \vek{\vek{\sigma}} (1+\frac{\partial V}{\partial U})
\end{eqnarray}

In addition to using potential energy as a {\em global} parameter Quigley and
Rodger advocate its use as a {\em local} parameter \cite{quigley-09a}, which
pertains to a specific subset of atoms in the system - namely those that form
the central atoms in the definitions of the Steinhardt or tetrahedral order
parameters discussed in the following sections. This allows the use of
potential energy in association with these order parameters. This approach has
the advantage that it allows the user to drive structural changes in parts of
the system that are of greatest interest and not (say) the solvent or
substrate. In implementing this, a corresponding calculation of the local
potential energy and stress tensor needs to be added to each DL\_POLY force
routine. It turns out that it is not practical to do this in all of DL\_POLY's
force routines, in particular those that determine many-body forces e.g.
Tersoff and metal potentials do not have this capability. A similar omission
occurs with the reciprocal space term of the Ewald sum.

\subsection{Steinhardt Order Parameters}
\index{metadynamics!Steinhardt parameters}
\index{metadynamics!order parameters}
The parameters of Steinhardt, Nelson and Ronchetti \cite{steinhardt-83a}
employ spherical harmonics to describe the local order of atoms of type
$\beta$ surrounding an atom of type $\alpha$, thus:
\begin{equation}
Q_{\ell}^{\alpha\beta}=\left [\frac{4\pi}{2\ell+1} \sum_{m=-\ell}^{\ell}
\left | \frac{1}{N_{c}N_{\alpha}}\bar{Q}_{\ell m}^{\alpha\beta} \right |^{2}
\right ]^{1/2} \label{stein1}
\end{equation}
where
\begin{equation}
\bar{Q}_{\ell m}^{\alpha\beta}=\sum_{b=1}^{N_{b}} f_{c}(r_{b})Y_{\ell
  m}(\theta_{b},\phi_{b}).\label{stein2}
\end{equation}
The summation in equation (\ref{stein2}) runs over all $N_{b}$ atoms of type
$\beta$ within a prescribed cutoff surrounding an atom of type $\alpha$ and
$r_{b}$ represents the scalar distance between the $\alpha$ and $\beta$ atoms.
The function $f_{c}(r_{b})$ is a switching function that sets the cutoff range
at the required separation in a continuous (and therefore differentiable)
manner. It has the form:
\begin{equation}
f_{c}(r) = \left\{ \begin{array} {l@{\qquad:\qquad}l}
1 & r \le r_{1} \\
\frac{1}{2} \left \{\cos \left
    [\frac{\pi~(r-r_{1})}{(r_{2}-r_{1})}\right]+1\right \} & r_{1} < r \le r_{2} \\
0 & r > r_{2}
\end{array} \right.\label{stein3}
\end{equation}
The parameters $r_{1}$ and $r_{2}$ define a range over which the $\beta$ atoms
gradually cease to count towards the overall sum.  Note that the numbers
$N_{c}$ and $N_{b}$ in the above formulas are formally expected to be the same
in a perfect crystal.  However, while $N_{c}$ remains fixed, $N_{b}$ may
fluctuate according to circumstance. (In fact the switching function replaces
the strict cut off in the original definition by Steinhardt {\em et al} in
which $N_{c}$ would be equivalent to $\sum_{b=1}^{N_{b}}f_{c}(r_{b})$ rather
than a constant.)  Quigley and Rodger also note that order parameter is not
scale invariant between systems of different numbers of atoms
\cite{quigley-09a}, however this does not matter for simulation where the
numbers are fixed.  The
spherical harmonic parameter $\ell$ is confined to the values $4$ and $6$ in
the \D{} implementation, giving the order parameters $Q_{4}^{\alpha\beta}$ and
$Q_{6}^{\alpha\beta}$.

The forces arising from the Steinhardt parameters are given by:
\begin{eqnarray}
\vek{f}_{ij}&=&-\hat{r}_{ij}\frac{\partial V}{\partial Q_{\ell}^{\alpha\beta}}
\frac{1}{Q_{\ell}^{\alpha\beta}}\frac{4\pi}{2\ell+1}\left
  (\frac{1}{N_{c}N_{\alpha}} \right )^{2}\times \nonumber \\
 & & \sum_{m=-\ell}^{\ell} \left \{\Re (\bar{Q}_{\ell m}^{\alpha\beta})
\frac{d}{dr_{ij}}[f_{c}(r_{ij}) \Re (Y_{\ell m}(\theta_{ij},\phi_{ij}))]
\right . \nonumber \\
 & & \left . +\Im (\bar{Q}_{\ell m}^{\alpha\beta})
\frac{d}{dr_{ij}}[f_{c}(r_{ij})
\Im (Y_{\ell m}(\theta_{ij},\phi_{ij}))]\right \}. \label{stein4}
\end{eqnarray}
where $\Re$ and $\Im$ indicate the {\em Real} and {\em Imaginary} parts of
complex quantities.

The stress tensor contributions arising from these forces are given by
\begin{equation}
\sigma_{\alpha\beta} \rightarrow
\sigma_{\alpha\beta}-f_{ij}^{\alpha}r_{ij}^{\beta}.
\end{equation}

\subsection{Tetrahedral Order Parameters}
\index{metadynamics!Tetrahedral parameters}
\index{metadynamics!order parameters}
The form of the tetrahedral order parameter in \D{} is that of Chau and
Hardwick \cite{chau-98a}, which quantify the degree to which atoms
surrounding a chosen atom are arranged tetrahedrally. When the chosen atom and
its surrounding neighbours are of the same type ($\alpha$) the parameter is
defined by formula
\begin{equation}
\zeta_{\alpha}=\frac{1}{N_{c}N_{\alpha}}\sum_{i=1}^{N_{\alpha}}
\sum_{j\ne i}^{N_{\alpha}}\sum_{k>j}^{N_{\alpha}}
f_{c}(r_{ij})f_{c}(r_{ik})(cos \theta_{jik}+1/3)^{2}, \label{tetra1}
\end{equation}
where indices $i$, $j$ and $k$ run up to $N_{\alpha}$ atoms of type
$\alpha$. Integer $N_{c}$ and function $f_{c}(r)$ are as for the Steinhardt
parameters (i.e. $f_{c}$ once again replaces a fixed cut off and $N_{c}$ is a
fixed constant). This order parameter is maximal for tetrahedral atomic
arrangements. The atomic forces that arise from this order parameter can be
expressed in terms of pair forces between atoms $i$ and $j$ and between $i$
and $k$ which are given by
\begin{eqnarray}
\vek{f}_{ij}&=&-\frac{\partial V}{\partial \zeta_{\alpha}} \left \{
  \frac{2}{r_{ij}}f_{c}(r_{ij})f_{c}(r_{ik})(cos \theta_{jik}+1/3)
(\hat{r}_{ik}-\hat{r}_{ij}cos \theta_{jik}) \right . \nonumber \\
& & +\left .(cos \theta_{jik}+1/3)^{2}
\frac{df_{c}(r_{ij})}{dr_{ij}}f_{c}(r_{ik})\hat{r}_{ij} \right \}
\label{tetra2}\\
\vek{f}_{ik}&=&-\frac{\partial V}{\partial \zeta_{\alpha}} \left \{
  \frac{2}{r_{ik}}f_{c}(r_{ij})f_{c}(r_{ik})(cos \theta_{jik}+1/3)
(\hat{r}_{ij}-\hat{r}_{ik}cos \theta_{jik}) \right . \nonumber \\
& & +\left . (cos \theta_{jik}+1/3)^{2}
\frac{df_{c}(r_{ik})}{dr_{ik}}f_{c}(r_{ij})\hat{r}_{ik} \right \}.
\label{tetra3}
\end{eqnarray}

The stress tensor contributions can be described in terms of these forces:
\begin{equation}
\sigma_{\alpha\beta} \rightarrow \sigma_{\alpha\beta}
-f_{ik}^{\alpha}r_{ik}^{\beta}-f_{ij}^{\alpha}r_{ij}^{\beta}.\label{tetra4}
\end{equation}

\subsection{Order Parameter Scaling}
\index{metadynamics!order parameter scaling}
\label{metascaling}
The order parameter vector $\vek{s}^{M}$ consists of an ordered set of
different order parameters and it is not generally the case that all of them
return numbers of the same order of magnitude. This is particularly true for
the potential energy. It is therefore sensible that when using the order
parameters collectively to define the state of a system, that they should be
scaled to give numbers of simular magnitudes. So when specifying order
parameters to define the metadynamics \D{} allows the user to include a scale
factor in the definition. This appears in the STEINHARDT and ZETA data files
described in the following section.

\section{Running Metadynamics Simulations}
\index{metadynamics!running simulations}

The recommended procedure for running metadynamics with \D{} is as follows.
\begin{enumerate}
\item Scope out the appropriate choices of order parameters for your system
  following the method of Quigley and Rodger outlined above in section
  (\ref{metadtheory}) and in \cite{quigley-09a}. You should use one of the
  (equilibrated) REVCON files as the starting configuration for your
  metadynamics study.
\item Decide a suitable interval $\tau_{G}$ for depositing the Gaussians
  (\ref{depgauss}) and an appropriate Gaussian height $w$ and width $\delta h$
  to ensure accuracy for the free energy calculation. See \cite{laio-05a} and
  \cite{quigley-09a} for details. Along with this the user must also choose a
  Gaussian deposition/convergence scheme (see section \ref{metadextras}).
\item Set up the metadynamics option in the CONTROL file as follows:
\begin{enumerate}
\item Set the metadynamics directive:\newline {\bf metadynamics} \newline or
  use the equivalent directive:\newline {\bf metafreeze} \newline Then
  enter the metadynamics control variables one per record as indicated below.
  Comment records may be inserted if the first character is the hash symbol
  (\#) or the ampersand (\&).
\item Set the number of order parameters to be used ({\em ncolvar}):\newline
  {\bf ncolvar} $n$ \newline where $n$ is an integer.
\item If Steinhardt order parameters are required, activate the option with
  the directive: \newline {\bf lstein}
\item If tetrahedral order parameters are required, activate the option with
  the directive: \newline {\bf ltet}
\item If the global potential energy order parameter is required, activate the
  the option with the directive: \newline {\bf lglobpe}
\item If local potential energy order parameters are required, activate the
  option with the directive (note that global and local potential energy are
  mutually exclusive options): \newline {\bf llocpe}
\item Set the scale factor for the global potential energy order parameter:
  \newline {\bf globpe\_scale} $f$ \newline where $f$ is a real number.
\item Set the scale factor for the local potential energy order parameter:
  \newline {\bf locpe\_scale} $f$ \newline where $f$ is a real number.
\item Set the number of Steinhardt $Q_{4}$ parameters required:
  \newline {\bf nq4} $n$ \newline where $n$ is an integer.
\item Set the number of Steinhardt $Q_{6}$ parameters required:
  \newline {\bf nq6} $n$ \newline where $n$ is an integer.
\item Set the number of tetrahedral $\zeta$ parameters required:
  \newline {\bf ntet} $n$ \newline where $n$ is an integer.
\item Set the Gaussian potential deposition interval (in units of time steps): \newline
  {\bf meta\_step\_int} $n$ \newline where $n$ is an integer.
\item Set the height ($w$) of the Gaussian potentials (in units of $k_{B}T$,
  where $T$ is the simulation temperature): \newline
  {\bf ref\_W\_aug} $f$ \newline where $f$ is a real number.
\item Set the Gaussian potential width parameter: \newline
  {\bf h\_aug} $f$ \newline where $f$ is a real number.
\item Set the Gaussian control key: \newline
  {\bf hkey} $n$ \newline where $n$ is an integer. See section
  (\ref{metadextras}) for guidance.
\item Set the control parameter for well-tempered dynamics
  (required when {\bf hkey} is 2): \newline
  {\bf wt\_Dt} $f$ \newline where $f$ is a real number.
\item Close the metadynamics specification: \newline
  {\bf endmet}
\end{enumerate}
\item Set other CONTROL file directives as follow:
\begin{enumerate}
\item Select the {\bf restart noscale} option if the CONFIG file was
      pre-equilibrated, otherwise leave out the {\bf restart} keyword
      altogether.
\item Set the length of the simulation required ({\bf steps}) and the
      equilibration period ({\bf equil}) (both in time steps). The
      equilibration can be short or absent if the system was pre-equilibrated.
\item You {\bf must} select one of the Nos\'{e}-Hoover NVT, NPT or
      N$\sigma$T ensembles.  Metadynamics is only available for one of these
      options. The program automatically defaults to velocity Verlet
      integration, if you use shell model electrostatics.
\item If you wish to follow the structural changes, set the {\bf
      trajectory} option in the CONTROL file. This will produce a HISTORY file
      you can view or analyse later.
\item Set the remaining CONTROL keywords as for a normal molecular dynamics
  simulation.
\end{enumerate}
\item Prepare, if required, the file STEINHARDT, which defines the
  control variables for the Steinhardt order parameters. The file
  specification is as follows.  \index{metadynamics!STEINHARDT file}
\begin{enumerate}
\item The file contains data for both  $Q_{4}$ and $Q_{6}$ order
  parameters.
\item The records describing the $Q_{4}$ entries appear first. There is
  one information record followed by $nq4$ data records. ($nq4$ is defined in
  item 3(i) above.) No $Q_{4}$ entries appear if $nq4$ is zero. After this,
  the records for the  $Q_{6}$ parameters appear.  There is one information
  record, followed by $nq6$ data records. ($nq6$ is defined in item
  3(j) above.) The records are free format and the content of
  the information records is ignored by the program.
\item Each data record (for both $Q_{4}$ and $Q_{6}$) consists of
  (in order)\newline
  - The name of the atom type $\alpha$ (max. 8 characters); \newline
  - The name of the atom type $\beta$ (max. 8 characters); \newline
  - The control parameter $r_{1}$ for the function $f_{c}(r)$ in
    equation (\ref{stein3}) (real); \newline
  - The control parameter $r_{2}$ for the function $f_{c}(r)$ in
    equation (\ref{stein3}) (real); \newline
  - The scale factor for the order parameter (real); \newline
  - The number $N_{c}$ of expected atoms of type $\beta$ around the $\alpha$
  atom (integer).
\end{enumerate}
\item Prepare, if required, the file ZETA, which defines the
  control variables for the tetrahedral order parameters. The file
  specification is as follows. \index{metadynamics!ZETA file}
\begin{enumerate}
\item The file consists of 1 information record followed by $ntet$ data
  records. The data records are free format and the content of the information
  record is ignored by the program.
\item Each data record consists of (in order)\newline
  - The name of the atom type $\alpha$ (max. 8 characters); \newline
  - The cutoff parameter $r_{1}$ for the function $f_{c}(r)$ (real); \newline
  - The cutoff parameter $r_{2}$ for the function $f_{c}(r)$ (real); \newline
  - The scale factor for the order parameter (real); \newline
  - The number $N_{c}$ of expected atoms of type $\alpha$ around each
    $\alpha$ atom (integer).
\end{enumerate}
\item Run the metadynamics simulation. This will perform a simulation at the
   temperature and pressure requested. When the simulation ends, proceed
   as follows.
\begin{enumerate}
\item Check the OUTPUT file. Make sure the simulation behaved sensibly and
  terminated cleanly. If the required number of time steps has not been
  reached, the simulation can be restarted from the REVCON and REVIVE files
  (renaming them as CONFIG and REVOLD for the purpose) and
  setting the directive {\bf restart} (with no qualifier) in the CONTROL
  file.
\item Use the DL\_POLY Java GUI to plot the system energy and temperature
  for the whole of the simulation and make sure no key variables misbehave.
\item The HISTORY file, if requested, contains the trajectory of the
  system. This may be viewed, or otherwise investigated, by appropriate viewing
  or analysis software.
\item The METADYNAMICS file produced by the run contains the data describing
  the evolution of the time dependent bias potential (equation \ref{metah}).
  \index{metadynamics!METADYNAMICS file}
  This file consists of a series of records, the content of each is:\newline
  - The {\em meta-step} (current time step number divided by the Gaussian
  deposition interval {\em meta\_step\_int}). Format (i8).\newline
  - All {\em ncolvar} order parameters selected. Format {\em ncolvar}*(e15.6).\newline
  - The height of the deposited Gaussian divided by $k_{b}T$. Format (e15.6).
\end{enumerate}

\end{enumerate}

\subsection{Additional Considerations}
\label{metadextras}
\begin{enumerate}
\item {\bf Choosing the Gaussian convergence scheme}. \D{} offers three
  different schemes for handling the addition of the Gaussians to the
  bias potential. Further details on these schemes can be obtained from
  Quigley and Rodger \cite{quigley-09a}.
\begin{enumerate}
\item {\bf Simple addition}. Gaussians with fixed height and width parameters
  are simply added to the bias
  potential. This option is selected by setting {\bf hkey=0} in the
  metadynamics section of the CONTROL file.
\item {\bf Wang-Landau recursion}. Starting from a given Gaussian height a
  histogram is accumulated with each Gaussian addition recording the visits to
  each square of the order parameter space. Once this histogram is
  approximately (say 80\%) flat, the Gaussian height is halved, the histogram
  is reset to zero and then the process continues until a higher degree of
  flatness has been achieved, and so on. The procedure is meant to
  ensure that the added Gaussians make progressively finer contributions as
  convergence is approached. Set {\bf hkey=1} for this option. (Note this
  option has been implemented only for the case where {\bf ncolvar=1}.)
\item {\bf Well-tempered dynamics}. The well-tempered scheme uses a maximum
  energy criterion. A threshold energy $V_{max}$ is set above the largest
  expected energy barrier and at each step the Gaussian deposition height is
  given by \[ w=w_{0}exp[-V_{aug}(\vek{s}^{M})/V_{max}] \] where $V_{aug}$ is
  the current value of the bias energy. The parameters
  $w_{0}$ and $V_{max}$ are defined by the input directives {\bf ref\_W\_aug}
  and {\bf wt\_Dt} in the CONTROL file (see above). Set {\bf hkey=2} for
  this option.
\end{enumerate}
\item {\bf Defining the switching function $f_{c}(r)$}\newline
  The switching function is determined by the parameters $r_{1}$ and $r_{2}$
  in formula (\ref{stein3}). These must be chosen so that $r_{2}$ absolutely
  excludes near-neighbouring atoms that are not intended to be considered part
  of the sum in equation (\ref{stein1}) or (\ref{tetra1}). $r_{1}$ should not
  be so short that is sometimes does not include atoms that should be fully
  counted. The range $r_{1}\rightarrow r_{2}$ should be set to correspond
  to the minimum in the appropriate pair correlation functions in the relevant
  system states. This choice minimises spurious forces that can arise from
  order parameters that have different ranges \cite{quigley-09a}.
\item {\bf Using \D{} to help select appropriate order parameters}\newline
  \index{metadynamics!order parameters}
  Section (\ref{metadtheory}) outlined a protocol for deciding suitable order
  parameters for a particular metadynamics study, which required the
  construction of distribution functions for candidate order parameters. These
  may be obtained from \D{} by perform simulations of the relevant system
  states with the {\bf metadynamics} directive set in the CONTROL file, but
  with the Gaussian accumulation disabled by setting the Gaussian height
  parameter {\bf ref\_W\_aug} to zero. The resultant dynamics
  will be {\em time independent} and the METADYNAMICS file will tabulate
  values of the order parameters at regular intervals. It remains then to
  construct histograms of these parameters to determine the degree of overlap
  between them as required.
\item {\bf Choosing order parameter scaling factors}\newline
  The widths of the histograms for the order parameter distributions
  described in the previous paragraph should also be used to set the
  appropriate order parameter scaling factors referred to in section
  (\ref{metascaling}).
\item {\bf Deciding the simulation length}\newline
  Deciding that the metadynamics simulation has been long enough
  is a matter of judging whether the system is diffusing like a random walk in
  the space of the order parameters i.e. that it is sampling all the
  available parameter space. This is somewhat easier in the cases of
  Wang-Landau recursion and well-tempered dynamics as it is indicated by the
  parameter {\em w} (the Gaussian height) becoming relatively small. In
  general a degree of experience in the technique is required to make a good
  judgement.
\item {\bf Contra-indications}\newline
  A useful point to note is that if a simulation does not reach a state
  where transitions between minima occur rapidly without residual hysteresis,
  then by implication the original choice of order parameters was poor.
\end{enumerate}

\subsection{Analysing the Metadynamics Results}

Analysis of the results of a metadynamics simulation can take a number of
forms, some of which are outlined here.

\begin{enumerate}
\item {\bf Determination of free energy} \newline The information in the
  METADYNAMCS file is sufficient to define the system free energy surface
  through equation \ref{metafree}. The free energy is a function of the $M$
  order parameters in the vector $s^{M}(\vek{r}^{N})$. This information can be
  used to determine the free energy of activation and free energy differences
  between states in the following manner. Firstly the free energy is projected
  down to a smaller subset of order parameters (usually about 2) by
  integrating $exp(-F/k_{B}T)$ over the other order parameters
  and then Boltzman-inverting. (This is an expensive operation if $M>3$).
  Once the free energy is mapped onto fewer dimensions the free energy
  barrier heights and free energy differences can be read off directly.
\item {\bf Following the system trajectory} \newline It is often useful to
  track the trajectory of the system in the space of the order parameters to
  see how well the simulation is exploring that space. For this purpose is is
  possible to plot the contents of the METADYNAMICS file graphically in a
  selection of 2D sections. Simple graphics are generally sufficient for this
  purpose. Alternatively, the HISTORY file may be viewed as a movie, using
  packages such as VMD \cite{VMD} to show the transformations that occur.
\end{enumerate}