\chapter{Hyperspectral}

%TODO translate using  Hyperspectral_tr.txt
An hyperspectral image contains a collection of spectral pixels or
equivalently, a collection of spectral bands.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Cube_HPX.eps}
  \itkcaption[Hyperspectral cube]{Illustration of an hyperspectral cube, spectral pixel and a spectral layer.}
  \label{fig:cube}
\end{figure}

An hyperspectral system \ref{fig:cube} acquired
radiance, each pixel contains fine spectral information
fine that depends of:

\begin{itemize}
\item{Spectrum of the light source (in practice, the
sun) and atmospheric disturbances.}
\item{Spectral responses
of different materials in the overlap zone and of the nature of the mixture.}
\end{itemize}

Preliminary treatments allow to perform
atmospheric correction for estimating a reflectance cube
spectral by subtraction of information extrinsic of the
scene (see also \ref{secAtmosphericCorrections}).

\section{Unmixing}

\subsection{Linear mixing model}

Reflectance information depends only of the materials spectral responses in the scene. When the mixture between
materials is macroscopic, the linear mixing model of spectra
is generally admitted. In this case, the image typically looks like this:

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Linear_Unmixing_HPX.eps}
  \itkcaption[Linear mixing model]{Zone which verify the LM model.}
  \label{fig:linear_unmixing}
\end{figure}

We notice \ref{fig:linear_unmixing} the
presence of pure pixels, and pixel-blending. The LMM acknowledges that
reflectance spectrum associated with each pixel is a linear combination of pure materials in the recovery area, commonly known as ``''endmembers. This is illustrated in \ref{fig:decomp_mml}

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Decomposition_MML_HPX.eps}
  \itkcaption[Decomposition of the LMM]{Decomposition of a hyperspectral cube according to the LMM.}
  \label{fig:decomp_mml}
\end{figure}
The `` left'' term represents the different spectral bands of
data cube. The `` right'' term represents a `` product''
between the reflectance spectra of endmembers and their respective abundances. Abundance band of endmembers is
image grayscale between $0$ and $1$. The pixel i of the
abundance band of endmember j is $s_ {ji}$. This value is the
abundance of endmember j in the pixel i. Under certain conditions
~\cite{Huck2009}, this value can be interpreted as the ratio
surface of the material in the overlap zone (\ref{fig:linear_unmixing}). In
practice, one can reasonably expect that:

\begin{itemize}
\item{a limit number
of pure materials compose the scene.}
\item{the scene contains pure pixels if the spatial resolution is sufficient and
do not necessarily contains them otherwise.}
\end{itemize}

Many techniques of unmixing in hyperspectral image analysis
are based on geometric approach where each pixel is seen as a spectral
vector of L (number of spectral bands). The
spectral bands can then be written as vectors.

\begin{figure}[h]
\begin{tikzpicture}
\pgfmathsetmacro{\cubex}{2}
\pgfmathsetmacro{\cubey}{2}
\pgfmathsetmacro{\cubez}{2}
\draw[black,fill=white] (0,0,0) -- ++(-\cubex,0,0) -- ++(0,-\cubey,0) -- ++(\cubex,0,0) -- cycle;
\draw[black,fill=white] (0,0,0) -- ++(0,0,-\cubez) -- ++(0,-\cubey,0) -- ++(0,0,\cubez) -- cycle;
\draw[black,fill=white] (0,0,0) -- ++(-\cubex,0,0) -- ++(0,0,-\cubez) -- ++(\cubex,0,0) -- cycle;
\node[draw] (R) at (2,0.5) {R};
\draw[->] (2,0.5) -- (R.west);
\end{tikzpicture}
\itkcaption[Hyperspectral cube vectorization]{``Vectorization'' of hyperspectral cube. The spectral pixels
are stored in the columns of the matrix R and, in
equivalently, the spectral bands are assigned to the lines of R.}
\label{fig:mml}
\end{figure}
%% Figure 5: "Vectorisation" du cube hyperspectral. Les pixels spectraux
%% sont ranges dans les colonnes de la matrice R et, de maniere
%% equivalente, les bandes spectrales sont rangees dans les lignes de R.

By deduction of \ref{fig:decomp_mml} et de \ref{fig:mml}, the LMM needs to decompose R as:
\begin{center}
$R= A.S + N = X + N$
\end{center}

J is the number of endmembers and the number of spectral pixels I:

\begin{itemize}
\item{J columns of the matrix A contain the spectra of endmembers.}
\item{J rows of the matrix S contain the abundance maps
vectorized, we call the columns vectors of abundances of
matrix S.}
\item{The matrix N, of dimensions $LXI$ is a matrix of additive noise.}
\end{itemize}

The unmixing problem is to estimate matrices A and S
from R or possibly of \"X, an estimate of the denoised matrix signal.

Several physical constraints can be taken into
account to solve the unmixing problem:

\begin{itemize}
\item{C1: reflectance spectra are positives (non negative matrix A).}
\item{C2: positivity abundances are positive (non-negative matrix S).}
\item{C3: additivity
abundance (the sum of the coefficients of each column of the matrix S is 1).}
\item{Independence between the ``algebraic spectral vectors'' of endmembers associated with linearity and mixtures of spectra, so that the simplex property described in paragraph below.}
\end{itemize}

\subsection{Simplex}
Recent unmixing algorithms based on the ``property of
simplex.'' In a vector space of dimension $J-1$, we can
associate to J vectors algebrically independent, J points which define the vertices of a J-simplex \ref{fig:simplex}.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Simplex_HPX.eps}
  \itkcaption[Simplex]{Illustration of a 2-simplex, a 3-simplex and a 4-simplex.}
  \label{fig:simplex}
\end{figure}

Hyperspectral cube of L bands, based on J endmembers,
may be contained in a affine subspace of dimension $J-1$.
A relevant subspace in the sense of signal-to-noise ratio is
generally obtained by Principal Component Analysis (PCA) \ref{sec:RoadExtraction}.
In practice, the $J-1$ eigenvectors associated with highest values are
the columns of a projection matrix V of dimension $(Lx(J-1))$. Reduced
data Z, of dimensions $((J-1)xI)$ are obtained by the operation:
$Z=V^{T}(\tilde{R})$

where each column of $\tilde{R}$ where the average spectrum is subtracted,
generally estimated under maximum likelihood. In the
subspace carrying the column-vectors Z, endmembers spectra oare associated to the top of the simplex. If the noise is
negligible, the simplex circumscribed reduced data.

This property shows that the endmembers research are the vertices of a
simplex that circumscribes the data. However, an infinity of different
simplices can identify the same data set. In fact, the problem of
unmixing generally does not have a unique solution. This degeneration can also be
demonstrated by the formalism of the non-negative
matrices factorization \cite{Huck2010a}.

It is therefore necessary to choose the most physically
relevant solution. All unmixing techniques based on this simplex property admit that the best solution is defined by the allowable
minimum volume simplex, or the notion of volume is extended to all
finite dimensions (possibly different from 3).

\subsection{State of the art unmixing algorithms selection}
The more recent linear unmixing algorithms exploit the
simplex property. It is possible to classify these methods into several families:

\subsubsection{Family 1}
A first family of unmixing algorithms are based on research of the
endmembers ``among'' data. This means that a minimum of one pure pixel must
be associated with each endmembers. If this hypothesis is not
verified, it will produce an estimation error of the endmembers spectra. The historical advantage of these algorithms are their low
algorithmic complexity. The three best known are :
\begin{itemize}
\item PPI (Pixel Purity Index)
\item VCA NFINDER (Vertex Component Analysis) \cite{Nascimento2005}
\end{itemize}

In addition to its success recognized by the
community and very competitive algorithmic complexity, the endmembers estimation is unbiased in absence of noise (and when there are pure pixels).

\paragraph{VCA}
The VCA algorithm is systematically
used to initialize various studied algorithms (except
MVES, based on a different initialization).

Important elements on the operation of VCA:
\begin{itemize}
\item {The VCA algorithm
is based on iterative projections of the data orthogonal to
the subspace already held by the endmembers.}
\item {Biaised when degree of purity is lower than 1.}
\end{itemize}

\subsubsection{Family 2}
A second family is composed of algorithms which are looking for the simplex of minimum
volume circumscribing the data. Phase initialization consists in determining
an initial simplex any circumscribing the data. Then, a numerical
optimization scheme minimizes a functional, increasing function of the
volume generalized, itself dependent of estimated endmembersin the
current iteration. The optimization scheme is constrained by the fact that the
data have remained on the simplex and possibly by constraints C1, C2 and C3.

Existing algorithms are:
\begin{itemize}
\item {MVSA (Minimum Volume Simplex
Analysis) \cite{Li2008}.}
\item { MVES (Minimum Volume Enclosing Simplex)
[Chan2009].}
\item {SISAL (Simplex Identification via Split Augmented
Lagrangian) \cite{Dias2009}.}
\end{itemize}

Main differences between algorithms are:
\begin{itemize}
\item {The numerical optimization scheme.}
\item {The way
constraints are taken into account.}

\end{itemize}
These issues impact the computational complexity and the precision of the estimation.

\paragraph{MVSA \cite{Li2008} }
MVSA key points:
\begin{itemize}
\item {Initialization by VCA.}
\item {All spectral pixels included in the simplex estimate
  (approximatively) by VCA does not impact the constraint of  data to belong to the researched simplex, they are
  simply delete the data used in the minimization of the
  simplex to reduce the algorithmic complexity.}
\item {The highly developed optimization technique uses
  sequential quadratic programming (Sequential Quadratic
  Programming - SQP) and more specifically of the category
  ``Quasi-Newton''under constraint.}
\end{itemize}

\paragraph{MVES \cite{Chan2009}}
 MVES key points:
\begin{itemize}
\item {Initialization by non-trivial
  iterative method (LP Linear Programming for Linear
  Programming), different from VCA.}
\item {Resolution of problem by LP.}
\end{itemize}

\paragraph{SISAL \cite{Dias2009}}
SISAL key points:
\begin{itemize}
\item {Initialisation by VCA.}
\item {Selection of similar spectral pixels as MVSA to reduce computational complexity.}
\item {Advanced optimization technique
combining multiple features.}
\begin{itemize}
\item {Decomposition of the non convex problem
in convex set of problems.}
\item {Development of a
specific method of separation of variables for considering
Lagrangian increases with a good design properties.}
\end{itemize}
\end{itemize}

\subsubsection{Family 3}
Non negative matrix factorization algorithms (NMF for Non-negative Matrix Factorization). The purpose of this
branch of applied mathematics is to factor a non-negative matrix, X in our case, into a product of non negative matrices: AS by minimizing a distance between X and AS and
with an adapted regularization to lift the degeneration in an appropriate manner adapted to the physical problems associated with unmixing.

\paragraph{MDMD-NMF \cite{Huck2010b}}
Key points of MDMD-NMF:
\begin{itemize}
\item {VCA initialization.}
\item {Minimizing the norm of standard
Frobenius with a spectral regularization and a regularization ``Space''(the matrix of abundances).}
\begin{itemize}
\item {Minimum spectral
Dispersion: the spectral regularization encourages the  variance of
coefficients of a spectrum of endmembers to be low.}
\item {Maximum spatial dispersion: the spatial regularization
  encourages the vector of abundances to occupy all the admissible parts (more information in \cite{Huck2009}). it
  presents a certain analogy with the minimum volume constraint.}
\end{itemize}
\end{itemize}

\subsubsection{Further remarks}
Algorithms of families 1 and 2 estimate `` only'' the spectra of
endmembers. The estimated abundance maps held
\textit{a posteri} and requires the application of an algorithm like Fully
Constrained Least Square (FCLS)\doxygen{otb}{FCLSUnmixingImageFilter} \cite{Heinz2001}. VSS includes an specific algorithm
for the estimation of the abundances.
The unmixing is a general non-convex problem,
which explains the importance of the initialization of algorithms.

Overview of algorithms and related physical constraints:

\begin{center}
   \begin{tabular}{ | p{1.5cm} | | p{2cm} | p{2cm} | p{2cm} | p{2cm} | p{2cm} | }
     \hline
     & VCA & MVSA & MVES & SISAL & MDMD \\ \hline
     C1 $(A>0)$ & mute & mute & mute & mute & hard \\ \hline
     C2 $(S>0)$ & mute & hard & hard & soft & hard \\ \hline
     C3 (additivity) & Mute (FCLS) & hard & hard & hard & soft \\ \hline
     simplex & Endmembers in the data & Circumscribed to data &
     Circumscribed to data & Circumscribed to data &
     Indirectly by ``space'' regularization \\ \hline
   \end{tabular}
 \end{center}

\subsubsection{Basic hyperspectral unmixing example}
\input{HyperspectralUnmixingExample}

\section{Dimensionality reduction}

Please refer to chapter~\ref{chap:dimred}, page~\pageref{chap:dimred} for a presentation of dimension reduction methods available in OTB.

\section{Anomaly detection}
By definition, an anomaly in a scene is an element that does not
expect to find. The unusual element is likely different from its
environment and its presence is in the minority scene. Typically, a
vehicle in natural environment, a rock in a field, a wooden hut in a
forest are anomalies that can be desired to detect using a
hyperspectral imaging. This implies that the spectral response of the
anomaly can be discriminated in the spectral response of ``background
clutter''. This also implies that the pixels associated to anomalies,
the anomalous pixels are sufficiently rare and/or punctual to be
considered as such. These properties can be viewed as spectral and
spatial hypotheses on which the techniques of detection anomalies in
hyperspectral images rely on.

Literature on hyperspectral imaging
generally distinguishes target detection and detection
anomalies:

\begin{itemize}
\item {We speak of detection of targets when the spectral response
element of the research is used as input to
the detection algorithm. This is an \textit{a priori} information that
can, in theory, allow to construct algorithms with very high
detection score, such as for example the Adaptive Matched Filter (AMF) or Adaptive
Cosine/Coherence Estimator (ACE). Nevertheless,
thin enough knowledge of the researched spectrum is a difficult
information to hold in practice, leading the use of anomaly detection algorithms.}

\item {We speak of detection of anomalies (or outliers) when the spectrum
  of the unusual element is not required by the algorithm. For this reason, we often associate the term
  ``Unsupervised'' detection. Nevertheless, these algorithms depend
  generally of ``structural'' parameters. For example, in the case
  of detection on a sliding window, selecting the right
  dimensions is based on an \textit{a priori} knowledge of the anomalies size.
  We focus here on algorithms for \bf{anomaly detection}.}

\end{itemize}

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Anomaly_HYP.eps}
  \itkcaption[Concept of detection]{Notions on detection: ground truth mask, map detection, face detection.}
  \label{fig:anomaly_hyp}
\end{figure}

In \ref{fig:anomaly_hyp}, we introduce some notions that will be useful later. Anomaly detection algorithms have an image as input and consider a map
serving as a detection tool for making a decision. A
adaptive thresholding provides a estimated mask detection that
we hope is the most similar possible as the ground truth mask,
unknown in practice.

Two approaches dominate the state of the art in anomaly detection
in hyperspectral images. Methods which use Pursuit
Projection (PP) and methods based on a probabilistic modelization of the background and possibly of the target class
with statistical hypothesis tests.
The PP consists in projecting linearly spectral pixels on
vectors wi which optimizes a criterion sensitive to the presence of anomalies
(like Kurtosis). This gives a series of maps of projections
where anomalies contrast very strongly with the background. But the automatic estimation of map  detection have also major difficulties, including:

\begin{itemize}
\item How many projectors should I consider?
\item What (s) projector (s) to choose?
\item How to manage an inhomogeneous background?
\item Detection performances varies with the spatial dimensions of the image (number of samples)
\item These algorithms usually do not have a structure allowing parallelization
\item These algorithms do not generally have a ``constant false alarm rate''.
\end{itemize}

Algorithms described here are RX (presented in the first version
in \cite{Reed1990} and GMRF \cite{Schweizer2001}. They are based on probability models, statistics and hypothesis tests and
sliding window. These approaches consist in answering the following question : ``The pixel (or set of neighboring pixels) tests looks like background pixels?'', by a process of test
statistics. More fully, this approach requires:

\begin{itemize}
\item A model for the background
\item The choice of a statistic test
\item Eventually, a model for the class ``anomaly''
\item Estimators for the parameters \textit{a priori} unknown
\item Hypothesis of homogeneity for a satisfactory compromise between:
\begin{itemize}
\item The number of samples compared to the number of estimate parameters
\item The homogeneity, including the background
\item Algorithmic complexity (although the algorithms considered are
highly parallelizable)
\end{itemize}
\end{itemize}
Principle of RX and GMRF can be resume with \ref{fig:workflow_anomaly}.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Anomaly_Detection_Workflow.eps}
  \itkcaption[Anomaly detection block diagram]{Basic workflow of anomaly detection in hyperspectral images.}
  \label{fig:workflow_anomaly}
\end{figure}

An optional first step is to reduce the size of spectral data
while maintaining information related to anomalies. Then, the spectral pixels
are tested one by oneturn (parallelizable task). The pixel
test works on a sliding window (see \ref{fig:sliding_anomaly}). This
window consists of two sub windows, centered on the pixel test of  dimensions L and LL, with $L<LL$.

\begin{itemize}
\item Pixels belonging to the annulus formed the class ``local background''. The statistical parameters of the background,
  required by statistical tests, are estimated from these
  spectral pixels. One of the challenges is to find a
  compromise on the thickness of the annulus (and therefore the number
  statistical sampling) where:

\begin{itemize}
\item If the annulus is too thick, the local background  is no longer homogeneous
\item If the number of samples is too low, the precision of
  the statistical estimation of the model parameters for the background is too
  low
\end{itemize}
\item Pixels of the central part of the sliding window
  belong to the class ``unknown''. If the test pixel (central pixel) is an anomaly, it is possible that its neighbors
  pixels are also anomalies, which imply to not to choose
  a too small value for L if it is known that anomalies
  can be distribute over several pixels.
\end{itemize}
Once all the parameters are estimated, a statistical test is performed and assigns a
value $\Lambda(i)$ to the tested pixel i. A map of detection is thus
formed.

\begin{figure}[h]
  \centering
  \includegraphics[width=0.7\textwidth]{Sliding_Window_Anomaly_Algo.eps}
  \itkcaption[Sliding window and parameters definitions]{Principle of the sliding window and definitions of
parameters L and LL of the sliding window. RX algorithms and
GMRF, taken herein in the following sections.}
  \label{fig:sliding_anomaly}
\end{figure}

The RX algorithm is available in OTB through the \doxygen{otb}{LocalRxDetectorFilter} class.

%\subsection{Local RX}


%methods

\acresetall

\part{Methods}
\label{sec:methods}

\chapter{Program Package \textsc{Spheres}}
\label{sec:theprogramspheres}

For the projects outlined in
chapters~\ref{sec:fromstickyhardspheretoLJtypeclusters} and
\ref{sec:thegregorynewtonclusters} a program to optimise 3-dimensional
coordinates of cluster structures with respect to a two-body potential was
required to be created. The optimisation routine needed to be flexible in the
way that it would be easy to implement different two-body potentials like
\acf{LJ} or \acf{eLJ}. Furthermore, the program needed to be able to analyse the
results regarding structure, energy and the matrix of second derivatives. The
resulting program was written in \Cpp with standard library version 11 and was
tested to compile in a Linux environment with the \texttt{clang++} compiler
version 6.0. In the following sections, main features are explained and more
important functionalities are outlined in more detail.

\section{Structural Optimisation and Analysis}
\label{sec:generalstructure}

The main functionality of the program is implemented in three different
executables. Each carries out a different task, i.e. optimising given input
structures and output the results, removing duplicate structures from a set of
input structures and finding differences of two sets of input structures. The
programs are set up in such a way that their outputs can be used as input files
for the other program parts. This allows the programs to be used sequentially
while retaining the flexibility of using each executable separately. This
chain-like execution scheme is depicted in
figure~\ref{fig:OptimisationAnalyseScheme}.
%
\begin{figure}[htb]\centering
    \begin{tikzpicture}[node distance = 3cm, scale=0.8, auto, transform shape]%, every node/.style={scale=0.8}]
    \node [cloud] (Input1) {Input structures};
    \node [block, right of=Input1] (Optimise) {Optimise};
    \node [decision, right of=Optimise] (Hessian) {Negative values in Hessian?};
    \node [block, below of=Optimise] (reopt) {Displace geometry};
    \node [cloud, right=1.5cm of Hessian] (print) {Optimised structures};
    \node [block, below of=print] (Analyse) {Analyse $\{r_{ij}\}$};
    \node [cloud, below of=Analyse] (Analysed) {Duplicates removed};
    \node [block, left of=Analysed] (Match) {Match sets};
    \node [cloud, left of=Match] (Input2) {Input structures};
    \node [cloud, below of=Match] (missing) {Missing clusters};

    \path [line] (Input1) -- (Optimise);
    \path [line] (Optimise) -- (Hessian);
    \path [line] (Hessian) |- node {yes} (reopt);
    \path [line] (reopt) -- (Optimise);
    \path [line] (Hessian) -- node {no} (print);
    \path [line] (print) -- (Analyse);
    \path [line] (Analyse) -- (Analysed);
    \path [line] (Analysed) -- (Match);
    \path [line] (Input2) -- (Match);
    \path [line] (Match) -- (missing);

    \node (OptProg) [transform shape=false,draw, color=blue, dashed, thick, %
        fit=(Optimise) (Hessian) (reopt)] {};
    %\node [above=0.1cm of OptProg] {\texttt{Optimize}};

    \node (InputBox1) [draw,color=gray,thick,transform shape=false, fit=(Input1)] {};
    \node (InputBox2) [draw,color=gray,thick,transform shape=false, fit=(Input2)] {};
    \node (OutputBox1) [draw,color=gray,thick,dashed,transform shape=false, fit=(print)] {};
    \node (OutputBox2) [draw,color=gray,thick,dashed,transform shape=false, fit=(Analysed)] {};
    \node (OutputBox3) [draw,color=gray,thick,dashed,transform shape=false, fit=(missing)] {};
    \node [above=0.1cm of InputBox1] {Input 1};
    \node [below=0.1cm of InputBox2] {Input 2};
    \node [above=0.1cm of OutputBox1] {Output 1};
    \node [below=0.1cm of OutputBox2] {Output 2};
    \node [below=0.1cm of OutputBox3] {Output 3};

\end{tikzpicture}
    \caption{Schematic representation of the optimisation, analysis and
    matching procedure. Red circles: program input or output, blue ellipses:
    program executions.}
    \label{fig:OptimisationAnalyseScheme}
\end{figure}
%
A set of input structures (Input 1) is provided to the first executable program
\texttt{optimize}, which optimises the structures and generates Output 1, which
is a list of optimised coordinates in the same format as the input. This can be
used as input for the second executable program \texttt{analyze}, which uses
the distance matrix to identify duplicate structures and outputs a list of
optimised structures with no duplicates (Output 2). In combination with a
secondary set of structures (Input 2) the third executable program
\texttt{match} can be used to compare both sets of structures and output
coordinates of structures that are missing from either set 1 or 2 (Output 3).

\subsection{Optimisation of Input Structures}
\label{sec:optimisationofinputstructures}

The optimisation of input structures with a chosen potential can be carried out
with the program \texttt{optimize}. The coordinates of the input structures have
to be provided in a single file where each line contains three numbers
representing the position of one sphere. Multiple structures can be included by
separating the list of coordinates by single blank lines. The program
automatically moves the input structures to their respective centres of mass,
and rotates them onto the principal axis system. To speed up the optimisation
the environment variable \texttt{OMP\_NUM\_THREADS} can be set to a value
greater than one to enable \ac{OMP} parallelisation. Each optimisation is
carried out in a separate thread, while the number of simultaneously running
threads is equal to \texttt{OMP\_NUM\_THREADS}.

Various parameters of the program run can be controlled using a
\texttt{settings} file in the working directory, which is parsed with help of
the library
\textit{libconfig}.\autocite{Lindner_libconfiglibraryprocessing_2018} For
example, the optimisation can be controlled using the \texttt{opt} group. In
this group, the optimisation model can be chosen with the \texttt{name} setting,
which currently can be set to either \texttt{BFGS} for the \acf{BFGS} algorithm
or \texttt{CG} for the conjugate gradient method. The optimiser uses the machine
learning library dlib\autocite{King_DlibmlMachineLearning_2009} as a back-end,
which allows for the implementation of additional optimisation models with
relative ease. The energy termination criterion is defined using the
\texttt{dforce} setting and the maximum number of steps can be set with the
\texttt{nsteps} setting. An example is given in the box below.
%\begin{figure}[ht]
%
\begin{Verbatim}[frame=single,label=settings file - opt tag]
opt: {
    name = "BFGS";
    dforce = 1e-10;
    nsteps = 100;
};
\end{Verbatim}
%\end{figure}
%
A potential model has to be chosen for the optimisation procedure. Custom models
can be added easily, which will be explained later. The current implementation
allows the selection of three pair potentials, \ac{LJ}, \ac{eLJ} and a \ac{LJ}
potential with a cut-off range. For the \ac{LJ} potential four parameters have
to be given to the \texttt{potential} group as shown in the box below.
%
%\begin{figure}[htb]
\begin{Verbatim}[frame=single,label=settings file - potential tag]
potential: {
    name = "LJ";
    epsilon = 1.0;
    rm =  1.0;
    exp1 = 12.0;
    exp2 = 6.0;
};
\end{Verbatim}
%\end{figure}
%
The \texttt{name} setting enables the \ac{LJ} potential and the four parameters
\texttt{epsilon}, \texttt{rm}, \texttt{exp1} and \texttt{exp2} define the
potential. The values shown in the example are the default values, which the
program will fall back to if no values are provided by the user. The \ac{eLJ}
potential can be chosen by setting the \texttt{name} setting to \texttt{ELJ} and
a range cut-off value can be chosen with the \texttt{RangeLJ} potential. For the
\ac{eLJ} potential the $c_n$ coefficients have to be provided in a separate file
located in the working directory and named \texttt{ext}, where each line
contains two numbers. The first integer represents $n$ and the second floating
point number defines the corresponding coefficient.

After each structural optimisation the program checks if the result is a true
minimum by calculating the Hessian matrix eigenvalues. If this check fails the
eigenvectors of the Hessian matrix are calculated and the non-minimum structure
is displaced in both possible directions according to the eigenvector belonging
to the first negative eigenvalue in the Hessian matrix. The algorithm tries to
re-optimise until there are no negative eigenvalues in the Hessian matrix or
until a maximum number of five re-optimisations is reached.

The resulting structures are printed in the file \texttt{coord} in the same
format as the input. To analyse the optimisation procedure the additional files
\texttt{opt} and \texttt{reopt} are created, which contain the intermediate
coordinates of the optimisation and re-optimisation procedures for each individual
input structure.

\subsection{Removing Duplicate Structures}
\label{sec:analysingresults}

Duplicated geometries in a set of input structures can be identified using the
second program \texttt{analyze}. The input has to be provided in the same format
as for the \texttt{optimize} program. The read-in procedure is equivalent to the
\texttt{optimize} program. The program uses two methods to identify unique
structures, one of which uses the energy of the cluster as a criterion,
therefore the \texttt{potential} group has to be set in the \texttt{settings}
file.

The first method uses four values to uniquely identify a structure. Those are
the values for energy and the three eigenvalues of the moment of inertia tensor.
This sorting procedure uses a \verb|map| container from the \Cpp standard
library, which is an implementation of a binary search tree. The \verb|map|
stores key-value pairs, which are guaranteed to be stored in order with respect
to the key.\autocite{Stroustrup_programminglanguage_2000} In this specific case,
the ordering is implemented as a custom comparator function that sorts by energy
first, then smallest moment of inertia tensor eigenvalue followed by the two
larger moment of inertia tensor eigenvalues. The comparator function returns
\texttt{true} if the first argument is considered smaller than the second
argument. This procedure is shown as pseudocode in algorithm~\ref{algo:sortEI}.
%
\begin{algorithm}[htp]
    \caption{Comparator function for sorting by energy $E$ and moment of inertia eigenvalues $I_1\leq I_2\leq I_3$.}
    \label{algo:sortEI}
    \begin{algorithmic}[1]
        \Procedure{Compare}{structure $a$, structure $b$}
            \If{$E^a < E^b$} \State \textbf{return} true
            \ElsIf{$E^a = E^b$ \textbf{and} $I_1^a < I_1^b$} \State \textbf{return} true
            \ElsIf{$E^a = E^b$ \textbf{and} $I_1^a = I_1^b$ \textbf{and} $I_2^a < I_2^b$} \State \textbf{return} true
            \ElsIf{$E^a = E^b$ \textbf{and} $I_1^a = I_1^b$ \textbf{and} $I_2^a = I_2^b$ \textbf{and} $I_3^a < I_3^b$} \State \textbf{return} true
            \Else \State \textbf{return} false
            \EndIf
        \EndProcedure
    \end{algorithmic}
\end{algorithm}
%
The key is mapped to a value, which is simply the number of structures.
The advantage of this method is its great scalability with respect to the number
of input structures as values can be retrieved quickly based on the key.

The second method uses the \ac{EDM} as the differentiation criterion. The
\ac{EDM} is the matrix of all inter-particle distances $d_{ij}$ where
each entry is defined as the Euclidean norm $||\cdot||$ between two spheres.
%
\begin{align}
    d_{ij}=||\mathbf{x}_i-\mathbf{x}_j||^2
\end{align}
%
Each unique embedding of the cluster in space can be represented by an \ac{EDM},
however information about the absolute position, orientation and chirality is
not contained in this representation. That means rigid transformations of
clusters (translations, rotations, reflections) don't affect the \ac{EDM} as
they don't change fixed distances between points in
space.\autocite{Dokmanic_EuclideanDistanceMatrices_2015} If at least one
inter-particle distance is different, the structures are said to be not equal.
The algorithm is implemented in such a way that a structure's distance matrix is
compared to the distance matrix of all other already sorted structures and is
added to the matching group if they are equal up to a set threshold. If not, the
structure is added to the array as a new group. If the optimisation results in
many unique structures, this method of sorting becomes slow, as each trial
structure has to be compared to all other already sorted structure groups.

\subsection{Matching Structures}
\label{sec:matchingstructures}

To compare the results from the optimisations procedures to previously
published sets of clusters the program \verb|match| can be used. It takes two
files as input that each contain a set of structures of equal or different size
and compares them based on the \ac{EDM}. The number of atoms in each set must
be equal, otherwise the program will be terminated. If both sets are found to
be identical no output files are created. In case there are unmatched
structures they will be printed in \verb|xyz| format in the \verb|output|
directory for further analysis.

\section{Graph-Theoretical Analysis}
\label{sec:GraphTheoreticalAnalysis}

Graph theoretical analysis of cluster structures can be done with
\verb|ico-subgraph|, which is a very specialised program designed to compare
contact graphs of \ac{GN} clusters to the icosahedral graph. The input format
for the structures remains unchanged.\footnote{This part of the program was used
in chapter~\ref{sec:thegregorynewtonclusters}. For a definition of contact
graphs and \ac{GN} clusters refer to the introduction in that part of the
thesis.}

All graph objects are handled by the \textit{Boost Graph
Library}\autocite{Siek_BoostGraphLibrary_2002} and are implemented as undirected
graphs as the direction of the connectivity in clusters is not meaningful. The
graphs are automatically generated from three-dimensional coordinate input given
by an object of type \verb|structure|. Two types of graphs can be generated from
these: (1) the graph of the complete structure and (2) the contact graph
containing all spheres but the central one (if it exists). The latter is the
more important one as it was used to carry out the analyses in
chapter~\ref{sec:thegregorynewtonclusters}. The decision whether two spheres are
connected is based on two parameters: the equilibrium distance $r_e$ and a
threshold value $\epsilon$. The default values for these are $r_e=1$ and
$\epsilon=10^{-10}$, respectively.

\begin{align}
    d_{ij}-r_e < \epsilon
\end{align}

The icosahedral graph is compared to the input structures via their graphs and
the \textit{VF2} graph matching
algorithm.\autocite{Cordella_ImprovedAlgorithmMatching_2001,Cordella_SubGraphIsomorphism_2004}
The algorithm finds all mappings of the vertices of the icosahedral graph to the
graph of the input structure. As the icosahedron represents the complete planar
graph for 12 vertices, every graph that is a subgraph of the icosahedral graph
can be represented by the number and type of edges removed from the icosahedral
graph. For this application the mapping was chosen based on the \ac{RMS} value
of the distances between the spheres corresponding to removed edges. From all
the possible mappings of the investigated graphs to the icosahedral graph the
one with the lowest \ac{RMS} value was chosen.

The graphs of the input structures are analysed with respect to their vertex
and face degrees. Vertex degrees are calculated directly by the library and can
be accessed with the \verb|degree| function. For face degrees the
\verb|planar_face_traversal| function has to be invoked. This algorithm
iterates over all faces in the planar embedding counting the number of vertices
constructing each face. The graphs of the input structures are then sorted
based on the calculated degree values, starting with the vertex degree in
descending order and followed by the face degree in ascending order. If two or
more graphs have the same amount of face and vertex degrees they are grouped
together. The sorted graphs are printed to the standard output in the same form
as table~\ref{tab:verticesandfaces} (page~\pageref{tab:verticesandfaces}).
Additionally, the investigated graphs are printed in terms of removed edges
from the icosahedral graph as shown in table~\ref{tab:icosubgraphs}
(page~\pageref{tab:icosubgraphs}).

\section{Additional Functionalities}
\label{sec:AdditionalFunctions}

Besides the main parts of the program, which have been described on the previous
pages, a few script-like executable programs are provided. These were used to
calculate various different properties of the investigated clusters.

\paragraph{Analyse Bond Lengths}%app-bondvariance app-shortestbond (app-N14?)
The bond variance in optimised structures can be calculated with
\texttt{app-bondvariance}. The bond variance is simply defined as the difference
between the shortest and longest bond of a cluster structure. In clusters
optimised by soft potentials a bond is not as well defined as for \ac{SHS}
clusters. Therefore, it has to be determined with respect to a threshold value
and the variance of the bond lengths can not be larger than this threshold. This
application has been used in
chapter~\ref{sec:612LennardJonesClustersfromBasinHopping} to calculate the bond
variance of the optimised $(6,12)$-\ac{LJ} structures. A more specialised
version called \texttt{app-shortestbond} is also provided that can be used to
find the cluster with the shortest bond distance.

\paragraph{Sort Structures by $N_c$}%app-GN and app-Nc

Analysing the total contact numbers $N_c$ or specific kissing numbers can be
done with the programs \texttt{app-Nc} and \texttt{app-GN}. The latter looks
for clusters with a central atom that has exactly 12 spheres arranged around
it, so called Gregory-Newton clusters.

\section{Implementations in Detail}
\label{sec:ImplementationsinDetail}

In the following sections the basic implementation of cluster structures and
two-body potentials will be explained in more detail.

\subsection{Treating Cluster Structures}
\label{sec:thestructureclass}

The main purpose of program \textsc{Spheres} was to optimise an input set of
structures with given Cartesian coordinates and analyse the results of the
optimisation based on properties of the resulting structures. The handling of
those structures and respective properties was therefore crucial to the
functioning of the program.

The Cartesian coordinates are read from a file with blank lines separating
different starting geometries. For each of these individual sets of coordinates
a \verb|structure| object is created. The \verb|structure| class, however, has
much more functionality than storing 3D coordinates. In fact, it serves as a
complex data type storing properties besides coordinates as well as member
functions to calculate those properties. They are stored in data members defined
in private fields, such that they can only be manipulated by functions owned by
the \verb|structure| class. The \verb|structure| class is designed to store
values for:
\begin{itemize}
    \item Cartesian coordinates
    \item energy (depending on the chosen potential),
    \item an integer number for labelling,
    \item moment of inertia,
    \item the \ac{EDM} and adjacency matrix representations and
    \item an \verb|undirectedGraph| that contains the connectivity information.
\end{itemize}

The Cartesian coordinates of each individual sphere are stored in an object of
type \texttt{coord3d}, which is a modified version of the \verb|coord3d|
implementation used in program
\textsc{Fullerene}.\autocite{Schwerdtfeger_Programfullerenesoftware_2013}

The moment of inertia tensor determines the torque needed to accelerate a rigid
body to spin around a rotational axis through the origin of the coordinate
system. It is therefore analogous to mass in case of linear, translational
acceleration. For cluster structures the tensor is equal to the sum over the
moments of inertia of all constituent particles. As the clusters investigated in
this thesis are only made up of one type of particle the mass term was set to
unity. The inertia tensor $\mathbf{I}$ can then be calculated via the equations
below.
%
\begin{gather}
    \begin{gathered}
    \mathbf{I}=
    \begin{pmatrix}
        I_{xx} & I_{xy} & I_{xz}\\
        I_{yx} & I_{yy} & I_{yz}\\
        I_{zx} & I_{zy} & I_{zz}
    \end{pmatrix}\\
    I_{xx}=\sum_i(y_i^2+z_i^2) \quad I_{yy}=\sum_i(x_i^2+z_i^2) \quad I_{xx}=\sum_i(y_i^2+x_i^2)\\
    I_{xy} = I_{yx} = -\sum_ix_iy_i \quad I_{xz} = I_{zx} = -\sum_ix_iz_i \quad I_{yz} = I_{zy} = -\sum_iy_iz_i
    \end{gathered}
\end{gather}
%
Here, $x_i$, $y_i$ and $z_i$ denote the respective coordinate of sphere $i$.
Diagonalising the inertia tensor yields a set of eigenvalues and eigenvectors,
with the latter representing the principal axis system. Upon creation of a
\verb|structure| object the coordinates are transformed, such that the
coordinate origin lies at the centre of mass and the structures principal axis
are aligned with the basis vectors of the Cartesian coordinate system.

The class is designed to ensure that when any of the particle coordinates
change, all properties are recalculated such that there is never a mismatch
between the properties and the coordinates they refer to.

\subsection{Treating Two-Body Potentials}
\label{sec:thepairpotentialclass}

For the geometry optimisation of the cluster structures, methods to calculate the
energy and gradient need to be provided to the optimisation library. As shown in
section~\ref{sec:PracticalImplementationForPotentialsDependingOnPairDistances}
the gradient can be expressed as in equation~\eqref{eqn:gradient}.
%
\begin{align}
    \pdv{E(\mathbf{X})}{x_m}=\sum_{j>i}^N\pdv{\varepsilon(r_{ij})}{r_{ij}}\ \pdv{r_{ij}}{\mathbf{r}_{ij}}\ \pdv{\mathbf{r}_{ij}}{x_m}\label{eqn:gradient}
\end{align}
%
The last two terms of the sum will be the same independent of the choice for the
potential function $\varepsilon(r_{ij})$. The implementation therefore
focused on reducing redundancies by making use of class inheritance features. As
explained in the following paragraphs, this makes exchanging the type of
two-body potential trivial.

For this, a base class called \verb|pairPotential| was defined. It is an
abstract class and therefore cannot be instantiated. Its private fields hold
declarations of virtual methods for calculating energy $E(r)$, first derivative
$\dv*{E}{r}$ and second derivative $\dv*[2]{E}{r}$ based on particle distance
$r$. They are declared virtual, because they will be overwritten with the
respective functions in the derived classes of the actual potentials. In the
bare \verb|pairPotential| class those functions are only declared but never
defined and cannot be used for calculations. The public members of the class are
the constructor and the user-accessible functions for calculating energy,
gradient vector and Hessian matrix as well as the optimiser.

As an example, the \Cpp implementation for the member function that calculates
the energy of the system is shown in listing~\ref{code:energy}.
%
\begin{lstlisting}[caption={Implementation of the redundant part of the energy calculation.},label=code:energy,float=htb]
double pairPotential::calcEnergy (structure &S) {
  double f(0);
  for (int i = 0; i < S.nAtoms(); i++) {
    for (int j = i + 1; j < S.nAtoms(); j++) {
      f += this->E (coord3d::dist (S[i],S[j]));
    }
  }
return f;
}
\end{lstlisting}
%
The method takes a \verb|structure| object as input and uses the virtual energy
function to calculate the energy contributions of all unique pairs of spheres.
The method \texttt{nAtoms()} returns the number of spheres in the
\texttt{structure} and \texttt{dist} calculates the Euclidean distance between
two spheres \texttt{i} and \texttt{j}.

\begin{lstlisting}[caption={Implementation of the distance dependant energy for the Lennard-Jones potential.},label=code:LJenergy,float=htb]
class LJ : public pairPotential {
  private:
    double E (double distance) {
      return (_epsilon / (_exp1/_exp2 - 1))
             * ( (pow (_rm / distance, _exp1)) - (_exp1/_exp2)
             * (pow (_rm / distance, _exp2)) );
    }
    //methods for first and second derivatives go here
};
\end{lstlisting}

To define a potential, a derived class that overwrites the virtual function
declarations is required. The virtual member functions of the base class are
overwritten in the private field of the derived class by providing properly
defined methods. Additionally, any parameters, that the potential form depends
on, are declared in the private fields. In the public fields, constructors for
the respective potential as well as a function that reads the potential
parameters from a user provided file need to be declared. The last function is
important as it also creates a pointer to the potential object on heap memory,
which is necessary, because the exact nature of the potential is not known at
compile time. For the \ac{LJ} potential the energy function can be defined as
shown in listing~\ref{code:LJenergy}. The variables starting with an underscore
are data members defined on object creation and refer to the two exponents
\texttt{\_exp1} and \texttt{\_exp2} the equilibrium distance \texttt{\_rm} and
the depth of the potential energy well \texttt{\_epsilon}. A different potential
can be implemented in the same way. A new derived class has to be defined,
containing the methods to calculate energy, first and second derivative.
Additionally, a function that reads parameters that define the respective
potential form has to be provided. By design, the function needs to return a
pointer to an instance of the class on heap memory. In case of the \ac{LJ}
potential this can be achieved as shown in listing~\ref{code:readPotential}.
%
\begin{lstlisting}[caption={Minimal example for the method \texttt{readPotential()}.},label=code:readPotential,float=htb]
LJ *LJ::readPotential () {
  //instructions to read parameters from file go here

  LJ *potential = new LJ(epsilon, rm, exp1, exp2);
  return potential;
}
\end{lstlisting}
%
Because memory on heap has to be managed by the user, the pointer should be used
in conjunction with \verb|unique_ptr| to ensure destruction of the object upon
exiting the scope. An example for this is given below.
%
\begin{lstlisting}
//read potentialName from settings file

std::unique_ptr< pairPotential > potential;
if (potentialName = "LJ") {
  potential.reset( LJ::readPotential() );
}
\end{lstlisting}

%\chapter{Computational Details}
%\label{sec:ComputationalDetails}
%
%The following sections detail the methodologies used in part~\ref{sec:results}.
%They have been altered slightly from the original
%publications\autocite{Trombach_HollowGoldCages_2016,%Trombach_stickyhardsphereLennardJonestypeclusters_2018,%Trombach_GregoryNewtonproblemkissing_2018}
%to match the style of this thesis.
%
%\section[Computational Details for
%Chapter~\ref{sec:goldendualfullerenes}]{Computational Details for
%Chapter~\ref{sec:goldendualfullerenes}\footnote{This section has been published
%previously in
%\citetitle*{Trombach_HollowGoldCages_2016}\autocite{Trombach_HollowGoldCages_2016}
%and is reproduced with permission from the publisher \textcopyright 2016
%WILEY-VCH Verlag GmbH \& Co. KGaA.}}
%\label{sec:ComputationalDetailsforGoldenDualFullerenes}

%\section[Computational Details for
%Chapter~\ref{sec:fromstickyhardspheretoLJtypeclusters}]{Computational Details
%for Chapter~\ref{sec:fromstickyhardspheretoLJtypeclusters}\footnote{This
%section has been published previously in
%\citetitle*{Trombach_stickyhardsphereLennardJonestypeclusters_2018}\autocite%{Trombach_stickyhardsphereLennardJonestypeclusters_2018}
%and is reproduced with permission from the publisher \textcopyright 2018
%American Physical Society.} }

%\section[Computational Details for
%Chapter~\ref{sec:thegregorynewtonclusters}]{Computational Details for
%Chapter~\ref{sec:thegregorynewtonclusters}\footnote{This section has been
%published previously in
%\citetitle*{Trombach_GregoryNewtonproblemkissing_2018}\autocite%{Trombach_GregoryNewtonproblemkissing_2018}
%and is reproduced with permission from the publisher \textcopyright 2018
%American Physical Society.}}


\chapter{Extending \Egraphs with \Eclass Analyses}
\label{sec:extensions}

As discussed so far, \egraphs and equality saturation provide an efficient way
  to implement a term rewriting system.
Rebuilding enhances that efficiency, but the approach remains designed for
  purely syntactic rewrites.
However, program analysis and optimization typically require more than just
  syntactic information.
Instead, transformations are \emph{computed} based on the input terms and also semantic facts
  about that input term, e.g., constant value, free variables, nullability,
  numerical sign, size in memory, and so on.
The ``purely syntactic'' restriction has forced existing equality saturation
  applications~\cite{eqsat, eqsat-llvm, herbie} to
  resort to ad hoc passes over the \egraph
  to implement analyses like constant folding.
These ad hoc passes require manually manipulating the \egraph,
  the complexity of which could prevent the implementation of more sophisticated
  analyses.
  % naturally expressed in the more conventional term rewriting setting,
  % but must implemented as complicated \textit{ad hoc} passes over the \egraph.

We present a new technique called \textit{\eclass analysis},
  which allows the concise
  expression of a program analysis over the \egraph.
An \eclass analysis resembles abstract interpretation
  lifted to the \egraph level,
  attaching \textit{analysis data} from a semilattice to each \eclass.
The \egraph maintains and propagates this data as
  \eclasses get merged and new \enodes are added.
Analysis data can be used directly to modify the \egraph, to inform
  how or if rewrites apply their right-hand sides, or to determine the cost of
  terms during the extraction process.

\Eclass analyses provide a general mechanism to replace what previously
  required ad hoc extensions that manually manipulate the \egraph.
\Eclass analyses also fit within the equality saturation workflow,
  so they can naturally cooperate with the equational reasoning provided by
  rewrites.
Moreover, an analysis lifted to the \egraph level automatically benefits from a
  sort of ``partial-order reduction'' for free:
  large numbers of similar programs may be analyzed for little additional cost
  thanks to the \egraph's compact representation.

This section provides a conceptual explanation of \eclass analyses as well
  as dynamic and conditional rewrites that can use the analysis data.
The following sections will provide concrete examples:
  \autoref{sec:impl} discusses the \egg implementation and a complete example of a
  partial evaluator for the lambda calculus;
  \autoref{sec:case-studies} discusses how three published projects have used
  \egg and its unique features (like \eclass analyses).

% An \eclass analysis may,  modify the \egraph itself to inject new based on the computed data,
% In this way, the \eclass analysis can work with the rewrites as opposed

% \Egg offers many convenient ways to interact with the \egraph that are difficult
%   or impossible in other implementations.
% These tools give \egg the flexibility highlighted by the diverse case studies
%   in \autoref{sec:case-studies}
%   and the lambda calculus example in \autoref{sec:lambda}.

% Like the rest of \egg, these extensions are generic over the language and
%   rewrites that the user is working with.
% These are all novel in practice, as \egg is the first (to our knowledge)
%   general-purpose, reusable \egraph library.
% \Eclass analyses appear to be conceptually novel as well.

% \Chandra{this entire section is a bit light in details and doesn't
% always connect back to the goals of egg the right way. For example,
% without a concrete advantage of runners, it's not clear why it is useful.}

\section{E-Class Analyses}
\label{sec:analysis}

An \eclass analysis defines a domain $D$ and associates a value $d_{c} \in D$ to
  each \eclass $c$.
The \eclass $c$ contains the associated data $d_{c}$,
  i.e., given an \eclass $c$, one can get $d_{c}$ easily, but not vice-versa.

The interface of an \eclass analysis is as follows,
  where $G$ refers to the \egraph,
  and $n$ and $c$ refer to \enodes and \eclasses within $G$:

% We will use the following metavariables and syntax in this section:

% \begin{center}
%   $G$: \egraph, $c$: \eclass, $n$: \enode,
%   $d_{c}$: the analysis data associated with \eclass $c$
% \end{center}
\vspace{1em}
\begin{tabular}{lp{0.7\linewidth}}
  $\textsf{make}(n) \to d_{c}$ &
    When a new \enode $n$ is added to $G$ into a new, singleton \eclass $c$,
    construct a new value $d_{c} \in D$ to be associated with $n$'s new \eclass,
    typically by accessing the associated data of $n$'s children.
  \\
  $\textsf{join}(d_{c_1}, d_{c_2}) \to d_{c}$ &
    When \eclasses $c_{1}, c_{2}$ are being merged into $c$,
    join $d_{c_1}, d_{c_2}$ into a new value $d_{c}$ to be associated with the
    new \eclass $c$.
  \\
  $\textsf{modify}(c) \to c'$ &
    Optionally modify the \eclass $c$ based on $d_{c}$, typically by adding an
      \enode to $c$.
    Modify should be idempotent if no other changes occur to the \eclass, i.e.,
      $\textsf{modify}(\textsf{modify}(c)) = \textsf{modify}(c)$
    % Returning $c$ unmodified ($c = c'$) suffices in many cases, and is the
    % default implementation for analyses in \egg.
\end{tabular}
\vspace{1em}

The domain $D$ together with the \textsf{join} operation should form a join-semilattice.
The semilattice perspective is useful for defining the \textit{analysis invariant}
  (where $\vee$ is the \textsf{join} operation):
\[
  \forall c \in G.\quad
  d_{c} = \bigvee_{n \in c} \textsf{make}(n)
  \quad \text{and} \quad
  \textsf{modify}(c) = c
\]

The first part of the analysis invariant states that the data associated with
  each \eclass must be the \textsf{join} of the \textsf{make} for every \enode
  in that \eclass.
Since $D$ is a join-semilattice, this means that
  $\forall c, \forall n \in c, d_{c} \geq \textsf{make}(n) $.
The motivation for the second part is more subtle.
Since the analysis can modify an \eclass through the \textsf{modify} method,
  the analysis invariant asserts that these modifications are driven to a fixed
  point.
When the analysis invariant holds, a client looking at the analysis data can be
  assured that the analysis is ``stable'' in the sense that
  recomputing \textsf{make}, \textsf{join}, and \textsf{modify} will not
  modify the \egraph or any analysis data.

\subsection{Maintaining the Analysis Invariant}

\begin{figure}
  \begin{minipage}[t]{0.47\linewidth}
    \begin{lstlisting}[gobble=4, numbers=left, numberstyle=\color{black}, basicstyle=\scriptsize\ttfamily\color{black!40}, escapechar=|,
   xleftmargin=13pt,
   numbersep=7pt,
]
    def add(enode):
      enode = self.canonicalize(enode)
      if enode in self.hashcons:
        return self.hashcons[enode]
      else:
        eclass = self.new_singleton_eclass(enode)
        for child_eclass in enode.children:
          child_eclass.parents.add(enode, eclass)
        self.hashcons[enode] = eclass
        |\color{black}\label{line:add1} eclass.data = analysis.make(enode)|
        |\color{black}\label{line:add2} analysis.modify(eclass)|
        return eclass

    def merge(eclass1, eclass2)
      union = self.union_find.union(eclass1, eclass2)
      if not union.was_already_unioned:
        |\color{black}\label{line:merge1}d1, d2 = eclass1.data, eclass2.data|
        |\color{black}\label{line:merge2}union.eclass.data = analysis.join(d1, d2)|
        self.worklist.add(union.eclass)
      return union.eclass
    \end{lstlisting}
  \end{minipage}
  \hfill
  \begin{minipage}[t]{0.47\linewidth}
    \begin{lstlisting}[gobble=4, numbers=left, firstnumber=21, numberstyle=\color{black}, basicstyle=\scriptsize\ttfamily\color{black!40}, escapechar=|,
    numbersep=5pt,
]
    def repair(eclass):
      for (p_node, p_eclass) in eclass.parents:
        self.hashcons.remove(p_node)
        p_node = self.canonicalize(p_node)
        self.hashcons[p_node] = self.find(p_eclass)

      new_parents = {}
      for (p_node, p_eclass) in eclass.parents:
        p_node = self.canonicalize(p_node)
        if p_node in new_parents:
          self.union(p_eclass, new_parents[p_node])
        new_parents[p_node] = self.find(p_eclass)
      eclass.parents = new_parents
    \end{lstlisting}
    \vspace{-3mm}
    \begin{lstlisting}[gobble=4, numbers=left, firstnumber=34, basicstyle=\scriptsize\ttfamily, escapechar=|,
    numbersep=5pt,
]

      # any mutations modify makes to eclass
      # will add to the worklist
      |\label{line:repair1}|analysis.modify(eclass)
      for (p_node, p_eclass) in eclass.parents:
        new_data = analysis.join(
          p_eclass.data,
          analysis.make(p_node))
        if new_data != p_eclass.data:
          p_eclass.data = new_data
          |\label{line:repair2}|self.worklist.add(p_eclass)
    \end{lstlisting}
  \end{minipage}
  \caption{
    The pseudocode for maintaining the \eclass analysis invariant is largely
      similar to how rebuilding maintains congruence closure
      (\autoref{sec:rebuilding}).
    Only lines \ref{line:add1}--\ref{line:add2},
      \ref{line:merge1}--\ref{line:merge2},
      and \ref{line:repair1}--\ref{line:repair2} are added.
    Grayed out or missing code is unchanged from \autoref{fig:rebuild-code}.
  }
  \label{fig:rebuild-analysis}
\end{figure}

We extend the rebuilding procedure from \autoref{sec:rebuilding} to restore the
  analysis invariant as well as the congruence invariant.
\autoref{fig:rebuild-analysis} shows the necessary modifications to the
  rebuilding code from \autoref{fig:rebuild-code}.

Adding \enodes and merging \eclasses risk breaking the analysis invariant in
  different ways.
Adding \enodes is the simpler case; lines \ref{line:add1}--\ref{line:add2}
  restore the invariant for the newly created, singleton \eclass that holds the
  new \enode.
When merging \enodes, the first concern is maintaining the semilattice portion of the
  analysis invariant.
Since \textsf{join} forms a semilattice over the domain $D$ of the analysis
  data, the order in which the joins occur does not matter.
Therefore, line \ref{line:merge2} suffices to update the analysis data of the
  merged \eclass.

Since $\textsf{make}(n)$ creates analysis data by looking at the data of $n$'s,
  children, merging \eclasses can violate the analysis invariant in the same way
  it can violate the congruence invariant.
The solution is to use the same worklist mechanism introduced in
  \autoref{sec:rebuilding}.
Lines \ref{line:repair1}--\ref{line:repair2} of the \texttt{repair} method
  (which \texttt{rebuild} on each element of the worklist)
  re-\textsf{make} and \textsf{merge} the analysis data of the parent of any
  recently merged \eclasses.
The new \texttt{repair} method also calls \textsf{modify} once, which suffices
  due to its idempotence.
In the pseudocode, \textsf{modify} is reframed as a mutating method for clarity.

\Egg's implementation of \eclass analyses assumes that the analysis domain $D$
  is indeed a semilattice and that \textsf{modify} is idempotent.
Without these properties, \egg may fail to restore the analysis invariant on
  \texttt{rebuild}, or it may not terminate.

% \subsection{Modifying the \Egraph from an \Eclass Analysis}
\subsection{Example: Constant Folding}

The data produced by \eclass analyses can be
  usefully consumed by other components of an equality saturation system
  (see \autoref{sec:rewrites}),
  but \eclass analyses can be useful on their own thanks to the
  \textsf{modify} hook.
Typical \textsf{modify} hooks will either do nothing, check some invariant about
  the \eclasses being merged, or add an \enode to that \eclass
  (using the regular \texttt{add} and \texttt{merge} methods of the \egraph).

As mentioned above, other equality saturation implementations have implemented
  constant folding as custom, ad hoc passes over the \egraph.
We can formulate constant folding as an \eclass analysis that highlights the
  parallels with abstract interpretation.
Let the domain $D = \texttt{Option<Constant>}$, and let the \texttt{join}
  operation be the ``\texttt{or}'' operation of the \texttt{Option} type:
\begin{figure}[h!]
\begin{lstlisting}[language=Rust, basicstyle=\ttfamily\footnotesize, xleftmargin=35mm]
match (a, b) {
  (None,    None   ) => None,
  (Some(x), None   ) => Some(x),
  (None,    Some(y)) => Some(y),
  (Some(x), Some(y)) => { assert!(x == y); Some(x) }
}
\end{lstlisting}
\end{figure}
Note how \textsf{join} can also aid in debugging by checking properties about
  values that are unified in the \egraph;
  in this case we assert that all terms represented in an \eclass should have
  the same constant value.
The \textsf{make} operation serves as the abstraction function, returning the
  constant value of an \enode if it can be computed from the constant values
  associated with its children \eclasses.
The \textsf{modify} operation serves as a concretization function in this
  setting.
If $d_{c}$ is a constant value, then $\textsf{modify}(c)$ would add
  $\gamma(d_{c}) = n$ to $c$, where $\gamma$ concretizes the constant value into
  a childless \enode.

Constant folding is an admittedly simple analysis, but one that did not formerly
  fit within the equality saturation framework.
\Eclass analyses support more complicated analyses in a general way, as
  discussed in later sections on the \egg implementation and case studies
  (Sections \ref{sec:impl} and \ref{sec:case-studies}).

\section{Conditional and Dynamic Rewrites}
\label{sec:rewrites}

In equality saturation applications, most of the rewrites are purely
  syntactic.
In some cases, additional data may be needed to determine if or how to perform
  the rewrite.
For example, the rewrite $x / x \to 1$ is only valid if $x \neq 0$.
A more complex rewrite may need to compute the right-hand side dynamically based
  on an analysis fact from the left-hand side.

The right-hand side of a rewrite can be generalized to a function
  \textsf{apply} that takes a substitution and an \eclass generated from
  e-matching the left-hand side, and produces a term to be added to the \egraph
  and unified with the matched \eclass.
For a purely syntactic rewrite, the \textsf{apply} function need not inspect the
  matched \eclass in any way; it would simply apply
  the substitution to the right-hand pattern to produce a new term.

\Eclass analyses greatly increase the utility of this generalized form of
  rewriting.
The \textsf{apply} function can look at the analysis data for the matched
  \eclass or any of the \eclasses in the substitution to determine if or how to
  construct the right-hand side term.
These kinds of rewrites can broken down further into two categories:
\begin{itemize}
  \item \textit{Conditional} rewrites like $x / x \to 1$ that are purely
  syntactic but whose validity depends on checking some analysis data;
  \item \textit{Dynamic} rewrites that compute the right-hand side based on
  analysis data.
\end{itemize}

Conditional rewrites are a subset of the more general dynamic rewrites.
Our \egg implementation supports both.
The example in \autoref{sec:impl} and case studies in \autoref{sec:case-studies}
  heavily use generalized rewrites, as it is typically the most convenient way
  to incorporate domain knowledge into the equality saturation
  framework.

\section{Extraction}
\label{sec:tricks-extraction}

Equality saturation typically ends with an extraction phase that selects an
  optimal represented term from an \eclass according to some cost function.
In many domains \cite{herbie, szalinski}, AST size
  (sometimes weighted differently for different operators) suffices as a simple,
  local cost function.
We say a cost function $k$ is local when the cost of a term $f(a_{1}, ...)$ can be
  computed from the function symbol $f$ and the costs of the children.
With such cost functions, extracting an optimal term can be efficiently done
  with a fixed-point traversal over the \egraph that selects the minimum cost
  \enode from each \eclass \cite{herbie}.

Extraction can be formulated as an \eclass analysis when the cost function
  is local.
The analysis data is a tuple $(n, k(n))$ where $n$ is the cheapest \enode
  in that \eclass and $k(n)$ its cost.
The $\textsf{make}(n)$ operation calculates the cost $k(n)$ based on
  the analysis data (which contain the minimum costs) of $n$'s children.
The \textsf{merge} operation simply takes the tuple with lower cost.
The semilattice portion of the analysis invariant then guarantees that the
  analysis data will contain the lowest-cost \enode in each class.
Extract can then proceed recursively;
  if the analysis data for \eclass $c$ gives $f(c_{1}, c_{2}, ...)$ as the optimal \enode,
  the optimal term represented in $c$ is
  $\textsf{extract}(c) = f( \textsf{extract}(c_{1}), \textsf{extract}(c_{2}), ... )$.
% The optimal term represented in an \eclass can then be built recursively,
%   starting with the optimal \enode from the analysis data.
% Extraction can be completed by starting from the desired \eclass and
%   building the term recursively based on the \enode from the analysis data.
This not only further demonstrates the generality of \eclass analyses, but also
  provides the ability to do extraction ``on the fly''; conditional and dynamic
  rewrites can determine their behavior based on the cheapest term in an \eclass.

Extraction (whether done as a separate pass or an \eclass analysis) can also
  benefit from the analysis data.
Typically, a local cost function can only look at the function symbol of the
  \enode $n$ and the costs of $n$'s children.
When an \eclass analysis is attached to the \egraph, however, a cost function
  may observe the data associated with $n$'s \eclass, as well as the data
  associated with $n$'s children.
This allows a cost function to depend on computed facts rather that just purely
  syntactic information.
In other words, the cost of an operator may differ based on its inputs.
\autoref{sec:tensat} provides a motivating case study wherein an \eclass
  analysis computes the size and shape of tensors, and this size information
  informs the cost function.

%%% Local Variables:
%%% TeX-master: "../thesis"
%%% End:


Our aim is to show that the agents controlling an industrial micro-grid can learn complex behaviors and coordination strategies that can lead to an optimization of the global energy consumption. Therefore, we train the agents in different competitive and cooperative scenarios. In each scenario, the environment consists of multiples agents with discrete action space, discrete time space and mixed observation spaces. Every agent controls a grid component and may take physical actions in the environment by changing the operational state of the corresponding grid component. We do not assume that all agents have identical action and observation spaces, or act according to the same policy $\pi_{\theta}$. The environment of each experiment were implemented as an extension of OpenAI environment Gym \cite{Brockman01540}. We found this framework useful for creating environments involving complex interactions between agents, while keeping the control and perception problems simple, as we are primarily interested in addressing agent strategies.

Unless otherwise specified, the actor and the critic of each agent is parameterized by a two-layer ReLU MLP with respectively $256$ units per layer for the actor and $512$ units per layer for the critic. In all of our experiments, we use the Adam optimizer with a learning rate of $0.0001$ and the discount factor $\gamma$ is set to be $0.9$. The size of the replay buffer is $10240$ and the sampling time horizon is $1024$. We update the network parameters after every $1024$ sampling steps and we use a batch size of $512$. We train with $10$ random seeds for environments with stark success/fail conditions (\textit{experiment 2} and \textit{experiment 3}) and with only $2$ seeds for \textit{experiment 1}. For the PPO-algorithm, we set the clipping parameter $\epsilon$ to $0.1$, the entropy coefficient to $0.005$ and the exploration noise to $0.2$.

This section describes in detail the designed experiments as well as the specific training parameters and analyzes various aspects of the learned behaviors. Source codes for the environments as well as learned policy parameters of the agents in all environments are open-sourced and available at \cite{Bakakeu2019}.

\subsection{Experiment 1: Autocurricula and Emergent Behavior}
\label{subsec:51}
In this scenario, we consider a micro-grid of $4$ identical production machines (energy loads) executing the same production task. The grid is operated in connected mode. However, the main grid can only provide enough energy for $3$ production machines to operate. Therefore, the production machines are in full competition. The exploration reward of the agents is set so that the agent is penalized if the production machine is shut down or suspended. The competition reward is based on the performance of the machines meaning that the agent will receive a positive incentive every time a production task is accomplished. In addition, all agents are penalized if the total energy demand is greater than the energy provided by the main grid. The exploration annealing factor $\alpha$ is set to $0.2$ so that the agent will first learn how to execute a production task during the initial $20\%$ of the training time. During the training each agent observed the full environment including the observations and the actions of the other agents. During the execution, the agents only observe the amount of energy available as well as the current operational state of the machine it controls. We train the agents with $2$ random seeds for $500$ sampling horizons.

% For figures use
%
\begin{figure}[b]
\sidecaption
% Use the relevant command for your figure-insertion program
% to insert the figure file.
% For example, with the graphicx style use
\includegraphics[scale=.48]{images/experiment_1_results}
%
% If no graphics program available, insert a blank space i.e. use
%\picplace{5cm}{2cm} % Give the correct figure height and width in cm
%
\caption{Emergent Skill Progression From Multi-Agent Autocurricula. Through the reward signal
agents go through different distinct stages of emergence. (a) Episode 1-30: the different agents learn basic skills (how to execute a production task) and the productivity rise. (b) Episode 30-160: strategy exploitation and competition take place between the agents and stabilize only when \textit{machine 1} adopts a very bad strategy. (c) Episode 160-220: the first coordination strategy emerge. \textit{Machine 2}, \textit{Machine 3} and \textit{Machine 4} start to produce more frequently while \textit{Machine 1} remains in the $idle$-state more often. (d) Episode 220-500: a second coordination strategy emerge.  \textit{Machine 2}, \textit{Machine 4} reached their highest productivity while \textit{Machine 3} is frequently halted.}
\label{fig:scenario_1_results}       % Give a unique label
\end{figure}

Fig. \ref{fig:scenario_1_results} shows a typical progression of the emergent strategies learned by all agents during the course of the training measured in this case by the respective average cumulative rewards. Clearly, training the agents against each other creates many different strategies, each of which introduces a previously non-existent pressure on other agents to do better in order to move to the next stage. Note that there are no direct incentives for agents to explore, but rather the emergent strategies are solely a result of the auto-curriculum induced by multi-agent competition.

As illustrated in Fig. \ref{fig:scenario_1_results}, the agents initially learn simultaneously how to control the corresponding production machine so that it can reach the $execute$-state as often as possible. After approximately 30 episodes, all agent have mastered this task and the energy demand of the whole grid begin to surpass the energy provided by the main grid. Consequently, the agents are then frequently penalized leading to a significant reduction of the production performance and the accumulated reward. At this point some agents begin to take advantage of the observations and actions of other agents and intentionally modify their policies accordingly. We can see, for example, some agents wait in the $idle$-state and then switch to the $execute$-state, when other agents are stopped or halted. However, the more the new strategy becomes efficient, the more it becomes predictable and the other agents begin to defend themselves against this strategy. After another 5-20 episodes, all agents have now learned to exploit the strategy of other agents and the energy demand of the production machines again surpass the energy supply and the agents are penalized again more frequently. At this point a new adaptation/exploitation cycle restarts. Because of random exploration factors, the best performing agent during this phase changes from cycle to cycle. After approximately 3-6 cycles (which corresponds to episode 30 - 160), the training stabilized when at least one agent adopts a very bad policy e.g. only switching between $stopped$- and $idle$- state. The first acceptable coordination strategy emerges from episode 160 to approximately episode 220: \textit{Machine 2}, \textit{Machine 3} and \textit{Machine 4} start to produce more frequently while \textit{Machine 1} remains in the $idle$-state oftenly. It is a valid strategy since the energy required for the production can be supplied by the main grid and the overall productivity is  relatively high. The second and final coordination strategy emerge at episode 220: while \textit{Machine 2} and \textit{Machine 4} reached their highest productivity,  \textit{Machine 3} is frequently halted.

In this experiment, we found that the emergent auto-curriculum is fairly robust, as similar observations can be made with different initialization strategies. Out of $10$ experimental runs, more than $7$ runs were able to converge to an equilibrium. However, many design choices can drastically reduce the convergence time and the sample complexity such as the surrogate loss and the entropy loss of the underlying PPO-algorithm. We also experimented with different configurations (for example by including energy storage and generator components) and found that these configurations also lead to emergent behaviors, providing further evidence that multi-agent interaction is a promising path towards self-supervised energy management.

\subsection{Experiment 2: Fully Cooperative Production}
\label{subsec:52}

% For figures use
%
\begin{figure}[h!]
\sidecaption
% Use the relevant command for your figure-insertion program
% to insert the figure file.
% For example, with the graphicx style use
\includegraphics[scale=.48]{images/experiment_2_results}
%
% If no graphics program available, insert a blank space i.e. use
%\picplace{5cm}{2cm} % Give the correct figure height and width in cm
%
\caption{Average rewards on cooperative production. A coordination strategy emerges rapidly after $100$ episodes. While the energy generators rapidly learn how to generate energy, the energy storage component adopted to discharge when the production machines in the $execute$-state and thus allowing the production machine to increase the productivity.}
\label{fig:scenario_2_results}       % Give a unique label
\end{figure}

In this scenario, the micro-grid is operated in isolated mode and consists of two production machines (a CNC-machine and a welding machine), an energy generator (a wind turbine) and an energy storage system. The energy generated by the wind turbine is lower than the energy needed by the production machines. Therefore, the energy storage system has to be turned on in order to execute a production task. All the components should therefore cooperate. The exploration rewards is similar to the previous experiment and the team-based cooperation reward of the agents is based on the performance of the production machines. We train the agents for $300$ episodes. The agents also used the same policy network architecture as in experiment 1 (see section \ref{subsec:51}).

As we can observe in Fig. \ref{fig:scenario_2_results}, the components rapidly learn the required basic skills and a coordination strategy emerge around the episode $100$. While the energy generator is constantly switched on, the energy storage component learns to charge when the production machines are standing still and to discharge when the production machines perform a production task and thus supports production. Once the energy generator and the energy storage component have adopted a strategy, the production machines rapidly learn to increase their productivity which reach its highest level around episode $250$.

To determine the seed dependence of the skill emergence, we ran this environment with $10$ random seeds and looked at the variability of the onset of behavioral shifts across runs. We find that the behavioral shifts happen at very similar times around episode $100$ in training. Rewards and environment statistics are very consistent in early as well as in later times in training.

%\subsection{Experiment 3: Effect of Exploration Curriculum}
%\label{subsec:53}
%... yet to come!!! [TODO: Jupiter Bakakeu]