\chapter{Practical Hartree-Fock approaches}\label{chap:hartreefock}

\abstract{This chapters aims at catching two birds with a stone;  to introduce to you essential features of the programming languages
C++ and Fortran with a brief reminder on Python specific topics, and to stress problems like
overflow, underflow, round off errors and eventually loss of precision due to the finite amount
of numbers a computer can represent.
The programs we discuss are tailored to these aims.}

\section{Getting Started}

The Hartree-Fock method, initially developed by Hartree \cite{hartree} and improved by Fock \cite{Fock},
is probably the most popular \emph{ab initio} method of quantum chemistry. There are mainly two reasons for this.
Firstly, it  provides an excellent first approximation to the wave function and energy of the system, often accounting
for about 90\%-99\% of the total energy. Secondly, in cases where an even higher degree of precision is needed, the result from a Hartree-Fock calculation
is a very good starting point for other so-called \emph{post-Hartree-Fock methods}. We will look into one such method, namely perturbation
theory, in chapter \ref{chapter:perturbation_theory}.

The general form of the equations is
\begin{equation}
 \mathcal{F}\psi_k = \varepsilon_k \psi_k,
\end{equation}
where $\mathcal{F}$ is the Fock operator, defined as
\begin{equation}
\begin{split}
 \mathcal{F}(\vec x)\psi_k(\vec x)  = & \Big[-\frac{1}{2}\nabla^2 - \sum_{n=1}^K\frac{Z_n}{|\vec R_n - \vec r|}\Big]\psi_k(\vec x) \\
                                      & +  \sum_{l=1}^N\int d\vec x'|\psi_l(\vec x')|^2\frac{1}{|\vec r - \vec r'|}\psi_k(\vec x) \\
                                      & - \sum_{l=1}^N \int d\vec x'\psi^*_l(\vec x')\frac{1}{|\vec r - \vec r'|}\psi_k(\vec x')\psi_l(\vec x).
\end{split}
\end{equation}
These are a set of coupled one-electron eigenvalue equations for the spin orbitals $\psi_k$.
The equations are non-linear because the orbitals we are seeking are
actually needed in order to obtain the operator $\mathcal{F}$ which determine them. They are therefore often referred to as self consistent field (SCF)
equations, and they must be solved iteratively.

The term in the square brackets is the one-body operator which we have called $h(\vec r)$ in equation (\ref{eq:H1}). The two extra sums are due to the interactions between
the electrons. The first of these represent the Coulomb potential from the mean field set up by the electrons of the system. The last is similar to the first
except that the indices of two orbitals have been switched. This is a direct consequence of the fact that in the derivation of the equations, the state
is assumed to be a Slater determinant. Note that due to the last sum, the Fock operator is non-local, that is to say, the value of $\mathcal{F}(\vec x)\psi_k(\vec x)$
depends on the value of $\psi_k(\vec x')$ at all coordinates $\vec x' \in \mathbb{R}^3 \oplus \{\uparrow, \downarrow\}$.

In the following section the general Hartree-Fock equations presented above are derived. Thereafter, we will see how to reformulate the equations to a more
implementation friendly form. We do this by removing the spin part so that the resulting equations are in terms of spatial orbitals only.
However, before doing this, it is necessary to decide how to relate the spin orbitals to the spatial orbitals. We discuss the two most common ways to
do this. These result in the so-called restricted and unrestricted Slater determinants, which are discussed in section \ref{sec:res_and_unres_dets}.
In section \ref{sec:RHF} we show how the restricted determinant leads to the restricted Hartree-Fock (RHF) equations, which is a set of integro-differential equations for
the spatial orbitals. In order to solve the equations, the spatial orbitals are expanded in a known basis, which leads to a set of self consistent algebraic equations
called the \emph{Roothaan equations}. Thereafter, in section \ref{sec:UHF}, we derive the unrestricted Hartree-Fock (UHF)
equations from the unrestricted determinant. These are also solved by introducing a set of known basis functions, leading to the so-called
\emph{Pople-Nesbet} equations, which is the unrestricted analogue of the Roothaan equations.

The theory of this chapter is covered by Szabo and Ostlund \cite{Szabo} and Thijssen \cite{Thijssen}.

% The Hartree-Fock method is a popular method for calculating the ground state and ground state energy of many-particle systems.
% It is a variational method which aims to find the minimum of the expectation value of the Hamiltonian. However, instead of minimizing in
% the space of all possible wave functions, the search is restricted to the set of all Slater determinants.
%
% The Hartree-Fock equations are given by
% \begin{equation}
%  \mathcal{F}\psi_k = \varepsilon_k \psi_k,
% \end{equation}
% where $\mathcal{F}$ is the Fock operator, defined as
% \begin{equation}
% \begin{split}
%  \mathcal{F}(\vec x)\psi_k(\vec x)  = & \Big[-\frac{1}{2}\nabla^2 - \sum_{i=1}^N\frac{Z_n}{|\vec R_n - \vec r|}\Big]\psi_k(\vec x) \\
%                                       & +  \sum_{l=1}^N\int d\vec x'|\psi_l(\vec x')|^2\frac{1}{|\vec r - \vec r'|}\psi_k(\vec x) \\
%                                       & - \sum_{l=1}^N \int d\vec x'\psi^*_l(\vec x')\frac{1}{|\vec r - \vec r'|}\psi_k(\vec x')\psi_l(\vec x).
% \end{split}
% \end{equation}
% These are a set of coupled one-electron eigenvalue equations for the spin orbitals $\psi_k$.
% The equations are non-linear because the orbitals we are seeking are
% actually needed in order to obtain the operator $\mathcal{F}$ which determine them. They are therefore often referred to as self consistent field (SCF)
% equations and must be solved iteratively. The procedure goes along the following lines. First make an initial guess for the spin orbitals $\psi_k$ and calculate the Fock
% operator $\mathcal{F}$. Then, solve the Hartree-Fock equations to obtain a new set of spin orbitals and use these as input to calculate a new Fock operator.
% The process continues like this until a convergence criterium is reached.
%
% The term in the square brackets is the one-body operator which we have called $h(\vec r)$ in equation (\ref{eq:H1}). The two extra sums are due to the interactions between
% the electrons. The first of these represent the Coulomb potential from the mean field set up by the electrons of the system. The last is similar to the first
% except that the indices of two orbitals have been switched. This is a direct consequence of the fact that in the derivation of the equations, the state
% is assumed to be a Slater determinant. Note that due to the last sum, the Fock operator is non-local, that is to say, the value of $\mathcal{F}(\vec x)\psi_k(\vec x)$
% depends on the value of $\psi_k(\vec x')$ at all coordinates $\vec x' \in \mathbb{R}^3 \oplus \{\uparrow, \downarrow\}$.
%
% As discussed in section \ref{sec:fermion_basis}, any state $\ket{\Psi}$ can be written as a linear combination of Slater determinants
% \begin{equation}
%  \ket{\Psi} = C_0\ket{c} + \sum_{ia}C^a_i\ket{\Psi^a_i} + \sum_{ijab}C^{ab}_{ij}\ket{\Psi^{ab}_{ij}} + \dots
% \end{equation}
% The most crude approximation is to neglect all terms except the reference Slater $\ket{c}$. The Hartree-Fock method answers the following problem:
% Given the approximation $\ket{\Psi} \approx \ket{c}$, what is the optimal set of spin orbitals to choose as constituents of $\ket{c}$, and what is the resulting approximation
% for the energy, $E_{HF}$? The energy $E_{HF}$ is defined as the energy found from an exact solution of the Hartree-Fock equations (which presupposes a complete basis).
%
% In most cases the Hartree-Fock method provides an excellent first approximation to the wave function and energy of the system, and it often accounts
% for about 99\% of the total energy. However, in quantum chemistry an even higher degree of precision is sometimes needed. In such cases, the solution of
% the Hartree-Fock equations is a very good starting point to use as input to other so-called \emph{post-Hartree-Fock methods}. The other
% methods discussed in this thesis can be considered to belong to this class of methods.
%
% The difference between the exact energy $E$ and the Hartree-Fock energy $E_{HF}$ is referred to as the correlation energy $\Delta E_{corr}$:
% \begin{equation}
%  \Delta E_{corr} = E - E_{HF}.
% \end{equation}

\section{Derivation of the Hartree-Fock equations}
\label{sec:hartree_fock_derivation}
As discussed in section \ref{sec:fermion_basis}, the exact ground state can be written as a
linear combination of Slater determinants
\begin{equation}
 \ket{\Phi_0} = C_0\ket{\Psi_0} + \sum_{ia}C^a_i\ket{\Psi^a_i} + \sum_{i<j,a<b}C^{ab}_{ij}\ket{\Psi^{ab}_{ij}} + \dots
\end{equation}
where $\ket{\Psi_0}$ is some chosen reference determinant. In the Hartree-Fock method all determinants except $\ket{\Psi_0}$ are neglected, and the
spin orbitals from which $\ket{\Psi_0}$ is constructed are chosen in such a way that the expectation value $E_0 = \bra{\Psi_0}H\ket{\Psi_0}$ comes as
close to the excact energy $\mathscr E_0$ as possible. According to the variational principle, the expectation value $E_0$ is an upper bound
to the exact energy $\mathscr E_0$, which means that the optimal choice of spin orbitals are those which minimise $E_0$. When $E_0$ is at its minimum, any
infinitesimal variation of the spin orbitals will leave $E_0$ unchanged, which in mathematical terms means that
\begin{equation}
 \delta E_0 = \sum_{k=1}^N[\bra{\delta\psi_k}\mathcal F\ket{\psi_k} + \bra{\psi_k}\mathcal F\ket{\delta\psi_k}] = 0.
\end{equation}
If each variation could be chosen independently, this would immediately give us $N$ equations to be solved for the $N$ spin orbitals.
Unfortunately, the spin orbitals cannot be varied independently, but must remain orthonormal throughout the variation. The orthonormality condition reads
\begin{equation}
 \langle\psi_k|\psi_l\rangle - \delta_{kl} = 0.
\end{equation}
This type of \emph{constrained} optimisation problem can be solved elegantly by the method of Lagrangian multipliers. A good review of the
method is given in Boas \cite{boas2005mathematical}. The method simply says that if we construct the new functional
\begin{equation}
\label{eq:L_operator}
 \mathscr L = E_0 - \sum_{k,l=1}^N\Lambda_{lk}[\langle\psi_k|\psi_l\rangle - \delta_{kl}],
\end{equation}
we can find the stationary value of $E_0$ by solving the \emph{unconstrained} variational problem for $\mathscr L$. By unconstrained we mean that the variation
of each spin orbital can be chosen freely. The multipliers $\Lambda_{lk}$ are called Lagrange multipliers and will also be determined as part of the solution.

% In the Hartree-Fock method,
% all but the first term is neglected and the orbitals from which this determinant is
% constructed are chosen such that the reference energy $E_0 = \bra{\Psi_0}H\ket{\Psi_0}$ comes
% as close as possible to the exact energy. The variational principle says that the reference
% energy $E_0$ gives an upper bound to the exact energy. We are therefore seeking the spin orbitals
% which minimize $E_0$.

Recall from equation (\ref{eq:E_ref}) that the reference energy is given by
\begin{equation}
 E_0 = \bra{\Psi_0}H\ket{\Psi_0} = \sum_{k=1}^N\bra{\psi_k} h\ket{\psi_k} + \frac{1}{2}\sum_{k,l=1}^N[\bra{\psi_k\psi_l}g\ket{\psi_k\psi_l}
                                    - \bra{\psi_k\psi_l}g\ket{\psi_l\psi_k}].
\end{equation}
Taking the variation of this yields
\begin{equation}
\begin{split}
 \delta E_0  = & \sum_{k=1}^N\bra{\delta \psi_k}h\ket{\psi_k} \\
          &   + \frac{1}{2}\sum_{k,l = 1}^N[\bra{\delta \psi_k \psi_l}g\ket{\psi_k \psi_l} + \bra{\psi_k \delta \psi_l}g\ket{\psi_k \psi_l} \\
          &  \hspace{12mm} - \bra{\delta \psi_k \psi_l}g\ket{\psi_l \psi_k} - \bra{\psi_k \delta \psi_l}g\ket{\psi_l \psi_k}] + \text{c.c.}\\
        = & \sum_{k=1}^N\bra{\delta \psi_k}h\ket{\psi_k} \\
          & + \frac{1}{2}\sum_{k,l = 1}^N[\bra{\delta \psi_k \psi_l}g\ket{\psi_k \psi_l} + \bra{\delta\psi_l \psi_k}g\ket{\psi_l \psi_k} \\
          & \hspace{12mm} - \bra{\delta \psi_k \psi_l}g\ket{\psi_l \psi_k} - \bra{\delta\psi_l \psi_k}g\ket{\psi_k \psi_l}] + \text{c.c.}\\
        = & \sum_{k=1}^N\bra{\delta \psi_k}h\ket{\psi_k} + \sum_{k,l = 1}^N[\bra{\delta \psi_k \psi_l}g\ket{\psi_k\psi_l} - \bra{\delta \psi_k \psi_l}g\ket{\psi_l \psi_k}] + \text{c.c.}
\end{split}
\end{equation}
where c.c. represents complex conjugate terms.
In the expression after the second equality sign, we have used that $\bra{\psi_k\delta\psi_l}g\ket{\psi_k\psi_l} = \bra{\delta\psi_l\psi_k}g\ket{\psi_l\psi_k}$, and the last
line follows from the fact that the indices $k$ and $l$ can be switched since they are dummy indices.
We next define the two operators
\begin{eqnarray}
\label{eq:J_operator}
 \mathcal{J}(\vec x)\psi_k(\vec x) & = & \sum_{l=1}^N\Big[\int d\vec x' \psi^*_l(\vec x') g(\vec r, \vec r')\psi_l(\vec x')\Big]\psi_k(\vec x), \\ \label{eq:K_operator}
 \mathcal{K}(\vec x)\psi_k(\vec x) & = & \sum_{l=1}^N\Big[\int d\vec x' \psi^*_l(\vec x') g(\vec r, \vec r')\psi_k(\vec x')\Big]\psi_l(\vec x),
\end{eqnarray}
so that the variation of the energy can be written more compactly as
\begin{equation}
 \delta E_0 = \sum_{k=1}^N\bra{\delta \psi_k} h + \mathcal J - \mathcal K \ket{\psi_k} + \text{c.c.}
\end{equation}
Next we consider the variation of the constraint:
\begin{equation}
 \sum_{k,l=1}^N[\Lambda_{lk}\langle\delta\psi_k|\psi_l\rangle + \Lambda_{lk}\langle\psi_k|\delta\psi_l\rangle].
\end{equation}
We will show that the second term of this equation is in fact the complex conjugate of the first. To realise this, consider the functional $\mathscr L$
of equation (\ref{eq:L_operator}). First note that it must be real because $E_0$ is real and the added constraints are equal to zero.
Taking the complex conjugate of $\mathscr L$ therefore gives
\begin{equation}
\begin{split}
 \mathscr L = & E_0 - \sum_{k,l=1}^N\Lambda^*_{lk}[\langle\psi_k|\psi_l\rangle^* - \delta_{kl}] \\
            = & E_0 - \sum_{k,l=1}^N\Lambda^*_{lk}[\langle\psi_l|\psi_k\rangle - \delta_{lk}] \\
            = & E_0 - \sum_{k,l=1}^N\Lambda^*_{kl}[\langle\psi_k|\psi_l\rangle - \delta_{kl}].
\end{split}
\end{equation}
This form of $\mathscr L$ is identical to the original of (\ref{eq:L_operator}) except that $\Lambda_{lk}$ has been replaced by $\Lambda^*_{kl}$.
Since both will yield exactly the same Lagrange multipliers (assuming that the solution is unique), this means that
\begin{equation}
 \Lambda_{lk} = \Lambda^*_{kl},
\end{equation}
that is to say, $\Lambda_{lk}$ are the elements of a Hermitian matrix. Thus we can write the variation of the constraint as
\begin{equation}
\begin{split}
 \sum_{k,l=1}^N[\Lambda_{lk}\langle\delta\psi_k|\psi_l\rangle + \Lambda_{lk}\langle\psi_k|\delta\psi_l\rangle] = &
 \sum_{k,l=1}^N\Lambda_{lk}\langle\delta\psi_k|\psi_l\rangle + \sum_{k,l=1}^N\Lambda^*_{kl}\langle\delta\psi_l|\psi_k\rangle^* \\
= & \sum_{k,l=1}^N\Lambda_{lk}\langle\delta\psi_k|\psi_l\rangle + \sum_{k,l=1}^N\Lambda^*_{lk}\langle\delta\psi_k|\psi_l\rangle^* \\
= & \sum_{k,l=1}^N\Lambda_{lk}\langle\delta\psi_k|\psi_l\rangle + \text{c.c.}
\end{split}
\end{equation}
Putting it all together, the variation of $\mathscr L$ is now
\begin{equation}
\begin{split}
 \delta \mathscr L &  = \sum_{k=1}^N\bra{\delta \psi_k}\Big[ (h + \mathcal J - \mathcal K)\ket{\psi_k}
                     - \sum_{l=1}^N\Lambda_{lk}\ket{\psi_l}\Big] + \text{c.c.} \\
                   & = 0.
\end{split}
\end{equation}
By defining the Fock operator
\begin{equation}
\label{eq:Fock_operator}
\mathcal{F} = h + \mathcal J - \mathcal K
\end{equation}
the above equation can be written even more compactly as
\begin{equation}
\begin{split}
  \delta \mathscr L & =\sum_{k=1}^N\bra{\delta \psi_k}\Big[\mathcal F\ket{\psi_k}
                       - \sum_{l=1}^N\Lambda_{lk}\ket{\psi_l}\Big] + \text{c.c.} \\
                    & = 0.
\end{split}
\end{equation}
Since the variations of the spin orbitals can be chosen freely, each term
in the square brackets must be equal to zero, which implies that
\begin{equation}
\label{eq:non_canonical_hartree_fock}
 \mathcal F\psi_k = \sum_{l=1}^N\Lambda_{lk}\psi_l.
\end{equation}
This equation\footnote{We are actually talking about a set of $N$ equations for all the spin orbitals
$\{\psi_k\}_{k=1}^N$, but since we observe that they are all equal, we may refer to \emph{the equation}.}
is not on the same form as the one introduced at the beginning of the chapter. This reason is as follows.
Given a solution $\{\psi_k\}$ of the equation above, it is possible to obtain a new set of spin orbitals
$\{\psi'_k\}$ via a unitary transformation
\begin{equation}
 \psi'_k = \sum_l \psi_l\ U_{lk}
\end{equation}
which keeps the expectation value $E_0 = \bra{\Psi'_0}H\ket{\Psi'_0}$ as well as the form of the Fock operator
unchanged, see Szabo and Ostlund \cite{Szabo}. Thus there is some flexibility in the choice of spin orbitals.
One particular choice of spin orbitals are the eigenfunctions of the Fock operator
\begin{equation}
\label{eq:canonical_hartree_fock}
 \mathcal F\psi_k = \varepsilon_k\psi_k,
\end{equation}
which are guaranteed to exist since $\mathcal F$ is Hermitian.
These particular spin orbitals are solutions of equation (\ref{eq:non_canonical_hartree_fock}) for the specific case where
$\Lambda_{lk} = \varepsilon_k\delta_{lk}$. Equation (\ref{eq:canonical_hartree_fock}) is called the canonical Hartree-Fock
equation.

If we are studying a molecular system, that is, if the system has more than a single nucleus,
the eigenfunctions of the Hartree-Fock equations are called \emph{molecular orbitals} (MOs). The word
molecular is used to emphasise that the orbitals are characterisitic of the molecular system.
It is important to distinguish between these and the familiar atomic orbitals because they are usually
very different. This means that for molecular systems it is no longer helpful to think of the
electrons as occupying atomic orbitals; the atomic orbitals are solutions of the Hartree-Fock equations
for \emph{isolated atoms}, but the molecule is an entirely different system with often entirely
different solutions.

When the Slater determinant $\ket{\Psi_0}$ is composed of the $N$ lowest
eigenfunctions of the Hartree-Fock equations we will call it the Hartree-Fock determinant,
and we will refer to $E_0$ as the Hartree-Fock energy. This will be the case for the
remainder of the thesis unless stated otherwise.

% Note that the Hartree-Fock equations are the same for all orbitals, which is reassuring. Recall that the Slater determinant was motivated from the fact that we wanted
% a many-particle wave function which did not distinguish between the particles. Had we gone through the exact same variational procedure as above, but instead started with the energy
% corresponding to the Hartree product, the equations would be different for different orbitals.

The Hartree-Fock energy can be written in terms of the operators $\mathcal J$ and $\mathcal K$ as
\begin{equation}
 E_0 = \sum_{k=1}^N\bra{\psi_k}h + \frac{1}{2}(\mathcal J - \mathcal K) \ket{\psi_k},
\end{equation}
which shows that the eigenvalues of the Hartree-Fock equations (\ref{eq:canonical_hartree_fock}) do not add up to the ground state energy; the term $(\mathcal J-\mathcal K)$ in the Fock operator
is a factor of two too large. However, the energy can be calculated via the eigenvalues in the following two equivalent ways:
\begin{equation}
\label{eq:E_ref2}
\begin{split}
 E_0 & = \frac{1}{2}\sum_{k=1}^N[\varepsilon_k + \bra{\psi_k}h\ket{\psi_k}] \\
         & = \sum_{k=1}^N[\varepsilon_k - \frac{1}{2}\bra{\psi_k}\mathcal J - \mathcal K\ket{\psi_k}].
 \end{split}
\end{equation}

\section{Restricted and unrestricted determinants}
\label{sec:res_and_unres_dets}
In (\ref{eq:canonical_hartree_fock}) the Hartree-Fock equations are written on their most general form. The unknowns are the eigenvalues $\varepsilon_k$ and the spin orbitals $\psi_k$.
However, before solving the equations, it is useful to rewrite them in terms of spatial orbitals $\phi_k$ instead of spin orbtitals $\psi_k$. This is done by
integrating out the spin part, as will be shown in the next section. But first we must decide how to construct the spin orbitals from the spatial orbitals. There are two
ways to do this: One can either form so-called restricted spin orbitals or unrestricted spin orbitals. The two approaches will lead to two different Hartree-Fock methods,
namely the restricted Hartree-Fock method (RHF) and the unrestricted Hartree-Fock method (UHF), respectively.

\subsection{Restricted determinants}
Recall from equation (\ref{eq:spin_orbital}) that a spin orbital $\psi_k$ is a spatial orbital $\phi_k$ multiplied with a spin function which is either spin up, $\alpha$,
or spin down, $\beta$. This means that we can create spin orbitals in the following way
\begin{equation}
\label{eq:restricted_spinorbitals}
\psi_k(\vec x) = \left\{\begin{array}{c} \phi_l(\vec r)\alpha(s) \\
                                          \text{or} \\
                                          \phi_l(\vec r)\beta(s). \\
                       \end{array}\right.
\end{equation}
Spin orbitals on this form are called restricted spin orbitals, and the Slater determinants they form are called restricted determinants. In such determinants, a spatial orbital is either
occupied by a single electron or two electrons, see figure \ref{fig:restricted_determinant}. A determinant which has
every spatial orbital doubly occupied, is called a closed shell determinant (left figure), whereas a
determinant that has one or more partially filled spatial orbitals, is called an open shell determinant (right figure).
If the system has an odd number of electrons, the determinant will always be open shell.
However, an even number of electrons does not imply that the determinant is closed shell; if degeneracies apart from that due to spin are present, it can still be open shell.
% Recall from equation (\ref{eq:spin_orbital}) that a spin orbital $\psi_k$ is a spatial orbital $\phi_k$ multiplied with a spin function which is either spin up, $\alpha$,
% or spin down, $\beta$. This means that a set of $N$ spin orbitals can be generated from a set of $\lceil N/2\rceil$ spatial orbitals in the following way:
% \begin{equation}
% \label{eq:restricted_spinorbitals}
%  \{\psi_{2k-1}(\vec x),\,\psi_{2k}(\vec x)\} = \{\phi_k(\vec r)\alpha(s),\,\phi_k(\vec r)\beta(s)\}, \qquad k = 1,\dots,\lceil N/2\rceil.
% \end{equation}
% Spin orbitals on this form are called restricted spin orbitals, and the Slater determinants they form are called restricted determinants. In such determinants, a spatial orbital is either
% occupied by a single electron or two electrons, one with spin up and one with spin down, see figure \ref{fig:restricted_determinant}. A determinant which has
% every spatial orbital doubly occupied, is called a closed shell determinant (left figure), whereas a
% determinant that has one or more partially filled spatial orbitals, is called an open shell determinant (right figure).
% If the system has an odd number of electrons, the determinant will always be open shell.
% However, an even number of electrons does not imply that the determinant is closed shell; if degeneracies apart from that due to spin are present, it can still be open shell.
\begin{figure}
 \begin{center}
  \includegraphics[scale=0.7]{hartree_fock/figures/restricted_determinant.pdf}
  \caption{Illustration of the restricted determinant comprised of spin orbitals on the form (\ref{eq:restricted_spinorbitals}).
  The left and right figures illustrate closed and open shell determinants, respectively.}
  \label{fig:restricted_determinant}
 \end{center}
\end{figure}

Throughout this thesis we will limit the use of restricted determinants (and restricted Hartree-Fock) to closed shell systems. This means that the
spin orbitals are given by
\begin{equation}
\label{eq:restricted_spinorbitals2}
 \{\psi_{2k-1}(\vec x),\,\psi_{2k}(\vec x)\} = \{\phi_k(\vec r)\alpha(s),\,\phi_k(\vec r)\beta(s)\}, \qquad k = 1,\dots,N/2,
\end{equation}
where $N$ is the number of electrons.

\subsection{Unrestricted determinants}
In equation (\ref{eq:restricted_spinorbitals}) the spin-up electrons are described by the same set of spatial orbitals as the spin-down electrons.
For closed shell systems this is often a good assumption.
However, consider the open shell determinant illustrated to the right of figure \ref{fig:restricted_determinant}. The electron occupying the spin orbital
$\phi_3\alpha$ will have an exchange interaction with the other spin-up electrons, but not with the spin-down electrons.
Hence, it could be energetically favourable to let the spin-up levels shift with respect to the spin-down levels, as shown in figure \ref{fig:unrestricted_determinant}.
This can be accomplished by letting the spin-up and spin-down electrons be described by different sets of spatial orbitals.
Spin orbitals formed in this way, are called unrestricted spin orbitals, and the Slater determinants they form are called unrestricted determinants.
\begin{equation}
\label{eq:unrestricted_spinorbitals}
\psi_k(\vec x) = \left\{\begin{array}{c} \phi^\alpha_l(\vec r)\alpha(s) \\
                                          \text{or} \\
                                          \phi^\beta_l(\vec r)\beta(s). \\
                       \end{array}\right.
\end{equation}
% In addition to  the case of open shell systems, unrestricted determinants will give lower energies when studying the dissociation of molecules. To see why, consider the
% $H_2$-molecule as an example. The restricted orbitals will force both electrons to be located at either of the nuclei with equal probability, and although
% this is reasonable for small internuclear distances, as the distance increases, the electrons will in fact be located at differenct nuclei. This can only be
% realised by the unrestricted spin orbitals which allow the electrons to have different spatial orbitals.
\begin{figure}
 \begin{center}
  \includegraphics[scale=0.7]{hartree_fock/figures/unrestricted_determinant.pdf}
  \caption{Illustration of the unrestricted determinant comprised of spin orbitals on the form (\ref{eq:unrestricted_spinorbitals}).
  The left and right figures illustrate closed and open shell determinants, respectively.}
  \label{fig:unrestricted_determinant}
 \end{center}
\end{figure}

% \subsection{Determinants and spin operators}
% The Hamiltonians which are considered in this thesis have no spin dependence. This means that it commutes with the total spin operator $\mathcal S^2$ and the spin operator
% in the z-direction $\mathcal S_z$. If we assume that the ground state of the Hamiltonian $\ket{\Psi_0}$ is nondegenerate, this means that $\ket{\Psi_0}$ is
% an eigenstate of $\mathcal S^2$ and $\mathcal S_z$ as well. To see this consider the following:
% \begin{equation}
% \begin{split}
%  H(\mathcal S^2\ket{\Psi_0}) = & ([\mathcal S^2,H] + \mathcal S^2H)\ket{\Psi_0} \\
%                              = & (0 + \mathcal S^2H)\ket{\Psi_0} \\
%                              = & E_0 \mathcal S^2\ket{\Psi_0}.
% \end{split}
% \end{equation}
% This shows that $\mathcal S^2\ket{\Psi_0}$ is also an eigenstate of $H$ with eigenvalue $E_0$, and since the ground state of $H$ is nondegenerate, this means that
% \begin{equation}
%  \mathcal S^2\ket{\Psi_0} = A \ket{\Psi_0}
% \end{equation}
% for some constant $A$.

\section{Slater determinants and the spin operators}
\label{sec:determinants_and_spin}
In this section we define the total spin operator for a system of $N$ particles and discuss how it acts on the restricted
and unrestricted determinants.
The reader is referred to the texts by Griffiths \cite{griffiths} and Shankar \cite{shankar} for introductory treatments of spin.

\subsection{Single-particle spin operators}
The spin operator for a single particle is given by
\begin{equation}
 \vec s = s_x\vec i + s_y\vec j + s_z\vec k,
\end{equation}
where $s_x$, $s_y$ and $s_z$ are the operators for the spin components along the coordinate axes. The latter satisfy the commutation relations
\begin{equation}
\label{eq:spin_commutation}
 [s_x,\, s_y] = i s_z, \qquad [s_y,\, s_z] = i s_x, \qquad [s_z,\, s_x] = i s_y.
\end{equation}
The squared magnitude of the spin operator is a scalar operator
\begin{equation}
 \vec s^2 = s_x^2 + s_y^2 + s_z^2.
\end{equation}
It commutes with all of the component operators, and it is therefore possible to find a common set of eigenstates for
$\vec s^2$ and \emph{one} of the operators $s_x$, $s_y$ or $s_z$. The standard choice in the literature is $s_z$.
A particle with spin $s$ has eigenvalues $s(s+1)$ and $m_s$
\begin{equation}
\begin{split}
 s^2 \ket{s,m_s} & = s(s+1) \ket{s,m_s}, \\
 s_z \ket{s,m_s} & = m_s \ket{s,m_s},
\end{split}
\end{equation}
where $m_s$ can take the values $\{-s,\, -s+1,\dots, s-1,\,s\}$. Electrons have spin 1/2, and the Hilbert space describing the
spin of electrons is therefore spanned by the two states $\ket{1/2,1/2}$ and $\ket{1/2,-1/2}$, which we until now have simply called $\ket{\alpha}$
and $\ket{\beta}$ for convenience.

\subsection{Many-particle spin operators}
Analogously, the total spin operator for a system of $N$ particles is given by
\begin{equation}
 \vec S = \sum_{i=1}^N \vec s(i),
\end{equation}
where $\vec s(i)$ is the spin operator of particle $i$. Also, the squared magnitude of the total spin operator is given by
\begin{equation}
 \vec S^2 = S_x^2 + S_y^2 + S_z^2,
\end{equation}
where
\begin{equation}
 S_I = \sum_{i=1}^N s_I(i), \qquad I\in\{x,\,y,\,z\}.
\end{equation}
Since the Hamiltonian does not depend on any of the spin coordinates, it commutes with $\vec S^2$ as well as $S_z$
\begin{equation}
 [H,\,\vec S^2] = [H, S_z] = 0.
\end{equation}
This means that the exact eigenstates of the Hamiltonian $\ket{\Phi_i}$ are also eigenstates of $\vec S^2$ and $S_z$ \cite{Szabo}
\begin{align}
 \vec S^2\ket{\Phi_i} & = S(S+1)\ket{\Phi_i},\\
 S_z\ket{\Phi_i} & = M_S\ket{\Phi_i},
\end{align}
where $S$ and $M_S$ are the quantum numbers for the total spin and its projection along the $z$-axis, respectively. A natural question
to ask at this point is whether or not the restricted and unrestricted determinants are eigenstates of $\vec S^2$ and $S_z$. The answer to this question is as follows:
\begin{enumerate}
 \item Both restricted and unrestricted determinants are eigenstates of $S_z$.
 \item The restricted closed shell determinant is an eigenstate of $\vec S^2$ with eigenvalue 0, making it a pure singlet state.
 \item The unrestricted determinant is generally not an eigenstate of $\vec S^2$.
\end{enumerate}
A consequence of point three is that unrestricted determinants can often have a total spin which is larger than the exact value. This is often referred
to as spin contamination of the unrestricted determinant. The reader is referred to appendix \ref{chapter:appendix_spin} for a further discussion of this topic.

\section{Restricted Hartree-Fock (RHF)}
\label{sec:RHF}
The most general form of the Hartree-Fock equations is given in (\ref{eq:canonical_hartree_fock}), where the Fock operator $\mathcal{F}$ is given by (\ref{eq:Fock_operator}) and
(\ref{eq:J_operator}) - (\ref{eq:K_operator}). We will now derive the equations which result from the restricted assumption in (\ref{eq:restricted_spinorbitals}).
Without loss of generality, we may assume that the spin orbital on which the Fock operator acts is spin up:
\begin{equation}
\mathcal F(\vec x)\phi_k(\vec r) \alpha(s) = \varepsilon_k\phi_k(\vec r)\alpha(s).
\end{equation}
Writing out the Fock operator explicitly:
\begin{equation}
\begin{split}
 \mathcal{F}(\vec x)\phi_k(\vec r)\alpha(s)  = &\ h(\vec r)\,\phi_k(\vec r)\alpha(s) \\
                      &  + \Big[\sum_{l=1}^N\int d\vec x'\psi^*_l(\vec x')g(\vec r, \vec r')\psi_l(\vec x')\Big]\phi_k(\vec r)\alpha(s) \\
                       &    - \Big[\sum_{l=1}^N\int d\vec x'\psi^*_l(\vec x')g(\vec r, \vec r')\phi_k(\vec r')\alpha(s)\Big]\psi_l(\vec x) \\
                       = &\  \varepsilon_k\phi_k(\vec r)\alpha(s).
%                    & = & h(\vec r)\,\phi_k(\vec r)\xi_k(s) + 2\Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_l(\vec r')\Big]\phi_k(\vec r)\xi_k(s) \\
%                    &   & - \Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_k(\vec r')\Big]\phi_l(\vec r)\xi_k(s).
\end{split}
\end{equation}
Our goal is to integrate out the spin of this equation. To do this, we must also express the spin orbitals in the sums in terms of spatial orbitals. We will assume that
$N$ is an even number of electrons and that we are dealing with a closed shell determinant. This means that both sums, which run from 0 to $N$, can be split
into two sums which run from 0 to $N/2$:
\begin{equation}
\begin{split}
 \mathcal{F}(\vec x)\phi_k(\vec r)\alpha(s)  = &\ h(\vec r)\,\phi_k(\vec r)\alpha(s) \\
                      &  + \Big[\sum_{l=1}^{N/2}\int d\vec x'\phi^*_l(\vec r')\alpha^*(s')g(\vec r, \vec r')\phi_l(\vec r')\alpha(s')\Big]\phi_k(\vec r)\alpha(s) \\
                      &  + \Big[\sum_{l=1}^{N/2}\int d\vec x'\phi^*_l(\vec r')\beta^*(s')g(\vec r, \vec r')\phi_l(\vec r')\beta(s')\Big]\phi_k(\vec r)\alpha(s) \\
                       & - \Big[\sum_{l=1}^{N/2}\int d\vec x'\phi^*_l(\vec r')\alpha^*(s')g(\vec r, \vec r')\phi_k(\vec r')\alpha(s')\Big]\phi_l(\vec r)\alpha(s) \\
                       & - \Big[\sum_{l=1}^{N/2}\int d\vec x'\phi^*_l(\vec r')\beta^*(s')g(\vec r, \vec r')\phi_k(\vec r')\alpha(s')\Big]\phi_l(\vec r)\beta(s) \\
                     = &\ \varepsilon_k\phi_k(\vec r)\alpha(s).
\end{split}
\end{equation}
We note that the first two sums are in fact equal when the spin coordinate $s'$ is integrated out. Furthermore, the last sum is equal to zero because the integration
is over unequal spins. Thus we are left with
\begin{equation}
\begin{split}
 \mathcal{F}(\vec x)\phi_k(\vec r)\alpha(s)  = &\ h(\vec r)\,\phi_k(\vec r)\alpha(s) \\
                      &  + 2\Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_l(\vec r')\Big]\phi_k(\vec r)\alpha(s) \\
                       & - \Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_k(\vec r')\Big]\phi_l(\vec r)\alpha(s) \\
                       = &\ \varepsilon_k\phi_k(\vec r)\alpha(s).
\end{split}
\end{equation}
If we now multiply both sides of the equation by $\alpha^*(s)$ and integrate over the spin coordinate $s$ we finally arrive at
\begin{equation}
\begin{split}
 \big[\sum_{s=\uparrow\downarrow}\alpha^*(s)\mathcal F(\vec x)\alpha(s)\Big]\phi_k(\vec r) = &\ h(\vec r)\,\phi_k(\vec r) \\
                       & + 2\Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_l(\vec r')\Big]\phi_k(\vec r) \\
                       & - \Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_k(\vec r')\Big]\phi_l(\vec r) \\
                     = &\ \varepsilon_k\phi_k(\vec r).
\end{split}
\end{equation}
Defining the restricted spatial Fock operator as
\begin{equation}
 F(\vec r) = \sum_{s=\uparrow\downarrow}\alpha^*(s)\mathcal F(\vec x)\alpha(s),
\end{equation}
the equation can be written as
\begin{equation}
\label{eq:RHF}
 F(\vec r)\phi_k(\vec r) = \varepsilon_k\phi_k(\vec r),
\end{equation}
where
\begin{equation}
\begin{split}
 F(\vec r)\phi_k(\vec r) = & h(\vec r)\,\phi_k(\vec r)  + 2\Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_l(\vec r')\Big]\phi_k(\vec r) \\
                       & - \Big[\sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_k(\vec r')\Big]\phi_l(\vec r).
\end{split}
\end{equation}
The spatial Fock operator can be written more compactly as
\begin{equation}
\label{eq:restricted_Fock_operator}
 F(\vec r) = h(\vec r) + 2 J(\vec r) - K(\vec r),
\end{equation}
where
\begin{align}
 J(\vec r)\phi_k(\vec r) & = \sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_l(\vec r')\Big]\phi_k(\vec r), \\
 K(\vec r)\phi_k(\vec r) & = \sum_{l=1}^{N/2}\int d\vec r'\phi^*_l(\vec r')g(\vec r, \vec r')\phi_k(\vec r')\Big]\phi_l(\vec r).
\end{align}
Note the factor 2 in front of the direct term, or more to the point, the absence of the factor 2 in front of the exchange term. This is due to the fact that
the exchange interaction is only present between electrons of equal spins.

To find the energy, consider the first line of equation (\ref{eq:E_ref2}):
\begin{equation}
\begin{split}
 E_0 = & \frac{1}{2}\sum_{k=1}^N[\varepsilon_k + \bra{\psi_k}h\ket{\psi_k}] \\
         = & \frac{1}{2}\sum_{k=1}^{N}\bra{\psi_k}(\mathcal{F} + h)\ket{\psi_k}.
\end{split}
\end{equation}
If, as we did above, insert the assumption (\ref{eq:restricted_spinorbitals}) and split the sum in two sums, one with spin up and one with spin down, we get
\begin{equation}
\label{eq:E_ref3}
 E_0 = \sum_{k=1}^{N/2}\bra{\phi_k}(F + h)\ket{\phi_k}.
\end{equation}

\subsection{Introducing a basis}
We have now eliminated the spin from the general Hartree-Fock equations (\ref{eq:canonical_hartree_fock}) and arrived at equation (\ref{eq:RHF}), which represents
a set of integro-differential equations for the spatial orbitals. There is presently no feasible way to solve these as they stand.
However, by expressing the orbitals in terms of some known basis
\begin{equation}
\label{eq:linear_expansion}
 \phi_k(\vec r) = \sum_{\mu=1}^M C_{\mu k}\chi_\mu(\vec r),
\end{equation}
where $M$ is the number of basis functions, the equations can be converted to a set of algebraic equations. Inserting this expansion into equation (\ref{eq:RHF}) gives
\begin{equation}
 \sum_{\nu=1}^M F(\vec r)\chi_\nu(\vec r) C_{\nu k} = \varepsilon_k \sum_{\nu=1}^M C_{\nu k} \chi_{\nu}(\vec r).
\end{equation}
If we now multiply by $\chi^*_\mu(\vec r)$ and integrate with respect to $\vec r$, we get the so-called \emph{Roothaan equations} \cite{roothan}
\begin{equation}
 \sum_{\nu=1}^M F_{\mu\nu}C_{\nu k} = \varepsilon_k\sum_{\nu=1}^M S_{\mu\nu}C_{\nu k},
\end{equation}
or in matrix notation
\begin{equation}
\label{eq:Roothaan}
 \mathbf{FC}_k = \varepsilon_k\mathbf{SC}_k,
\end{equation}
where
\begin{equation}
 F_{\mu\nu} = \int d\vec r \chi^*_\mu(\vec r)F(\vec r)\chi_\nu(\vec r)
\end{equation}
is the Fock matrix and
\begin{equation}
 S_{\mu\nu} = \int d\vec r \chi^*_\mu(\vec r)\chi_\nu(\vec r)
\end{equation}
is the overlap matrix. If the basis is orthonormal, the overlap matrix is the identity matrix. However, in molecular calculations Gaussian functions are
most often used, and these are not orthogonal. In section \ref{sec:gen_eig} we show how equation (\ref{eq:Roothaan}) can be transformed to a regular
eigenvalue problem
\begin{equation}
 \mathbf{F'C'_k} = \varepsilon_k\mathbf C'_k.
\end{equation}

Let us take a closer look at the Fock matrix:
\begin{equation}
 \begin{split}
  F_{\mu\nu} = & \int d\vec r \chi^*_\mu(\vec r) F(\vec r)\chi_\nu(\vec r)\\
                  = & \int d\vec r \chi^*_\mu(\vec r)[h(\vec r) + 2J(\vec r) - K(\vec r)]\chi_\nu(\vec r) \\
                  = & \bra\mu h\ket\nu + 2 \bra\mu J\ket\nu - \bra\mu K\ket\nu.
 \end{split}
\end{equation}
To get the final expression for the matrix, we insert the expansion (\ref{eq:linear_expansion}) into the operators $J(\vec r)$ and $K(\vec r)$.
For $J(\vec r)$ this gives
\begin{equation}
\begin{split}
 J(\vec r)\chi_\nu(\vec r) & = \Big[\sum_{k=1}^{N/2}\int d\vec r'\phi^*_k(\vec r')g(\vec r, \vec r')\phi_k(\vec r')\Big]\chi_\nu(\vec r) \\
                             & = \Big[\sum_{k=1}^{N/2}\sum_{\sigma,\lambda=1}^M\int d\vec r'\chi^*_\sigma(\vec r')g(\vec r, \vec r')\chi_\lambda(\vec r')\Big]C^*_{\sigma k}C_{\lambda k}\chi_\nu(\vec r),
\end{split}
\end{equation}
so that
\begin{equation}
 \bra\mu J\ket\nu = \sum_{k=1}^{N/2}\sum_{\sigma,\lambda=1}^M\bra{\mu\sigma}g\ket{\nu\lambda}C^*_{\sigma k}C_{\lambda k},
\end{equation}
where the matrix element $\bra{\mu\sigma}g\ket{\nu\lambda}$ is defined in equation (\ref{eq:pqgrs_def}).
The expression for $\bra\mu K\ket\nu$ is similar, except that the indices $\nu$ and $\lambda$ switch places in the integral:
\begin{equation}
 \bra\mu K\ket\nu = \sum_{k=1}^{N/2}\sum_{\sigma,\lambda=1}^M\bra{\mu\sigma}g\ket{\lambda\nu}C^*_{\sigma k}C_{\lambda k}.
\end{equation}
Hence, the Fock matrix is given explicitly in terms of the basis functions by
\begin{equation}
 F_{\mu\nu} = \bra\mu h\ket\nu + \sum_{k=1}^{N/2}\sum_{\sigma,\lambda=1}^M[2\bra{\mu\sigma}g\ket{\nu\lambda} - \bra{\mu\sigma}g\ket{\lambda\nu}]C^*_{\sigma k}C_{\lambda k}.
\end{equation}
It is useful to define the so-called density matrix
\begin{equation}
\label{eq:density_matrix}
 P_{\lambda\sigma} = 2\sum_{k=1}^{N/2}C_{\lambda k}C^*_{\sigma k},
\end{equation}
which allows us to write the Fock matrix more compactly as
\begin{equation}
\label{eq:Fock_matrix}
 F_{\mu\nu} = \bra\mu h\ket\nu + \frac{1}{2}\sum_{\sigma,\lambda=1}^M P_{\lambda\sigma}[2\bra{\mu\sigma}g\ket{\nu\lambda} - \bra{\mu\sigma}g\ket{\lambda\nu}].
\end{equation}

The energy is found by expanding equation (\ref{eq:E_ref3}) in the known basis:
\begin{equation}
 \label{eq:E_ref4}
 E_0 = \frac{1}{2}\sum_{\mu,\nu=1}^M P_{\nu\mu}[\bra\mu h \ket\nu + F_{\mu\nu}].
\end{equation}

% Inserting this into equation (\ref{eq:RHF}), multiplying by $\chi^*_p(\vec r)$ and integrating with respect to $\vec r$ leads to the following set of eigenvalue equations
% \begin{equation}
%  \mathbf{FC_k} = \varepsilon_k\mathbf{SC_k}
% \end{equation}
% where the elements of the Fock matrix $\mathbf{F}$ are
% \begin{equation}
%  F_{pq} = \bra{p}h\ket{q} + \sum_l\sum_{rs}(2\bra{pr}g\ket{qs} - \bra{pr}g\ket{sq})C^*_{rl}C_{sl}
% \end{equation}
% and the elements of the overlap matrix $\mathbf{S}$ are
% \begin{equation}
%  S_{pq} = \langle p|q\rangle.
% \end{equation}
% It is convenient to introduce the density matrix $P_{rs} = 2\sum_lC^*_{rl}C_{sl}$. The Fock matrix is then written more compactly as
% \begin{equation}
%  F_{pq} = \bra{p}h\ket{q} + \frac{1}{2}\sum_{rs}P_{rs}(2\bra{pr}g\ket{qs} - \bra{pr}g\ket{sq}).
% \end{equation}
% The energy is given by
% \begin{equation}
%  E_{ref} = \sum_{pq}P_{pq}\bra{p}h\ket{q} + \frac{1}{2}\sum_{pqrs}P_{pq}P_{rs}(\bra{pr}g\ket{qs} - \frac{1}{2}\bra{pr}g\ket{sq})
% \end{equation}

\section{Unrestricted Hartree-Fock (UHF)}
\label{sec:UHF}
We now derive the unrestricted Hartree-Fock equations which result from the assumption (\ref{eq:unrestricted_spinorbitals}). The Hartree-Fock equations (\ref{eq:canonical_hartree_fock})
are now split into two sets of equations
\begin{align}
\mathcal F^\alpha(\vec x) \phi^\alpha_k(\vec r)\alpha(s) & = \varepsilon^\alpha_k\phi^\alpha_k(\vec r)\alpha(s), \qquad k=1,2,\dots, N_\alpha, \\
\mathcal F^\beta(\vec x) \phi^\beta_k(\vec r)\beta(s) & = \varepsilon^\beta_k\phi^\beta_k(\vec r)\beta(s), \qquad k=1,2,\dots, N_\beta,
\end{align}
where $N^\alpha$ and $N^\beta$ are the number of electrons with spin up and spin down, respectively.
Again, we want to take out the spin part of the equations. We can insert the unrestricted spin orbitals (\ref{eq:unrestricted_spinorbitals}) and do the calculations
explicitly in the same way as we did for the restricted case. However, using the insight gained during our previous calculation, we can come up with the answer directly. Consider
for example the operator $\mathcal F^\alpha(\vec x)$. It contains a kinetic term and the potential due to the atomic nuclei. Furthermore, it contains
a direct interaction term due to the mean field set up by all electrons, both spin up and spin down. Finally it contains an exchange term due to the mean field
set up by the electrons \emph{with spin up only}. Thus we can conclude that the equations for the spatial orbitals $\phi^\alpha_k$ are given by
\begin{equation}
\label{eq:HF^alpha}
 F^\alpha(\vec r)\phi^\alpha_k(\vec r) = \varepsilon^\alpha_k\phi^\alpha_k(\vec r),
\end{equation}
where the unrestricted spin up Fock operator is defined as
\begin{equation}
\label{eq:Foperator^alpha}
 F^\alpha(\vec r) = h(\vec r) + [J^\alpha(\vec r) - K^\alpha(\vec r)] + J^\beta(\vec r),
\end{equation}
with
\begin{align}
 J^\alpha(\vec r)\phi^\alpha_k(\vec r) = & \sum_{l=1}^{N^\alpha}\Big[\int d\vec r' \phi^\alpha_l(\vec r')g(\vec r, \vec r')\phi^\alpha_l(\vec r')\Big]\phi^\alpha_k(\vec r), \\
 J^\beta(\vec r)\phi^\alpha_k(\vec r) = & \sum_{l=1}^{N^\beta}\Big[\int d\vec r' \phi^\beta_l(\vec r')g(\vec r, \vec r')\phi^\beta_l(\vec r')\Big]\phi^\alpha_k(\vec r), \\
 K^\alpha(\vec r)\phi^\alpha_k(\vec r) = & \sum_{l=1}^{N^\alpha}\Big[\int d\vec r' \phi^\alpha_l(\vec r')g(\vec r, \vec r')\phi^\alpha_k(\vec r')\Big]\phi^\alpha_l(\vec r).
\end{align}
The equations for the spatial orbitals $\phi^\beta_k$ are the same, except that the indices $\alpha$ and $\beta$ switch places.

\subsection{Introducing a basis}
To solve the unrestricted Hartree-Fock equations we expand the spatial orbitals in terms of a known basis
\begin{align}
 \phi^\alpha_k(\vec r) & = \sum_{\mu=1}^M C^\alpha_{\mu k} \chi_\mu(\vec r), \label{eq:phi^alpha_expansion}\\
 \phi^\beta_k(\vec r) & = \sum_{\mu=1}^M C^\beta_{\mu k} \chi_\mu(\vec r), \label{eq:phi^beta_expansion}
\end{align}
just as we did in the restricted case. Inserting the expansion (\ref{eq:phi^alpha_expansion}) into equation (\ref{eq:HF^alpha}) gives
\begin{equation}
 \sum_{\nu=1}^M F^\alpha(\vec r)\chi_\nu(\vec r)C^\alpha_{\nu k} = \varepsilon^\alpha_k\sum_{\nu=1}^M C^\alpha_{\nu k}\chi_\nu(\vec r).
\end{equation}
If we multiply this equation by $\chi^*_\mu(\vec r)$ and integrate over $\vec r$, we get
\begin{equation}
 \sum_{\nu=1}^M F^\alpha_{\mu\nu}C^\alpha_{\nu k} = \varepsilon^\alpha_k\sum_{\nu=1}^M S_{\mu\nu}C^\alpha_{\nu k},
\end{equation}
where $S_{\mu\nu}$ is the overlap matrix and $F^\alpha_{\mu\nu}$ is the matrix representation of the unrestricted spatial Fock operator
\begin{equation}
\label{eq:Fmatrix^alpha}
 F^\alpha_{\mu\nu} = \int d\vec r \chi^*_\mu(\vec r) F^\alpha(\vec r)\chi_\nu(\vec r).
\end{equation}
The corresponding equations for the spin down particles are derived in exactly the same way, of course. In total we have the two sets of equations
\begin{align}
 \mathbf F^\alpha \mathbf C^\alpha_k = & \varepsilon^\alpha_k\mathbf{SC}^\alpha_k, \label{eq:pople-nesbet_up}\\
 \mathbf F^\beta \mathbf C^\beta_k = & \varepsilon^\beta_k\mathbf{SC}^\beta_k, \label{eq:pople-nesbet_down}
\end{align}
called the \emph{Pople-Nesbet equations} \cite{pople_nesbet}, which are the unrestricted generalisation of the Roothaan equations derived in the previous section. They are nonlinear
and coupled since the matrices $\mathbf F^\alpha$ and $\mathbf F^\beta$ are functions of both $\{\mathbf C^\alpha_k\}$ and $\{\mathbf C^\beta_k\}$.

The final expression for the Fock matrix $\mathbf F^\alpha$ is obtained by inserting the expansion (\ref{eq:phi^alpha_expansion}) into equation (\ref{eq:Fmatrix^alpha}).
Recalling that the Fock operator $F^\alpha(\vec r)$ is given by (\ref{eq:Foperator^alpha}) this leads to
\begin{equation}
 F^\alpha_{\mu\nu} = \bra{\mu}h\ket{\nu} + \sum_{k=1}^{N^\alpha}\sum_{\sigma,\lambda=1}^M\langle\mu\sigma||\nu\lambda\rangle(C^\alpha_{\sigma k})^* C^\alpha_{\lambda k}
                                         + \sum_{k=1}^{N^\beta}\sum_{\sigma,\lambda=1}^M\bra{\mu\sigma}g\ket{\nu\lambda}(C^\beta_{\sigma k})^* C^\beta_{\lambda k},
\end{equation}
and similarly we find
\begin{equation}
 F^\beta_{\mu\nu} = \bra{\mu}h\ket{\nu} + \sum_{k=1}^{N^\beta}\sum_{\sigma,\lambda=1}^M\langle\mu\sigma||\nu\lambda\rangle(C^\beta_{\sigma k})^* C^\beta_{\lambda k}
                                         + \sum_{k=1}^{N^\alpha}\sum_{\sigma,\lambda=1}^M\bra{\mu\sigma}g\ket{\nu\lambda}(C^\alpha_{\sigma k})^* C^\alpha_{\lambda k},
\end{equation}
where $\bra{\mu\sigma}g\ket{\nu\lambda}$ and $\langle\mu\sigma||\nu\lambda\rangle$ are defined in equations (\ref{eq:pqgrs_def}) and (\ref{eq:pqgrs_as_def}),
respectively. If we introduce the density matrices
\begin{align}
  P^\alpha_{\lambda\sigma} = & \sum_{k=1}^{N^\alpha}C^\alpha_{\lambda k} (C^\alpha_{\sigma k})^*, \label{eq:density_matrix_up}\\
  P^\beta_{\lambda\sigma} = & \sum_{k=1}^{N^\beta}C^\beta_{\lambda k} (C^\beta_{\sigma k})^*, \label{eq:density_matrix_down}\\
  P^T_{\lambda\sigma} = & P^\alpha_{\lambda\sigma} + P^\beta_{\lambda\sigma},
\end{align}
the Fock matrices can be written more compactly as
\begin{equation}
\label{eq:Fock_matrix_up}
 F^\alpha_{\mu\nu} = \bra{\mu}h\ket{\nu} + \sum_{\sigma,\lambda=1}^M\left[\langle\mu\sigma||\nu\lambda\rangle P^\alpha_{\lambda\sigma}
                                                                      + \bra{\mu\sigma}g\ket{\nu\lambda}P^\beta_{\lambda\sigma}\right],
\end{equation}
and
\begin{equation}
\label{eq:Fock_matrix_down}
 F^\beta_{\mu\nu} = \bra{\mu}h\ket{\nu} + \sum_{\sigma,\lambda=1}^M\left[\langle\mu\sigma||\nu\lambda\rangle P^\beta_{\lambda\sigma}
                                                                  + \bra{\mu\sigma}g\ket{\nu\lambda}P^\alpha_{\lambda\sigma}\right].
\end{equation}

The energy can be found from the expectation value of these matrices, keeping in mind that the double counting of the interaction terms must be taken into account.
The expression is
\begin{equation}
 E_0 = \frac{1}{2}\sum_{k=1}^{N^\alpha}\sum_{\mu,\nu=1}^M \big[\bra\mu h\ket\nu + F^\alpha_{\mu\nu} \big](C^\alpha_{\mu k})^* C^\alpha_{\nu k}
          +\frac{1}{2}\sum_{k=1}^{N^\beta}\sum_{\mu,\nu=1}^M\left[\bra\mu h\ket\nu + F^\beta_{\mu\nu}\right](C^\beta_{\mu k})^* C^\beta_{\nu k},
\end{equation}
or in terms of the density matrices
\begin{equation}
 E_0 = \frac{1}{2}\sum_{\mu,\nu=1}^M\left[P^T_{\nu\mu}\bra\mu h\ket\nu + P^\alpha_{\nu\mu}F^\alpha_{\mu\nu} + P^\beta_{\nu\mu}F^\beta_{\mu\nu}\right].
\end{equation}

\section{Solving the generalised eigenvalue problem}
\label{sec:gen_eig}

In this section we show how the generalised eigenvalue problem
\begin{equation}
 \mathbf{FC}_k = \varepsilon_k\mathbf{SC}_k,
\end{equation}
can be transformed to the regular eigenvalue problem
\begin{equation}
 \mathbf F'\mathbf C'_k = \varepsilon_k\mathbf C'_k.
\end{equation}
We can achieve this if there exists a matrix $\mathbf X$ such that
\begin{equation}
 \mathbf{X^\dagger S X = I},
\end{equation}
because, if we then let $\mathbf{C_k = X C'_k}$, we get
\begin{equation}
\begin{split}
 \mathbf{FC}_k & = \varepsilon_k\mathbf{SC}_k \\
 \mathbf{FXC}'_k & = \varepsilon_k\mathbf{SXC}'_k \\
 \mathbf{X^\dagger FXC}'_k & = \varepsilon_k\mathbf{X^\dagger SXC}'_k \\
 \mathbf F' \mathbf C'_k & = \varepsilon_k\mathbf C'_k,
\end{split}
\end{equation}
where
\begin{equation}
 \mathbf{F' = X^\dagger FX}.
\end{equation}
It only remains to show that the matrix $\mathbf X$ does indeed exist and how to construct it. First note that the overlap matrix $\mathbf S$ is Hermitian:
\begin{equation}
\begin{split}
 S_{\mu\nu} = & \int d\vec r \chi^*_\mu(\vec r) \chi_\nu(\vec r) \\
            = & \int d\vec r \chi_\nu(\vec r) \chi^*_\mu(\vec r) \\
            = & S^*_{\nu\mu}.
\end{split}
\end{equation}
This means that there exists a unitary matrix $\mathbf U$ such that
\begin{equation}
\label{eq:similarity_transf}
 \mathbf{U^\dagger S U = s},
\end{equation}
where $\mathbf s =$ diag$(s_1, s_2, \dots, s_M)$ is a diagonal matrix containing the eigenvalues of $\mathbf S$, which are all real, and the columns of $\mathbf U$ are the eigenvectors of $\mathbf S$.
Furthermore, the eigenvalues $\{s_i\}$ are positive. To see this, consider the expansion of some function $f(\vec r)$, not identically equal to zero, in terms of the basis functions $\chi_\mu(\vec r)$:
\begin{equation}
 f(\vec r) = \sum_\mu A_\mu\chi_\mu(\vec r).
\end{equation}
No matter how the coefficients are chosen, the norm of $f(\vec r)$ will be positive. In particular, if we choose $\mathbf{A}=[A_\mu]$ to be equal to the $i$'th eigenvector of $\mathbf S$, we get
\begin{equation}
\begin{split}
 0 < \langle f|f\rangle = & \sum_{\mu\nu}A^*_\mu S_{\mu\nu} A_\nu = \mathbf{A^\dagger S A} \\
                    = & s_i\mathbf{A^\dagger A} =  s_i ||\mathbf A||^2.
\end{split}
\end{equation}
Now, since all eigenvalues are positive, one can define the matrix
\begin{equation}
 \mathbf s^{-1/2} = \left[\begin{array}{c c c c}
                          s_1^{-1/2} &&& \\
                          & s_2^{-1/2} && \\
                          && \ddots & \\
                          &&& s_M^{-1/2}
                         \end{array}\right].
\end{equation}
Multiplying equation (\ref{eq:similarity_transf}) from the left and right by $\mathbf s^{-1/2}$ yields
\begin{equation}
\begin{split}
 \mathbf s^{-1/2} \mathbf U^\dagger \mathbf{S U s}^{-1/2} =  \mathbf I \\
 (\mathbf{Us}^{-1/2})^\dagger \mathbf S (\mathbf{Us}^{-1/2}) =  \mathbf I,
\end{split}
\end{equation}
which means that
\begin{equation}
 \mathbf X = \mathbf{Us}^{-1/2}.
\end{equation}


%!TEX root = ../../dissertation.tex

\section{Calibration} \label{sec:calibration}

The structural model laid out in Section~\ref{sec:sdmodel} is dependent on many parameters and several
distributional assumptions. In an attempt to prevent ourselves from leveraging this fact too much
and to avoid overfitting the data, we use, where possible, parameter values and distributional
assumptions that have been previously used in peer-reviewed research anywhere possible. This won't
work for all of the model parameters since certain parameters are sensitive to the model
specificiation. For example, the discrete choice shocks, $\pi_e$ and $\pi_c$, need to be determined
for our particular model because the effect they have on choices is dependent on the model's Bellman
equations. For parameters that we can't determine externally, we use a simulated method of moments
approach and aim to match certain moments from the data.

We describe the data sources we use and how we computed the relevant moments in
Appendix~\ref{sec:data_appendix}.

\subsection{Distributional Assumptions}

  We discretize the types by drawing $N_j=200$ types, $j := (m^j, k_1^j, q^j, z^j)$, distributed
  according to the following distribution:

  Let $\{ \varepsilon_j^m, \varepsilon_j^k, \varepsilon_j^q, \varepsilon_j^z \}_j$ be independent
  and identically distributed standard normals\footnote{In practice, we draw the $\varepsilon$s
  using Sobol sequences rather than randomly to provide better coverage of the type space}. Then

  \begin{align*}
    k^j &= \max \{0, \mu_k + \sigma_k \frac{\varepsilon_j^k + \alpha_{km} \varepsilon_j^m}{\sqrt{1 + \alpha_{km}^2}} \} \\
    m^j &= \frac{\varepsilon_j^m + \alpha_{mq} \varepsilon_j^q + \alpha_{mz} \varepsilon_j^z}{\sqrt{1 + \alpha_{mq}^2 + \alpha_{mz}^2}} \\
    q^j &= \mu_q + \sigma_q \frac{\varepsilon_j^q + \alpha_{qz} \varepsilon_j^z}{\sqrt{1 + \alpha_{qz}^2}} \\
    z^j &= \max \{0, \mu_z + \sigma_z \varepsilon_j^z \}
  \end{align*}

  Additionally, we assume that an individual's ability, conditional on their GPA ($m$) is drawn from

  $$a \sim N(\rho m, \sqrt{1 - \rho^2})$$

  where $\rho := \frac{\alpha_{am}}{\sqrt{1 + \alpha_{am}^2}}$ is the correlation between $a$ and
  $m$. We do this by choosing $N_a=9$ discrete levels of $a$\footnote{We choose these values of $a$
  such that $a \in \{\bar{a}_1, \dots, \bar{a}_{N_a}$ with $\bar{a}_1 < \bar{a}_2 < \dots <
  \bar{a}_{N_a}$ such that each ability level is drawn with equal probability} and computing the
  implied joint probability distribution over the discrete $a, j$ pairs.

  We discretize $\zeta$ into $N_\zeta=4$ possible values. A value $\zeta_i$ determines two financial
  states of college $dq^i$ and $\Omega_v^i$. We choose the form of the per-period cost of college as
  $q(j, \zeta_i) = q_j + dq^i$. $\Omega_v$ represents the choice set of hours that an individual can
  choose to work while in school and is chosen to match various levels of part-time to
  full-time\footnote{For more details on how these values were determined, see
  \cite{HendricksLeukhina2017}}. $\zeta$ follows a Markov chain, so the cost of college and the
  hours an individual can work while in college potentially fluctuate while they are enrolled.

  We assume the age specific components of income for each education level, $\theta^s(t)$, follows a
  cubic polynomial in age as in \cite{Guvenen2009} and others

  $$\theta^s(t) = \theta^s_0 + \theta^s_1 \text{age} + \theta^s_2 \frac{\text{age}^2}{100} + \theta^s_3 \frac{\text{age}^3}{1000}$$

\subsection{Parameters}

  The model parameters are determined in three stages:

  \subsubsection{Previous Knowledge}

    The previous knowledge parameters consist of parameters that are either widely used or come from
    specific research papers. The model does not seem to be particularly sensitive to small
    deviations of these parameters.

    These parameters are listed in Table~\ref{table:pk}.

  \subsubsection{Externally Calibrated}

    Our externally calibrated parameters are parameters that have some form of a data counterpart
    that can be used to line the model up with the data. This ensures that the model will match a
    particular piece of the data, but that it cannot use these parameters to match unrelated pieces
    of data. We now briefly describe how we determined parameters that fell in this category and
    the associated values can be found in Table~\ref{table:ec}.

    We find the age specific component of income by following standard procedures for
    estimating a dynamic income process. We use PSID data on family income and restrict the sample
    to working age (24-60) males and remove those who have certain irregularities such as too low
    of yearly earnings or too many hours worked\footnote{We follow decisions made by
    \cite{Guvenen2009} and \cite{Hryshko2012} when selecting a subsample to work with}. We needed
    to compute the age specific components on our own because the income dynamics literature
    typically focuses on the residual income so these values were not reported.

    We choose to set the number of classes per year to 6 with each class worth 6 hours of
    credit\footnote{This in effect means that each course in our model corresponds to 2 courses in
    the data since most courses in college are worth 3-4 hours of credit} in the data. This means
    the maximum number of hours an individual can earn in a given year is 36 which, according to
    \cite{HendricksLeukhina2017}, is the 90th percentile of credits earned. Additionally, we set the
    maximum amount of time that students can stay in school to 6 years since less than 5\% of
    college graduates take more time than this to graduate.

    The data used in \cite{HendricksLeukhina2017} corresponds to students who were in college in the
    mid to late 1980s. We follow their lead and set the cumulative student loan limit to -\$19,750
    (in 2000 dollars) which approximately matches the 1986 limit of \$12,500.

    A list of the externally calibrated parameters can be found in Table~\ref{table:ec}

  \subsubsection{Internally Calibrated}

    The final group of parameters are parameters that do not have any data based counterpart. We
    determine values for these parameters by choosing certain features of the data that we think are
    informative about these parameters and then numerically minimize the difference between these
    features in the data and in our model.

    This includes the parameters $\delta_c$, $\delta_v$, $\pi_e$, $\pi_c$, $U_{HS}$, $U_{CD}$,
    $U_{CG}$, $\alpha_{km}$, $\mu_k$, and $\sigma_k$. We choose these jointly to match statistics on
    enrollment by HS GPA quartiles, completion by HS GPA quartiles, yearly dropout rates, yearly
    dropout rates by HS GPA quartiles, yearly fractions of college students with student loans,
    yearly average student loans, and college earnings. The parameter values for these parameters
    can be found in Table~\ref{table:ic}.

\subsection{Model Fit}

  The data and model values for the moments we described are displayed in
  Tables~\ref{table:moments}. Overall, we find that our model is able to accurately capture the
  moments of the data that we are interested in. We discuss some specific examples below:

  A mixture of the discrete choice shocks and disutility of being a college dropout/graduate helps
  us successfully match the enrollment and drop out rates quite accurately. While the overall yearly
  drop out rate matches quite well, we do slightly worse for the dropout rates by year and HS GPA
  quartile. In particular, our model predicts that those in the low GPA quartiles drop out a little
  slower than in the data, but those in the high GPA quartiles drop out a little more quickly than
  in the data. Additionally, we consider it a success that using the credit accumulation parameters
  from \cite{HendricksLeukhina2017} results in us accurately matching the educational fractions and
  completion rates.

  One feature that we struggled to match was that in our model too few people accumulated debt
  during their first year of college (11\% of students in our model had debt after the first year
  whereas the data says that 26\% of students have debt after their first year).


\documentclass[a4paper]{article}
\usepackage{amsmath, amssymb, textcomp}
\usepackage{graphicx}
\usepackage{cite}
\usepackage[hidelinks]{hyperref}
\usepackage[section]{placeins}
\usepackage[font=footnotesize,labelfont=bf]{caption}
\newcommand{\Part}[3][ ]{\ensuremath{\frac{\partial^{#1} #2}{{\partial #3}^{#1}}}}
\newcommand{\Dif}[3][ ]{\ensuremath{\frac{d^{#1} #2}{{d #3}^{#1}}}}
\renewcommand{\O}[1]{\ensuremath{\mathcal{O}\left(#1\right)}}
\renewcommand{\vec}{\bold}

\begin{document}
\title{Plasma Analysis}
\author{Tobias de Jong \& David Kok}
\date{\today}
\maketitle

\section{Introduction}
%{\bf Fysische achtergrond, motivatie voor numeriek. Bonuspunten voor gebruik van het woord Topologisch }
In this project some topological properties of magnetic field lines in a plasma are determined.
The numerical computations involved are heavily parallelizable.
For this reason computations are performed on a GPU, which can run thousands threads in parallel, instead of the CPU which on modern computers can only run $\sim$10 parallel processes.
We use CUDA to write code to efficiently calculate the topological properties of magnetic field lines as generated in recent full-magnetohydrodynamical numerical studies of magnetic reconnection.~\cite{PhysRevLett.115.095001}

This work concentrates on plasmas in which the field lines have a non-trivial topological order, such as interlocking rings.
As linking is a conserved quantity in MHD, this topological order enhances the stability of the plasma.

The studied system consists of the evolution through time of the plasma, with an initial configuration of two or more interlocked rings. These rings evolve into a set of nested tori.
In this work we will compute topological properties of these tori, foremost the winding number.

To implement the code we were inspired by Marek Fiser.\footnote{\url{http://www.marekfiser.com/Projects/Real-time-visualization-of-3D-vector-field-with-CUDA}} He uses Runge-Kutta integration to visualize (air) flows around a Delta wing, but the idea of integration of a vector field is the same.

\section{Theory and Methods}
\subsection{Theory of field lines}
We want to calculate properties of the magnetic field, using the field lines associated to the field. In order to do this we need (finite segments of) field lines. To describe these field lines it is sufficient to note that field lines are the solutions $\vec x(s)$ to the following ordinary differential equation:
\[\Dif{\vec x}{s} = \vec B(\vec x),\]
where $s$ is the parametrization variable.\\
An interesting observation made by \cite{taylor1986relaxation} is that in an ideal plasma the total helicity of the field $H_{\text m}$, defined through the volume integral:
\[
	H_{\text m} = \int\vec{A}\cdot\vec{B}\ d\vec{x} = \int\vec{A}\cdot (\nabla\times\vec{A})d\vec{x}
\]
is conserved. Therefore the initial value of this integral restricts the possible long-term dynamics of a plasma.

In particular we are working with a dataset of highly helical initial data, in which case the field lines will align themselves along the surfaces of nested tori.
A line on the surface of a torus can be associated with a winding number, which is the ratio of the winding along the toroidal axis (denoted with $\alpha$) and the winding along the poloidal axis (denoted $\beta$, see Figure \ref{fig:coordsys}).
To calculate a field line and its winding numbers we will use numerical integration as described in more detail below.

\subsection{Methods}
To compute the properties of interest we use a CUDA program.\footnote{Full code available on Github: \url{https://github.com/TAdeJong/plasma-analysis}}
Our code is split into two main segments.

Firstly we numerically integrate the magnetic field to obtain an approximation of a field line in the plasma.
Secondly we compute several properties of the magnetic field using this field line, such as the length of the line (which estimates the average field strength), the center of the line (estimating the origin of the torus on which it lies) and its winding number, i.e. its contribution to the total helicity of the magnetic field.

Both steps can be parallelized heavily. To parallelize the numerical integration of the magnetic field we simply consider more than one field line at a time.
In the second step we not only consider multiple field lines at once, but also parallelize the computation of the aforementioned parameters per field line.

Both the fact that we need to calculate many field lines and the fact that all field lines are independent, with almost no conditional steps in the property calculation, indicate that a GPU will be a powerful instrument to tackle this problem.

\section{Numerical Aspects}
\subsection{Vector field representation}
Note that we are interested in the field lines for fixed moments in time, so all our analysis will not take into account any time evolution of the plasma.
Thus the magnetic field is a pure 3D vector field $\vec{B}: \mathbb{R}^3 \to \mathbb{R}^3$. To represent this field in a computer a discretization is needed.
This is done by defining a rectangular grid in space and saving the components of the field at each grid point.

There are two ways to save all the components: either separate the components and save a three-dimensional array for each component of the vector field (three $N\times N\times N$ arrays), or save one large array of which each component contains the whole vector at the corresponding gridpoint (one $N\times N\times N$ array of $3D$ vectors).

Since numerical integration algorithms will use all the components of the field at a point at the same time we use the second method.
This choice of representation is both physically intuitive as well as efficient, since it will improve cache coherency and therefore access times.

\subsubsection{Interpolation}
In order to accurately integrate we need accurate estimates of the original vector field at arbitrary locations.
It is unfeasable to have a discretization fine enough such that we have a value for each location of interest.
We therefore want to interpolate the vector field to get a good approximation of the value of the vector field at an arbitrary location.
The simplest method for interpolation is linear interpolation, for 3D data known as trilinear interpolation.

Interpolation of two- and even three-dimensional data is used in graphics processing in order to map images onto three-dimensional objects.
In GPU-parlance this is known as texturing. As this problem is very important in graphics processing, modern GPU's have efficient texturing units, performing hardware accelerated trilinear interpolation.

Using CUDA enables us to use this hardware accelerated interpolation by storing our vector field data as a texture.
Since every RK4-step needs at least 4 vector field fetches, this acceleration significantly speeds up the numerical integration.

\subsection{Runge-Kutta method}
For line integration the fourth order Runge-Kutta method (RK4) was used.  This method is known as a good allround integration method.
Compared to standard Forward Euler, RK4 uses four values of the magnetic field to calculate the next point on the field line.
This increase in calculation time per step is compensated by a vastly reduced error per step, allowing for a much larger step size $\Delta s$, since RK4 has a much higher order total accumulated error $\O{\Delta s^4}$.

\section{Algorithm}
At a high level our algorithm consists of three steps:
\begin{enumerate}
	\item Calculate the field line
	\item Determine a toroidal coordinate system for this field line
	\item Calculate the required properties in this new coordinate system
\end{enumerate}
We will further eleborate on these steps below and comment on the complexity.\\

\subsection{Step by step}

To calculate the winding number of a field line passing through a certain starting point we first use RK4 to integrate a length of field line from the magnetic field data. This length should be large enough such that we have a high number of revolutions around the torus.

This integration process is inherently sequential, i.e. computations on later RK4-steps can not be performed before the results from all earlier steps have been computed, leaving no room for parallelization on a per step basis.
Due to the hardware optimized interpolation we are memory access bound in this part of the algorithm, which we have verified by using the NVIDIA profiler.
Thus there was nothing to be gained from parallelizing the integration further than one-thread-per-field-line.

To compute the winding number of a field line we need to convert to a toroidal coordinate system. Since not all tori of interest are perfectly aligned this means we first have to find the position and orientation of the torus of interest before we can calculate any other relevant parameters.

By averaging the coordinates over a whole field line we find (a good approximation of) the origin of our torus. This summation can be done efficiently in parallel as a reduction. For implementation we implemented most of the methods described by Harris et al.~\cite{harris2007optimizing}.

Next we need to calculate the orientation of the torus, in particular the normal to the surface plane of the torus.
We can find this normal by taking the cross product of the direction of a field line (the $\vec{B}$-field) at a point and the location vector of that point. We perform this local calculation for each point on our field line and implement a similar parallel reduction as before to find the orientation of our torus.

The last part of the torus coordinate system needed is the (center) radius of the torus. For this we calculate the average distance from the origin in the projection onto the plane of the torus. %This is error prone, see discussion.

\begin{figure}[!htb]
\minipage{0.47\textwidth}
  \includegraphics[width=\linewidth]{Figures/TorusAlphaBeta.pdf}
\endminipage\hfill
\minipage{0.47\textwidth}
  \includegraphics[width=\linewidth]{Figures/torusfit3.png}
\endminipage
	\caption{Illustration of the toroidal coordinate system and the normal finding procedure. The red circle is a circle through the torus, lying in a plane perpendicular to the torus normal and with radius equal to the center radius of the torus. From this circle we can compute the $\alpha$- and $\beta$-coordinates, i.e. the toroidal and poloidal coordinates respectively, of a point on the torus surface.}\label{fig:coordsys}
\end{figure}

With this coordinate system calculating the winding numbers of the field lines is as simple as calculating the difference in both poloidal and toroidal coordinates between consecutive data points and summing these along a field line. This is once again implemented as a parallel reduction.

In the implementation it was needed to take care of unwrapping the coordinates modulo $2\pi$ using some inefficient if-statements. %Discussion: One could probably devise a parallel unwrap algorithm, but would it be worth it?
As a last step, the ratio of the total polodoidal and toroidal change is computed to find the winding number.

As mentioned before this entire algorithm is parallelized by considering multiple field lines simultaneously. In particular we pick a 2D-grid of initial locations for the field lines instead of a single point. Using this data we can show the winding number as a function of position in the nested tori.\\
\subsection{Complexity}
In order to analyse the theoretical (time) complexity let's introduce the following variables:
\begin{itemize}
	\item $p$, the maximum number of concurrent threads on the GPU
	\item $m$, the number of RK4-steps per field line.
	\item $N$, the number of pixels in our picture, equal to the total number of field lines calculated.
\end{itemize}
As is clear from the definitions, the time needed for the integration will be linear in $m$. As soon as the number of field lines is larger than the number of threads, a linear factor in $N/p$ comes in as well, as we have to divide the calculations over several rounds of computation:
\[t_{RK4}\in\O{m\left\lceil\tfrac{N}{p}\right\rceil}\]
For the reduction of a calculated value along a field line, we need $\frac m2$ threads. The reduction itself is logarithmic, so we have the following complexity:
\[t_\textit{reduc} \in \O{\left\lceil\frac{Nm}{2p}\right\rceil\log\left(m\right)} \]
Note that this is theoretically slower than performing a serial reduction per field line for a large enough number of field lines ( which would again have $t \in \O{\left\lceil N/p\right\rceil m}$ ), but cache locality and coherency favor this algorithm heavily.

Any local operations on datapoints of field lines are of course linear in the number of datapoints $mN$ and, as they are local (i.e. fully parallelizable), reduced by a factor of $p$.\\
On top of all this we expect our code to have a fixed contribution to the runtime from allocating memory, loading the magnetic field and initialising the hardware accellerated structures. We therefore expect the runtime of our code to be approximately linear in the numbers $m$ and $N$, but we do not expect this linear dependence to go through the origin.
\newpage
\section{Results}
%{\bf plaatjes, timings + scaling, portability, meer plaatjes: lines, lengths, windings, masking}
\subsection{Simulated field lines}
Using our code we computed magnetic field lines for given magnetic fields. Included are three examples of the winding numbers computed with this code, and one example of the computed line lengths.

\begin{figure}[!htb]
\minipage{0.49\textwidth}
  \includegraphics[width=\linewidth]{Figures/Rings_Papertwist_twist1_82_steps32k.png}
\endminipage\hfill
\minipage{0.49\textwidth}
  \includegraphics[width=\linewidth]{Figures/Rings_Papertwist_twist1_125_steps32k.png}
\endminipage
	\caption{Winding numbers of the field lines of a cross-section through the torus for the dataset \texttt{Papertwist1}. The color scale was cut on both sides to optimize for the structure inside the Torus. Both images have a resolution of $1024\times 1024$ pixels and use $32768$ points per field line. The left image is at time $t=82$, the right one at $t=125$.}\label{fig:125-32k}\label{fig:82-32k}
\end{figure}

In order to make a good image the data is restricted in two ways.
We firstly changed the colour scale to ensure that the changes in the nearly-constant winding number are visible.
Secondly we have discarded (masked) all points with a total length of the field line below a certain threshold, reasoning that the winding numbers of short field lines can be wildly inaccurate.
In this category fall nearly all field lines that leave the simulation box, as the texture clamps all vector field accesses to zero outside it.
For completeness we include a plot of the length of the field lines and the unmasked (but still colour-clipped) data in Figure \ref{fig:125-32k} below.\\

\begin{figure}[!hb]
\minipage{0.32\textwidth}
  \includegraphics[width=\linewidth]{Figures/Rings_Papertwist_twist1_125_lengths_steps32k.png}
	\caption{Lengths of the field lines of a cross-section through the torus. The vector field used is \texttt{Papertwist1-125}. The image is $1024\times 1024$ pixels, with $32768$ points per field line.}\label{fig:125-lengths}
\endminipage\hfill
\minipage{0.32\textwidth}
  \includegraphics[width=\linewidth]{Figures/Rings_Papertwist_twist1_125_nomask_steps32k.png}
	\caption{Unmasked winding numbers of the field lines of a cross-section through the torus. The vector field used is \texttt{Papertwist1-125}. The image is $1024\times 1024$ pixels, with $32768$ points per field line.}\label{fig:125-unmasked}
\endminipage\hfill
\minipage{0.32\textwidth}
  \includegraphics[width=\linewidth]{Figures/Rings_Papertwist_twist1_125_steps8k.png}
	\caption{Identical to Figure~\ref{fig:125-32k}, albeit with $8196$ points per field line. Although most of the features reproduce, numerical `ripples' arise due to the too short integration time, as dicussed in Section \ref{section:numerrors}.}\label{fig:125-8k}
\endminipage
\end{figure}

\FloatBarrier
\subsection{Speed and scaling}
We have recorded the runtime of the code on a NVIDIA GTX1080 for the dataset \texttt{Shafranov\_Rot3\_ampl\_0.05\_Animation50}, generating and analysing datasets as in Figure \ref{fig:82-32k}. For the first set of timing data we fixed the image size at $1024\times 1024$ pixels (field lines) and varied the number of steps per field line. After that we fixed the number of steps per field line (at $32768$) and varied the number of pixels in an image.\\
\begin{figure}[!ht]
	\includegraphics{Figures/timings_1.pdf}
	\caption{Linear fit of the computation time as a function of the number of RK4-steps, with the deviation from the fit plotted below. This was done for a resolution of $1024 \times 1024$}\label{fig:timings1}
\end{figure}
\begin{figure}[!ht]
	\includegraphics{Figures/timings_2.pdf}
	\caption{Linear fit of the computation time as a function of number of field lines for 32768 RK4-steps each, with the deviation from the fit plotted below.}\label{fig:timings2}
\end{figure}

%\FloatBarrier

We compare the performance of our program with the prevous method used, which also computed winding numbers for plasma field lines.
This existing Python code simulated a $1\times 1100$ array of field lines with 4000--16000 RK6-steps per line, and took approximately $36$ hours for this computation.
Comparing this with our own results we conclude that our program is faster by a factor of approximately $3.5\cdot 10^6$.\\

Lastly we used the NVIDIA visual profiler to see how much memory our code was using while active.
While generating a $1024\times 1024$ dataset with $32768$ steps per field line the program was using $6513$ MiB while in use, whereas running the code with identical parameters except for halving the number of steps per line (to $16384$) the program was using $3441$ MiB.

\section{Discussion}
In this section we interpret the results of our code, and suggest possible improvements.
In particular we discuss the accuracy of the used method for calculating the magnetic structure and how numerical errors effect the computed structure.

\subsection{Numerical errors}\label{section:numerrors}
There are two features in the results that merit further discussion.

\subsubsection{Coordinate system inaccuracy}

\begin{figure}[!ht]
	\centering
	\includegraphics{Figures/TorusMismatch.pdf}
	\caption{ Illustration of the coordinate system mismatch between the calculated torus centre (red) and actual, elliptical, torus.}\label{fig:mismatch}
\end{figure}

Firstly we remark that near the center of each of the cross-sections the winding numbers behave oddly (the small white dots in the centers of Figure \ref{fig:82-32k}, better visible in the zoom, Figure \ref{fig:artefacts}).
Closer inspection reveals that there is a sign flip in the winding number of the field lines here, which shows up as white due to the colour scale.

We suspect that this sign flip is due to a numerical error in the code, possibly caused by the change in coordinate systems.
Since our transformation of Cartesian coordinates to Toroidal coordinates is based on the line data, this coordinate change is likely to be inaccurate for small tori.
In particular we determine a center circle for each of the nested tori (the red circle in Figure \ref{fig:coordsys}), and assume that this circle lies inside the torus in question. For perfect tori this is always the case, but the field lines fill a surface that is only approximately toroidal. Furthermore numerical errors influence our estimate of the center circle. Due to both these effects the center circle might not fully lie in the center of the torus, as depicted in Figure \ref{fig:mismatch}. This copmletely invalidates all calculations done in the toroidal coordinate system, explaining the wildy different value of the winding number.

Since in practice the tori on which the field lines lie may be elliptical (instead of perfectly circular), the only way to completely avoid this error may be through more thorough analysis of these surfaces to find a coordinate system which takes this non-circularity into account.

\subsubsection{Finite length}
Secondly it is interesting to consider the differences between Figure \ref{fig:125-8k} and Figure \ref{fig:125-32k}, in particular the apparent shapes near the center of the cross section. These centers are reproduced below in Figure \ref{fig:artefacts}.
\begin{figure}[!ht]
	\centering
\minipage{0.42\textwidth}
  \includegraphics[width=\linewidth]{Figures/artefact32k.png}
\endminipage ~
\minipage{0.42\textwidth}
  \includegraphics[width=\linewidth]{Figures/artefact8k.png}
\endminipage\hfill
	\caption{Zoom of Figures \ref{fig:125-32k} and \ref{fig:125-8k}, corresponding to 32k RK4-steps and 8k RK4-steps respectively.}\label{fig:artefacts}
\end{figure}

In Figure \ref{fig:125-8k} we clearly see shapes with twofold rotational symmetry around the white center, which are much less pronounced and different in shape in Figure \ref{fig:125-32k}.
We suspect that these patterns in the winding numbers may be due to the finite length of the simulated field lines.

As mentioned above, the surfaces on which our field lines lie are not perfect tori but rather are slightly elliptical. This implies there are sections of field line where the local winding number is not equal to the global (average) winding number along a line.
Therefore after a finite length of field line the average of our simulated line may disagree with the true winding number (corresponding to an infinitely long line).
This problem is made worse by the fact that our toroidal coordinate system is not optimized for correcting this eccentricity, which explains why these features are most prominent at the center of the cross section.

A possible solution might be to use a more sophisticated method to extract the winding number from the local data than the (weighed) average that is currently in place, although it should be noted that most of these methods are hard to parallellize and will therefore significantly slow down the algorithm.\\

We would furthermore like to remark that, as always when numerically integrating a vector field, there is a tradeoff between small step size (and low truncation error in the numberical method) and large step size (and low total number of steps for fixed length, so a low cumulative rounding error).
In our code we fix the step size inversely proportional to the average magnitude of the magnetic field in the dataset.
Therefore the spatial step size, with a length of our step size times the local magnitude of the magnetic field, is on average constant between different input fields.
A possible improvement on this would be to dynamically scale the step size at each integration step, ensuring that all spatial steps taken are of very similar length. Regular adaptive step size algorithms however branch depending on whether a step was determined to be too large or too small, making them ill-fitted for the SIMT (Single Instruction Multiple Threads) parallelism of GPU's.\\

\subsection{Speed and scaling}
We note that the memory usage agrees well with our theoretical predictions, where we consider that we store (for the $32768$ steps per line):
\begin{itemize}
	\item The vector field, $256\times256\times256$ \texttt{Float4} objects ($16$ Byte each), for a total of $256$ MiB.
\item The line data itself, $64\times 64$ lines (this part of the code is serialized so not all lines need to be stored at the same time) times $32768$ \texttt{Float4} points per line, for a total of $2048$ MiB.
\item One array to compute and store the normal vectors to the tori of interest, which needs to be of the same size as the line data array, so $2048$ MiB total.
\item Two scalar fields to compute and store the toroidal and poloidal windings of $64\times 64\times 32768$ Floats, which equals $512$ MiB per field or $1024$ MiB total.
\item One scalar field to compute and store the length of the vector fields, for $512$ MiB total.
\item One scalar field to compute and store the middle radius of the tori, for $512$ MiB total.
\end{itemize}
In total this adds up to $6400$ MiB, which is in good agreement with the observed $6513$ MiB.
To find the theoretical memory use for the simulation with $16384$ steps per field line we repeat the calculation above and find a total of $3328$ MiB in theory, which agrees well with the $3441$ MiB observed.
In both cases we are using $113$ MiB more than was predicted in theory, which we attribute to general overhead.\\

Secondly we observe that, as was predicted, the runtime of our code is highly linear in both the number of pixels/field lines and the number of steps per field line, with an offset of about two seconds.
We conclude that this is the amount of time that it takes to load the vector field to the memory on the graphics card and allocate all relevant arrays (the two different datasets above give a different time offset because they correspond to different array sizes that need to be allocated), and that the complexity of the algorithm is as predicted.

\section{Suggestions for future work}

\subsection{Interpolation}
Better algorithms exist for interpolation, most notably cubic interpolation and interpolation schemes which preserve the zero divergence of the magnetic field~\cite{McNally01052011}. It is possible that these algorithms could further reduce numerical artefacts.
Notably, efficient and open source implementations in CUDA of cubic interpolation exist~\cite{Ruijters01012012}.

\subsection{Integration}
As mentioned the code was shown to be largely memory access bound. Implementation of a multi-step method such as Adams-Bashforth could reduce the number of texture fetches per step, while maintaining a high order.\\
Also worth mentioning is that the RK4 integration method uses no special properties of the magentic field, such as the divergencelessness or the Hamiltonian description. It might for example be possible to exploit the knowledge that the system is Hamiltonian by implementing a symplectic method.

\subsection{Optimization}
Because the numerical integration is the only inherently sequential part of the algorithm, we made the choice to minimize calculations during this part of the algorithm.
Profiling of the code shows however that it is both memory bound and memory access bound.

A possible optimization would be to increase the computational load during the integration of the magnetic field, for example pre-calculating some of the properties required later, thus improving use of the other computational resources and overall efficiency during this part of the algorithm.

Another optimization opportunity is the memory usage. In the current code a lot of memory is used for computing the normal vectors and scalar fields. Efficient reuse of memory could reduce total memory usage per field line, enabling more field lines to be treated in parallel.

\newpage
\bibliographystyle{plain}
\bibliography{introduction}
\end{document}


\section{Bayesian Posterior Probability Density Functions}
\subsection{Model-Dependent Search}
\subsection{Model-Independent Search}


\documentclass{standalone}
\begin{document}
	\subsection*{Lung Extraction}

	This preliminary step is performed before both training and labelling and involves the managing of the HU, the isolation of lung regions and the removal of the main bronchial structures.
	The registration of the HU on a common space is necessary to overcome the issues that may arise from the different padding values used by the different manufacturer of the CT scans. Moreover, some noise may change air values of some regions, which will be not exactly $-1000$. During the scan acquisition, all the regions outside the CT tube aren't sampled, so to obtain a square $N\times N$ image for each slice some padding values are added, which different values according to the scan manufacturer. This registration allows overcoming some issues that may arise during the conversion of the image from $16$ to $8$ bit unsigned integers, required to operate with \textsc{OpenCV} functions.

	Lung segmentation is a pivotal pre-processing step in many image analyses such as identification and classification of lung pathologies~\cite{ART:Johannes}. The lung isolation allows to find a mask for the lung regions, and thus excluding all the body regions, the CT tube and the extra-lung organs like intestine and heart, avoiding the formation of false positives.

	Automatic lung segmentation algorithms are typically developed and tested on limited datasets and usually over a limited spectrum of visual variability by containing mainly cases without severe pathologies~\cite{ART:Johannes}. Rule-based approach, like thresholding, region growing, etc., usually fails for CT scans of patients with severe Interstitial Lung Disease (ILD), as we can see in \figurename\,\ref{fig:UNetVSThr}. For these reasons to achieve the lung segmentation, I've used a pre-trained U-Net~\cite{ART:Johannes}~\cite{REP:lungmask}.
	\begin{figure}[h!]
		\centering
		\includegraphics[width=\linewidth]{UNetVSThr.png}
		\caption{From left to right the original CT scan of a patient with severe ILD, the lung segmented by threshold and it's connected components, and lung segmentation achieved by the pre-trained U-Net. We can clearly see the missing areas in the first segmentation, correctly identified in the U-Net one.}\label{fig:UNetVSThr}
	\end{figure}

	This kind of segmentation includes in the lung region also motion artefacts and bronchial structures, which are the main source of false positives. To achieve a better segmentation a refinement process is performed, which aims to remove the main bronchial structures form the selected lung regions.

	Bronchi have elongated shape respect to the other structure which usually is rounded: the basic idea was to use this kind of information. To perform this task, I have computed the covariant matrix of the derivative in a neighbourhood (\ref{eq:CovMat}) and the corresponding eigenvalues.

	\begin{equation}\label{eq:CovMat}
		M = \begin{bmatrix} \sum _{S(p)}(dI/dx)^2 & \sum _{S(p)}dI/dx dI/dy \\ \sum _{S(p)}dI/dx dI/dy & \sum _{S(p)}(dI/dy)^2 \end{bmatrix}
	\end{equation}

 	If a particular region has an elongated shape, one of the eigenvalues (corresponding to the eigenvector in the direction of the structure) will be higher than the other, otherwise, both eigenvalues have a low one. I have applied this filter on each slice of the scans and took a map of the maximum eigenvalues.
	\begin{figure}[h!]
		\centering
			\includegraphics[width=\linewidth]{MaxEigenVal.png}
		\caption{From left to right: lung regions selected by the U-Net, maximum eigenvalues map of the lung. As we can see the U-Net does not exclude the bronchial structure from the lung, on the other hand, the maximum eigenvalues map delineates very well these regions. I have used this map to remove the unwanted bronchial regions.}\label{fig:MaxEigenval}
	\end{figure}

	In \figurename\,\ref{fig:MaxEigenval}, I've displayed the image after the lung segmentation by the neural network and the corresponding eigenvalues map. As we can see the higher values of the map correspond to the edges of the main bronchial structures. To create the mask for these structures, I have applied a fixed threshold to the map. Since the main bronchial structures are large, this process can remove only part of the edges, but the inner structure is preserved. To refine the segmentation, this process is repeated a second time, allowing a fine removal of the structures.

\end{document}


\subsection{\texttt{bingle\_vis.py}}\label{code:bingle_vis}

\begin{verbatim}
#bingle_vis.py: A two-particle collision simulation
#               with visualisation using povray
#	Author: D. Weatherley
#	Date: 15 May 2007
#	Organisation: ESSCC, University of Queensland
#	(C) All rights reserved, 2007.
#
#
#import the appropriate ESyS-Particle modules:
from esys.lsm import *
from esys.lsm.util import Vec3, BoundingBox
from esys.lsm.vis import povray

def snapshot(particles=None, index=0):
   pkg = povray
   scene = pkg.Scene()
   for pp in particles:
      povsphere = pkg.Sphere(pp.getPosn(), pp.getRadius())
      povsphere.apply(pkg.Colors.Red)
      scene.add(povsphere)

   camera = scene.getCamera()
   camera.setLookAt(Vec3(0,0,0))
   camera.setPosn(Vec3(0,0,20))
   camera.setZoom(0.1)

   scene.render(
      offScreen=True,
      interactive=False,
      fileName="snap_{0:04d}.png".format(index),
      size=[800,600]
   )
   return

#instantiate a simulation object
#and initialise the neighbour search algorithm:
sim = LsmMpi(numWorkerProcesses=1, mpiDimList=[1,1,1])
sim.initNeighbourSearch(
   particleType="NRotSphere",
   gridSpacing=2.5,
   verletDist=0.5
)

#set the number of timesteps and timestep increment:
sim.setNumTimeSteps(10000)
sim.setTimeStepSize(0.001)

#specify the spatial domain for the simulation:
domain = BoundingBox(Vec3(-20,-20,-20), Vec3(20,20,20))
sim.setSpatialDomain(domain)

#add the first particle to the domain:
particle=NRotSphere(id=0, posn=Vec3(-5,5,-5), radius=1.0, mass=1.0)
particle.setLinearVelocity(Vec3(1.0,-1.0,1.0))
sim.createParticle(particle)

#add the second particle to the domain:
particle=NRotSphere(id=1, posn=Vec3(5,5,5), radius=1.5, mass=2.0)
particle.setLinearVelocity(Vec3(-1.0,-1.0,-1.0))
sim.createParticle(particle)

#specify the type of interactions between colliding particles:
sim.createInteractionGroup(
   NRotElasticPrms(
      name = "elastic_repulsion",
      normalK = 10000.0,
      scaling = True
   )
)

#compute the specified number of timesteps:
N_max = sim.getNumTimeSteps()
n=0
while (n < N_max):
   #compute a single timestep:
   sim.runTimeStep()

   # Take a snapshot of the simulation every 100 timesteps:
   if (n%100==0):
      particles = sim.getParticleList()
      snapshot(particles=particles, index=n)

   # update the total number of timesteps computed (n):
   n += 1

#Exit the simulation.
sim.exit()
\end{verbatim}