\documentclass[11pt,a4paper]{scrartcl}
\usepackage[utf8]{inputenc}
\usepackage[english]{babel}
\usepackage{microtype}
\usepackage{amsmath}
\usepackage{mathtools}
%\usepackage[leqno]{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{graphicx}
\usepackage{indentfirst}
\usepackage{fancyvrb}
%\usepackage{subfigure}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{enumitem}
\usepackage{booktabs}
\usepackage{algorithm}
\usepackage{algpseudocode}
\usepackage[onehalfspacing]{setspace}
\usepackage[hidelinks]{hyperref}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\fancyhead[R]{\thepage}

\usepackage{amsmath}
\usepackage{nccmath}
\newenvironment{mpmatrix}{\begin{medsize}\begin{pmatrix}}%
{\end{pmatrix}\end{medsize}}%

\date{}
\title{STDP-based spiking deep convolutional neural networks for object recognition}
\begin{document}

\input{Frontespizio}

\tableofcontents

\clearpage
\section*{Introduction}
Second generation artificial neural networks -- mainly characterized by fully connected layers that deal with continuous output values -- have allowed us to make enormous progresses in many relevant scientific fields. In the same way, also the third generation of artificial neural networks seems to be very promising and it is expected to bridge the gap between neuroscience and machine learning, by using more realistic \textit{spiking} neuron models. These 3rd generation networks are called \textquotedblleft Spiking Neural Networks\textquotedblright and are different from usual networks, since they do not deal with continuous values, but they communicate through spikes, that are discrete events that occur at certain points in time. In fact, when a (membrane of a) neuron reaches a certain potential value, it spikes, so it resets its value.
This behaviour is inspired by real biological processes and is expected to better model reality. This may seem not so \textquotedblleft innovative\textquotedblright, since discrete signals are used instead of continuous ones. Instead, this modelling allows us to handle spatio-temporal data, that are real-world sensory data. These spatial and temporal aspects refer to the fact that neurons are connected to neurons local to them and that temporal information of the spikes (that are lost in binary encoding) are now preserved. Indeed, it has been proven that spiking neurons are computationally more powerful than traditional artificial neurons.\\
In this report we describe a project in the frame of the \textquotedblleft Neural Networks\textquotedblright course of Sapienza Università di Roma, that is inspired from the \textquotedblleft STDP-based spiking deep convolutional neural networks for object
recognition\textquotedblright paper of Kheradpisheh, Ganjtabesh, Thorpe and Masquelier. In particular, the implementation of a Spiking Convolutional Neural Network for object classification is described.

\section{Network architecture}

An STDP-based spiking deep neural network (SDNN) with a spike-time neural coding has been implemented. In the very beginning of this network, the input image is passed to the temporal-coding layer,that uses Difference of Gaussians (DoG) filters to convert the image into an asynchronous spike train. This is used as input for the successive layer, that is a convolutional one,
in which input spikes are integrated and new spikes are emitted if a certain threshold is exceeded, that is when a visual feature is detected. Learning is done through STDP, that is used to learn visual features, and so the weights of the network. Neurons that fire earlier in this phase prevent the others from firing. The convolutional layer is followed by a pooling one, where the visual information is compressed. Three convolutional and pooling layers (arranged in an alternate and consecutive order) have been implemented to handle larger and more complex visual features through the network. In the end, in the last pooling layer, a global max pooling is performed for the classification, to train a linear classifier. The results are used to find the proper category to which the input image belongs.\\
\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{images/architecture}
	\caption{SDNN architecture}
	\label{fig:architecture}
\end{figure}

\subsection{Temporal coding}

In the very first part of the neural network, the input signal is encoded into temporal-discrete spike events. A Difference of Gaussians (DoG) filter is used to detect the contrasts in the input image and emit a spike, accordingly. The higher the contrast in a certain cell, the more strongly this one is activated in order to fire. The firing time (i.e. the time when the spike is emitted) of a DoG cell is inversely proportional to its activation value. In particular, if an \textit{r} output is obtained in a cell after DoG, its corresponding firing time will be $ \tau = 1/r $. DoG cells are sensitive to both positive and negative contrasts, depending on their two ON-centre and OFF-centre maps and they emit when their activation values exceed a predefined threshold.\\
An example of DoG filter output is shown in figure \ref*{fig:dog}. In this picture, a face and a motor (image categories that compose the learning and training dataset) are encoded into discrete spike events.

\subsection{Convolutional and pooling layers}

A convolutional layer is represented by several neuronal maps, whose role is to detect visual features at different locations. To this extent, the synaptic weights assigned to the same neuronal map will be the same. Each neuron in the convolutional layer generates spikes according to what it receives from the pre-synaptic neurons and when its internal membrane potential reaches a certain threshold. So, at each time step \textit{t}:
\begin{equation*}
	V_{i}(t) = V_{i}(t-1) + \sum_{j}W_{j,i}S_{j}(t-1)
\end{equation*}
the internal potential $ V_{i}(t) $ of the \textit{i-th} convolutional neuron is updated by adding the spike train $ S_{j}(t-1) $ of the \textit{j-th} pre-synaptic neuron, multiplied to the synaptic weight $ W_{j,i} $ between the \textit{j-th} pre-synaptic neuron and \textit{i-th} convolutional neuron. In particular, the potential $ S_{j}(t-1) $ can be equal to 1 if the neuron has fired at previous time, or 0 otherwise. If the neuron potential $ V_{i}(t) $ exceeds the threshold $ V_{th} $, then it emits a spike ($ S_{i}(t) = 1 $) and it resets its potential to zero. After it fires, the neuron can not fire again and it also inhibits other neurons in its same location (but belonging to other neuronal maps). In this way it is easier to detect a visual feature in a certain location.\\
In the pooling layer instead, each neuron performs a max pooling operation over a window in the corresponding neuronal map of the previous layer. Here no learning occurs and, as before, each neuron can emit only one spike. In this phase the visual information is compressed. These convolution-pooling phases are repeated for three times, to handle larger and always more complex data.

\subsection{STDP-based learning}

Spike-timing-dependent plasticity (STDP) is actually a biological process used by brain to modify its own synapses. This same idea has been also used in neural networks, so as to have stronger synapses in case they contribute to the firing of a post-synaptic neuron and weaker synapses if they do not give any contribute to the firing.\\
\begin{figure}[h]
	\centering
	\includegraphics[width=0.75\linewidth]{images/spikes}
	\caption{Contributon to the firing of a post-synaptic neuron}
	\label{fig:spikes}
\end{figure}
This process is easy to understand by looking at figure \ref{fig:spikes}. In this case, four different pre-synaptic neurons are connected to one neuron (the blue one) by synapses. Each of them is characterized by its own firing rate. These spikes are sent forward by the corresponding synapses increasing the membrane potential of the post-synaptic neuron. When this spikes, it is possible to see which pre-synaptic neuron has contributed and its membrane potential is increased. In the same way, the potential of the neurons that have not contributed to the firing of the post-synaptic one, is weakened.\\
As already mentioned, learning occurs only in convolutional layers, so that neurons compete to fire earlier than others and use STDP to learn input patterns. This competition is crucial to encourage neurons of different neuronal maps to learn different features. In this case, STDP is represented as:
\begin{equation*}
	\Delta w_{ij} = \begin{cases}
		a^{+}w_{ij}(1-w_{ij}), & \textit{if }\; t_{j}-t_{i} \leq 0,\\
		a^{-}w_{ij}(1-w_{ij}), & \textit{if }\; t_{j}-t_{i} > 0,
	\end{cases}
\end{equation*}
where \textit{i} and \textit{j} refer to the post-synaptic and pre-synaptic neurons, respectively, $ \Delta w_{ij} $ is the synaptic weight modification and $ a^{+} $ and $ a^{-} $ are the learning rate parameters. The term $ w_{ij}(1-w_{ij}) $ is needed to bound the weight values between 0 and 1. The best choice for the learning rate parameters is with $ a^{+} $ greater than $ a^{-} $, but with small values for both. This, to prevent that neurons learn more than one pattern and/or that do not reach the predefined threshold to fire.\\
It could happen that different neurons fire at the same time, since the time is discrete. In that case, the neuron with the highest potential is selected, since it should define a higher similarity between its learned feature and input pattern.\\
Weights are randomly initiated with mean 0.8 and standard deviation 0.05 and the learning convergence of the \textit{l-th} convolutional layer is represented by:
\begin{equation*}
C_{l} = \sum_{f}\sum_{i}w_{f,i}(1-w_{f,i})/n_{w}
\end{equation*}
where $ n_{w} $ is the total number of synaptic weights in that layer and $ w_{f,i} $ is the \textit{i-th} synaptic weight of the \textit{f-th} feature.

\subsection{Classification}
In regards of the images classification section, a dataset of labeled features is built with the output of the global max pooling layer, which in this case gives a vector of twenty elements that represent the greatest potentials reached by the neuronal maps in the third convolutional layer.\\
In fact the activation threshold of the neurons belonging to the last layer has been set to an infinite positive value, since has been choosen not to incurr in the reset of the potentials due to neurons spiking.\\
The resulting dataset then is standardised and splitted in training and testing sets in order to be feeded in a linear SVM.

%The third pooling layer is used for the classification, so that each neuron belonging to this layer performs a global max pooling over the corresponding neuronal maps in the previous convolutional layer. Then, one feature can be described by only one output value that is used to train a linear SVM classifier and determining the object's category.

\section{Results}

The spiking convolutional network has been applied over two main image categories: human faces and motorbikes. This dataset of 470 images has been divided into three main subfolders, for the learning, training and testing phases, respectively.
It is possible to analyse what happens during the process, through the network.\\
At the beginning, the input image (face or motor) is passed to the DoG filter that encodes the temporal information in 10 different layers. In the two upper images of figure \ref{fig:face}, it is possible to see input image and the contrasts highlighted by the DoG. In the remaining four images instead, the first four outputs of the filter are shown. The information is accumulated by augmenting the encoding layers.\\
\begin{figure}[h]
	\centering
	\begin{minipage}[b]{\textwidth}
		\includegraphics[width=\textwidth]{images/face_temp}
	\end{minipage}
	\\
	\begin{minipage}[b]{\textwidth}
		\includegraphics[width=\textwidth]{images/face_dog_out}
	\end{minipage}
	\caption{DoG temporal coding (face). Top-left: input image. Top-right: contrasts. Four images: first four encoding layers.}
	\label{fig:face}
\end{figure}\\
In the same way, it is possible to see the DoG outputs for the \ref{fig:motor} category.\\
\begin{figure}[h]
	\centering
	\begin{minipage}[b]{\textwidth}
		\includegraphics[width=\textwidth]{images/motor_temp}
	\end{minipage}
	\\
	\begin{minipage}[b]{\textwidth}
		\includegraphics[width=\textwidth]{images/motor_dog_out}
	\end{minipage}
	\caption{DoG temporal coding (motor). Top-left: input image. Top-right: contrasts. Four images: first four encoding layers.}
	\label{fig:motor}
\end{figure}
Once the outputs of the DoG filter are obtained, these are used in the first convolutional layer. The latter is defined by a $ 16\times250\times1 $ input dimension, $ 5\times5 $ filter,  input and output channel equal to 1 and 4, respectively.
By iterating over the DoG output images, these are passed in sequence to the convolutional layer, that produces four different output images. These are shown in figure \ref{fig:conv_out}.
\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{images/conv_output_slice3_cut}
	\caption{Conv out}
	\label{fig:conv_out}
\end{figure}\\
It can be interesting to see what are the neurons that spike right after the convolution. This information is reported in figure \ref{fig:s_k}, in which it is possible to see the spiking neurons before
S and K values:
\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{images/S_K_values_slice2}
	\caption{S and K}
	\label{fig:s_k}
\end{figure}\\
\begin{figure}[h]
	\centering
	\begin{minipage}[b]{0.32\textwidth}
		\includegraphics[width=\textwidth]{images/K_1}
	\end{minipage}
	\hfill
	\begin{minipage}[b]{0.32\textwidth}
		\includegraphics[width=\textwidth]{images/K_2}
	\end{minipage}
	\hfill
	\begin{minipage}[b]{0.32\textwidth}
		\includegraphics[width=\textwidth]{images/K_3}
	\end{minipage}
	\caption{K}
	\label{fig:k}
\end{figure}
Pooling:
\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{images/pool_slice1}
	\caption{Pooling}
	\label{fig:pool}
\end{figure}\\
Weights:
\begin{figure}[h]
	\centering
	\includegraphics[width=\textwidth]{images/weights_slice1}
	\caption{Weights}
	\label{fig:weights}
\end{figure}\\
\section{Conclusions}

\clearpage

\begin{thebibliography}{5}

\bibitem{STPD}
S. R. Kheradpisheh, M. Ganjtabesh, S. J. Thorpe, and
T. Masquelier, "Stdp-based spiking deep convolutional neural
networks for object recognition," Neural Networks, vol. 99,
pp. 56–67, 2017.

\bibitem{Next-gen}
Spiking Neural Networks, the Next Generation of Machine Learning, https://towardsdatascience.com/spiking-neural-networks-the-next-generation-of-machine-learning-84e167f4eb2b

\bibitem{GitHub}
Spiking-Neural-Network, https://github.com/Shikhargupta/Spiking-Neural-Network
\end{thebibliography}

\end{document}


\subsection{Prospect theory}

Cumulative prospect theory.


\chapter{Summary and outlook}\label{ch:summary}

The aim of this thesis was to investigate
the contributions of induced three-body operators
in the IMSRG.
To this end,
we studied the IMSRG(3) truncation
of the IMSRG.
The uncoupled ($m$-scheme) implementation of the IMSRG(3)
is too expensive for nuclear applications,
so we exploited the spherical symmetry
of the IMSRG
when applied to closed-shell nuclei
to arrive at the $J$-scheme IMSRG(3)
fundamental commutators.
We implemented an IMSRG(2)/(3) solver
using these fundamental commutators
that is more performant than
the previous $m$-scheme implementation.
With this implementation in hand,
we were able to do first explorations
of the effects of approximations to the IMSRG(3) truncation
when compared with the IMSRG(2) truncation in ${}^{4}\text{He}$
and ${}^{16}\text{O}$.

To allow for the exploration of different single-particle bases,
Hamiltonians, and oscillator frequencies,
we introduced several approximate IMSRG(3) truncation schemes
that leave out the most expensive fundamental commutators,
one of which was the IMSRG(3)-$N^7$.
We find that in ${}^{4}\text{He}$,
the IMSRG(3)-$N^7$ truncation
offers a small correction at the sub-percent level
to the IMSRG(2) result
for the ground-state energy
in the $\emax=2$ model space considered.
The corrections to the charge radius are larger,
on the order of a few percent.
We see that the IMSRG(3)-$N^7$
performs somewhat better than the IMSRG(2)
away from the optimal HF oscillator frequency,
a trend that needs to be verified
for larger model spaces and different systems.

For ${}^{16}\text{O}$,
we considered
various approximate IMSRG(3) truncation schemes.
These schemes were set up to include
one or several fundamental commutators
on top of the previous truncation scheme,
starting from the IMSRG(2).
The organization of these schemes was
partially physically motivated
(in particular the IMSRG(3)-A scheme),
but most commutator inclusions
were organized by computational cost.
We found that this organization
did not deliver the desired results.
While the IMSRG(3)-A truncation
provided a relatively significant shift
from the IMSRG(2) results,
the inclusion of the remaining
$\mathcal{O}(N^6)$ and
$\mathcal{O}(N^7)$ commutators
provided only small corrections
to the ground-state energy of ${}^{16}\text{O}$.
The final truncation scheme we considered,
which included the first $\mathcal{O}(N^8)$
commutator,
produced another large shift over the previous truncation.
This suggests that computational cost
is not a good proxy for importance in the formation of
an approximate IMSRG(3) truncation.

Looking to the future,
there are clear objectives for our IMSRG(3) implementation.
The first is tuning the performance of the implementation
to put full IMSRG(3) $\emax=6$ and hopefully $\emax=8$
within reach.
Additional restrictions can be placed on
for example the range of three-body channels,
potentially allowing IMSRG(3) calculations
to reach even larger single-particle model space sizes.
This will allow us to perform converged IMSRG(3) calculations
for medium-mass nuclei
and investigate what effects it has on observables.
Parallel to this,
obtaining exact (full configuration interaction) results
in $\emax=2$
would allow us to see how IMSRG(3)
compares with IMSRG(2)
relative to the exact solution
of the Schr{\"o}dinger equation
in $\emax=2$.
A secondary implementation goal
is the support of tensor operators,
which would give our implementation access to many more observables,
such as low-lying spectroscopy.

A key result of this thesis is that the organization by computational cost,
at least for the limited calcuations discussed here
for which we lack exact results
provided by full configuration interaction calculations,
is not a good strategy for developing an IMSRG(3) approximation.
To develop the IMSRG(3) further,
a more careful look into the behavior and structure of the fundamental commutators
is important
to understand the deficiencies in the naive organization by cost.
One path to doing this is the perturbative analysis
of the IMSRG(3) flow equations.
This could allow one to develop an approximate IMSRG(3) truncation
that is potentially fourth- or fifth-order exact in MBPT.
This analysis will need to be adapted
to account for non-HF reference states
and also cases beyond the NO2B approximation
where the residual normal-ordered three-body Hamiltonian
is included at the start of the IMSRG calculation.
In this analysis of the IMSRG(3),
it will be interesting to look at how the IMSRG(3)
performs relative to the IMSRG(2) under variation of the reference state.
If one finds that the IMSRG(3) is more successful
for sub-optimal reference states,
understanding this feature of the IMSRG(3)
may allow for constructions of IMSRG(3) approximations
that share this behavior.

This thesis represents a first step
towards full IMSRG(3) calculations
of light and medium-mass nuclei
and the systematic development and testing
of approximate IMSRG(3) truncation schemes.
Accessing full IMSRG(3) calculations
will provide access to high-precision many-body results,
hopefully bringing the many-body uncertainty error
for ground-state energies robustly into the per-mille range.
Well-motivated approximations of the IMSRG(3)
will make most of the accuracy benefits of the IMSRG(3)
available at a lower computational cost,
necessary for calculations where large model spaces
are required to converge calculations,
for example for heavy nuclei or hard, unevolved nuclear Hamiltonians.


\chapter{Introduction}

\section{Problem}

In Information Retrieval (IR) the main objective is to retrieve and rank relevant documents that satisfy a user’s search query, which represents an instance of a user’s information need.
Most traditional information retrieval models, like TF-IDF and BM25, are designed to estimate document relevance by counting the word resp. term frequency in documents.
These models utilize an inverted index for holding a posting list for every term in the model’s vocabulary.
The posting list for a term saves tuples of a document id and its term frequency of the documents, in which the term occurs.
A posting list only holds the document ids for a term where the term frequency is non-zero, and therefore the data storage requirements are reduced.
Additionally, these traditional models take advantage of efficient data access during the retrieval phase.

However, a drawback of these models is that they consider only the documents which contain exact matches of the query terms.
These IR models ignore the aboutness of a document and the semantic concepts it depicts, because they do not comprise relationships between terms. \cite{mitra:2018:introduction-neural-ir}

Users may express their search intent by formulating a query with words, that do not exactly match the words that occur in the relevant documents for their information need.
Circumlocutory queries are also used by people for seeking relevant medical information for the self-diagnosis of symptoms they observe, for instance.
These queries are formulated as a description of their symptoms (e.g. hives all over body).
Zuccon et al. revealed in their work \cite{zuccon:2015:diagnose-this} that current search engines are still poorly equipped to answer such queries, which might result in misleading advice, escalation of medical concerns and medication errors.

\section{Motivation and Goal}

A vocabulary gap occurs, if the set of terms used in a query compared to those used in documents are different, but describe the same concepts. \cite{van-gysel:2017:neural-vector-spaces}
\\Neural IR models were proposed to bridge the gap between query and document vocabulary.
These models employ neural networks for the information retrieval task.
Current neural IR models use supervised machine learning techniques for learning to rank that require a large collection of labelled data, which sometimes are not sufficiently available.
\\In an alternative setting where labelled data is not needed, unsupervised learning can be used to learn vector representations (embeddings) of query and document text that capture different notions of text similarity and relatedness between terms.
These representations can then be used in conjunction with a similarity metric (e.g. cosine similarity) to estimate the relevance of a document to a query in a vector space. \cite{mitra:2018:introduction-neural-ir}

Words that are used in a document in similar context tend to be semantically similar, according to the distributional hypothesis.
In observed feature spaces, a term can be represented differently based on the selected distributional features (e.g. by neighboring terms), and consequently captures different semantic relationships between terms.
A drawback of observed feature spaces is, that representations are highly sparse and high-dimensional.
By contrast, an embedding is a simplified representation of items in a new vector space and usually learned from observed features in a way such that relationships and properties of the items are preserved.
The features of an embedding’s vector space are not observable resp. not interpretable, but latent.
Latent vector spaces are dense and lower-dimensional and learnt from data. \cite{mitra:2018:introduction-neural-ir}

As stated in the work of Zamani et al. \cite{zamani:2018:from-neural-reranking-to-neural-ranking} current neural IR ranking models learn dense latent representations for query and document terms, which is highly inefficient and infeasible for large document collections, if every query and document representation must be matched during scoring.
To avoid this, neural re-ranking models have been proposed, which are integrated in a multi-stage ranking system and only re-rank a small subset of potentially relevant documents retrieved by an efficient different first stage ranker beforehand. \cite{zamani:2018:from-neural-reranking-to-neural-ranking}
As a first stage ranker a lexical matching method can be used. \cite{van-gysel:2017:neural-vector-spaces}
Re-ranking a small subset of the document collection might be more efficient, but a downside of re-ranking models is their acting as a document filter.
Some relevant documents might be filtered out by the first stage ranker in an early stage, that could have been retrieved by a neural IR model. \cite{zamani:2018:from-neural-reranking-to-neural-ranking}
\\Scientific research and work on neural IR approaches have been limited mostly to re-ranking the top-k documents. \cite{mitra:2018:introduction-neural-ir}
This is motivating to explore new neural IR models.

In a recent work from Van Gysel et al. \cite{van-gysel:2017:neural-vector-spaces} they introduce the Neural Vector Space Model (NVSM) suited for document collections of medium scale.
According to the authors, their model performs better at document ranking than existing latent semantic vector space methods.
Their neural IR model learns low-dimensional representations of words and documents with unsupervised learning from scratch by gradient descent.
Additionally, they learn a linear transformation function that maps from the word representation vector space to the document representation vector space, due to different vector space dimensionality.
NVSM is different from other neural semantic matching models, because instead of matching every document in the document collection to a query, they use nearest neighbor search algorithms within the vector space framework to rank the top-k documents. \cite{van-gysel:2017:neural-vector-spaces}

Zamani et al. recently proposed \cite{zamani:2018:from-neural-reranking-to-neural-ranking} a standalone neural ranking model (SNRM).
Their model does not require a first stage ranker and the authors stress retrieval using large-scale document collections as efficient as conventional term matching models.
The SNRM is designed to first learn low-dimensional dense latent representations of queries and documents to capture semantic meaning, and then learns a function that transforms the representation in a high-dimensional representation by enforcing and rewarding sparsity.
The resulting sparse representations then allow the construction of an inverted index for the document collection. \cite{zamani:2018:from-neural-reranking-to-neural-ranking}


\documentclass[aos]{imsart}
\setattribute{journal}{name}{}
\pdfminorversion=4
\usepackage[left=1in,top=1in,right=1in,bottom=1in]{geometry}
\usepackage{amsmath}
\usepackage{amsbsy}
\usepackage{hyperref}
\usepackage{graphicx, amsmath, amssymb, fullpage, amsfonts, bbm, bbold}
\usepackage{lineno}
\usepackage{color}
\usepackage[round]{natbib}
\usepackage{enumerate}
\linespread{1.5}
\newcommand{\abs}[1] {|#1|}
\newcommand{\logit}{\mbox{logit}}
\newcommand{\B}{\mathbf{B}}
\newcommand{\bPsi}{\mathbf{\Psi}}
\newcommand{\bbeta}{\boldsymbol{\beta}}
\newcommand{\X}{\mathbf{X}}
\newcommand{\M}{\mathbf{M}}
\newcommand{\Y}{\mathbf{Y}}
\newcommand{\tepsilon}{\tilde{\epsilon}}
\newcommand{\blambda}{\mathbf{\lambda}}
\newcommand{\Z}{\mathbf{Z}}
\newcommand{\Norm}{\mathcal{N}}
\newcommand{\br}{\mathbf{r}}
\newcommand{\bv}{\mathbf{b}}
\newcommand{\f}{\mathbf{f}}
\newcommand{\btheta}{\boldsymbol{\theta}}
\newcommand{\epsy}{\epsilon}
\newcommand{\epsz}{\nu}
\newcommand{\z}{\mathrm{z}}
\newcommand{\bSigma}{\boldsymbol{\Sigma}}
\newcommand{\probit}{\mbox{probit}}
\newcommand{\hiw}{\mbox{{\small\textsc{HIW}}}}
\newcommand{\iw}{\mbox{{\small\textsc{IW}}}}
\newcommand{\N}{\mbox{{\small\textsc{N}}}}
\newcommand{\T}{\mbox{{\small\textsc{T}}}}
\newcommand{\C}{\mbox{{\small\textsc{C}}}}
\newcommand{\Ga}{\mbox{{\small\textsc{Ga}}}}
\newcommand{\IG}{\mbox{{\small\textsc{IG}}}}
\newcommand{\Be}{\mbox{{\small\textsc{Be}}}}
\newcommand{\Ber}{\mbox{{\small\textsc{Ber}}}}
\newcommand{\dd}{\mbox{d}}
\newcommand{\E}{\mbox{E}}
\newcommand{\V}{\mathbf{V}}
\newcommand{\I}{\mathbf{I}}
\newcommand{\Tau}{\mathrm{T}}
\newcommand{\x}{\mathrm{x}}
\newenvironment{example}[1][Example]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newenvironment{remark}[1][Remark]{\begin{trivlist}
\item[\hskip \labelsep {\bfseries #1}]}{\end{trivlist}}
\newcommand\independent{\protect\mathpalette{\protect\independenT}{\perp}}
\def\independenT#1#2{\mathrel{\rlap{$#1#2$}\mkern2mu{#1#2}}}
\usepackage{bm}

\begin{document}
\begin{frontmatter}
\title{Semi-supervised propensity score estimation}

\author{Andrew Herren} \and \author{P. Richard Hahn}
\affiliation{Arizona State University}
\date{}

\begin{abstract}
We extend the notion of semi-supervised learning to causal inference by considering data as ``unlabeled"
if treatment assignment and covariates are observed but outcomes are unobserved.
We demonstrate how to make use of unlabeled data in estimating the average treatment effect,
and validate the results using simulation studies as well as public survey data.
\end{abstract}

%% keywords here, in the form: keyword \sep keyword
\begin{keyword}
Bayesian; Causal inference; Machine learning; Semi-supervised learning; Propensity score; Unlabeled data
\end{keyword}

\end{frontmatter}
% Activate to display a given date or no date
\section{Introduction}
The challenge of inferring causal effects from observational data has led to decades of methodological research.
\cite{rosenbaum1983central} introduce the propensity score as an individual's probability of receiving treatment and show that
it can produce unbiased estimates of treatment effects when used with matching or stratification estimators.
Propensity score methods have been put to successful use in many domains, including economics (\cite{heckman1997matching}),
medicine (\cite{luo2010applying}), and political science (\cite{fong2018covariate}).

We show that treatment effect estimates can be improved by using unlabeled treatment assignment
and covariate data (that is, data points for which outcomes are not observed, but covariates and treatment status are observed).
With insight from the semi-supervised learning literature, we use the unlabeled data to calculate more accurate propensity
scores and then use the predicted propensity scores to estimate treatment effects among the labeled observations.

Much work has been done on propensity score methods in the general case of missing data.
\cite{d2000estimating} extend the ECM algorithm (\cite{meng1993maximum}) to learn propensity scores with missing covariates.
\cite{williamson2012doubly} introduce doubly robust estimators that can handle missing data in the outcome, treatment, or covariates.
\cite{zhang2016causal} use data with missing treatment assignments to motivate a triply robust estimator of outcome, propensity, and missingness.
The goal of these papers, broadly, is to impute missing data before estimating treatment effects.

\cite{liang2007use} review the general topic of semi-supervised learning and articulate the conditions
under which marginal, or unlabeled, data can aid in parameter inference, and when it is unnecessary.
The semi-supervised learning framework has been applied productively in the machine learning literature
(see for example, \cite{kingma2014semi}).
\cite{cheng2018efficient} propose a semi-supervised learning framework for estimating average treatment effects.
However, their method differs from our approach in several ways.
Most prominently, they aim to build a doubly robust estimator of the ATE, while our aim is to
learn propensity scores which can be used flexibly in a variety of estimators.

\section{Problem statement and notation}

\subsection{Notation}
\label{subsection:notation}

Consider a dataset with $n$ observations. Let $\Y$ denote the outcome of interest, which we assume to be only partially observed.
We refer to the data points with observed outcomes as ``labeled data," and let $n_o$ refer to the number of such labeled observations
and $n_m = n - n_o$ refer to the number of data points with missing outcomes.
Let $\X$ denote covariates, which includes information that may predict treatment assignment, outcomes, or both.
In many applied settings, covariates include demographics, health markers, or various self-reported behaviors.
$\Z$ refers to a binary treatment assignment. In a controlled experiment, the treatment assignment would be randomized according to a pre-specified design.
Observational studies are characterized by a lack of direct manipulation of treatment assignment.
Finally, an outcome's status as missing or observed is indexed by $\M$. Central to our approach is the assumption that outcomes are only
observed for a fraction of a given dataset.

Following the notation of \cite{liang2007use}, we denote unlabeled, or marginal, data as

$\X^m = \{x_i; i = n_o + 1, ... , n_m \}$, and

$\Z^m = \{z_i; i = n_o + 1, ... ,n_m \}$,

where $n_m$ denotes the number of unlabeled covariate data points. We observe that $\M = 1$ where $x \in \X^m, z \in \Z^m$ and $\M = 0$ elsewhere.

\subsection{Average treatment effects and their estimators}

One common goal in causal inference with observational data is to recover an unbiased estimate of the average treatment effect (ATE). Following \cite{rosenbaum1983central},
we define $\textrm{ATE} = \E_X[\E[Y | X, Z = 1] - \E[Y | X, Z = 0]]$ as the population expectation of the outcome for those who received treatment less the expected outcome for those who did
not receive treatment. We assume that $Z$ is exchangeable conditional on $X$. The ATE can be estimated by a number of procedures. This paper will focus on several
estimators which make use of estimated propensity scores.

The first method we evaluate is the Inverse Propensity Weighted (IPW) estimator of the ATE (\cite{hirano2003efficient}), defined as
\[ \textrm{ATE}_{\textrm{IPW}} = \frac{1}{n_o} \sum_{i = 1}^{n_o} \left( \frac{Y_i Z_i}{\hat{p_i}} - \frac{Y_i (1 - Z_i)}{(1 - \hat{p_i})} \right) \]
where the propensity score, $\hat{p_i} = \hat{p_i}(Z_i = z | X_i)$, is an estimate of the probability of receiving treatment conditional on covariates $X_i$ (\cite{rosenbaum1983central}).

Next, we consider the targeted maximum likelihood estimator (TMLE) (\cite{van2010targeted}, \cite{van2010targeted2}, \cite{gruber2009targeted})
\[\textrm{ATE}_{\textrm{TMLE}} = \frac{1}{n_o} \sum_{i = 1}^{n_o} \left( Q^*\left( Z_i = 1, X_i \right) - Q^*\left( Z_i = 0, X_i \right) \right)\]
where $Q^*\left( Z_i, X_i \right)$ represents a semi-parametric model of the outcome $Y$ which incorporates an estimate of the propensity model.

Finally, we consider Bayesian causal forests (BCF) (\cite{hahn2020bayesian})
\[\textrm{ATE}_{\textrm{BCF}} = \frac{1}{n_o} \sum_{i = 1}^{n_o} \left( \bar{f}(X_i, Z_i = 1) - \bar{f}(X_i, Z_i = 0) \right) \]
where $\bar{f}(X_i, Z_i)$ is an average of posterior simulations (across all simulations) of two BART (\cite{chipman2010bart}) models, defined as $f(X_i, Z_i) = \mu(X_i, \hat{p_i}) + \tau(X_i) Z_i$. $\hat{p_i}$ is an estimate of the propensity score.

Each of these methods is evaluated according to three approaches:

\begin{itemize}
\item \textbf{Complete case analysis}: We discard all unlabeled data samples, estimate $\hat{p}$, and then compute average treatment effects on only the labeled observations.
\item \textbf{Semi-supervised}: We use the labeled and unlabeled $X$ and $Z$ values to estimate $\hat{p}$ and then use the $\hat{p}$ predictions to compute average treatment effects on the labeled observations.
\item \textit{[Simulation only]} \textbf{Ground truth}: Simulation studies provide us with actual probabilities of receiving treatment, so we use these true $p$ to compute average treatment effects on the labeled observations. It is evident that in empirical evaluations tested on real world data, such ground truth will not be available.
\end{itemize}

\section{Empirical results}

Given that most propensity scores model a binary treatment assignment, logistic regression is a natural choice for
estimating propensity scores, however ~\cite{lee2010improving} point out that the assumptions required for logistic
regression are not always warranted and examine propensity score estimation using a number of machine learning
methods, including CART (\cite{breiman1984classification}) and Random Forest (\cite{breiman2001random}).

This paper proceeds similarly, but instead relies on Bayesian Additive Regression Trees (BART) (\cite{chipman2010bart}).
BART is a nonparametric Bayesian tree ensemble method that learns complex functions via a sum of weak regression
or classification trees. In the case of our IPW estimator, we also consider a propensity model specified using logistic regression
in order to investigate how inferences can be harmed with a mis-specified propensity model.

We construct confidence intervals for our simulations as follows:

\begin{itemize}
	\item \textbf{IPW}: We compute the standard error for a logit propensity model as in \cite{cerulli2014treatrew}
	\item \textbf{TMLE}: We use the 95\% confidence interval produced by the \texttt{tmle} R package
	\item \textbf{BCF}: We construct a 95\% credible interval using posterior samples of the treatment effect
\end{itemize}

\subsection{Evaluation metrics}

We evaluate the results of our simulation using the following metrics.

\begin{itemize}
	\item \textbf{RMSE}: root mean squared error of estimated treatment effects
	\item \textbf{Bias}: difference between true ATE and average estimated ATE
	\item \textbf{Coverage}: share of 95\% confidence intervals that contain the true ATE
\end{itemize}

We will see that some estimators produce biased estimates of the ATE, others produce largely unbiased estimates
but attain poor interval coverage, and in some cases, both phenomena occur.

\subsection{Simulations}
\label{section:simstudy}

We first test our method using simulated data. We model a number of data generating processes, each of which is designed to
test the conditions under which propensity scores trained using unlabeled data are helpful or harmful to the ultimate
goal of estimating average treatment effects.

\subsubsection{Data Generating Process}

For simulated covariates, outcomes, and treatment effects, we use a subset of the data generating processes tested in \cite{hahn2020bayesian}.

\begin{align*}
Y &= \mu(X) + \tau Z + \varepsilon\\
\mu(X) &=
\begin{cases}
3 + x_1 x_3,&  x_5 = 1,\\
x_1 x_3,&  x_5 = 2\\
-3 + x_1 x_3,&  x_5 = 3
\end{cases} \\
x_1, x_2, x_3 &\sim \mathcal{N}(0, 1)\\
x_4 &\sim \textrm{Bernoulli}(0.5)\\
x_5 &\sim \textrm{Categorical}(0.25, 0.5, 0.25)\\
\tau &= 3\\
\varepsilon &\sim \mathcal{N}(0, 1)
\end{align*}
There are $p = 5$ covariates, the first three of which are independent standard normal variables, the fourth of which is binary,
and the fifth is an unordered categorical variable with values $1, 2, 3$. Treatment effects are homogeneous ($\tau = 3$), and
the outcome is determined by a combination of treatment effects and a piecewise interaction function of three of the covariates
($\mu(x) = 1 + g(x_5) + x_1 x_3$, where $g(x) = 2$ if $x = 1$, $g(x) = -1$ if $x = 2$, and $g(x) = -4$ if $x = 3$).
$\mu(x)$ is often referred to as a \textit{prognostic effect} and can be conceptualized as the expected value of
the outcome for individuals who do not receive the treatment.
Sample sizes in our simulations vary between $500$, $1000$ and $5000$.

For treatment assignment, we consider two cases:

\begin{enumerate}
\item $P(Z = 1 | X) = 0.5$
\item $P(Z = 1 | X) = 0.8\Phi(3 \mu(X) / s - 0.5 x_1) + 0.05 + u / 10$
\begin{itemize}
	\item $\Phi(\cdot)$ is the standard normal CDF
	\item $s$ is the sample standard deviation of simulated observation of $\mu(X)$
	\item $u \sim \mathrm{Uniform}(0, 1)$
\end{itemize}
\end{enumerate}

The second treatment assignment mechanism is described as ``Targeted Selection" in \cite{hahn2020bayesian}, and
refers to the simple phenomenon of assigning treatment based on the expected value of the outcome for those who don't receive
treatment. This could be motivated quite simply as a physician's mental calculus of ``how much worse will these symptoms get if I don't
assign treatment."

In order to simulate the process by which outcomes are unlabeled, we assume that 90\% of the data is unlabeled and that the status of $\M = 0$ or $\M = 1$
does not correlate with $\Y$, $\X$, or $\Z$. This parallels the notion of missing completely at random (MCAR) (\cite{little2019statistical}) in the missing data
literature, though it is worth noting that our problem is not a traditional missing data problem. We treat our unlabeled $X^m$ and $Z^m$ as auxiliary data that
may be helpful in estimating the ATE rather than treating our unobserved $Y^m$ as missing data to be imputed or otherwise estimated.

\subsubsection{Simulation results under targeted selection}

We first summarize the key takeaways from our simulations before proceeding to a more detailed discussion.

\begin{itemize}
	\item Using unlabeled data will harm inferences in the case of a mis-specified propensity model, as confidence intervals are narrow around a biased estimate
	\item BCF and IPW/BART both work best in the targeted selection scenario
	\item While randomized treatment assignment makes ATE estimation much easier, unlabeled data can still be useful in this case for variance reduction purposes
\end{itemize}

We first examine the targeted selection case, in which treatment assignment is correlated with the
prognostic effect. The table below shows the results for 500 simulations using an IPW estimator and propensities
estimated by logistic regression. Logistic regression cannot capture the complexity of the simulated
treatment assignment in the targeted selected scenario, and we see that all of the results
which use estimated propensities are badly biased. Furthermore, we observe that using unlabeled data
and increasing sample size is harmful to proper inference of the true ATE, as the variance is reduced
around a biased estimate, leading to poor interval coverage.

\input{../../outputs/simulations_IPW_LIN_HOM_MCAR_90.tex}

The table below shows the results for 500 simulations using an IPW estimator and propensities
estimated by BART. In this case, the propensity model is not mis-specified, and we observe that
the use of unlabeled data reduces bias and increases interval coverage.

\input{../../outputs/simulations_IPW_BART_LIN_HOM_MCAR_90.tex}

The table below shows the results for 500 simulations using a TMLE estimator and propensities
estimated by BART. In this case, the propensity model is not mis-specified, however, we observe
that the TMLE estimator achieves poor interval coverage, even in high sample sizes when the
average bias is lower.

\input{../../outputs/simulations_TMLE_LIN_HOM_MCAR_90.tex}

The table below shows the results for 500 simulations using a BCF estimator and propensities
estimated by BART. BCF was designed in part to handle the case of targeted selection, and we
see that it produces unbiased estimates with high coverage as more unlabeled data is incorporated.

\input{../../outputs/simulations_BCF_LIN_HOM_MCAR_90.tex}

\subsubsection{Simulation results under randomized assignment}

We now turn to the randomized treatment assignment scenario.
The table below shows the results for 500 simulations using the Complete Case approach
across all four estimators. We see in this case that all methods produce unbiased estimates
of the ATE as sample size increases, and with the exception of the TMLE estimator, the approaches
all achieve high coverage of the true treatment effect.

\input{../../outputs/simulations_CC_ALL_EXP_LIN_HOM_MCAR_90.tex}

We now test the same simulations using the semi-supervised approach, and observe that
each method produces unbiased estimates of the ATE with larger sample sizes, but that using
unlabeled data reduces the variance of the IPW estimates, so that coverage is closer to 95\%.

\input{../../outputs/simulations_SSL_ALL_EXP_LIN_HOM_MCAR_90.tex}

\subsection{Real Data Example}

\subsubsection{NHEFS}

Following \cite{hernan2020causal}, we use data from the National Health and Nutrition Examination Survey Data I Epidemiologic Follow-up Study (NHEFS)
to estimate the effect of quitting smoking on weight gain.
We use a modified version of the NHEFS data available on the book website for \cite{hernan2020causal}\footnote{https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/}.
Following the discussion in Chapter 12, we remove missing values for any of the data fields which will be used in estimating the ATE: age, sex, race, weight in 1971, weight in 1982, height,
education level, frequency of alcohol consumption, number of daily cigarettes smoked in 1971, and the change in number of daily cigarettes smoked between 1971 and 1982.

After making these adjustments, we summarize the differences in the treatment and control groups across a number of variables and see that the results closely match those of \cite{hernan2020causal}.

\begin{table}[ht]
\centering
\begingroup\small
\begin{tabular}{lllrrr}
  \hline
Treatment Status & Age & \% Men & \% White & Weight (kg), 1971 & Cigarettes per day, 1971 \\
  \hline
Quit smoking & 46.2 & 54.6 & 91.1 & 72.4 & 18.6 \\
Have not quit smoking & 42.8 & 46.6 & 85.4 & 70.3 & 21.2 \\
   \hline
\end{tabular}
\endgroup
\caption{NHEFS - summary of covariates by treatment group}
\end{table}

Given the two approaches which showed the strongest finite sample properties in our simulation studies were IPW-BART and BCF, we use
both methods to estimate the average effect on weight gain of quitting smoking.
In each case, we use the same eleven potential confounders as covariates in a propensity model and estimate the overall ATE either via IPW or BCF.
With IPW-BART, we estimate an average treatment effect of 3.0 kilograms with a 95\% confidence interval of $(2.0, 3.9)$ and, with BCF we estimate an
average treatment effect of 3.4 kilograms and a 95\% credible interval of $(2.5, 5.9)$.

To explore the potential use of unlabeled data, we proceed by removing data at random from this dataset, estimating complete case and semi-supervised ATEs, and repeating the procedure
on many sub-samples of the data. Table 8 below reports the average treatment effects estimated at different levels of outcome missingness.
These results highlight the same trends seen in our simulation studies. Using unlabeled data to estimate propensity scores generally improves the quality of our estimates, but having an
adequate sample of observed outcomes is crucial. 80\% of outcomes missing translates to roughly 300 observed outcomes, and we see that in this case the semi-supervised approach comes
much closer to approximating the full data ATE.

\begin{table}[ht]
\centering
\begingroup\small
\begin{tabular}{ccc}
  \hline
\% Outcomes Missing & Complete Case & Semi-Supervised \\
  \hline
80\% & 2.5 & 3.0 \\
50\% & 2.8 & 3.0 \\
20\% & 2.9 & 3.0 \\
   \hline
\end{tabular}
\endgroup
\caption{IPW-BART Comparison of Complete Case and Semi-Supervised Approaches (200 sub-samples)}
\end{table}

In running this experiment with the BCF estimator, we see that the difference between the semi-supervised and complete case approaches is virtually indistinguishable.

\begin{table}[ht]
\centering
\begingroup\small
\begin{tabular}{ccccc}
  \hline
\% Outcomes Missing & Complete Case & Semi-Supervised \\
  \hline
80\% & 3.0 & 3.0 \\
50\% & 3.3 & 3.4 \\
20\% & 3.4 & 3.4 \\
   \hline
\end{tabular}
\endgroup
\caption{BCF Comparison of Complete Case and Semi-Supervised Approaches (200 sub-samples)}
\end{table}

\newpage
\section{Discussion}

\subsection{Theoretical motivation of semi-supervised propensity score results}
\label{subsection:theoretical}

To understand when and to what extent unlabeled data can be useful in estimating average treatment effects with propensity scores, we consider a theoretical example.
Let $Y$ be an outcome of interest, $X$ be a set of covariates, and $Z$ be treatment assignment. Following \cite{hernan2020causal}, we define the ATE according to:
$\E[Y | Z] = \theta_1 + \theta_2 Z$, where $\theta_2$ is the ATE estimand for a binary treatment variable $Z$.
We write the two parameters of this model as $\theta = [\theta_1, \textrm{ } \theta_2]$ and the matrix $U = [1, \textrm{ } Z]$ as the treatment assignments with a vector of
ones appended. Thus, we can formulate the conditional expected outcome as $\E[Y | Z] = U\theta$.
As \cite{hernan2020causal} note, as long as every individual has a non-zero probability of receiving treatment, the IPW estimator of the ATE is equivalent in expectation to
the weighted least squares estimator of $\theta_2$, given by the second element of the vector $\hat{\theta} = \left(U^TWU\right)^{-1}U^TWY$, where $W$ is a
diagonal matrix of weights computed from estimated propensity scores ($\hat{p}_i$) as follows:
\begin{equation*}
\begin{aligned}
w_i & =
\begin{cases}
\frac{1}{\hat{p}_i},&  Z_i = 1,\\
\frac{1}{1 - \hat{p}_i},&  Z_i = 0,
\end{cases} \\
\end{aligned}
\end{equation*}

\cite{gelman2006data} point out that the weighted least squares estimator is the maximum likelihood estimator for
$\theta$ in the data model $Y_i \sim \Norm(Z_i\theta, \frac{\sigma^2}{w_i})$. Consider the following data generating process:

\begin{equation*}
\begin{aligned}
X & \sim \Norm(0, \Sigma_1)\\
\beta & \sim \Norm(0, \Sigma_2)\\
\theta & \sim \Norm(0, \Sigma_3)\\
p_i & = \left(\frac{1}{1+\exp(-X_i\beta)}\right)\\
Z_i | p_i & \sim \textrm{Bernoulli}(p_i)\\
w_i & =
\begin{cases}
\frac{1}{p_i},&  Z_i = 1,\\
\frac{1}{1 - p_i},&  Z_i = 0,
\end{cases} \\
Y_i | Z_i, \theta & \sim \Norm(Z_i\theta, \frac{\sigma^2}{w_i})\\
\end{aligned}
\end{equation*}

As in Section \ref{subsection:notation}, we assume that, in addition to $Y$, $Z$, and $X$, we also have data $Z^m$ and $X^m$ for which no outcomes are observed.
We are interested in characterizing the posterior distribution $p(\theta, \beta | Y, Z, X)$ to determine whether information contained in $Z^m$ and $X^m$ will be helpful
in making inferences about $\theta$. We can see that
\begin{equation*}
\begin{aligned}
p(\theta, \beta | Y, Z, X) & \propto p(Y | Z, \theta, w, \sigma) p(Z | X, \beta) p(X | \Sigma_1) p(\theta | \Sigma_2) p(\beta | \Sigma_3)\\
\end{aligned}
\end{equation*}
The unlabeled data enter this posterior via the propensity model $p(Z | X, \beta)$ and the covariate likelihood $p(X | \Sigma_1)$.
We see that when the true propensities are unknown and must be estimated from the data, the marginal data $Z^m$ and $X^m$ are useful in inferring $\theta$.
This is driven by the fact that the weights in the outcome likelihood depend on $\beta$ which is learned from the labeled and unlabeled samples of ($X, Z$).

However, when $p(Z | X, \beta) = p$ (i.e. $Z \perp X$) for some known value $p$, as in the case of a randomized experiment, the weights $w$ in the outcome
likelihood will be constant in $X$, so the posterior for $\theta$ does not depend on $\beta$.
In this case, unlabeled data $Z^m$ and $X^m$ are irrelevant in making inferences about $\theta$, because $p(Y | Z, \theta, w, \sigma)$ is only informed by the known
probability $p$. This is also true of the more general case in which $Z \not\perp X$ but the true treatment probabilities are known for each value of $X$.

\subsection{Estimated versus true propensities}
\label{subsection:estimatedprop}

A recurrent theme in the causal inference literature is that treatment effect estimators which rely on fitted propensities attain a lower variance than estimators that
rely on known, true propensities. Taken at face value, this point would appear to invalidate our recommendation to use unlabeled data in treatment effect estimation.
Why use unlabeled data to better approximate the true propensities, when the estimated propensities lead to superior inference?
What appears at first glance as a conflict between our approach and the received statistical wisdom masks a series of interesting and subtle points. We feel that this topic
deserves a more detailed and consolidated elucidation, which we present in the following section.

\cite{rosenbaum1987model} provides the first detailed treatment of this question in the context of estimating population outcomes from sample data.
Consider a population of interest in which each of $S$ strata have (possibly different) probabilities of selection. He compares three estimators of a population outcome:
\begin{enumerate}
\item Empirical postratification: each stratum is weighted by its proportion of sampled units
\item Model-based postratification: each stratum is weighted by a parametric estimate of its sampling probability
\item Known weight postratification: each stratum is weighted by its known sampling probability
\end{enumerate}
He shows that, conditional on the observed sample, estimators 1 and 2 are unbiased, but 3 is biased. Citing \cite{holt1979post}, he notes that post-stratification estimators
which are unbiased conditional on an observed sample are preferable.

\cite{imbens1996efficient} develop an efficient estimator for stratified sampling data which necessitates a moment constraint on the selection probability less the observed
selection variable. When the true selection probabilities are known, they note that this approach is similar in nature to the Seemingly Unrelated Regression (SUR)
(\cite{zellner1963estimators}), in which models fit simultaneously can experience efficiency gains even when there are no common parameters.
\cite{hirano2003efficient} also show that estimators which use the true propensities can attain the efficiency bound by incorporating additional moment constraints on
the selection (or treatment) variable, covariates, and the true propensities. \cite{lunceford2004stratification} demonstrate that the IPW estimator
with estimated propensities has a lower large sample variance than the same estimator using true propensities.

To address these points and reconcile them to our approach, we note first and foremost that the results are primarily concerned with the asymptotic behavior of estimators.
It is certainly true in practice that the sample sizes required to obtain a minimum-variance treatment effect estimate depends on a number of factors, so that one can never
be sure an asymptotic result holds for their particular problem.
Indeed, our simulation results show that, for finite samples, using the true propensities (or more accurate estimates of the true propensities using unlabeled data) can lead
to reduced bias and greater interval coverage. Put differently, without enough data to attain asymptotic results, the risk of bias greatly outweighs the risk of inefficiency.

Second, we note that the results apply to weighting, stratification, and matching estimators. There are many practical alternatives to these methods, such as
BCF, which do not exhibit this behavior. Third, this phenomenon depends entirely upon being able to construct a propensity model that is an unbiased estimate of the true
propensities. Any amount of mis-specification (discussed the next section) will completely invalidate the efficiency gains by yielding a biased estimate of the average
treatment effect.

Finally, we demonstrate a modification to the weighting estimator which uses the true propensities and attains a lower variance than the fitted-propensity estimator.
Consider $X$, $Z$, and $Y$ as covariates, treatment assignment, and outcome, and $p$ as the true probability of receiving treatment. Inverting the logit function, we transform the true
propensities into $h = - \log(\frac{1}{p} - 1)$ and fit a logistic regression of $Z$ on $h$. Using the predicted probabilities from this regression model as weights in an IPW estimator,
we see in simulation studies that this method attains the lowest variance among three alternatives:
\begin{enumerate}
\item True propensities as weights
\item Estimated propensities as weights
\item Modified true propensities as weights
\end{enumerate}

In summary, we advise researchers who have access to true propensities, or a set of unlabeled data that will help in approximating the true propensities, to use that data in
estimating treatment effects. Researchers who wish to use a weighting estimator can simply modify the true propensities to reduce the variance of their estimates, and researchers
using more flexible techniques such as BCF should not worry about this result in any case.

\subsection{Propensity model specification}
\label{subsection:overfitprop}

Though there may be benefits to using estimated propensity scores, in practice, calculating propensity scores introduces a new set of risks. The simulations in Section \ref{section:simstudy}
show that a misspecified propensity model can bias estimates, sometimes drastically. It may seem natural, then, to treat a propensity model as any other supervised learning
problem, with variable selection, model validation, and diagnostics to ensure a proper fit.
Indeed, \cite{mccaffrey2004propensity} use generalized boosted models (GBM) to estimate propensities, in part for the automatic variable selection and flexible model specification that
tree ensembles provide. Their results show that using GBM for a propensity model leads to better covariate balance and lower standard errors than a logit propensity model.

However, \cite{hernan2020causal} caution that traditional model selection techniques may
be of dubious utility in constructing propensity models. They note that the purpose of the propensity score is to construct an alternate population in which exchangeability is not
conditional on any of the measured covariates. They are careful to point out that a purely data-driven approach to model selection runs the risk of including variables, such as
colliders, mediators, or post-treatment variables, that jeopardize identification of treatment effects. They also advise that, even when treatment effects are identified, including
variables which are not confounders, but are strongly associated with treatment assignment, in a propensity model can increase the variance of treatment effect estimates
by pushing estimated propensity weights closer to 0 and 1.

While it is certainly true that propensity models can be incorrect in ways that make estimation of treatment effects impossible or numerically unstable, a number of practical steps
can be taken to mitigate these concerns. First, estimated propensity scores should always be visualized or otherwise inspected to ensure that they are
not numerically close to 0 and 1. Second, methods like ridge regression or tree ensembles, which introduce some measure of regularization, can be used instead of logistic regression
to minimize the risk of estimating propensity scores close to 0 and 1. Finally, subject matter expertise should always play a role in building propensity models. However, we are aware that
for many applied problems, especially those with a high-dimensional covariate set, the challenge of constructing an accurate causal graph is highly non-trivial. Insofar as subject-matter
expertise can identify variables such as colliders, post-treatment variables, or non-confounders, such variables should of course be excluded from any propensity model.
Researchers who are concerned about an underlying causal graph invalidating their propensity model can also turn to the causal discovery literature (i.e. \cite{peters2017elements}). Specifically,
\cite{pmlr-v31-entner13a} present an approach to the problem of identifying confounders for adjustment in a causal model. In practice, the validity of assumptions about a causal graph
are often difficult to assess, and many researchers will have to assume that their study design (whether observational or experimental) is valid and be open about their assumptions.

In order to better understand the risk of including non-confounders in a propensity model, we test several possibilities in a simulation study. Letting $X_j$ be a variable in a set of $p$ covariates,
we consider four cases:
\begin{enumerate}[a)]
\item \textbf{Case 1}: $X_j$ is a confounder and is included in a propensity model
\item \textbf{Case 2}: $X_j$ is \underline{\textbf{not}} a confounder and is included in a propensity model
\item \textbf{Case 3}: $X_j$ is a confounder and is \underline{\textbf{not}} included in a propensity model
\item \textbf{Case 4}: $X_j$ is \underline{\textbf{not}} a confounder and is \underline{\textbf{not}} included in a propensity model
\end{enumerate}
Cases 1 and 4 are ideal, while cases 2 and 3 both fail to properly account for the true underlying causal model. The question we are concerned with is whether erring on the side
of including non-confounders in a propensity model is as harmful to inference as ignoring true confounders. To do this, we simulate 100 datasets of $n = $1,000 from the following data generating process:
\begin{align*}
Y &= X\beta + \tau Z + \varepsilon\\
X, \varepsilon &\sim \mathcal{N}(0, 1)\\
\tau &= 3\\
p & = \left(\frac{1}{1+\exp(-X\gamma)}\right)\\
Z & \sim \textrm{Bernoulli}(p)
\end{align*}
For all variables except $X_j$, the $\beta$ and $\gamma$ coefficients for the outcome and propensity models are drawn uniformly at random from a range of $(-0.5, 0.5)$ and $(-0.3, 0.3)$,
respectively. For the scenarios in which $X_j$ is a confounder (that is, $X_j$ impacts both $Y$ and $Z$), we set $\beta_{X_j} = 0.5$.
Each of our simulations assume some relationship between $X_j$ and $Z$, and we vary $\gamma_{X_j}$ between $\{0, 1, 10\}$ to test the extent to which conditioning on $X_j$ in a
propensity model separates the treatment and control groups.

Below we see the simulated bias, RMSE, and 95\% interval coverage of each of the four cases, using BCF with a BART propensity model and $\gamma_{X_j} = 1$

\begin{table}[ht]
\centering
\begingroup\small
\begin{tabular}{cccccc}
  \hline
Case & $X_j$ confounder? & $X_j$ in $\hat{p}$? & Bias & RMSE & Coverage \\
  \hline
1 & Yes & Yes & 0.004 & 0.098 & 89\% \\
2 & No & Yes & -0.008 & 0.096 & 88\% \\
3 & Yes & No & 0.068 & 0.109 & 84\% \\
4 & No & No & -0.007 & 0.087 & 87\% \\
   \hline
\end{tabular}
\endgroup
\caption{ATE estimation using BCF, $\gamma_{X_j} = 1$}
\end{table}
We see little difference in the results across each of the cases, because the influence of $X_j$ on $Y$ is modest in the confounded case.
Below is the same set of outputs for $\gamma_{X_j} = 10$.

\begin{table}[ht]
\centering
\begingroup\small
\begin{tabular}{cccccc}
  \hline
Case & $X_j$ confounder? & $X_j$ in $\hat{p}$? & Bias & RMSE & Coverage \\
  \hline
1 & Yes & Yes & 0.033 & 0.253 & 91\% \\
2 & No & Yes & -0.09 & 0.262 & 87\% \\
3 & Yes & No & 0.24 & 0.252 & 80\% \\
4 & No & No & -0.045 & 0.126 & 98\% \\
   \hline
\end{tabular}
\endgroup
\caption{ATE estimation using BCF, $\gamma_{X_j} = 10$}
\end{table}

We see that cases 1 and 4 exhibit the best performance (since they properly incorporate the true
causal graph), but even when $X_j$ and $Z$ are strongly related, conditioning on $X_j$ in a propensity mode
when $X_j$ is not a confounder is less harmful than excluding $X_j$ when it is a confounder.
To understand why, we briefly review the discussion of ``regularization-induced confounding" (RIC) covered in \cite{hahn2020bayesian}.

As we have seen in our simulations, classic parametric models of a treatment assignment are vulnerable to
misspecification. While this shortcoming suggests the use of flexible machine learning methods to estimate propensity scores, such methods run the risk of overfitting the data and estimating propensities close to 0 or 1.
In addition to numeric instability of common ATE estimators, such overfitting can also violate conditional exchangeability.
Researchers who want to fit flexible propensity models while avoiding overfitting can use regularized machine learning methods, such as BART. \cite{hahn2020bayesian}, building on \cite{hahn2018regularization}, show that a naive regularized model can bias treatment effect estimates.
Their proposed solution, BCF, controls this bias by incorporating the estimated propensities as a feature in a regression model predicting $\E(Y \mid Z = 0, \hat{p}, x)$.

In our case, since the simulation studies above are conducted using BCF with a BART propensity model, we avoid the problem of an overfit propensity model separating treated and control groups while also recovering unbiased estimates of the average treatment effect.
This explains why including a non-confounder, $X_j$, which is strongly predictive of $Z$ in the propensity model doesn't harm inference of the ATE by nearly as much as excluding $X_j$ when it is a
confounder.

\subsection{Connections to manifold learning theory}

\cite{belkin2006manifold} demonstrate a technique for semi-supervised learning which learns a low-dimensional representation of the feature space and incorporates that representation
as a penalty term in a loss function. \cite{bickel2007local} show that a sufficiently flexible locally-weighted regression will tend to rely on the relevant dimensions of the feature space
without making any explicit effort to learn a representation of the low(er) dimensional space. While these results may appear contradictory, it is worth noting that they have slightly different goals.
\cite{bickel2007local} show that in a supervised learning problem, a low-dimensional representation does not need to be explicitly learned in order to be incorporated into a regression function.
Their approach does not provide any method for making use of unlabeled data, however, while \cite{belkin2006manifold} explicitly aim to introduce unlabeled data in their learning problem.

The connection between propensity score methods and manifold learning is subtle. In manifold learning, the objective is to constrain a fitted function such that samples which are close
in the underlying manifold are also close in their predictions. In the context of treatment effect estimation, \cite{hahn2020bayesian} note that the propensity score can be viewed
as a one-dimensional representation of the feature space. Propensity scores could thus be considered as a constraint which ensures that samples which are close in the propensity space
will differ on average by treatment assignment and the associated causal effect. In either case, the key results of \cite{belkin2006manifold} show that, if a low dimensional representation is
desired for an estimation problem, using unlabeled data (from the same underlying distribution) can yield better estimates.

\subsection{Parallels to survey inference}

The use of auxiliary data in survey population inference has a rich history. \cite{horvitz1952generalization}
introduce estimators of survey totals based on measured quantities from a survey and auxiliary information about the population of interest.
\cite{GelmanLittle97} develop multilevel regression and post-stratification (MRP),
in which a multilevel model is used to estimate sample parameters for various demographic strata and population data
(often from public sources such as the census) is used to adjust those parameters to reflect their distribution in the population.
\cite{WANG2015980} show that a convenience sample with large sample size can be used along with
high quality auxiliary population data to accurately forecast election outcomes using MRP.
As response rates decline for traditional probability surveys (\cite{kennedy2019response}),
practitioners will be forced to rely increasingly on non-probability surveys.
\cite{mkks2017} have drawn connections between observational causal inference and non-probability surveys.
The semi-supervised propensity score method extends quite naturally to non-probability survey estimation
by treating auxiliary data (from federal surveys, the Census Bureau, etc...) as ``unlabeled."

\subsection{Acknowledgments}

This work was partially supported by NSF Award \#1502640.

\bibliographystyle{imsart-nameyear}
\bibliography{semi-supervised-propensity}

%\appendix
%\section{}

\end{document}


%!TEX root = ./thesis.tex
\graphicspath{{Figs/Burro/}{Figs/Falcon/}{Figs/BurroFigs/}}

\chapter{Dispersion of dense gas over obstacles}

\section{Simulations of Thorney Island test}
As the continuous spill of LNG in Burro 8 test, a transient simulation is employed. Initial values of all nodes in the domain must be specified prior to the simulation. These values are gained from a steady simulation of atmospheric condition before the spill.

The steady simulation uses the atmospheric inlet specified by Monin–Obukhov similarity theory. Two different turbulent model: standard $k-\epsilon$ with modifications and realizable $k-\epsilon$ are used to study the ability to simulate the atmospheric boundary of each model.

The transient simulation is divided into two steps. During the spill duration, from the time of zero to the time of \SI{107}{\second}, the LNG inlet is specified as in~\ref{sec:gas-inlet}. After that, the inlet is treated as the solid wall boundary.

To study the effect of turbulent intensity in the gas inlet, different value of turbulent intensity listed Table 9 will be used. The effect on the distance to LFL of these simulation will be recorded.

Three basic models of heat transfer from the ground in Table 10 will be simulated and compared with the point wise concentration profile and temperature profile from the experimental data of Burro 8.

\paragraph{Domain and grid generation}
To investigation the effects of the mesh on the solution, three different meshes which the number of nodes vary with the factor of two are compared. The best mesh achieved will then be used for further study.

A coarse mesh can be used to make the exploratory simulations to determine the adequately domain size \cite{Luketa-Hanlin2007}. The effect of the domain size to the gas concentration is observed to choose the sufficient domain for further study. The flow can be closely assumed symmetric, only half of the flow will be modelled. However, the symmetric plane of the flow do not coincide with the centreline of instrument array. Therefore, some conversion should be made when comparing the point-wise data of the simulation to the measurement. Equation (3.3)can be used for this purpose, where $\theta_{wind}$ is the deflection angle between the wind direction and the centreline of instrument array. The domain of (950, 500, 50) is used for the simulation of Burro tests.  The computational domain can be named by surfaces: pond area (gas vapour inlet), near-pond area (used for create a finer mesh region where there is an interaction between ambient air and gas vapour), ground, top boundary, side boundary, symmetry, inlet and outlet as Figure~\ref{fig:Domain} for ease of meshing process.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.8\textwidth]{Domain.png}
	\caption{The computational domain} \label{fig:Domain}
\end{figure}

The suitable type of the mesh majorly depends on the type of physics solved. For the dispersion of dense gas cloud simulation, structured mesh with hexahedral cells is proven to be much more computationally effective than unstructured mesh using tetrahedral cells \cite{Giannissi2013}. The mesh can be shown in Figure~\ref{fig:Mesh}.

\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.8\textwidth]{Mesh.png}
	\caption{The mesh} \label{fig:Mesh}
\end{figure}

The adequate number of nodal points using for the study can be determined using the mesh independence study. Successive grids are used for the simulation of the flow and the results are compared to see the effect of differencing grids. In case the resulting solution cannot be achievable due to the very large mesh, the Richardson’s extrapolation can be used to estimate the exact solution. \cite{Luketa-Hanlin2007}

The mesh independence on concentration of methane is tested using two mesh, the later mesh have double refinement in the pond area. Mesh statistics and quality is recorded in Table 8

\subsection{Boundary conditions}
\paragraph{Atmospheric inlet boundary}

Monin-Obukhov similarity theory is used to specify the wind velocity and temperature profile at the inlet. The velocity profile is calculated from Equation~\ref{eq:MO-velocity}. All required meteorological parameters are referred to Table~\ref{tab:Burro-meteo-params}.

\begin{equation}
u(z)=1.58 + 0.185\ln(z)+0.056z
\end{equation}

Using Equation~\ref{eq:MO-temperature}, the temperature profile is:
\begin{equation}
T(z) = 32.66+0.3625\ln(z)+0.11z
\end{equation}

Profile of momentum diffusivity $K_m$ and heat diffusivity $K_h$:
\begin{equation}
k_m = K_h = \frac{0.0296z}{1+0.303z}
\end{equation}

Calculation of the height of the atmospheric boundary layer using , $h_{ABL}=48 \si{\meter}$ . Using as the parameter for $k-\epsilon$ model, the profile of turbulent production rate k and dissipation rate $\epsilon$ can be calculated using Equation~\ref{eq:MO-k} and~\ref{eq:MO-epsilon}
\[
k =
\begin{cases}
0.033 					& (h \le 4.8) \\
0.033 \left(1-z/48\right)^{1.75}  	& (h>4.8)
\end{cases}
\]

\[
\epsilon =
\begin{cases}
\num{2.6e-4}+\num{1.24e-3}/z 				& (h \le 4.8) \\
(\num{2.6e-4}+\num{1.24e-3}/48)(1-0.85z/48)^{1.5}   	& (h>4.8)
\end{cases}
\]

\paragraph{Vapour gas inlet} \label{sec:gas-inlet}
Vapour gas inlet condition is usually obtained from separate source term modelling. There is not much information about the vaporisation of LNG from the experimental data. Therefore, uncertainty arises at the setting of this condition.

Burro 8 test was designed to maximize the vaporisation rate of LNG. Therefore, we can estimate the formation LNG vapour as equal to the spill rate of LNG. The velocity of LNG vapour, used as the inlet velocity in a simulation, thus is determined by equation:

Where the natural gas density $\rho_{NG}=1.76$ $Kg/m^{3}$ at the spill temperature 111 K. The density of LNG $\rho_{LNG}=424.1$ $Kg/m^{3}$ based on \cite{Zhang2015}.

The velocity of LNG is calculated from the spill rate and the pool area calculated using data from with the assumptions made in chapter 3.1. The natural gas inlet velocity is found to be $v_{NG}=0.1 m/s$.

For other circumstances, $\rho_{LNG}$ in equation (3.5) can be seen as the mass flux of LNG or the LNG vaporization rate. (Luketa-Hanlin, 2006) reviewed a number of experiments conducting to estimate the LNG vaporization rate of the spill on water, the range of this value varied between approximately \SIrange[range-units=single]{0.029}{0.195}{\kilogram\per\square\meter\per\second}. In the case of Burro 8, the simulated vaporisation rate is \SI{0.0426}{\kilogram\per\square\meter\per\second}

The continuous release of LNG demands the transient simulation. The spill duration is calculated from the spill volume and the spill rate. The duration of LNG spill in Burro 8 test is 107 $s$.

For turbulent modelling, the values of k and $\epsilon$ above the pool can be approximated using the length scale, D and a turbulent intensity $T_i=1-10 \%$.  The length scale, D, is the pool radius in this simulation.
\begin{equation}
k=\frac{3}{2}\left(v_{NG}T_i\right)^2
\nonumber
\end{equation}

\begin{equation}
\epsilon=C_\mu^{3/4}\frac{k^{3/2}}{0.07D}
\nonumber
\end{equation}

Turbulent intensity is the ratio of root-mean-square of the velocity fluctuations and the mean flow velocity. Low turbulent flow has value of turbulence intensity of 1\% or less. The low-turbulence wind tunnels test can have the value of free-stream turbulence intensity as low as 0.05\%. Turbulence intensities greater than 10\% are used in high turbulent flows. A good estimate of the turbulence intensity at the inlet boundary is usually obtained from external, measured data. As this value was not available in the experiment data, the sensitivity analysis should be performed on the value of the turbulent intensity to be ascertained about its effect on the result. Different values of inlet turbulent intensity are used to investigate its effect in the distance to LFL. These values are listed in Table below:

\begin{table}[h!]
	\caption{Setting of turbulence intensities} \label{tab:turb-intens-set}
	\centering
	\begin{tabular}{rrrr}
		\toprule
		& Case 1	& Case 2	& Case 3		\\
		\midrule
		Turbulent intensity (\si{\percent}) 		& 1 	& 3 	& 10 	\\
		\bottomrule
	\end{tabular}
\end{table}

\paragraph{Wall boundary}
In Fluent, two roughness parameters, i.e. roughness height $z_0$ and roughness constant, $C_s$ are used to model the roughness effect. From experiment data, $z_0=\SI{0.0002}{\metre}$. A proper roughness constant is dictated mainly by the type of given roughness. The default roughness constant was retained $C_s=0.5$, which indicating tightly-packed, uniform sand-grain.

Wall roughness value affects drag (resistance), heat and mass transfer on the wall. $f_r$ quantifies the shift of the intercept due to roughness effects. Non-dimensional roughness height
\begin{equation}
U^*=\frac{1}{\kappa}\ln(Ey^*)-\Delta B
\nonumber
\end{equation}

\begin{equation}
\Delta B = \frac{1}{\kappa}\ln(f_r)
\nonumber
\end{equation}

\begin{equation}
K_s^{+} = \frac{\rho K_s u^*}{\mu}
\nonumber
\end{equation}

\begin{equation}
u^* = C_{\mu}^{1/4}k_p^{1/2}
\nonumber
\end{equation}

ANSYS fluent offer the wide range of heat transfer models to simulate to heat addition from the substrate to the vapour cloud. Convective heat transfer boundary condition calculates the heat flux to the wall using external heat transfer coefficient $h_{ext}$ and external heat sink temperature $T_{ext}$ as:
\begin{equation}
q_s = h_{ext}(T_ext-T_w)
\nonumber
\end{equation}

Wall temperature boundary assume a constant temperature of wall, then the heat flux to the wall can be computed in the similar manner to heat conduction as below, $h_f$ is the local heat coefficient, $T_f$ is the fluid temperature.
\begin{equation}
q_s = h_{f}(T_w-T_f)
\nonumber
\end{equation}

Thermal wall boundary condition can be modelled as adiabatic wall, which ignoring the heat transfer effect from the ground. However, the sustainable heat adding to the vapour cloud is from the substrate. Heat flux from the ground should be taken into account. Convective and wall temperature models are used to study the effectiveness of heat addition to the cloud from the ground. There is an analogy between equation (3.6) and reported wall heat transfer equation (3.2), so $h_{ext}$ and $T_{ext}$ are used using reported data. The constant $T_w$ used for wall temperature model is chosen as the ambient air temperature.

\paragraph{Outlet}
The boundary condition is used as “out flow”. It assumes the flow is fully developed, unidirectional. All flow variables are supposed to be constant in this boundary. The placement of this boundary, therefore, is very important. As the placement is so close to the source, significant errors can be made due to the propagate of the source to other boundaries. Otherwise, if the placement is so far, the computational time will increase dramatically.
Top and side boundaries
Top and side boundaries represents the far field boundaries of the flow. However, when there is a change in the inlet condition, the constant pressure in these boundary cannot assure the desired velocity in these boundaries, which should be compatible with the wind profile at the inlet (Luketa-Hanlin et al., 2007).  The symmetry boundary condition, the normal velocity is zero and all others variables are set equal to the flow value. This condition emerged to fit well to the top and side boundaries which can reserve the wind profile to and eliminate the effect of the change of the inlet boundaries.

\paragraph{Numerical Schemes}
\begin{itemize}
	\item First order implicit discretization: VOF equation, time discretization
	\item QUICK: Energy, Momentum equation
	\item PRESTO: Interpolation of pressure
	\item  PISO: solving algorithm
\end{itemize}

\paragraph{Solving linear system}
Preconditioned bi-conjugate gradient method with incomplete-Cholesky preconditioner \textcite{Flores2013a}

\section{Burro series}
\subsection{Velocity prediction}
\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3ExpVelX140Z2}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7ExpVelX140Z2}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8ExpVelX140Z2}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9ExpVelX140Z2}
		\caption{}
	\end{subfigure}
	\caption{Centreline point-wise concentration (a) Burro 3 (b) Burro 7 (c) Burro 8 (d) Burro 9}
	\label{fig:Point-wise velocity}
\end{figure}

\subsection{Concentration prediction}
\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3ConPtX140}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7ConPtX140}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8ConPtX57}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9ConPtX140}
		\caption{}
	\end{subfigure}
	\caption{Centreline point-wise concentration (a) Burro 3 (b) Burro 7 (c) Burro 8 (d) Burro 9}
	\label{fig:Point-wise concentration}
\end{figure}

\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3SimConX140}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3ExpConX140}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7SimConX140}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7ExpConX140}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8SimConX140}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8ExpConX140}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9SimConX140}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9ExpConX140}
	\end{subfigure}
	\caption{Vertical isosurface at X = 140, Left: Sim, Right: Exp, Up to bottom:Burro 3, Burro 7, Burro 8 and Burro 9}
	\label{fig:IsoConX140}
\end{figure}

\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3SimConZ1}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3ExpConZ1}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7SimConZ1}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7ExpConZ1}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8SimConZ1}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8ExpConZ1}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9SimConZ1}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9ExpConZ1}
	\end{subfigure}
	\caption{Horizontal isosurface at Z = 1 (a) Burro 3 (b) Burro 7 (c) Burro 8 (d) Burro 9}
	\label{fig:IsoConZ1}
\end{figure}

\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro3ConMax}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro7ConMax}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro8ConMax}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Burro9ConMax}
		\caption{}
	\end{subfigure}
	\caption{Maximum arc-wise concentration (a) Burro 3 (b) Burro 7 (c) Burro 8 (d) Burro 9}
	\label{fig:ConMax}
\end{figure}

\section{Falcon series}
\subsection{Isosurface contour}
\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon1SimConX150}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon3SimConX150}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon4SimConX150}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line
	\caption{Vertical isosurface at X = 150, Left: Sim, Right: Exp, Up to bottom: Falcon 1, Falcon 3, Falcon 4}
	\label{fig:FalconIsoConX150}
\end{figure}

\subsection{Arcwise prediction}
\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon1ConMax}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon3ConMax}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon4ConMax}
		\caption{}
	\end{subfigure}

	\caption{Maximum arc-wise concentration (a) Falcon 1 (b) Falcon 3 (c) Falcon 4}
	\label{fig:FalconConMax}
\end{figure}

\subsection{Point-wise prediction}
\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon1ConPtX50}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon1ConPtX150}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon3ConPtX50}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon3ConPtX150}
		\caption{}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon4ConPtX50}
		\caption{}
	\end{subfigure}
	~
	\begin{subfigure}[]{0.49\textwidth}
		\includegraphics[width=\textwidth]{Falcon4ConPtX150}
		\caption{}
	\end{subfigure}
	\caption{Centreline point-wise concentration (a) Falcon 1 (c) Falcon 3 (e) Falcon 4}
	\label{fig:FalconPtCon}
\end{figure}

\section{Result and discussion}

\subsection{Grid sensitivity study}
Three meshes which the number of node differs in the factor of two are made for the study of mesh independence. The statistics of these meshes are in Table 8.

Meshes are used for steady state simulation which the only atmospheric inlet (the LNG vapour inlet is excluded). The wind velocity profile and turbulent profile at the inlet is modelled using Monin-Obukhov similarity theory as described in
\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.9\textwidth]{MeshIndepentStudy.png}
	\caption{The gas concentration contour from simulation (left) and experiment(right)} \label{fig:MeshIndepentStudy}
\end{figure}

\subsection{Ground heat transfer sensitivity study}
\paragraph{Distance to LFL and bifurcated structure}: Three different models of heat transfer from the ground are used to study their effect in numerical results, which summarised in Table~\ref{tab:wall-conditon}. The  distance to LFL of case 3 – the wall temperature boundary condition is closest to the validation data.

\begin{table}[h!]
	\caption{Wall thermal boundary condition} \label{tab:wall-conditon}
	\centering
	\begin{tabular}{lr}
		\toprule
		&  Distance to LFL (\si{\meter})\\
		\midrule
		Case 1: Adiabatic wall	 			&405.25	\\
		Case 2: Convective heat transfer 	&372.5	\\
		Case 3:	Wall temperature			&342.7\\
		\midrule
		Validation data						&320\\
		\bottomrule
	\end{tabular}
\end{table}

The isosurfaces of \SI{5}{\percent} concentration (LFL) at \SI{1}{\meter} height, \SI{200}{\second} of all three cases are shown at Figure~\ref{fig:Isosurface-wall-heat}. Figure~\ref{fig:LFLcontourConstTempWall} also shows the bifurcated structure at cloud. The further detail results from this case are presented in the Figure~\ref{fig:ConcentZ1mT200sCompar}. The gas concentration contour is plotted side by side with the contour analysing from experiment data. The bifurcated structure is shown in the simulation but less significant than observed from experiment. Besides, It can be seen that the gas move downwind faster. The high concentration can be found very far from the source than validation data.
\begin{figure}[htbp]
	\centering
	\begin{subfigure}[]{0.7\textwidth}
		\includegraphics[width=\textwidth]{LFLcontourAdiabatic.png}
		\caption{} \label{fig:LFLcontourAdiabatic}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.7\textwidth}
		\includegraphics[width=\textwidth]{LFLcontourConvectiveWall.png}
		\caption{} \label{fig:LFLcontourConvectiveWall}
	\end{subfigure}
	%a blank line to force the subfigure onto a new line

	\begin{subfigure}[]{0.7\textwidth}
		\includegraphics[width=\textwidth]{LFLcontourConstTempWall.png}
		\caption{} \label{fig:LFLcontourConstTempWall}
	\end{subfigure}
	\caption{Isosurfaces of \SI{5}{\percent} concentration (LFL) (a) Adiabatic wall. (b) Convective heat flux. (c) Constant temperature wall.}
	\label{fig:Isosurface-wall-heat}
\end{figure}

\paragraph{Point-wise profiles}
For further understanding the result, the point concentration and temperature data from experiment will be compared to the results. The first point is selected near the source, which is \SI{57}{\meter} downwind and the second point is \SI{140}{\meter} downwind.

Figure~\ref{fig:point-concentration} is the plot of gas concentration at \SI{1}{\meter} elevation at the of \SI{57}{\meter} downwind. It shows the good temporal trend of the simulation to the validation data. The concentration data is fairly matched excepting the peak duration (from 75 s to 100 s). The peak concentration is underestimated four times the experimental data. Figure 29 is the comparison of concentration at the point of 140m downwind. The temporal trend remains good. The concentration magnitude is really match with the validation data.is the temperature profile at 57 m downwind. This also so the gap between predictions and validation data. The difference of lowest temperature is 20 degree. The temperature profile shown in Figure 31 is better matched with validation data.

\subsection{Effect of obstacle}
An impoundment surrounding the source having the size of \SI{60x60x3}{\meter} is used to investigate effectiveness this method in reducing the distance to LFL of vapour cloud. Figure~\ref{fig:impoundment} compares isosurfaces of \SI{5}{\percent} concentration (LFL) at \SI{1}{\meter} height, at the time of \SIlist{50;100;150;200}{\second}. At \SI{200}{\second}, the distance to LFL of the case including impoundment is \SI{207}{\meter}, while the case with no impoundment is \SI{302}{\meter}. An impoundment can reduce the cloud distance to LFL \SI{40}{\percent}.
\begin{figure}[htbp]
	\centering
	\includegraphics[width=0.9\textwidth]{impoundment.png}
	\caption{Dispersion of the cloud with impoundment (left) and no impoundment(right)} \label{fig:impoundment}
\end{figure}

\section{Conclusion}
The good preliminary simulation results show the suitable of general CFD code like FLUENT for modelling physics associating in LNG spills. The atmospheric condition models including the velocity profile, turbulent kinetic energy and turbulent dissipation rate profiles by Monin-Obukhov similarity theory contributed to that successfulness.

Three ground heat transfer assumptions are simulated and compare with the validation data. The wall temperature model which assume isothermal ground (ground’s temperature is constant) gave the closest prediction of distance to LFL. This may be explained as the case had the largest heat flux from the ground to the vapour cloud so it can compensate other source of heat addition to the cloud which is not considered in the simulation.
The detail comparison with point-wise data shows that prediction at near-source point are quite conservative. The assumption of source term contribute major in this error.

The impoundment is shown to reduce the distance to LFL of the cloud to 40\%
It is shown that the preliminary results have some limitations of:

The simulation is overestimate the distance to LFL, which means the effect of ambient velocity to the cloud dispersion is overestimate and the turbulent damping effect within the cloud should pay more attention.

The concentration and temperature peak in near source distance is quite conservative in comparison with validation data. The source modelling should be more well defined for better predictions.


\subsection{NEGF Algorithm in TBtrans}

\begin{frame}[fragile,label=algorithm]
  \frametitle{Input options}
  \framesubtitle{NEGF Algorithm in TBtrans}

  \begin{itemize}
    \item Reduce Green function calculation to region of interest ($D$)

    \item Algorithms determined from user-requested quantities

  \end{itemize}

  \vskip 4em

  \begin{itemize}
    \item Algorithm:
  \end{itemize}
  \begin{center}
    \incg[]{tbt-algorithm}
  \end{center}

  \begin{tikzpicture}[remember picture,overlay]
    \node at ($(current page.center)+(2.5,1.7)$) {%
        \incg[width=.7\linewidth]{inv-block3}%
    };
  \end{tikzpicture}

  \doicite{Papior \etal: \doi{10.1016/j.cpc.2016.09.022}}

  \hfill\hyperlink{electrode<6>}{\beamergotobutton{Return to electrode-setup}}

\end{frame}

\begin{frame}
  \frametitle{Input options}
  \framesubtitle{NEGF Algorithm in TBtrans}

  \begin{itemize}[<+->]
    \item Define $D$ via input

    \begin{tikzpicture}[thick,fixed node]

      % Define the electrode and we are done
      \begin{scope}
        \matrix {
            \node[fdf] {\%block TBT.Atoms.Device}; \\
            \node[fdf,ind] {atom [ 1 -- 3 ]}; \\
            \node[fdf,ind] {atom 5 11}; \\
            \node[fdf,ind] {atom -19}; \\
            \node[fdf] {\%endblock}; \\
        };
      \end{scope}
      \begin{scope}[xshift=8cm,yshift=1cm]
        \node[anchor=north,fill=black!70!white,text=white,align=center,text width=4.5cm] (A)
        at (0,0) {Always!\\\mbox{}\\\texttt{tbtrans -fdf TBT.Analyze}};
      \end{scope}
    \end{tikzpicture}

    \item TBtrans does everything else by it-self
    \item A smaller device region drastically reduces required memory(!), and increases
    throughput

    \item In non-orthogonal basis ($\SO\neq \mathbf I$) \fdf{TBT.Atoms.Device.Connect} may
    be useful

  \end{itemize}

  \incg[]{tbt-algorithm}

  \doicite{Papior \etal: \doi{10.1016/j.cpc.2016.09.022}}

\end{frame}