%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% script_monad_combinators.tex
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The script monad is the runtime core of our DSL. A DSL can be augmented by defining a series of operators that automate or simplify common operations for the DSL developers. The best set of operators for a scripting DSL is strongly dependent upon the kind of game to be scripted. In this section we describe a general-purpose set of operators that make up a basic calculus of coroutines, but we would expect that other game developers would define additional operators that are a tighter fit to their games. The operators of our calculus of coroutines take as input one or more coroutines and return as output a new coroutine:

\begin{itemize}
\item \texttt{parallel} ($s_1 \wedge s_2$) executes two scripts in parallel and returns both results
\item \texttt{concurrent} ($s_1 \vee s_2$) executes two scripts concurrently and returns the result of the first to terminate
\item \texttt{guard} ($s_1 \Rightarrow s_2$) executes and returns the result of a script only when another script evaluates to \texttt{true}
\item \texttt{repeat} ($\uparrow s$) keeps executing a script over and over
\item \texttt{atomic} ($\downarrow s$) forces a script to run in a single tick of the \textit{discrete simulation engine}
\end{itemize}

We show here the implementation of one of these combinators with our monadic system. To see the other implementations, see \cite{FRIENDLY_FSHARP}.

\begin{lstlisting}
let rec parallel_ (s1:Script<'a>) (s2:Script<'b>)
          : Script<'a * 'b> =
  fun s ->
    match s1 s,s2 s with
    | Done x, Done y    -> Done (x,y)
    | Next k1, Next k2  -> parallel_ k1 k2
    | Next k1, Done y   -> parallel_ k1 (fun s -> Done y)
    | Done x, Next k2   -> parallel_ (fun s -> Done x) k2
\end{lstlisting}

We can now give another self-contained example that shows a producer and a consumer running in parallel. We start by defining the state as a single memory location, which can either be empty (\texttt{None}) or contain a value (\texttt{Some x}):

\begin{lstlisting}
type Buffer = { mutable Contents : Option<int> }
let buffer = { Contents = None }
\end{lstlisting}

We define some additional helper functions to access the state. A good engineering rule is that the more complex is the state, the less coroutines directly use the \texttt{get\_state} function; rather, when the state is complex, we define a series of additional accessor functions that help us manipulating the various aspects of the state:

\begin{lstlisting}
let set_buffer v : Script<Unit,Buffer> =
  script{
    let! s = get_state
    s.Contents <- Some v }
let is_buffer_empty : Script<bool,Buffer> =
  script{
    let! s = get_state
    return s.Contents = None }
\end{lstlisting}

We define the producer (the consumer is symmetric) as follows; notice that each access to the state automatically suspends the coroutine, so we do not need to explicitly invoke \texttt{suspend}:

\begin{lstlisting}
let rec wait_buffer_empty:Script<Unit,Buffer> =
  script{
    let! c = is_buffer_empty
    if c = false then
      do! suspend
      do! wait_buffer_empty }

let producer =
  let rec producer i : Script<Unit,Buffer> =
    script{
      do! wait_buffer_empty
      do! set_buffer i
      do! producer (i+1) }
\end{lstlisting}

The main loop is very similar to the main loop seen for the Fibonacci sample above; the only difference is that we invoke the producer and the consumer coroutines in parallel, by passing to the inner \texttt{main\_loop} function the value \texttt{parallel\_ producer consumer}.

\subsection{Scripting in Games}

We now discuss the introduction of our scripting system in games. In the following we outline how we have built most of the game logic of the upcoming RTS game Galaxy Wars (which is also released with a fully open source \cite{GALAXY_WARS}). In the game we are considering the players compete to conquer a series of \textit{star systems} by sending fleets to reinforce their systems or to conquer the opponent's.

\subsubsection{Game Patterns}

Thanks to our general combinators we can define a small set of recurring game patterns; by instantiating these game patterns one can build the actual game scripts with great ease. These patterns may be adapted for the specific domain of a game, or altogether new patterns may be created that better fit one's game.
The first game pattern we see is the most general, and for this reason it is called \texttt{game\_pattern}. This pattern initializes the game in a single tick, then performs a game logic (while the game is not over) and finally it performs the ending operation before returning some result. The initialization is performed by the \texttt{init} script, which returns a result of a generic type \texttt{'a}; this result is the state of the script, and contains data that may be helpful for tracking additional information that is useful to our scripts but which is not stored in the game state. The logic of the various game entities, such as their AI, is then performed, repeatedly, by the \texttt{logic} script. While the logic script is run, the \texttt{game\_over} script continuously checks to see if the game has been won or lost and thus must be terminated; when the termination condition is met, the \texttt{ending} script is invoked that may show some recap of the game that has just ended. The game pattern is implemented as follows:

\begin{lstlisting}
let game_pattern
      (init:Script<'a>)
      (game_over:'a -> Script<'bool>)
      (logic:'a -> Script<Unit>)
      (ending:'a -> Script<'c>)
      : Script<'c> =
  script{
    let! x = init |> atomic_
    let! (Left y) =
       concurrent_
         (guard_ (game_over x) (ending x))
         (logic x |> repeat_)
    return y }
\end{lstlisting}

Notice that with the introduction of appropriate operators we may remove many of the parentheses of the above sample if they are seen as a hindrance to readability. The main portion of the code above may then be rewritten as:

\begin{lstlisting}
(game_over x => ending x) .|| (logic x |> repeat_)
\end{lstlisting}

The game pattern above is very general, but not all scripts always need all of its parameters. We can build less general game patterns by reducing the number of parameters; for example, we may build a game pattern that has no initialization, logic or ending sequence; such a game pattern would implement the case of a game script whose sole responsibility is to check the termination conditions for a game (those that trigger the ``game over'' screen):

\begin{lstlisting}
let wait_game_over (game_over:Script<bool>) : Script<Unit> =
  let null_script = script{ return () }
  game_pattern
    null_script
    (fun () -> game_over)
    (fun () -> null_script)
    (fun () -> null_script)
\end{lstlisting}

Writing a script with our system will consist of instantiating one game pattern with specialized scripts as its parameters; these scripts will alternate accesses to the specific state of the game with invocations of combinators from the calculus seen above. In the next session we will see an example of this.


%% This file is part of the DiffractionMicroscopy project
%% Copyright 2015, 2016 David W. Hogg

% To-Do list
% ----------
% - title is a mess
% - need to discuss at length the issues of identifiability
% - discussion point: Could distinguish two similar models
% - perhaps re-phrase everything in terms of INVERSE variance?
% - possible confusions between the variance of the $k$-space Gaussian and the inverse variance of the $x$-space Gaussian!
% - confusion between the Gaussian describing the FT and its SQUARE.  I think the text may be wrong as it stands.
% - include divergences figure / results
% - add inferred variance tensor to figure captions
% - read and cite literature

% Style notes
% -----------

\documentclass[12pt]{article}
\usepackage{amsmath, amssymb, mathrsfs, hyperref, graphicx}
\input{vc}

% useless formatting
\linespread{1.08333} % 10/13 spacing
\setlength{\parindent}{2\baselineskip}\addtolength{\parindent}{-1.25ex}
\setlength{\parskip}{0ex}
\newcommand{\hoggcaption}[1]{\caption{\textsf{#1}}}
\renewcommand{\thefigure}{\textsf{\arabic{figure}}}
\renewcommand{\figurename}{\textsf{Figure}}
\newcommand{\equationname}{equation}

% text definitions
\newcommand{\doi}[1]{{\footnotesize \url{http://doi.org/#1}}}

% math definitions
\newcommand{\normal}{\mathscr{N}}
\newcommand{\Poisson}{\mathscr{P}}
\newcommand{\sqnorm}[1]{|{#1}|^2}
\newcommand{\unitvec}[1]{\hat{#1}}
\newcommand{\ehat}{\unitvec{e}}
\newcommand{\xhat}{\unitvec{x}}
\newcommand{\yhat}{\unitvec{y}}
\newcommand{\zhat}{\unitvec{z}}
\newcommand{\transpose}{^{\mathsf{T}}}
\newcommand{\given}{\,|\,}
\newcommand{\setof}[1]{\left\{{#1}\right\}}
\newcommand{\dd}{\mathrm{d}}
\DeclareMathOperator{\trace}{tr}

\begin{document}\sloppy\sloppypar

\section*{\raggedright%
  Reconstruction of three-dimensional diffraction imaging
  from noisy two-dimensional images taken at unknown orientations: The Gaussian case}
\noindent
David W. Hogg%
\footnote{Simons Center for Data Analysis, Simons Foundation}%
\footnote{Center for Cosmology and Particle Physics, Department of Physics, New York University}%
\footnote{Center for Data Science, New York University}%
\footnote{david.hogg@nyu.edu}

\bigskip

\paragraph{Abstract:}
In certain kinds of diffraction-microscopy problems, the sample being
imaged is only in the x-ray beam for a short time, and at random, unknown
orientation (Euler angles), leading to a noisy image of one 2-d slice
of the squared norm of the full 3-d Fourier transform.
In principle, many of these noisy slice images can be combined to
reconstruct the full 3-d function.
However, the fact that the orientations are unknown makes this
difficult.
Here we present a fully probabilistic approach to this problem, which
reconstructs the full 3-d function, marginalizing out the Euler
angles.
We show that, in the highly restricted case of a precisely Gaussian molecule,
the reconstruction can be performed, even in the absurd limit in which
there is only \emph{one photon detected in each image}.
That is, the reconstructon is possible even as the information content per
image approaches zero.
There are, however, requirements on the total information content in
the data set, and on the true distribution of Euler angles.
The problem setup is artificially simple, but nothing fundamental
about the method depends on the assumption of a Gaussian molecule; in
principle this method can work for molecules of arbitrary complexity.

\section{Introducion}

DWH: This problem, apparently, is an important problem.  State it clearly
and cite some literature (see references below).

DWH: State the basic reasoning behind the approach and why,
counter-intuitively, it should work.

DWH: Point out that although this case is extremely boring and
unrealistic, there is nothing in prinicple that makes it different
from much more complex cases.

\section{Problem statement}

In a three-dimensional (3-d) space there exists a 3-d multivariate
Gaussian-shaped density blob $f(x)$.
The Fourier transform $F(k)$ in 3-d reciprocal space $k$ will
also be a Gaussian-shaped density blob.
\begin{eqnarray}
  F(k) &=& a\,\normal(k\given 0, V)
  \quad ,
\end{eqnarray}
where $a$ is some kind of amplitude and $\normal(k\given\mu, V)$ is a
Gaussian pdf for $k$ with 3-vector mean $\mu$ and $3\times 3$ variance
tensor $V$.
Without loss of generality we are going to presume a variance tensor
$V$ with principal axes aligned with the coordinate axes $\ehat_1$,
$\ehat_2$, and $\ehat_3$ directions:
\begin{eqnarray}
  V &=& \sum_{d=1}^3 \sigma^2_d \, \ehat_d\cdot\ehat_d\transpose
  \quad ,
\end{eqnarray}
where $V$ is a symmetric $3\times3$ tensor, the vectors are column
orthonormal unit vectors (so the vector products
$\ehat_d\cdot\ehat_d\transpose$ are outer products), the $\sigma^2_d$ are
scalar eigenvalues (principal variances).
Still without loss of generality we can assume that the corresponding
eigenvalues of the tensor are ordered
\begin{eqnarray}
  \sigma^2_1 \geq \sigma^2_2 \geq \sigma^2_3 > 0
  \quad .
\end{eqnarray}
Just as a reminder, the eigenvalues $\sigma^2_d$ have dimensions of
inverse length squared, since they are $k$-space variances.

We don't get to observe this Gaussian blob or its variance tensor
directly.
Instead, we get $N$ two-dimensional (2-d) images $y_n$, each of which
contains a noisy 2-d slice of the 3-d Gaussian blob, through the origin
but along an unknown slice plane.
In the small-angle limit of diffraction imaging, this slice is along a
plane perpendicular to the illumination direction $\zhat$.
In full generality, this slice is along a Ewald sphere that kisses
that perpendicular plane at the origin.
Let's start in the small-angle limit, to keep things simple.

In detail, when we take the small-angle limit, we can think of each
image $y_n$ as being an array of $M$ pixels at
2-vector coordinates $\xi_m$ in the slice plane.
Each image pixel $m$ contains a number of photons that can be thought
of as a Poisson draw with expectation set by the evaluation of the
squared norm $\sqnorm{F(k)}$ of the three-dimensional Gaussian
blob, restricted to the $\zhat$ slice, arbitrarily rotated (in 2-d).
(Actually, in detail, each pixel has expectation set by an
\emph{integral} of the squared norm of the blob, over the surface of
the pixel; we will ignore this integral for now, and just evaluate.)
None of the projection directions $\zhat_n$ or 2-d rotations are
known; only the noisy images are given.

Amusingly, in the limit in which the image is filled densely with
tiny pixels, each pixel will (with arbitrarily high probability) be
either empty or contain exactly one photon.
In this limit, each image $y_n$ can be thought of as simply a photon
list, or a set $\setof{\xi_{nk}}_{k=1}^{K_n}$ of $K_n$ photons $k$,
each with position $\xi_{nk}$.

The variance tensor of the 2-d Gaussian blob appearing in image $y_n$
is given by a slice and rotation of the 3-d inverse-variance tensor:
\begin{eqnarray}
  V_n &=& [P_n\transpose\cdot V^{-1}\cdot P_n]^{-1}
  \quad ,
\end{eqnarray}
where $P_n$ is a $3\times 2$ rectangular projection matrix, that
performs the slice and rotation.
Each rectangular projection matrix $P_n$ has three degrees of freedom or three
parameters (in some sense), which can be thought of as the Euler
angles, which are two angles on the sphere to define the direction of
$\zhat_n$ and one more angle to define the direction of the x-axis of
the image in the projection plane.

In detail, if one has the orthonormal unit 3-vectors
$\xhat_n,\yhat_n,\zhat_n$ that define the x-direction on the image
slice plane (but in the 3-d $k$-space), the y-direction on the image
slice plane, and the normal to the image slice plane, the $3\times 2$
projection matrix $P_n$ becomes
\begin{eqnarray}
  P_n &\equiv& [\xhat_n\,,\,\yhat_n]
  \quad ,
\end{eqnarray}
which (because $\xhat_n,\yhat_n$ are 3-vectors) is a $3\times2$ matrix.
This object $P_n$ might look like it has six parameters, but it is
constructed from two orthonormal vectors, so it really only has three
degrees of freedom, which correspond to the Euler angles.
Some example artificial data, made from random slices of an
anisotropic 3-d Gaussian, are shown in
\figurename~\ref{fig:exampledata8}, \ref{fig:exampledata4}, and \ref{fig:exampledata0}.
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{data_08_08.png}
\hoggcaption{Some example artificial data, from a data set made with
  $N=256$ images, each with $K=256$ photons, random Euler angles, and
  an anisotropic 3-d Gaussian with axis-aligned variances
  $\setof{\sigma^2_d}_{d=1}^3 = (47, 13, 11)$.\label{fig:exampledata8}}
\end{figure}
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{data_12_04.png}
\hoggcaption{Same as \figurename~\ref{fig:exampledata8} but from a data set made with
  $N=4096$ images, each with $K=16$ photons.\label{fig:exampledata4}}
\end{figure}
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{data_16_00.png}
\hoggcaption{Same as \figurename~\ref{fig:exampledata8} but from a data set made with
  $N=65536$ images, each with $K=1$ photon!\label{fig:exampledata0}}
\end{figure}

The question is:
Given these data, \emph{can we infer the 3-d variance tensor $V$?}
And if we can, how accurately, and what does it depend on?  We are
going to attempt the inference by maximizing a marginalized
likelihood, marginalizing out all $3\,N$ unknown projection and
rotation parameters.

\section{Marginalized likelihood}

Our single-pixel likelihood will be based on the Poisson process:
\begin{eqnarray}
  p(y_{nm}\given P_n,V) &=& \Poisson(y_{nm}\given\lambda_{nm})
  \\
  \lambda_{nm} &\equiv& \sqnorm{a\,\normal(\xi_m\given 0,V_n)}\,\Delta_m
  \quad,
\end{eqnarray}
where $y_{nm}$ is the $m$th pixel value in image $y_n$,
$\lambda_{nm}$ is the evaluation of the squared norm of the Fourier
transform at the pixel center (just evaluation, not integral),
$\Delta_m$ is the effective $k$-space area of the $m$th pixel,
and $\Poisson(k\given\lambda)$ is the Poisson probability for obtaining
count $k$ given expectation $\lambda$.
The likelihood is a function of the projections $P_n$ and the
three-space tensor $V$, through their combination $V_n$.

If we drop (don't infer) normalizations, and presume that all pixels
have the same effective area $\Delta_m$ in $k$-space, then this leads
to a log-likelihood
\begin{eqnarray}
  \ln p(y_n\given P_n,V) &=& \sum_{m=1}^M \ln p(y_{nm}\given P_n,V)
  \\
  \ln p(y_{nm}\given P_n,V) &=& y_{nm}\,\ln\sqnorm{\normal(\xi_m\given 0,V_n)} + \mbox{const}
  \\
  \ln p(y_{nm}\given P_n,V) &=& y_{nm}\,\ln\normal(\xi_m\given 0,\frac{1}{2}\,V_n) + \mbox{const}
  \quad,
\end{eqnarray}
where we are ignoring constants, and where we have used that the square
of a Gaussian pdf is proportional to a Gaussian pdf with half the variance.
In the dense-image-of-small-pixels limit, this can be re-written as a
sum over photons:
\begin{eqnarray}
  \ln p(y_n\given P_n,V) &=& \sum_{k=1}^{K_n} \ln p(y_{nk}\given P_n,V)
  \\
  \ln p(y_{nk}\given P_n,V) &=& \ln\normal(\xi_{nk}\given 0,\frac{1}{2}\,V_n) + \mbox{const}
  \quad.
\end{eqnarray}

Now what we really want is to marginalize out the projections and
offsets.
We are going to think of the rectangular projection matrix $P_n$ as
being controlled by a set of Euler angles $\phi$ and we are going to
marginalize these out, using an isotropic prior.
\begin{eqnarray}
  \ln p(y_n\given V) &=& \ln\int p(y_n\given P_{\phi},V)\,p(\phi)\,\dd\phi
  \quad,
\end{eqnarray}
where now we subscript $P_{\phi}$ with the Euler angles and integrate
over them.
This marginalized likelihood has the horrifying form of a log of an
integral of an exponential of a simple log-likelihood.  That's bad.

We make things little better by replacing this integral with a
sampling approximation.
If we have $T$ samples $P_t$ of rectangular projection matrices, drawn
from an isotropic distribution of Euler angles $\phi$, the
marginalized likelihood can be approximated by
\begin{eqnarray}
  \ln p(y_n\given V) &=& \ln\left[\frac{1}{T}\,\sum_{t=1}^T p(y_n\given P_t,V)\right]
  \quad,
\end{eqnarray}
where now we have a log-sum-exp form (since it is $\ln p(y_n\given
P,V)$ that has the simple form).

DWH: Expression for the derivatives?

DWH: Implementation notes and details?

DWH: How dense does the sampling have to be?  One answer is that it
has to be dense enough that the inferences don't change as the density
function is rotated in $k$-space.  For our purposes, it means we
should get the same marginalized likelihood (to a precision that is
$\ll 1$) when we re-order the principal axes of the body.  DWH: FIGURE HERE.

DWH: Also concept of effective number of samples?

\section{Experiments}

\paragraph{Performance as a function of image and photon counts:}
DWH: Hello World!

DWH: As a performance metric, we will use the symmetrized K--L
divergence $S$, which in the case of 3-d, zero-mean, multivariate
Gaussians, is:
\begin{eqnarray}
  S &\equiv& \frac{1}{2}\,\left[\trace(V_1^{-1}\cdot V_2)
                              + \trace(V_2^{-1}\cdot V_1)
                              - 6\right]
  \label{eq:div}\quad,
\end{eqnarray}
where $V_1$ and $V_2$ are the variance tensors of the two Gaussians
being compared.
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{divergences.png}
\caption{The divergences---as defined in
  \equationname~(\ref{eq:div})---between the True Gaussian and the
  best-fit Gaussian for a set of different trials.  In each case, the
  thin lines are individual trials, and the thick lines are the median
  across 32 trials at the same number of images $N$ and number of
  photons per image $K$.  The short lines are trials that have the
  product $N\,K=256$~photons, the medium-length lines are trials that
  have $N\,K=4096$~photons, and the long lines are trials that have
  the product $N\,K=65536$~photons.  The thin grey lines show medians
  for the three different $N\,K$ values.  The take-home messages are:
  More photons is better, but it isn't far worse to have $K=1$ photon
  per image than 64 or 256, so long as the total number $N\,K$ of
  photons is large.\label{fig:divs}}
\end{figure}

\paragraph{Information content in low-photon images:}
DWH: Hello World!

DWH: Prediction: The effective number of samples gets smaller as the
number of photons per image increases.  The need for denser
$\phi$-space sampling increases as the number of photons per image
increases.
DWH: Prediction: The only $K=1$ images that have significant angle
information are images with photons at anomalously large radius or
anomalously small radius.
DWH: These predictions are confirmed in
\figurename~\ref{fig:sampling8}, \ref{fig:sampling4}, and \ref{fig:sampling0},
which show the posterior beliefs about individual-image 2-d variance
tensors, given a setting for the 3-d variance tensor.
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{sampling_08_08.png}
\hoggcaption{The same subset of artificial data as shown in
  \figurename~\ref{fig:exampledata8},
  but now with a posterior sampling of 2-d variance tensors overplotted in red.
  The posterior sampling is drawn with the 3-d variance tensor set to the
  maximum-marginalized-likelihood value for the full data set,
  which in this case is
  $N=256$ images, each with $K=256$ photons, random Euler angles, and
  an anisotropic 3-d Gaussian with axis-aligned variances
  $\setof{\sigma^2_d}_{d=1}^3 = (47, 13, 11)$.
  The variance tensors are depicted by two-sigma contour ellipses.\label{fig:sampling8}}
\end{figure}
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{sampling_12_04.png}
\hoggcaption{Same as \figurename~\ref{fig:sampling8} but for the data shown in
  \figurename~\ref{fig:exampledata4}
  ($N=4096$ images, each with $K=16$ photons).\label{fig:sampling4}}
\end{figure}
\begin{figure}[!tbp]
\includegraphics[width=\textwidth]{sampling_16_00.png}
\hoggcaption{Same as \figurename~\ref{fig:sampling8} but for the data shown in
  \figurename~\ref{fig:exampledata0}
  ($N=65536$ images, each with $K=1$ photon).\label{fig:sampling0}}
\end{figure}

\paragraph{Effects of wrong orientation priors:}
DWH: Hello World!

DWH: Prediction: A wrong orientation prior will hurt more as the
number of photons per image drops.

\section{Discussion}

DWH: Hello World!

DWH: Didn't deal with censoring; how would things change if I did?

DWH: Make the Jeremy Magland argument that you could have done this
restricted case with some clever empirical statistics!  Hence: No
mystery.  The mystery is that the arguments for this approach make no
assumptions about the form or complexity of the density.

\bigskip

It is a pleasure to thank
  Dan Foreman-Mackey (UW),
  Leslie Greengard (SCDA),
  Jeremy Magland (SCDA),
  and
  Marina Spivak (SCDA)
for valuable discussions.
This work was partially supported by [DWH GRANT NUMBERS].

Everything in this project, including this document itself, is
available at \giturl.  If you want this exact version, clone at git
hash \texttt{\githash\,(\gitdate)}.

\section*{Possibly relevant literature}\raggedright
\begin{trivlist}
\item
Ayyer,~K., Philipp,~H.~T., Tate,~M.~W., Elser,~V., Gruner,~S.~M., 2014,
``Real-Space x-ray tomographic reconstruction of randomly oriented objects with sparse data frames'',
Opt.\ Express 22, 2403--2413
\doi{10.1364/OE.22.002403}.
\item
Bortel,~G., Faigel,~G., Tegze,~M., 2009,
``Classification and averaging of random orientation single macromolecular diffraction patterns at atomic resolution'',
J Struct Biol 166(2) 226--233.
\item
Huldt,~G., Szoke,~A., Hajdu,~J., 2003,
``Diffraction imaging of single particles and biomolecules'',
J Struct Biol 144(1--2) 219--227.
\item
Martin,~A.~V., 2014,
``The correlation of single-particle diffraction patterns as a continuous function of particle orientation'',
Philos Trans R Soc Lond B Biol Sci 369(1647) 20130329
\doi{10.1098/rstb.2013.0329}.
\item
Tegze,~M., Bortel,~G., 2012,
``Atomic structure of a single large biomolecule from diffraction patterns of random orientations'',
J Struct Biol 179(1) 41--45
\doi{10.1016/j.jsb.2012.04.014}.
\end{trivlist}

\end{document}


\documentclass[a4paper, 12pt]{article}
\usepackage[utf8]{inputenc}
\usepackage{url}
\usepackage{amsmath, amssymb, amsfonts}
\usepackage{graphicx}

\begin{document}
\begin{center}
{\LARGE \bf Praktikum Wissenschaftliches Rechnen \\ \vspace{0.2cm} (CFD, Final Project)}\\
\vspace{0.6cm}
{\small Group 9: Breznik, E., Cheng, Z., Ni, W., Schmidbartl, N.}
\end{center}
\normalfont
\section{The Topic}\label{sec:topic}
For the final project, our group chose to extend the Navier Stokes code to be used in 3D for arbitrary geometries, with the ability of handling
the Free Surface flow scenarios.
We put a lot of attention to designing a code that will work in 3D for as many different scenarios as possible. That is why we did not code a
special set of boundaries for just a few more known and frequently simulated situations, but rather made them completely dependent on the input
geometry file.

%TODO:comment on not doing free surface
\subsection{Running the program}
Program is called in the same way as its 2D version;
\begin{center}
\begin{verbatim}
./sim parameter_file
\end{verbatim}
\end{center}
with parameter\_file denoting the usual (only slightly extended) parameter file, where (under variable name $problemGeometry$) also the geometry file is
given - see next section for more information on input.

\subsection{Input}\label{sec:input}
The parameter file that was used as input in our previous worksheets remained, with just slight changes due to the additional dimension.
Only notable difference is, that we now take as an input also a scalar $velIN$, and a vector $velMW$. First one represent the velocity at the
inflow boundaries, and the second one the wall velocity for the moving wall boundaries. %TODO: when describin these b.c. say more about the velocity, what it represents

To make our code able to handle truly arbitrary scenarios, we designed so that it allows for any sort of implemented boundary conditions to be employed in any
domain cell - so even the obstacles inside the domain can have arbitrary boundaries, as opposed to only allowing that on the domain walls (as in worksheet 3).
The standard boundary conditions, that we implemented, are
\begin{itemize}
\item no slip,
\item free slip,
\item inflow,
\item outflow,
\item moving wall.
\end{itemize}
To specify, where which of them is applied, we used special numbering of cells when generating our input pgm files, which can be seen from table
1, so our geometries are represented by a grayscale image with 7 levels of brightness.
\begin{table}[hb!]
\centering
\label{tab1}
\begin{tabular}{|l|c|}
\hline
{\bf Cell type} & {\bf Number code}\\
\hline
water & 0 \\
air & 1 \\
no-slip & 2 \\
free-slip & 3 \\
inflow & 4 \\
outflow & 5 \\
moving wall & 6 \\
\hline
\end{tabular}
\caption{Number representations of different possible cell types.}
\end{table}
Since we are working in three dimensions, this image consists of a sequence of 2D ($xy$ plane) images. The orientation of coordinate system and an
example picture for a small lid driven cavity case can be seen on figure 1. One geometry picture set thus needs to contain
$(imax+2)\times(jmax+2)\times(kmax+2)$ numbers in $(jmax+2)\times(kmax+2)$ rows, and is typically very big.
\begin{figure}[ht!]
\centering
\label{fig1}
\includegraphics[height=6.2cm]{coord.jpg}
\includegraphics[height=6.2cm]{cavity.png}
\includegraphics[height=6cm]{cavity20202.jpg}
\caption{The structure of our input pgm files and a lid driven cavity example case.}
\end{figure}

\section{Implementation}\label{sec:implement}
\subsection{Boundary conditions}\label{sec:boundary_vel}
As mentioned before, we have implemented 5 different boundary conditions for the 6 boundaries of our 3D domain, as well as for internal boundaries.
\subsubsection{No-Slip boundary condition}\label{sec:no-slip}
For no-slip conditions, the fluid vanishes at the boundary. As a consequence, both the velocity components are zero.
%Using staggered grid, the discrete velocity components normal to and parallel to the boundary lie directly at the boundary, whilst the one parallel to the boundary is the average of the boundary cell's and its neighbouring fluid cell's velocity components.

%Therefore, for the \texttt{B\_O} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{lll}
%u_{i,j,k} = 0, & v_{i,j-1,k} = -v_{i+1,j-1,k}, & v_{i,j,k} = -v_{i+1,j,k}, \\
%w_{i,j,k-1} = -w_{i+1,j,k-1}, & w_{i,j,k} = -w_{i+1,j,k},& \\
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 5 cases: \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}.
%
%For the \texttt{B\_NO} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{ll}
%u_{i,j,k} = 0, & u_{i-1,j,k} = -u_{i-1,j+1,k}, \\
%v_{i,j,k} = 0, & v_{i,j-1,k} = -v_{i+1,j-1,k}, \\
%w_{i,j,k} = -\frac{1}{2}(w_{i+1,j,k}+w_{i,j+1,k}), &
%w_{i,j,k-1} = -\frac{1}{2}(w_{i+1,j,k-1}+w_{i,j+1,k-1}), \\
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 11 cases: \texttt{B\_NW}, \texttt{B\_NU}, \texttt{B\_ND}, \texttt{B\_SO}, \texttt{B\_SW}, \texttt{B\_SU}, \texttt{B\_SD}, \texttt{B\_OU}, \texttt{B\_WU}, \texttt{B\_OD} and \texttt{B\_WD}.

For the \texttt{B\_NOU} case, we set the following boundary conditions:
\begin{equation}
\begin{array}{ll}
u_{i,j,k} = 0, & u_{i-1,j,k} = -\frac{1}{2}(u_{i-1,j+1,k}+u_{i-1,j,k+1}), \\
v_{i,j,k} = 0, & v_{i,j-1,k} = -\frac{1}{2}(v_{i+1,j-1,k}+v_{i+1,j-1,k+1}), \\
w_{i,j,k} = 0, & w_{i,j,k-1} = -\frac{1}{2}(w_{i+1,j,k-1}+w_{i,j+1,k-1}), \\
\end{array}
\end{equation}
and we analogously set the boundary conditions for the following 7 cases: \texttt{B\_NWU}, \texttt{B\_NOD}, \texttt{B\_NWD}, \texttt{B\_SOU}, \texttt{B\_SWU}, \texttt{B\_SOD}, and \texttt{B\_SWD}.

Boundary conditions of the cells that only have one or two fluid neighbours are analogous to their 2D counterparts, and are omitted here for the sake of brevity;
same applies for the discussion in  \ref{sec:free-slip}, \ref{sec:outflow}, and \ref{sec:boundary_pressure}.

%\texttt{B\_O}, \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U}, \texttt{B\_D}; \texttt{B\_NO} \texttt{B\_NW}, \texttt{B\_NU}, \texttt{B\_ND}, \texttt{B\_SO}, \texttt{B\_SW}, \texttt{B\_SU}, \texttt{B\_SD}, \texttt{B\_OU}, \texttt{B\_WU}, \texttt{B\_OD} and \texttt{B\_WD}.

\subsubsection{Free-Slip boundary condition}\label{sec:free-slip}
For free-slip conditions, the fluid flows freely parallel to the boundary, but does not cross the boundary. As a consequence, the velocity component normal to the boundary is zero, as well as the normal derivative of the velocity component parallel to the wall.
%Using staggered grid, the discrete velocity components normal to the boundary lie directly at the boundary, whilst the one parallel to the boundary is the average of the boundary cell's and its neighbouring fluid cell's velocity components.
%
%Therefore, for the \texttt{B\_O} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{lll}
%u_{i,j,k} = 0, & v_{i,j-1,k} = v_{i+1,j-1,k}, & v_{i,j,k} = v_{i+1,j,k}, \\
%w_{i,j,k-1} = w_{i+1,j,k-1}, & w_{i,j,k} = w_{i+1,j,k},& \\
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 5 cases: \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}.
%
%For the \texttt{B\_NO} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{ll}
%u_{i,j,k} = 0, & u_{i-1,j,k} = u_{i-1,j+1,k}, \\
%v_{i,j,k} = 0, & v_{i,j-1,k} = v_{i+1,j-1,k}, \\
%w_{i,j,k} = \frac{1}{2}(w_{i+1,j,k}+w_{i,j+1,k}), &
%w_{i,j,k-1} = \frac{1}{2}(w_{i+1,j,k-1}+w_{i,j+1,k-1}), \\
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 11 cases: \texttt{B\_NW}, \texttt{B\_NU}, \texttt{B\_ND}, \texttt{B\_SO}, \texttt{B\_SW}, \texttt{B\_SU}, \texttt{B\_SD}, \texttt{B\_OU}, \texttt{B\_WU}, \texttt{B\_OD} and \texttt{B\_WD}.

For the \texttt{B\_NOU} case, we set the following boundary conditions:
\begin{equation}
\begin{array}{ll}
u_{i,j,k} = 0, & u_{i-1,j,k} = \frac{1}{2}(u_{i-1,j+1,k}+u_{i-1,j,k+1}), \\
v_{i,j,k} = 0, & v_{i,j-1,k} = \frac{1}{2}(v_{i+1,j-1,k}+v_{i+1,j-1,k+1}), \\
w_{i,j,k} = 0, & w_{i,j,k-1} = \frac{1}{2}(w_{i+1,j,k-1}+w_{i,j+1,k-1}), \\
\end{array}
\end{equation}
and we analogously set the boundary conditions for the following 7 cases: \texttt{B\_NWU}, \texttt{B\_NOD}, \texttt{B\_NWD}, \texttt{B\_SOU}, \texttt{B\_SWU}, \texttt{B\_SOD}, and \texttt{B\_SWD}.

\subsubsection{Outflow boundary condition}\label{sec:outflow}
For outflow conditions, the normal derivatives of both the velocity components are zero.
%Using staggered grid, the discrete velocity component normal to the boundary lie directly at the boundary, whilst the one parallel to the boundary is the average of the boundary cell's and its neighbouring fluid cell's velocity components.
%
%Therefore, for the \texttt{B\_O} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{lll}
%u_{i,j,k} = u_{i+1,j,k}, & v_{i,j-1,k} = v_{i+1,j-1,k}, & v_{i,j,k} = v_{i+1,j,k}, \\
%w_{i,j,k-1} = w_{i+1,j,k-1}, & w_{i,j,k} = w_{i+1,j,k},& \\
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 5 cases: \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}.
%
%For the \texttt{B\_NO} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{ll}
%u_{i,j,k} = u_{i+1,j,k}, & u_{i-1,j,k} = u_{i-1,j+1,k}, \\
%v_{i,j,k} = v_{i,j+1,k}, & v_{i,j-1,k} = v_{i+1,j-1,k}, \\
%w_{i,j,k} = \frac{1}{2}(w_{i+1,j,k}+w_{i,j+1,k}), &
%w_{i,j,k-1} = \frac{1}{2}(w_{i+1,j,k-1}+w_{i,j+1,k-1}), \\
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 11 cases: \texttt{B\_NW}, \texttt{B\_NU}, \texttt{B\_ND}, \texttt{B\_SO}, \texttt{B\_SW}, \texttt{B\_SU}, \texttt{B\_SD}, \texttt{B\_OU}, \texttt{B\_WU}, \texttt{B\_OD} and \texttt{B\_WD}.

For the \texttt{B\_NOU} case, we set the following boundary conditions:
\begin{equation}
\begin{array}{ll}
u_{i,j,k} = u_{i+1,j,k}, & u_{i-1,j,k} = \frac{1}{2}(u_{i-1,j+1,k}+u_{i-1,j,k+1}), \\
v_{i,j,k} = v_{i,j+1,k}, & v_{i,j-1,k} = \frac{1}{2}(v_{i+1,j-1,k}+v_{i+1,j-1,k+1}), \\
w_{i,j,k} = w_{i,j,k+1}, & w_{i,j,k-1} = \frac{1}{2}(w_{i+1,j,k-1}+w_{i,j+1,k-1}), \\
\end{array}
\end{equation}
and we analogously set the boundary conditions for the following 7 cases: \texttt{B\_NWU}, \texttt{B\_NOD}, \texttt{B\_NWD}, \texttt{B\_SOU}, \texttt{B\_SWU}, \texttt{B\_SOD}, and \texttt{B\_SWD}.

\subsubsection{Inflow boundary condition}\label{sec:inflow}
For inflow conditions, we assume the inflow velocity is perpendicular to the inflow boundary. Thus, the input for the velocity is a positive scalar.
We implement the following 6 cases, \texttt{B\_O}, \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}, and forbid the other cases (see section \ref{sec:forb} for forbiden cells).
For the \texttt{B\_O} case, the boundary velocity that we set to the input value is $u_{i,j,k} = \texttt{velIN}$, while other velocities for this cell are set to zero.
 We analogously set the boundary conditions for the other 5 cases.

%For inflow conditions, we assume the inflow velocity is perpendicular to the inflow boundary. We implement the following 6 cases, \texttt{B\_O}, \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}, and forbid the other cases.
%
%For the \texttt{B\_O} case, the boundary velocities are same with those the no-slip condition, except that $u_{i,j,k} = \texttt{velIN}$, where the inflow velocity \texttt{velIN} is larger than zero.
%
%Similarly, for the \texttt{B\_W} case, the boundary velocities are same with those of the no-slip condition, except that $u_{i,j,k} = -\texttt{velIN}$, where \texttt{velIN} is larger than zero, and we analogously set the boundary conditions for the following 4 cases: \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}.

\subsubsection{Moving wall boundary condition}\label{sec:moving_wall}
For moving wall conditions, we need to restrict the moving wall directions.
When the boundary cell is like \texttt{B\_O}, we allow the moving wall direction to be $+y$ or $-y$. Consequently, the boundary velocities are equal to zero,
 except for the ones along this moving wall direction and parallel to the boundary. The only velocities that are thus set to a non-zero value in this case are:
\begin{equation}
\begin{array}{ll}
v_{i,j-1,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j-1,k}, & v_{i,j,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j,k},
\end{array}
\end{equation}
where $\texttt{velMW} = [\texttt{velMW}_x, \texttt{velMW}_y, \texttt{velMW}_z]$ is the moving wall velocity vector, and $\texttt{velMW}_x$ is its component along the $x$ axis. We analogously set the boundary conditions for the following 5 cases: .

When the boundary cell is like \texttt{B\_NO}, we set the directions of moving walls \texttt{N} and \texttt{O} in a ``circular'' way. For example, if \texttt{N} moves in the $+x/-x$ direction, then \texttt{O} moves in the $-y/+y$ direction. Therefore, the following boundary conditions differ from those of the no-slip condition:
\begin{equation}
\begin{array}{ll}
u_{i,j,k} = 2\cdot\texttt{velMW}_x-u_{i,j+1,k}, & u_{i-1,j,k} = 2\cdot\texttt{velMW}_x-u_{i-1,j+1,k}, \\
v_{i,j,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j,k}, & v_{i,j-1,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j-1,k}
\end{array}
\end{equation}

%When the boundary cell is like \texttt{B\_O}, we set the moving wall direction to the direction of next (first nonfixed) coordinate, i.e., if $x/y/z$ is fixed, the wall moving in $y/z/x$. For example, if the flag is \texttt{B\_O}, the moving wall direction is $+y$ or $-y$. Consequently, the velocities are same with those of the no-slip condition, except for the ones along this moving wall direction and parallel to the boundary. Therefore, for the \texttt{B\_O} case, the following boundary conditions differ from those of the no-slip condition:
%\begin{equation}
%\begin{array}{ll}
%v_{i,j-1,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j-1,k}, & v_{i,j,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j,k},
%\end{array}
%\end{equation}
%where $\texttt{velMW} = [\texttt{velMW}_x, \texttt{velMW}_y, \texttt{velMW}_z]$ is the moving wall velocity vector, and $\texttt{velMW}_x$ is its component along the $x$ axis. We analogously set the boundary conditions for the following 5 cases: .
%
%When the boundary cell is like \texttt{B\_NO}, we set the directions of moving walls \texttt{N} and \texttt{O} in a ``circular'' way. For example, if \texttt{N} moves in the $+x$ direction, then \texttt{O} moves in the $-y$ direction. Alternatively, if \texttt{N} moves in the $-x$ direction, then \texttt{O} moves in the $+y$ direction. Therefore, for the \texttt{B\_NO} case, the following boundary conditions differ from those of the no-slip condition:
%\begin{equation}
%\begin{array}{ll}
%u_{i,j,k} = 2\cdot\texttt{velMW}_x-u_{i,j+1,k}, & u_{i-1,j,k} = 2\cdot\texttt{velMW}_x-u_{i-1,j+1,k}, \\
%v_{i,j,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j,k}, & v_{i,j-1,k} = 2\cdot\texttt{velMW}_y-v_{i+1,j-1,k}
%\end{array}
%\end{equation}
We analogously set the boundary conditions for the following 16 (5 + 11) cases: \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U}, \texttt{B\_D}, \texttt{B\_NW}, \texttt{B\_NU}, \texttt{B\_ND}, \texttt{B\_SO}, \texttt{B\_SW}, \texttt{B\_SU}, \texttt{B\_SD}, \texttt{B\_OU}, \texttt{B\_WU}, \texttt{B\_OD}, \texttt{B\_WD},
and we forbid the boundary conditions for the following 8 cases: \texttt{B\_NOU}, \texttt{B\_NWU}, \texttt{B\_NOD}, \texttt{B\_NWD}, \texttt{B\_SOU}, \texttt{B\_SWU}, \texttt{B\_SOD}, \texttt{B\_SWD}.

\subsubsection{Boundary condition for the pressure}\label{sec:boundary_pressure}
On the other hand, for all the 5 aforementioned boundary conditions, the boundary values for the pressure are derived from the discretized
momentum equation and result in discrete \textit{Neumann} conditions (the same as in the 2D case).

%Therefore, for the \texttt{B\_O} case, we set the following boundary conditions:
%\begin{equation}
%\begin{array}{lll}
%F_{i,j,k} = u_{i,j,k}, & p_{i,j,k} = p_{i+1,j,k} &
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 5 cases: \texttt{B\_W}, \texttt{B\_N}, \texttt{B\_S}, \texttt{B\_U} and \texttt{B\_D}.
%
%For the \texttt{B\_NO} case, we set the following boundary conditions
%\begin{equation}
%\begin{array}{ll}
%F_{i,j,k} = u_{i,j,k}, \quad G_{i,j,k} = v_{i,j,k},
%& p_{i,j,k} = \frac{1}{2}(p_{i+1,j,k}+p_{i,j+1,k}))
%\end{array}
%\end{equation}
%and we analogously set the boundary conditions for the following 11 cases: \texttt{B\_NW}, \texttt{B\_NU}, \texttt{B\_ND}, \texttt{B\_SO}, \texttt{B\_SW}, \texttt{B\_SU}, \texttt{B\_SD}, \texttt{B\_OU}, \texttt{B\_WU}, \texttt{B\_OD} and \texttt{B\_WD}.

For the \texttt{B\_NOU} case, we set the following boundary conditions
\begin{equation}
\begin{array}{ll}
F_{i,j,k} = u_{i,j,k}, & G_{i,j,k} = v_{i,j,k}, \quad H_{i,j,k} = w_{i,j,k}, \\
 & p_{i,j,k} = \frac{1}{3}(p_{i+1,j,k}+p_{i,j+1,k}+p_{i,j,k+1})
\end{array}
\end{equation}
and we analogously set the boundary conditions for the following 7 cases: \texttt{B\_NWU}, \texttt{B\_NOD}, \texttt{B\_NWD}, \texttt{B\_SOU}, \texttt{B\_SWU}, \texttt{B\_SOD}, and \texttt{B\_SWD}.

\subsection{Allowed cells and their representation}\label{sec:forb}
As in the 2D case, we again forbade geometries with cells, that are of boundary type and border on fluid in two opposite directions. However,
we added two more types of forbidden cells, to make implementation easier and results more physical.
Firstly, the inflow cells that border on fluid on more than one side are forbidden due to the fact that we're reading the inflow velocity as a scalar.
Setting that for a cell with more fluid neighbours would mean having to choose the vector ($u$, $v$ or $w$) of the inflow.
And secondly, since an obstacle that has a corner and all of its sides moving isn't really physical, we decided not to deal with cells that have three
fluid neighbours and moving wall boundary condition.

Cells are in the code represented by flags, which consist of 16 bits, instead of 5 (see following table). % \ref{tab4}).
\begin{table}[hb!]
\label{tab4}
\centering
\begin{tabular}{|c||c|c|c|c|c|c|}
\hline
center & east & west & north & south & bottom & top \\
4bits & 2bits & 2bits & 2bits & 2bits & 2bits & 2bits\\
\hline
\end{tabular}
\end{table}
That extension was necessary because we planned to add the possibility of free flow scenarios,
fo which we needed the additional (apart from water and obstacle) state of cells - air. We also extended representation of center cell by three bits, to
be able to store the information on special boundary conditions.

For neighbour cells we used the following 2bit representation:
\begin{center}
\begin{tabular}{ccc}
01 & $\cdots$ & water \\
10 & $\cdots$ & air \\
11 & $\cdots$ & obstacle
\end{tabular}
\end{center}
And for the center cell we used 4bit sequences:
\begin{center}
\begin{tabular}{ccc}
$00|01$ & $\cdots$ & water \\
$00|10$ & $\cdots$ & air \\
$00|00$ & $\cdots$ & no slip \\
$00|11$ & $\cdots$ & free slip \\
$01|00$ & $\cdots$ & inflow \\
$01|11$ & $\cdots$ & outlow \\
$10|00$ & $\cdots$ & moving wall
\end{tabular}
\end{center}

\subsection{Output}
As previously, results for chosen times are written into vtk files with corresponding names. However, there is an additional output of csv files, for the
particle tracking part. Into those, particle position, velocity, and index of particle set are written.
The particle lines can be easily visualized from this data sets in paraview as well.

\section{Problems, current state and future work}\label{sec:prob_state_future}
Implementation of truly arbitrary geometries took us much longer than expected; we had problems at deciding which cells to forbid and how to calculate the
allowed ones for more complicated boundary conditions. We have therefore spent less time on free surfaces; and have not been able to finish it completely.
Currently, tracking of particles is implemented, and is done automatically within simulation (no special setting or call needed).
There is a lot of space for improvement of our code, for example finishing the free surface extention, parallelization, and so on.
\section{Results}\label{sec:results}
Here are a few show cases of what our code can handle: inflow through just part of the outer wall, inflow on the surface of interior obstacle,
particle tracking, etc.
\newpage
\begin{itemize}
\item Inflow through a pipe: geometry and streamlines.
\begin{center}
\includegraphics[height=4.1cm]{pipeinflow.png}\\
\includegraphics[height=7cm]{pipeinflw.png}
\end{center}

\item Inflow through part of wall, outflow through pipe.
\begin{center}
\includegraphics[height=6.5cm]{inoutpipe.jpg}
\end{center}

\newpage
\item Flow over step: streamlines and particle paths.\\
\begin{center}
\includegraphics[height=5.4cm]{flowoverstep.png}\\
\includegraphics[height=7.7cm]{Step-Particles.png}
\end{center}
\end{itemize}

\begin{thebibliography}{99}
\bibitem{Griebel}
Griebel, M., Dornsheifer, T., Neunhoeffer, T.: \emph{Numerical Simulation in
Fluid Dynamics: A Practical Introduction}. SIAM, {\bf 1998}.

\bibitem{Hirt}
Hirt, C. W., Nichols, B. D.: \emph{Volume of Fluid Method for the Dynamics of Free Boundaries}. Journal of Computational Physics {\bf 39} (1981).

\bibitem{sola}
Hirt, C. W., Nichols, B. D., Hotchkiss, R. S.: \emph{SOLA-VOF: A solution Algorithm for Transient Fluid Flow with Multiple Free Boudaries}. LASL, {\bf 1980}.
\end{thebibliography}

\end{document}


\section{Temporal difference learning for Markov Decision Processes}


\section{Results}

We report in this section results on standard examples, using both low and high
dimensional data. Unless specified otherwise, all the models were parameterized
using values given in table \ref{table:parameters}. These values were chosen to
be simple and do not really impact the performance of the model.
%%
\begin{table}[!ht]
  \begin{center}
    \begin{tabular}{ll}
        \textbf{Parameter} & \textbf{Value} \\
        \hline
        Number of epochs      ($t_f$)           & 25000\\
        Learning rate initial ($\varepsilon_i$) & 0.50\\
        Learning rate final   ($\varepsilon_f$) & 0.01\\
        Sigma initial         ($\sigma_i$)      & 0.50\\
        Sigma final           ($\sigma_f$)      & 0.01\\
    \end{tabular}
      \caption{\textbf{Default parameters} Unless specified otherwise, these are
        the parameters used in all the simulations.}
      \label{table:parameters}
  \end{center}
\end{table}
All simulations and figures were produced using the Python scientific stack,
namely, SciPy \citep{Jones:2001}, Matplotlib \citep{Hunter:2007} and
NumPy \citep{Walt:2011}. Sources are available at
\href{https://github.com/rougier/VSOM}{github.com/rougier/VSOM}.\\

\subsection{Low-dimensional data}

\subsubsection{One-dimensional data}

\begin{figure}
  \includegraphics[width=\columnwidth]{figures/vsom-scalar-1.pdf}

  \vspace{2mm}

  \includegraphics[width=\columnwidth]{figures/vsom-scalar-2.pdf}

  \vspace{2mm}

  \includegraphics[width=\columnwidth]{figures/colormap.pdf}

  \caption{Voronoidal SOM made of 1003 neurons with a 2-nearest neighbors
    induced topology. Model has been trained for 10,000 epochs on random
    uniform scalars in [0,1]. \textbf{A} Map topology in the neural
    space. \textbf{B} Map prototypes displayed in neural space using Voronoi
    cells (whose color indicates prototype according to colormap). \textbf{C to
      H} Map activity for some random data (\textbf{C}:0.0, \textbf{D}:0.2,
    \textbf{E}: 0.4, \textbf{F}:0.6, \textbf{G}:0.8, \textbf{H}:1.0).}
\end{figure}

\begin{figure}
  \includegraphics[width=\columnwidth]{figures/vsom-spatial-1.pdf}

  \vspace{2mm}

  \includegraphics[width=\columnwidth]{figures/vsom-spatial-2.pdf}

  \vspace{2mm}

  \includegraphics[width=\columnwidth]{figures/colormap.pdf}
  \caption{Voronoidal SOM made of 1003 neurons with a 3-nearest neighbors
    induced topology. Model has been trained for 25,000 epochs on random
    samples inside a torus. \textbf{A} Map topology in the neural
    space. \textbf{B} Map prototypes displayed in data space, blue points are
    the actual samples. \textbf{C to H} Map activity for some random data
    (shown on subplot \textbf{B}).}
\end{figure}

\begin{figure}
  \includegraphics[width=\columnwidth]{figures/vsom-colors-1.pdf}

  \vspace{2mm}

  \includegraphics[width=\columnwidth]{figures/vsom-colors-2.pdf}

  \vspace{2mm}

  \includegraphics[width=\columnwidth]{figures/colormap.pdf}

  \caption{}
\end{figure}

\subsection{High-dimensional data}

\begin{figure}
  \setlength{\fboxsep}{0pt}%
  \setlength{\fboxrule}{.25pt}%

  \fbox{\includegraphics[height=5.1cm]{figures/mucha.png}}
        \includegraphics[height=5.1cm]{figures/vsom-image.pdf}
  \fbox{\includegraphics[height=5.1cm]{figures/mucha-vsom.png}}

%  \includegraphics[width=\textwidth]{figures/vsom-image-1.pdf}

  \vspace{2mm}

  \includegraphics[width=\textwidth]{figures/vsom-image-2.pdf}

  \caption{Voronoidal SOM made of 1003 neurons with a 2-nearest neighbors
    induced topology. Model has been trained for 10,000 epochs on random
    uniform scalars in [0,1]. \textbf{A} Map topology in the neural
    space. \textbf{B} Map prototypes displayed in neural space using Voronoi
    cells (whose color indicates prototype according to colormap). \textbf{C to
      H} Map activity for some random data (\textbf{C}:0.0, \textbf{D}:0.2,
    \textbf{E}: 0.4, \textbf{F}:0.6, \textbf{G}:0.8, \textbf{H}:1.0).}
\end{figure}

\subsection{Reorganization}


\documentclass{article}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{csquotes}
\usepackage[colorlinks=true, citecolor = blue]{hyperref}      % hyperlinks
\usepackage{multirow, amssymb, amsmath, graphicx, arydshln, url}
\usepackage{algorithm, algorithmicx, algpseudocode}
\usepackage[T1]{fontenc}
\usepackage{natbib}

\title{Supporting Information for ``Modelling publication bias and \textit{p}-hacking''\\ by Jonas Moss and Riccardo De Bin}

\date{}

\begin{document}

\maketitle

\section*{Web Appendix A}

\begin{algorithm}[!h]
\begin{algorithmic}[1]
	\State $x^{0}\sim p(x)$.
	\For{$i$ in $i=0,1,\ldots$}
		\If{$s\mid x^i = 1$}
			\State Report $x^i$.
		\Else
			\State $x_{H}^{i+1}\sim p(x_{H}^{i}\mid x_{H^{c}}^{0})$.
		\EndIf
	\EndFor
\end{algorithmic}
\caption{\label{alg:Selection model}The selection model $q_{H}(x)$.}
\end{algorithm}

\section*{Web Appendix B}

\paragraph{Proof of Proposition 2}

We only need to show that $q_{H}(x)$ integrates to $1$. \begin{eqnarray*}
\int q_{H}(x)dx & = & \int\frac{p(s=1)}{p(s=1\mid x_{H^{c}})}p(x)dx\\
 & = & \int\frac{p(s=1\mid x_{H},x_{H^{c}})}{p(s=1\mid x_{H^{c}})}p(x_{H^{c}}\mid x_{H})p(x_{H})dx_{H^{c}}dx_{H}\\
 & = & \int\frac{p(s=1\mid x_{H^{c}})}{p(s=1\mid x_{H^{c}})})p(x_{H^{c}})dx_{H^{c}})\\
 & = & 1
\end{eqnarray*}

\section*{Web Appendix C}
As mentioned in Section 3, any \textit{p}-hacking model can be written on the form of a selection model. Observe that
\begin{eqnarray*}
\int_{[0,1]}f_\alpha^{\star}(x_{i}\mid\theta_{i},\eta_{i}, u)d\omega(\alpha) & = & \int_{[0,1]}f(x_{i}\mid\theta_{i})P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})^{-1}d\omega(\alpha)\\
 & = & f(x_{i}\mid\theta_{i})\int_{[0,u]}P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})^{-1}d\omega(\alpha).
\end{eqnarray*}
where $f_\alpha^{\star}$ is the density $f^{\star}$ truncated so that the \textit{p}-value $u\in\left[0,\alpha\right]$. This is a publication bias model if $$h(u)=\int_{[0,u]}P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})^{-1}d\omega(\alpha)$$ is bounded for each $u$ and $h(u)$ is independent of $\theta_{i},\eta_{i}$. While $h(u)$ can be bounded, it is typically dependent of $\theta_{i},\eta_{i}$, with the fixed effect model under complete selection for significance being a notable exception.

On the other hand, any selection model $f(x_{i};\theta_{i},\eta_{i})\rho(u)$ with $I =\int f(x;\theta_{i},\eta_{i})\rho(u)du<\infty$ can be written as a mixture model. For then there is a finite measure $d\omega(\alpha;\theta_{i},\eta_{i})$ satisfying
\[
\rho(u)=\int_{[0,u]}\frac{1}{P(u\in\left[0,\alpha\right]\mid\theta,\eta)}d\omega(\alpha;\theta_{i},\eta_{i})
\]
Just take $d\omega(\alpha;\theta_{i},\eta_{i})=d\rho(\alpha)P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})$, where $d\rho(\alpha)$ is defined by $\int_{0}^{u}d\rho(\alpha)=\rho(u)$. The size of the measure is
\begin{eqnarray*}
\int_{0}^{1}d\omega(\alpha;\theta_{i},\eta_{i}) & = & \int_{0}^{1}P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})d\rho(\alpha)\\
 & = & \int_{0}^{1}f(u;\theta_{i},\eta_{i})\int_{0}^{u}d\rho(\alpha)du\\
 & = & I
\end{eqnarray*}
Hence $I_{\theta,\eta}d\omega'(\alpha;\theta_{i},\eta_{i})$ is a probability measure. This probability measure makes
\[
I^{-1}f(x_{i};\theta_{i},\eta_{i})\rho(u)=\int_{[0,1]}f_\alpha(x_{i};\theta_{i},\eta_{i})d\omega'(\alpha)
\]
as can be seen by the following computation,
\begin{eqnarray*}
I^{-1}f(x_{i};\theta_{i},\eta_{i})\rho(u) & = & I^{-1}\int_{[0,u]}\frac{f(x_{i};\theta_{i},\eta_{i})}{P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})}d\omega(\alpha)\\
 & = & I^{-1}\int_{[0,1]}\frac{f(x_{i};\theta,\eta)1_{\left[0,\alpha\right]}(u)}{P(u\in\left[0,\alpha\right]\mid\theta_{i},\eta_{i})}d\omega(\alpha)\\
 & = & I^{-1}\int_{[0,1]}f_\alpha(x_{i};\theta_{i},\eta_{i})d\omega(\alpha)\\
 & = & \int_{[0,1]}f_\alpha(x;\theta_{i},\eta_{i})d\omega'(\alpha)
\end{eqnarray*}

Proposition 1 shows the form of the one-sided normal step function selection probability publication bias model when it is written as a mixture model of the form (5). But most such mixture models are not true \textit{p}-hacking models, as the mixing probabilities $\pi_{i}^{\star}$ depend on $\theta$. There is no way for the \textit{p}-hacker to know
$\theta$, so we cannot regard the publication bias model as a \textit{p}-hacking model.

\end{document}