\documentclass[12pt]{article}
\bibliographystyle{plain}
\usepackage[dvips]{graphicx}
\usepackage{program}
\newcommand{\institute}[1]{\emph{#1}}
\title{Blob-Tracker using FVision and XVision2}
\author{Jatin D. Shah}
\date{\today}

\newtheorem{defn}{Definition}[section]
\begin{document}
\maketitle
\section{Introduction}
\par A blob-tracker tracks the motion of a region of specific color in a video sequence. The region is generally described as a rectangle or a circle. In this project, the blob tracker assumes that the region to be tracked is rectangular. The tracker samples a small window where it expects to find the blob. Then the blob is identified using suitable functions from a vision library. This process is repeated every frame by changing the location of the window to match the motion of the blob tracker.
\par In this project, I have implemented a blob-tracker in Haskell. The vision library used is a C++ library, XVision2; these library functions were accessed in Haskell using FVision, which is an interface to the XVision2 through GreenCard.
\section{Parts of Blob Tracker}
\par As described above, once the user clicks on a blob, a small rectangular region around the blob of size {\tt blobSize} centered at the position clicked is sampled. The FVision function {\tt getColorSelector} returns the color of this region. The type of this function is: $$\mbox{\tt getColorSelector :: ImageRGB -> ColorSelector}$$ and it is defined in {\tt NXVision2.gc}. The feedback loop described in the next section described in the next section tracks the motion of a blob of the color returned by {\tt getColorSelector}.
\subsection{Feedback Loop}
\par This part of the program attempts to locate the present position of the blob using its position in the previous frame and the color of the blob. It is implemented using FRP primitive {\tt loopB} in file {\tt FRPExtra.hs}. A skeleton of the program is code is shown below\footnote{This is not the actual program code}:
\begin{verbatim}
newXY =
      loopB (\newXY ->
         let
           positionX = (centerX)
                            `delayB` (fst (derefB newXY))
           positionY = (centerY)
                            `delayB` (snd (derefB newXY))
           ...
           window = getRegion video positionX positionY windowSize windowSize
           (blobX,blobY,blobW,blobH) = stepBlob2 positionX positionY
                                                 window color
           ...
           newX = positionX + blobX + blobH/2
           newY = positionY + blobY + blobW/2
         in
           pairZ newX newY
          )
\end{verbatim}
\par In the code shown above {\tt centerX} and {\tt centerY} is the location clicked by the user. {\tt loopB} implements the feedback loop where position of the loop (represented by {\tt positionX/Y} and {\tt newX/Y})is fed back to the system.
\par The function {\tt getRegion} cuts out a window from the image and the function {\tt stepBlob2} find the blob in the window cut by {\tt getRegion}. These are FVision functions (defined in file {\tt NXVision2.gc}).
\subsection{Motion Prediction}
\par Here it is assumed that the blob is moving with a constant velocity so we only try to approximate the first derivative of position to find the position of the blob. The {\tt derivativeB} function from FRP is not used as John Peterson has doubts if it will work. Instead, last two positions of the blob are tracked and the first derivative is approximated from those positions. The basic structure of the new code is shown below:
\begin{verbatim}
newXY =
        loopB (\newXYref ->
            let
                    position0X = (centerX)
                                    `delayB` (fst (derefB newXYref))
                    position1X = position0X
                                    `delayB` (snd (derefB newXYref))
                    ...

                    dispX = position0X - position1X
                    predictedPositionX = position0X + dispX
                    ...
              )
\end{verbatim}
\par In this case, {\tt position0X} and {\tt position1X} are the two previous positions of the blob ({\tt position0X} being the more recent position)
\par These positions are used to find the {\tt predictedPositionX/Y} which are passed to the tracker instead of the previous positions.
\subsection{Adaptive Window Resizing}
\par It may happen that inspite of motion prediction, the blob tracker may lose track of the blob. In such a situation, one might try to find the blob in the entire image. But this can be very slow. A better strategy would be to increase the window size by a small factor until the tracker finds the blob. The window size can later be decreased once the tracker has begun to track the blobs properly.
\par The code shown below achieves that\footnote{In the actual code, we wait if we do not find a blob for {\tt threshold} consecutive frames. This has to be done because {\tt stepBlob2} sometimes erroneously returns that it has not found the blob even when it is tracking the blob properly.}
\begin{verbatim}
newXY = loopB (\newXYref ->
            let
                ...
                (blobX,blobY,blobH,blobW) = stepBlob2 ...
                dataError = (blobX<*0 ||* blobY<*0 ||* blobH<*0 ||* blobW<*0)
                         ||* (blobH==*0 &&* blobW==*0)
                newX=ifX dataError position0X (position0X+blobX+blobW/2)
                newWinSz = if dataError
                              (changeWinSz oldWinSz percent)
                              (changeWinSz oldWinSz (-percent))
                ...
              )
\end{verbatim}
\par If the function {\tt stepBlob2} returns negative values implying that blob is not found then position of the blob is not changed. Further, if {\tt stepBlob2} is unable to find the blob then window size is increased.
\section{Implementation Issues}
There are several issues regarding implementation of this blob tracker in Haskell. The program runs extremely slowly in both hugs and GHC implementation of Haskell.
\subsection{Space Leak in FRP}
\par It seems that the implementation of the program I have discussed in this paper suffers from some kind of space leak problem which I was unable to detect. The problem is aggravated if I used more graphics primitives. It should be noted that the space leak is present even if the program is compiled using GHC.
\subsection{Choosing a Window Size}
\par Ideally, the algorithm for changing the window size should converge at a that size which minimizes the number of {\it no blob found} returns by {\tt stepBlob2}. But the algorithm described in this paper is unable to do this. My algorithm, decreases the window size to minimum possible if it is tracking properly and increases it if it fails to track. Thus, in this implementation, proper tracking of by the program requires that the window size entered by the user is appropriate. Since we are using first order motion prediction, we will be able to track all blobs moving with constant velocities with window sizes about the size of the blob. Window sizes about the size of the blob will be sufficient for accelerated blobs once we incorporate second-order motion prediction.
\par If the window size is smaller than the object, then {\tt stepBlob2} returns the approximately entire window as blob. The window sizes should be increased in this situation as well.
\end{document}


    \filetitle{findname}{Find names of variables, shocks, or parameters by their descriptors}{model/findname}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
[Name,Name,...] = findname(M,Desc,Desc,...)
[List,List,...] = findname(M,'-rexp',Rexp,Rexp,...)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{M} {[} model {]} - Model object in which the names will be
  searched for.
\item
  \texttt{Desc} {[} char {]} - Variable, shock, or parameter descriptors
  that will be searched for.
\item
  \texttt{Rexp} {[} char {]} - Regular expressions that will be matched
  against variable, shock, and parameter descriptors.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\item
  \texttt{Name} {[} char {]} - First name found with the descriptor
  \texttt{Desc}.
\item
  \texttt{List} {[} cellstr {]} - List of names whose descriptors match
  the regular expression \texttt{Rexp}.
\end{itemize}

\paragraph{Description}\label{description}

\paragraph{Example}\label{example}


\section{Mutable Objects}
We now implement mutable objects. We start with inheritance:
\begin{lstlisting}
data Inherit x = Inherit x

instance (o ~ (BaseCons bo 'Cons' no), Record o ST s => Object o ST s where
  type Base o = bo
  get_base self_ref =
	StRef(do ((BaseCons base) 'Cons' tl) <- eval self_ref
           return base)
       (\base' -> do (_ 'Cons' tl) <- eval self_ref
           self_ref := ((Inherit base') 'Cons' tl)
           return ())

  ...

instance (o ~ (Unit 'Cons' so), Record o ST s => Object o ST s where
  type Base o = o
  get_base self_ref = self_ref

  ...
\end{lstlisting}
In our mutable encoding the first field of the object must be either the value of the inherited value or unit when the object does not inherit anything.

Methods enjoy the same implementation in both cases, so we just give one:
\begin{lstlisting}
data MethodCons t a b = MethodCons(t -> a -> (b,t))

instance (o ~ (Unit 'Cons' so), Record o ST s => Object o ST s where
  ...

  type Method ro st a b = MethodCons ro a b
  self_ref <=| (Label read write) =
             \x -> do self <- eval self_ref
                      let (MethodCons m) = read self
                          (res,self') = m self x
                      self_ref := self'
                      return res
\end{lstlisting}

It can prove very useful to take advantage of our existing infrastructure to create methods from references and statements, so that the user of our system will not be forced to define methods by explicitly tracking mutations to the value of $this$; for this reason we add a function to the $Object$ predicate that converts a method from reference to state into our format (the implementation here is the same for both instances of $Object$, so we provide only one:
\begin{lstlisting}
class (Record o st s, Recursive o, ro~Rec o) => Object o st s where
  ...
  mk_method :: (StRef o o -> a -> st o b) -> Method ro st a b

instance (o ~ (Unit 'Cons' so), Record o ST s => Object o ST s where
  ...
  mk_method m = Method(\this->\args->
                         let (ST res_st) = m state_ref args
                         in res_st this)
\end{lstlisting}

Before Giuseppe's PhD/Monads/ObjectiveMonad/MonadicObjects/trunk/tex v2/7.mutable_objects.tex


\documentclass[11pt]{scrartcl} % Font size
\input{structure.tex} % Include the file specifying the document structure and custom commands

%----------------------------------------------------------------------------------------
%	TITLE SECTION
%----------------------------------------------------------------------------------------

\title{
	\normalfont\normalsize
	\textsc{Harvard Privacy Tools Project}\\ % Your university, school and/or department name(s)
	\vspace{25pt} % Whitespace
	\rule{\linewidth}{0.5pt}\\ % Thin top horizontal rule
	\vspace{20pt} % Whitespace
	{\huge Randomness and Noise}\\ % The assignment title
	\vspace{12pt} % Whitespace
	\rule{\linewidth}{2pt}\\ % Thick bottom horizontal rule
	\vspace{12pt} % Whitespace
}

\author{} % Your name

\date{} % Today's date (\today) or a custom date

\begin{document}

\maketitle

\tableofcontents

\section{Overview}
This document describes the strategies the library uses for generation of randomness and noise.

\section{Preliminaries}
\begin{definition}
	\label{defn:differential_privacy}
	Differential Privacy \cite{DMNS06} \newline
	For $\epsilon, \delta \geq 0$, a randomized mechanism
	$\mathcal{M}: \mathcal{X}^n \times \mathcal{Q} \rightarrow \mathcal{Y}$ is
	$(\epsilon, \delta)$-DP if, for every pair of neighboring data sets $X, X' \in \mathcal{X}^n$ and
	every query $q \in \mathcal{Q}$ we have
	\[ \forall \mathcal{T} \subseteq \mathcal{Y}: \Pr[\mathcal{M}(X, \epsilon, \delta, q) \in \mathcal{T}] \leq e^{\epsilon} \Pr[\mathcal{M}(X', \epsilon, \delta, q) \in \mathcal{T}] + \delta. \]
\end{definition}
If $\delta = 0$, we call this \emph{Pure DP}. If $\delta > 0$, we call this \emph{Approximate DP}.
Note that, in practice, differential privacy could be thought of a bit more broadly -- as a bounded
distance between joint distributions over all entities that could be observable to an adversary.\footnote{For example, imagine that the
US government uses $\epsilon = 1$ if the President is in the data and $\epsilon = 10$
if not -- if anyone knew about this rule, the choice of epsilon would leak information not accounted for in the
traditional definition of DP.} The primary focus of this library is to respect the traditional notion of
differential privacy as defined in~\ref{defn:differential_privacy}. However, we are also working to
reduce the possibility of information leakage via differences in computational runtime. We will touch on this when relevant
during throughout the document.

\begin{definition}
	\label{defn:exact_rounding}
	Exact Rounding \newline
	Let $S \subset \R$ be some set.
	Let $\phi: \R^n \rightarrow \R$ be a function on the reals and $\phi_{S}: S^n \rightarrow S$ be its implementation over $S$.
	Then, $\phi_{S}$ respects \emph{exact rounding} for $(\phi, S)$ if
	\[ \forall s \in S: \phi_{S}(s) = round_{S}[\phi(s)], \]
	where $round_{S}(\cdot)$ rounds a real number to a member of $S$ according to some rounding rule.
\end{definition}
For our purposes, we will care only about the case where $S = \F$, the set of IEEE-754 floating-point numbers.

\begin{definition}
	\label{def:truncation_censoring}
	Truncation and Censoring \newline
	Throughout our noise functions, we use the terms ``truncated'' and ``censored''.
	Both are means of bounding the support of the noise distribution, but they are distinct. \newline

	Truncating a distribution simply ignores events outside of the given bounds, so
	all probabilities within the given bounds are scaled up by a constant factor.
	One way to generate a truncated distribution is via rejection sampling.
	You can generate samples from a probability distribution as you normally would (without any bounding),
	and reject any sample that falls outside of your bounds. \newline

	Censoring a distribution, rather than ignoring events outside of the given bounds, pushes the
	probabilities of said events to the closest event within the given bounds. One way to generate
	a censored distribution would be to generate samples from a probability distribution as you
	typically would, and then clamp samples that fall outside of your bounds to the closest element
	inside your bounds.
\end{definition}

\section{Randomness}
Nearly all of our random number generation involves uniform random sampling of bits via OpenSSL.
The exception is when we use methods from MPFR (see below) to sample from a probability distribution.
In this case, we feed the state of then OpenSSL RNG directly into the method. \newline

We will take as given that OpenSSL is cryptographically secure. We intend to support a broader set
of cryptographically secure sources of randomness at a later date.

\section{Noise Generation}

\subsection{Introduction to MPFR}
The \href{https://www.mpfr.org/}{GNU MPFR Library}\cite{FHL+07} is a C library with methods for carrying out a number
of floating-point operations with \emph{exact rounding}(see Definition~\ref{defn:exact_rounding}).
Among these are basic arithmetic operations and means of generating samples from basic probability distributions. \newline

\subsection{Biased bit sampling}
Recall that we are taking as given that we are able to sample uniform bits from OpenSSL.
For many applications, however, we want to be able to sample bits non-uniformly,
i.e. where $\Pr(bit = 1) \neq \frac{1}{2}$. To do so, we use the $sample\_bit$ function.
This function uses the unbiased bit generation from OpenSSL to return a single bit, where $\Pr(bit = 1) = prob$ --
there is a nice write-up of the algorithm \href{https://amakelov.wordpress.com/2013/10/10/arbitrarily-biasing-a-coin-in-2-expected-tosses/}{here}.
We will give a general form of the algorithm, and then talk about implementation details.
\begin{algorithm}[H]
	\caption{Biasing an unbiased coin (in theory)}
	\label{alg:biasing_a_coin_theory}
	\begin{algorithmic}[1]
		\State $p \gets \Pr(bit = 1)$
		\State Find the infinite binary expansion of $p$, which we call $b = (b_1, b_2, \hdots,)_2$.
		Note that $p = \sum_{i=1}^{\infty}\frac{b_i}{2^i}$.
		\State Toss an unbiased coin until the first instance of ``heads''. Call the (1-based) index where this occurred $k$.
		\State return $b_k$
	\end{algorithmic}
\end{algorithm}
Let's first show that this procedure gives the correct expectation:
\begin{align*}
	p &= \Pr(bit = 1) \\
		 &= \sum_{i=1}^{\infty} \Pr(bit = 1 \vert k = i) \Pr(k = i) \\
		 &= \sum_{i=1}^{\infty} b_i \cdot \frac{1}{2^i} \\
		 &= \sum_{i=1}^{\infty}\frac{b_i}{2^i}.
\end{align*}
This is consistent with the statement in Algorithm~\ref{alg:biasing_a_coin_theory}, so we know that
the process returns bits with the correct bias.
In terms of efficiency, we know that we can stop coin flipping once we get a heads,
so that part of the algorithm has $\E(\# flips) = 2$ because the probability of getting a heads
on any given flip is $1/2$. \newline

We now move to constructing the infinite binary expansion of $p$.
We start by noting that, for our purposes, we do not actually need an infinite binary expansion.
Because $p$ will always be a 64-bit floating-point number, we need only get a binary
expansion that covers all representable numbers in our floating-point standard that are
also valid probabilities.
Luckily, the underlying structure of floating-point numbers makes this quite easy. \newline

In the 64-bit standard, floating-point numbers are represented as
\[ (-1)^s(1.m_1m_2 \hdots m_{52})_2 * 2^{(e_{1}e_2 \hdots e_{11})_2 - 1023}, \]
where $s$ is a sign bit we ignore for our purposes.
Our binary expansion is just the mantissa $(1.m_1m_2 \hdots m_{52})_2$, with
the radix point shifted based on the value of the exponent.
We can then index into the properly shifted mantissa and check the value of the $k$th element.
We end up with the following algorithm:
\begin{algorithm}[H]
	\caption{Biasing an unbiased coin (in practice): \newline sample\_bit(p: f64)}
	\label{alg:biasing_a_coin_practice}
	\begin{algorithmic}[1]
		\State We know that $p$ is respresentable as an IEEE-754 64-bit floating point number.
		\State $m, x \gets $ mantissa and exponent of the floating-point representation of $p$.
		We ensure that the mantissa gets the implicit leading 1 and the exponent is the ``unbiased'' version.
		So $m \in \{1\} \bigcup \{0, 1\}^{52}$ and $x \in \{0, 1, \hdots, 1022\}$
		\State Toss an unbiased coin until the first instance of ``heads''. Call the (0-based) index where this occurred $k$.
		\State n\_leading\_zeros $\gets \max(0, 1022 - x)$
		\If{$k <$ n\_leading\_zeros}
			\State return 0
		\Else
			\State $i \gets k -$ n\_leading\_zeros
			\If{$i > \text{len}(m)$}
				\State return 0
			\Else
				\State return $i^{th}$ element of $m$ (using 0-based indexing)
			\EndIf
		\EndIf
	\end{algorithmic}
\end{algorithm}
Note that $x \in \{0, 1, \hdots, 1022\}$ above because these are the biased exponent values that correspond to $p \in [0,1]$.
When $x=0$, $p$ is subnormal.

\subsection{Sampling from censored Geometric}
The Geometric distribution is a building block for many of our other mechanism, either as the basis of the
noise distribution (as for the Geometric mechanism) or as a component of a larger algorithm (as we will show in
section~\ref{subsec:unif_sampling}). For now, the library supports sampling only from a censored Geometric distribution. \newline

The function accepts three arguments. $p$ is the probability of success on any given trial. max\_trials is the maximum number of
trials for which the algorithm will run -- if no success has occurred by the time max\_trials has been reached, the algorithm
will return max\_trials as its answer.
The $ect$ boolean stands for \emph{enforce constant time} and tells the algorithm to run for the maximum number
of trials, regardless of when success is achieved. This is useful for reducing variability in algorithm runtime that could be used
for a timing attack.
\begin{algorithm}[H]
	\caption{Generating draws from Censored Geometric: \newline sample\_geometric\_censored(p: f64, max\_trials: i64, ect: bool)}
	\label{alg:censored_geom_p}
	\begin{algorithmic}[1]
		\State trial\_index $\gets 1$
		\State geom\_return $\gets 0$
		\While trial\_index $\leq$ max\_trials
			\State bit $\gets sample\_bit(p)$
			\If{bit $== 1$}
				\If{geom\_return $== 0$} \Comment{Update result from Geometric only if we have not already seen a 1}
					\State geom\_return $\gets$ trial\_index
					\If{ect $==$ False} \Comment{If we do not care to enforce constant time...}
						\State return geom\_return \Comment{return the result}
					\EndIf
				\EndIf
			\Else
				\State trial\_index $+= 1$
			\EndIf
		\EndWhile
		\If{geom\_return $== 0$} \Comment{If Geometric result $>$ censoring bound...}
			\State return max\_trials \Comment{have it return the value of the bound}
		\EndIf
	\end{algorithmic}
\end{algorithm}

\subsection{Sampling from Uniform[min, max)}
\label{subsec:unif_sampling}
In this method, we start by generating a floating-point number $y \in [0,1)$,
where each is generated with probability relative to its unit of least precision (ULP).\footnote{The ULP is the value
represented by the least significant bit of the mantissa if that bit is a 1.}
That is, we generate $y \in [2^{-g}, 2^{-g+1})$ with probability $\frac{1}{2^i}$
for all $g \in \{1,2,\hdots,1022\}$ and $y \in [0, 2^{-1022})$ for $g = 1023$.
At the end, we will scale our output from $[0,1)$ to be instead in $[min, max)$. \newline

The algorithm is as follows:
\begin{algorithm}[H]
	\caption{Generating draws from Uniform[min, max)}
	\label{alg:unif_min_max}
	\begin{algorithmic}[1]
		\State $m \gets \{0, 1\}^{52}$ from OpenSSL (or other cryptographically-secure RNG)
		\State $g \gets min(1023, sample\_geometric\_censored(p = 0.5, max\_trials = 1023, ect = True))$
		\State $u \gets (1.m_1m_2 \hdots m_{52})_2 * 2^{-g} * (max - min) + min$
		\State return $u$
	\end{algorithmic}
\end{algorithm}

This method was proposed in \cite{Mir12} as a component of a larger attempt to create
a version of the Laplace mechanism that is not susceptible to floating-point attacks.
Note that the original method generates values $\in [0,1)$ rather than arbitrary $[min, max)$
and does not give guidance on what to do if the sample from the Geometric is $> 1023$.
There is no universally agreed upon method for generating uniform random numbers (for privacy
applications or otherwise), but this method seems to approximate the real numbers better than many
others because of the sampling relative to the ULP. \newline

\begin{tcolorbox}[colback = {green}, title = {Known Privacy Issues}, colbacktitle = black]
	When $g=1023$ we are sampling from subnormal floating-point numbers. Because processors do not typically support
	subnormals natively, they take much longer to sample and open us up to an easier timing attack, as
	seen in \cite{AKM+15}. \newline

	We are incurring some floating-point error when converting from $[0,1)$ to $[min, max)$ which
	could jeopardize privacy guarantees in ways that are difficult to reason about.\cite{Mir12} \cite{Ilv19}
\end{tcolorbox}

We have a method for generating uniform samples via MPFR that respects exact rounding, but it is being used sparingly in the library.
We are working to figure out if and how we can use this method as a building block for floating-point safe methods of drawing from
other distributions.

\subsection{Problems with sampling from other continuous distributions}
In principle, we can generate draws from non-uniform continuous distributions (e.g. Laplace, Gaussian)
by using \href{https://en.wikipedia.org/wiki/Inverse_transform_sampling}{inverse transform sampling}.
To draw from a distribution $f$ with CDF $F$,
we sample $u$ from $Unif[0,1)$ and return $F^{-1}(u)$. \newline

\begin{tcolorbox}[colback = {green}, title = {Known Privacy Issues}, colbacktitle = black]
	Carrying out the inverse probability transform employs floating-point arithmetic,
	so we run into the same problems as were described in the uniform sampling section.
	This is potentially a very significant problem, and one for which we do not
	currently have a good general solution.
\end{tcolorbox}
Because of the vulnerabilities inherent in using floating-point arithmetic, we would
like to avoid using inverse transform sampling. We do not have a completely general way of ensuring
that algorithms designed to be private when drawing noise from $\R$ remain private when they have
access only to floating-point numbers. Instead, we will take each relevant distribution individually
and discuss potential solutions. \newline

\subsubsection{Potential fixes for Laplace}
We have a \href{https://github.com/opendifferentialprivacy/smartnoise-core/tree/CC_add_snapping}{branch} with an implementation of
the Snapping mechanism from \cite{Mir12}. We are currently working to verify the theory and associated implementation, as well
as consider how to use it effectively in practice. \newline

\subsubsection{Potential fixes for Gaussian}
As mentioned for the Uniform, we have a method for generating Gaussian samples via MPFR that respects exact rounding.
It is not currently being used in the library. \newline

\cite{DKM+06} proposes using the binomial approximation to the Gaussian, and notes that
an additive noise mechanism drawing noise from a $Binomial(n, p = 0.5)$ respects $(\epsilon, \delta)$-DP
provided that $n \geq 64 \frac{\log(2/\delta)}{\epsilon^2}$. This seems promising and we already
have the infrastructure for generating a Binomial without floating-point arithmetic, but it
involves manually sampling bits. For small $\epsilon$, this method could quickly become
computationally unwieldy. For example, having the mechanism respect $(10^{-2}, 10^{-9})$-DP
requires sampling nearly 14 million bits.

\bibliographystyle{alpha}
\bibliography{noise}

\end{document}


\section{Introduction}

Classical \emph{probabilistic verification} techniques rely on iterative approximation algorithms for linear equation systems and linear programs, such as \emph{value iteration (VI)}, e.g.\  \cite{Puterman}.
However, the scalability of such numeric analyses is severely limited, compared to standard non-quantitative (hardware or software) verification, since exact transformations, such as abstraction or partial-order reduction, are more difficult to use.
Consequently, weaker guarantees such as \emph{probably approximately correct (PAC)} results become acceptable even for completely known systems (\emph{white box}) and not only in contexts where the system is executable but unknown (\emph{black box}), and where thus absolute guarantees are principally impossible.

\begin{example}
	Consider the task of model checking a reachability property of a~probabilistic communication protocol, which starts by generating a few, say $k$, random bits.
	Thus the execution immediately branches into $2^k$ states.
	If there are only few or hard-to-find symmetries in the behaviour, standard analysis quickly becomes infeasible.
	In the following, we discuss drawbacks of previously studied alternative approaches; then we suggest a new one that overcomes the difficulties for a wide class of models.
\end{example}

%prob.verif. (say reach/LTL) of a protocol with wide branching but hard-to-find/few symmetries, e.g. starting with generating a random number? ! standard impossible, what are the answers? don't work, we give a better one SMC with white; "we tell the story now"

The exponential state-space explosion quickly renders explicit VI unable to propagate information by more than a single step.
Besides, if the transition probabilities depend on the generated bits, even the symbolic variants of VI \cite{DBLP:conf/icalp/BaierCHKR97} cannot help much.
There have been two major alternatives proposed, both relying on extensive use of simulations.

%standard = vi, better than LP or oten SI but simple does not scale that much

$\bullet$ (I) For large and possibly \emph{unknown} systems, \emph{statistical model checking (SMC)} \cite{Younes02} reincarnates the Monte Carlo method.
It runs simulations of the system; the resulting statistics then yields confidence intervals, i.e.~PAC results.
However, for unbounded-horizon properties, such as reachability or linear temporal logic (LTL) \cite{pnueli1977temporal}, performing simulations of finite length requires some information about the model \cite{isola-survey}:
\begin{enumerate}
	\item Either the second eigenvalue of the transition matrix can be bounded \cite{DBLP:journals/apal/LassaigneP08,YCZ10}, which requires essentially the complete knowledge of the system ({white box}) and is as hard as solving the model checking problem, or
	\item the topology of the underlying state-graph is known \cite{YCZ10,ase10} (sometimes called \emph{grey box}, e.g. \cite{cav19})  and the whole system is preprocessed, which beats the purpose of sublinear analysis, or
	\item a bound on the minimum transition probability $\pmin$ is known as is the case in~\cite{atva14,DHKPjournal}. This is the closest to black box, thus called \emph{black SMC} here.
\end{enumerate}

In black SMC, long enough simulations can be run to ensure the system passes from the transient to the recurrent part and reliable information on the whole infinite run is obtained.
While the a-priori length is practically infeasible \cite{atva14}, early detection of recurrent behaviour has been proposed \cite{DHKPjournal} as follows.
Based on the observed part of a simulation run, a hypothesis on the topology of the system is made, answering what bottom strongly connected component (BSCC) this run ends up in.
With repetitive observations of transitions over the run, the confidence grows that what currently looks as a BSCC indeed is a BSCC.
Since quite a few repetitions of \emph{all} transitions in the BSCC are required, this approach turns out practical only for systems with small BSCCs and not too small $\pmin$.

%- large or unknown - smc and pac is ok - unbounded required essentially white (globally hence slow) or too long sim (slow) - fast smc (idea) - larger bsccs? ! locally to make them shorter

In this paper, assuming knowledge of the system (white-box setting), we twist the technique to a more efficient one as follows.
After quickly gaining (unreliably low) confidence that the run entered a BSCC, we use the knowledge of the topology to confirm this information---again very quickly since not the whole model is built but only the \emph{local} BSCC.
Consequently, BSCCs are detected fast even in the case with larger BSCCs or small $\pmin$.
As the information used turns out quite limited, corresponding to the grey-box setting, we call this approach \emph{grey SMC}.

$\bullet$ (II) The other alternative to VI, now in the context or large but \emph{known} systems, is the \emph{asynchronous value iteration}, e.g. \cite{10.5555/59912}, a generalization of the Gauss-Seidel method and the core of reinforcement learning and approximate dynamic programming. There, the VI updates on states of the system are performed in varying orders, in particular possibly entirely skipping some states.
The class of algorithms providing guarantees is represented by \emph{bounded real-time dynamic programming (BRTDP)} \cite{BRTDP,atva14,cav19} where the states to be updated at each moment are those appearing on a current simulation run.
Consequently, states with low probability of visiting and thus low impact on the overall value are ignored.
While this allows for treating very ``wide'' systems with lots of unimportant branches, the scalability problem persists as soon as the branching is very uniform (see also Example \ref{ex:adv} on Fig.~\ref{fig:adv}).
From this perspective, grey SMC relaxes the rigorous approximation in the transient part and replaces it with a statistical estimate.

%- large (wide) but known - brtdp - (uniformly very) wide? ! relax to PAC but still use white

Overall, grey SMC fills the gap in the following spectrum:

\smallskip
\begin{tabular}{|@{\hskip 5mm}c@{\hskip 9mm}c@{\hskip 9mm}c@{\hskip 9mm}c@{\hskip 5mm}|}
	\hline
	VI&BRTDP&grey SMC&black SMC\\
	\multirow {2}{*}{analysis} & analysis with   & simulation with  &  \multirow {2}{*}{simulation}\\
	&simulation inside& analysis inside &\\\hline
\end{tabular}
\smallskip

\noindent
On the one end, numeric analysis (VI) provides reliable results; in BRTDP, simulations are additionally used in the analysis to improve the performance while preserving the guarantees.
On the other end, simulations (SMC) provide PAC guarantees; grey SMC then improves the performance by additional analysis in the simulation.

%analysis (VI) / simulation inside analysis (BRTDP) / analysis inside simulation (white SMC) / simulation (SMC)

\emph{Our contribution} can be summarized as follows:
\begin{itemize}
	\item We modify the black SMC for unbounded properties of \cite{DHKPjournal} to perform better in the white-box (and actually also in the so-called grey-box) setting.
	\item We compare our grey SMC to black SMC, BRTDP and VI both conceptually, illustrating advantages on examples, as well as experimentally, comparing the runtimes on standard benchmarks.
	\item We present all algorithms within a unified framework, which in our opinion eases understanding and comparison, provides a more systematic insight, and is pedagogically more valuable.
\end{itemize}

\emph{Outline of the paper:}
After recalling necessary definitions in Section~\ref{sec:prelim}, we describe and exemplify the algorithms in Section \ref{sec:algos} and the respective key sub-procedure in Section \ref{sec:stuck}.
Then we compare the algorithms and other related work in Section \ref{sec:discussion}, discussing the expected implications, which we confirm experimentally in Section \ref{sec:experiments}.
For a broader account on related work on SMC in the context of unbounded-horizon properties, we refer the interested reader to the survey \cite{isola-survey}.

%\todo{Jan: Write intro. Below is first the notes of Pranav and then the intro of CAV19 paper as inspiration. Also, the intro of FSMC \cite{DHKP16} probably is very relevant.}
%
%\subsection{Pranav's notes}
%White FMSC - SMC for unbounded
%
%DIfference bw brtdp/smc
%
%- Extrmeme stochastic branching (brtdp will fail because it has to explore everything)
%WhiteFSMC better
%
%- FSMC Reviewers complained FSMC was only working with trivial end components
%
%White SMC
%- Built partial model and explore a few more unseen states.
%
%
%SMC is not entirely reliable, but sometimes its the only choice.
%New thing was BRTDP, but it doesn't always work.
%We now use guaranteed thing inside SMC.
%
%Unbounded reachability
%- He et. al. 2010 - take a look in a bit more detail
%- Preprocess model to find 0 states. Stop on reaching such states. (unable to find ref)
%- Weighing with expectation, but bounded reach (?)
%
%Check FMSC related work for related work
%
%Approaches before were not using local whitebox info. We want to use that.
%
%\subsection{CAV19 intro}
%
%\para{Statistical model checking (SMC)} \cite{YS02} is an analysis technique for probabilistic systems based on
%\begin{compactenum}
%	\item simulating finitely many finitely long runs of the system,
%	\item statistical analysis of the obtained results,
%	\item yielding a confidence interval/probably approximately correct (PAC) result on the probability of satisfying a given property, i.e., there is a non-zero probability that the bounds are incorrect, but they are correct with probability that can be set arbitrarily close to $1$.
%\end{compactenum}
%One of the advantages is that it can avoid the state-space explosion problem, albeit at the cost of weaker guarantees.
%Even more importantly, this technique is applicable even when the model is not known (\emph{black-box} setting) or only qualitatively known (\emph{grey-box} setting), where the exact transition probabilities are unknown such as in many cyber-physical systems.
%
%In the basic setting of Markov chains \cite{norris1998markov} with (time- or step-)bounded properties, the technique is very efficient and has been applied to numerous domains, e.g.\ biological \cite{DBLP:conf/cmsb/JhaCLLPZ09,DBLP:conf/cmsb/PalaniappanG0HT13}, hybrid \cite{DBLP:conf/hybrid/ZulianiPC10,DBLP:journals/corr/abs-1208-3856,DBLP:conf/formats/EllenGF12,DBLP:conf/formats/Larsen12} or cyber-physical \cite{DBLP:conf/forte/BasuBBCDL10,DBLP:conf/atva/ClarkeZ11,DBLP:conf/nfm/DavidDLLM13} systems and a substantial tool support is available \cite{DBLP:conf/tacas/JegourelLS12,DBLP:journals/corr/abs-1207-1272,DBLP:conf/qest/BoyerCLS13,DBLP:conf/mmb/BogdollHH12}.
%In contrast, whenever either (i)~infinite time-horizon properties, e.g. reachability, are considered or (ii)~non-determinism is present in the system, providing any guarantees becomes significantly harder.
%
%Firstly, for \emph{infinite time-horizon properties} we need a stopping criterion such that the infinite-horizon property can be reliably evaluated based on a finite prefix of the run yielded by simulation.
%This can rely on the the complete knowledge of the system (\emph{white-box} setting) \cite{sbmf11,DBLP:journals/apal/LassaigneP08}, the topology of the system (grey box) \cite{sbmf11,ase10}, or a lower bound $\pmin$ on the minimum transition probability in the system (black box) \cite{DHKP16,BCC+14}.
%
%Secondly, for Markov decision processes (MDP) \cite{Puterman} with (non-trivial) \emph{non-determinism}, \cite{DBLP:conf/qest/HenriquesMZPC12} and \cite{LP12} employ reinforcement learning~\cite{SB98} in the setting of bounded properties or discounted (and for the purposes of approximation thus also bounded) properties, respectively.
%The latter also yields PAC guarantees.
%
%Finally, for MDP with unbounded properties, \cite{DBLP:conf/forte/BogdollFHH11} deals with MDP with spurious non-determinism, where the way it is resolved does not affect the desired property.
%The general non-deterministic case is treated in \cite{DBLP:conf/rss/FuT14,BCC+14}, yielding PAC guarantees.
%However, the former requires the knowledge of mixing time, which is at least as hard to compute; the algorithm in the latter is purely theoretical since before a single value is updated in the learning process, one has to simulate longer than the age of universe even for a system as simple as a Markov chain with 12 states having at least 4 successors for some state.
%
%
%\para{Our contribution} is an SMC algorithm with PAC guarantees for (i) MDP and unbounded properties, which runs for realistic benchmarks \cite{qcomp} and confidence intervals in orders of minutes, and (ii) is the first algorithm for stochastic games (SG).
%It relies on different techniques from literature.
%\begin{enumerate}
%	\item The increased practical performance rests on two pillars:
%	\begin{itemize}
%		\item extending early detection of bottom strongly connected components in Markov chains by \cite{DHKP16} to end components for MDP and simple end components for SG;
%		\item improving the underlying PAC Q-learning technique of \cite{Strehl}:
%		\begin{enumerate}
%			\item learning is now model-based with better information reuse instead of model-free, but in realistic settings with the same memory requirements,
%			\item better guidance of learning due to interleaving with precise computation, which yields more precise value estimates.
%			\item splitting confidence over all relevant transitions, allowing for variable width of confidence intervals on the learnt transition probabilities.
%		\end{enumerate}
%	\end{itemize}
%	\item The transition from algorithms for MDP to SG is possible via extending the over-approximating value iteration from MDP \cite{BCC+14} to SG by \cite{KKKW18}.
%\end{enumerate}
%To summarize, we give an anytime PAC SMC algorithm for (unbounded) reachability.
%It is the first such algorithm for SG and the first practical one for MDP.
%
%
%
%
%
%
%
%
%
%
%
%\subsection*{Related work}
%
%
%Most of the previous efforts in SMC have focused on the analysis of properties
%with \emph{bounded} horizon
%\cite{Younes02,Sen04,DBLP:journals/sttt/YounesKNP06,DBLP:conf/cmsb/JhaCLLPZ09,DBLP:conf/tacas/JegourelLS12,DBLP:journals/corr/abs-1207-1272}.
%
%SMC of \emph{unbounded} properties was first considered in \cite{vmcai04} and the first approach was proposed in \cite{cav05}, but observed incorrect in \cite{ase10}.
%Notably, in~\cite{sbmf11} two approaches are described.
%The {first approach} proposes to terminate sampled paths at every step with some probability $p_{term}$ and re-weight the result accordingly.
%In order to guarantee the asymptotic convergence of this method, the second eigenvalue $\lambda$ of
%the chain and its mixing time must be computed, which is as hard as the verification problem itself and requires the complete knowledge of the system ({white box} setting).
%The correctness of \cite{DBLP:journals/apal/LassaigneP08}
%relies on the knowledge of the second eigenvalue $\lambda$, too.
%The {second approach} of~\cite{sbmf11} requires the knowledge of the chain's
%topology (grey box), which is used to transform the chain so that all potentially
%infinite paths are eliminated.
%In \cite{ase10}, a similar transformation is performed, again requiring
%knowledge of the topology.
%In \cite{DHKP16}, only (a~lower bound on) the minimum transition probability $\pmin$ is assumed and PAC guarantees are derived.
%While unbounded properties cannot be analyzed without any  information on the system, knowledge of $\pmin$ is a relatively light assumption in many realistic scenarios \cite{DHKP16}.
%For instance, bounds on the rates for reaction kinetics in chemical reaction systems are typically known; for models in the PRISM language \cite{prism}, the bounds can be easily inferred without constructing the respective state space.
%In this paper, we thus adopt this assumption.
%
%
%
%	In the case with general \emph{non-determinism},
%	one approach is to give the non-determinism a probabilistic semantics,
%	e.g., using a uniform distribution instead, as for timed automata
%	in \cite{DBLP:conf/formats/DavidLLMPVW11,DBLP:conf/cav/DavidLLMW11,DBLP:conf/ifm/Larsen13}.
%	Others~\cite{LP12,DBLP:conf/qest/HenriquesMZPC12,BCC+14} aim to quantify over all strategies and produce an $\epsilon$-optimal strategy.
%	In \cite{DBLP:conf/qest/HenriquesMZPC12}, candidates for optimal strategies are generated and gradually improved, but ``at any given point we cannot quantify how close to optimal the candidate scheduler is'' (cited from~\cite{DBLP:conf/qest/HenriquesMZPC12}) and the algorithm
%	%does not estimate the maximum probability of the property''
%	``does not in general converge to the true optimum''
%	(cited from~\cite{DBLP:conf/sefm/LegayST14}). Further, \cite{DBLP:conf/sefm/LegayST14,DBLP:journals/sttt/DArgenioLST15,DBLP:conf/isola/DArgenioHS18} randomly sample compact representation of strategies, resulting in useful lower bounds if $\varepsilon$-schedulers are frequent.
%	\cite{DBLP:conf/tacas/HahnPSSTW19} gives a convergent model-free algorithm (with no bounds on the current error) and identifies that the previous \cite{DBLP:conf/cdc/SadighKCSS14} ``has two faults, the second of which also affects approaches [...] \cite{DBLP:journals/corr/abs-1801-08099,DBLP:journals/corr/abs-1902-00778}''.
%%
%
%	Several approaches provide SMC for MDPs and unbounded properties with \emph{PAC guarantees}.
%Firstly, similarly to \cite{DBLP:journals/apal/LassaigneP08,sbmf11}, \cite{DBLP:conf/rss/FuT14} requires (1) the mixing time $T$ of the MDP. The algorithm then yields PAC bounds in time polynomial in $T$ (which in turn can of course be exponential in the size of the MDP). Moreover, the algorithm requires (2) the ability to restart simulations also in non-initial states, (3) it only returns the strategy once all states have been visited (sufficiently many times), and thus (4) requires the size of the state space $|S|$.
%Secondly, \cite{BCC+14}, based on delayed Q-learning (DQL) \cite{Strehl}, lifts the assumptions (2) and (3) and instead of (1) mixing time requires only (a bound on) the minimum transition probability $\pmin$.
%Our approach additionally lifts the assumption (4) and allows for running times faster than those given by $T$, even without the knowledge of $T$.
%
%
%
%    Reinforcement learning (without PAC bounds) for stochastic games has been considered already in \cite{DBLP:journals/mor/LakshmivarahanN81,DBLP:conf/icml/Littman94,DBLP:conf/ijcai/BrafmanT99}.
%	\cite{DBLP:conf/ijcai/WenT16} combines the special case of almost-sure satisfaction of a specification with optimizing quantitative objectives.
%	We use techniques of \cite{KKKW18}, which however assumes access to the transition probabilities.
%
%
%
%
%


\documentclass{article}
  \usepackage[utf8]{inputenc}
  \usepackage{mathtools}
  \usepackage{listings}
  \usepackage{graphicx}
  \usepackage{url}
  \title{The on-chain claiming mechanism economics}
  \author{Piotr Kosiński\thanks{Thanks to Vijay Kandy, Piper Merriam, Logan Saether for the contribution.}\\ChronoLogic}
  \date{\today\\v1.1}
  \begin{document}
  \maketitle
  \begin{abstract}
    Running a service like a TimeNode can be profitable for the operators, however, given the many variables that influence its payout,  it's not trivial to estimate the expected profits. The proposed model and simulation code has been created to address that problem.
  \end{abstract}
  \section{Introduction}
  This article describes the economic model for on-chain claiming mechanism used by the Ethereum Alarm Clock protocol. This mechanism has been implemented in order to improve the economics of running the \textbf{TimeNode}\footnote{TimeNode is an off-chain executing agent that scans the blockchain for new scheduled transactions and attempts the execution}.

  Given that scheduled transactions are expected to be executed at the exact time and the network of $n$ competing nodes exists, we expect to face the "swarming" problem which can be described as: \textit{uncoordinated attempts of execution by $n$ nodes at the same time}. This problem may result in unnecessary costs\footnote{due to the fact that all transactions on the Ethereum network costs gas, successful and failed} for \textbf{TimeNode} making the operations potentially not profitable as only 1-of-$n$ is going to earn the TimeBounty for the execution, other \textbf{TimeNodes} trying to execute at the same block will pay the failing transaction cost.

  By introducing payment modifier $P_{mod}$ \textbf{TimeNode} operators are able to pick theirs profitability point, this effectively allows to mitigate the "swarming" problem as we expect \textbf{TimeNodes} to try to claim on different blocks/moment of time.
  \section{Claiming mechanism}
  Claiming mechanism can be described as follows:
  For any transaction $Tx$ that has been deployed to the network and is expected to be executed by 1 of $n$ nodes in the network. The process of execution is divided into two steps:
  \textbf{claiming} and \textbf{execution}. Claiming is a process of reserving the transaction for further execution.
  \begin{figure}
    \includegraphics[width=\linewidth]{timeranges.png}
    \caption{Scheduled transaction life cycle}
    \label{fig:boat1}
  \end{figure}

  \subsection{Claiming}
  \begin{itemize}
  \item can happen before execution
  \item every node has the same chance to successfully claim $Tx$
  \item claiming requires a $Deposit$ to be locked by claimant
  \item $Deposit$ is lost by claimant when execution won't happen within \textit{exclusive execution window}
  \item every node can fail on claiming when $Tx$ was already claimed
  \item claiming requires sending transaction that has cost described as $C_{c}$ when success and $C_{f}$ when failure
  \item payment modifier $P_{mod}(t)=\begin{cases}
  0 & \quad \text{at beginning of claiming window}\\
  1 & \quad \text{at end of claiming window}
  \end{cases}$
  \item claiming is optional
  \end{itemize}

  \subsection{Execution}
  \begin{itemize}
  \item successful execution has reward described as $TimeBounty$
  \item execution cost is reimbursed by the scheduler when successful
  \item execution cost has cost described as $C_{e}$ when not successful
  \item $Deposit$ locked by the claimant can be acquired by a node when the claimant failed to execute
  \end{itemize}

  \subsection{Expected payout definition}
  Let's define the expected payout for node as
  \[
  P=P_{s}+P_{f}+P_{nf}
  \]
  where
  \\

  $P_{s}$ is expected payout after successful claiming and execution
  \\

  $P_{f}$ is expected payout after successful claiming and missed execution
  \\

  $P_{nf}$ is expected payout after other node losing the deposit
  \\

  \subsubsection{Network with $n=1$ nodes}
  For network of nodes with $n=1$ we can define expected payouts as:
  \[
  P_{s}(P_{mod})=P_{mod} \times TimeBounty-C_{C}
  \]
  \[
  P_{f}=-C_{C}-Deposit
  \]
  \[
  P_{nf}=TimeBounty-C_{C}
  \]
  \subsubsection{Network with $n>1$ nodes}
  For that case the expected reward will be $\frac{TimeBounty}{n}$ assuming that probability of successful claiming is equal for all nodes. Also in case of failing transaction node will pay the $C_{Tx}$
  \[
  P_{s}(P_{mod})=\frac{P_{mod} \times TimeBounty-C_{C}}{n} - (n-1) \times C_{Tx}
  \]
  \[
  P_{f}=-C_{C}-Deposit
  \]
  \[
  P_{nf}=\frac{TimeBounty+Deposit}{n} - (n-1) \times C_{Tx}
  \]

  In order to improve the cost of failing transactions let's introduce a mechanism $X$ that prevents sending the transaction that will fail, the accuracy of mechanism $X$ is defined as $A_{X} \in [0;1]$

  \[
  P_{s}(A_{X}, P_{mod})=\frac{P_{mod} \times TimeBounty-C_{C}}{n} - (1-A_{X}) \times (n-1) \times C_{Tx}
  \]
  \[
  P_{f}=-C_{C}-Deposit
  \]
  \[
  P_{nf}(A_{X})=\frac{TimeBounty+Deposit}{n} - (1-A_{X}) \times (n-1) \times C_{Tx}
  \]

  The last part is to introduce $P_{ld} \in [0;1]$ as probability of TimeNode loosing the $Deposit$

  \[
  \label{eq}
  P(A_{X}, P_{ld}, P_{mod})=P_{s}(A_{X}, P_{mod}) \times (1-P_{ld}) + (P_{f}(A_{X})+P_{nf}(A_{X})) \times P_{ld}
  \]
  \section{Simulation}
  We are going to simulate few cases using equation formulated in section \ref{eq}. $A_{X}, P_{ld}, P_{mod}$ are the variables that depends on reliability and running costs by TimeNode owners. All calculated results are going to be represented as \textit{gas}

  Let' now define the profitability threshold, we assume monthly running cost for the TimeNode is 7USD (this is based on current rates on Heroku cloud). Translating 7USD to \textit{gas} we get:

  \begin{align*}
  ETH/USD=700USD\\
  Gas Price = 10Gwei\\
  7USD \equiv 1000000gas
  \end{align*}

  This shows us that running costs are covered after acquiring 1000000\textit{gas}. Now let's take a look at how this translates to the amount of executed transactions. In order to calculate $P(A_{X}, P_{ld}, P_{mod})$ we use the script listed below. More over we are using these values describing TimeNode operations and network conditions.
  \begin{align*}
  &n \approx 8\\
  &TimeBounty \approx 300000gas\\
  &Deposit \approx 600000gas\\
  &A_{X} \approx 0.95 (95\%)\\
  &P_{ld} \approx 0.1 (1\%)\\
  &C_{C} = 90000gas\\
  &C_{Tx} = 25000gas\\
  &target = 1000000gas~or~7USD
  \end{align*}
  Results using these parameters are:
  \begin{verbatim}
       p_mod       res  tx
    15  0.70  1428.250 700
    16  0.75  3178.094 314
    17  0.80  4450.974 224
    18  0.85  5951.569 168
    19  0.90  7394.698 135
    20  0.95  8980.225 111
    21  1.00 10402.272  96
  \end{verbatim}
  Results data frame contains 3 columns:
  \begin{itemize}
  \item \textbf{p\_mod} - payment modifier ($P_{mod}$)
  \item \textbf{res} - expected amount of gas earned by TimeNode per transaction
  \item \textbf{num\_of\_tx} - number of transactions to be executed in order to cover running costs
  \end{itemize}

  The results achieved by running this simulation should be treated informational rather than something taken for granted. The expected payout depends on all of the variables described above, for a simulation purpose we picked values using our intuition.

  The variable controlled by the TimeNode operator is $P_{mod}$ which is the major component. Low enough $P_{mod}$ allows TimeNode to claim transaction before others, still, in order to be profitable using low $P_{mod} $, the TimeNode running cost has to be low.

  The TimeNode market has many characteristics of the perfect competition market\footnote{\url{http://www.economicsonline.co.uk/Business_economics/Perfect_competition.html}}: there is perfect information, no barriers to entry, they deliver the same service, TimeNode are the price takers. Based on that, long-term we may get in the situation where marginal cost is equal average cost.
  \appendix
  \section{Simulation source code}
  \lstinputlisting[language=R, caption=Siml.R]{onchain_claiming_mechanism.R}

  \end{document}


%\documentclass[useAMS, usenatbib,usegraphicx,letter]{mn2e}
%\documentclass[11pt]{article}
\documentclass[5p]{elsarticle}

\usepackage{aas_macros}
\usepackage{epsfig,amsmath,natbib}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{dsfont}
\usepackage{hyperref}
\usepackage{color}
\usepackage{pbox}
\usepackage{booktabs}

\bibliographystyle{elsarticle-num}

\hypersetup{
	colorlinks=false,
	citecolor=green
}
% \usepackage{graphicx}
% \usepackage{epstopdf}
% \usepackage{natbib}

%%%%%%%%%%%%%%%%%
%Custom commands%
%%%%%%%%%%%%%%%%%

\newcommand{\bb}[1]{\mathbf{#1}}
\newcommand{\bbh}[1]{\mathbf{\hat{#1}}}
\newcommand{\h}[1]{\hat{#1}}
\newcommand{\ttt}[1]{\texttt{#1}}
\newcommand{\LT}{\texttt{LensTools} }

\newcommand{\specialcell}[2][c]{\begin{tabular}[#1]{@{}c@{}}#2\end{tabular}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\title{Mocking the Weak Lensing universe: the LensTools python computing package}

\author[cu,bnl]{Andrea Petri}
\ead{apetri@phys.columbia.edu}

\address[cu]{Department of Physics, Columbia University, New York, NY 10027, USA}
\address[bnl]{Physics Department, Brookhaven National Laboratory, Upton, NY 11973, USA}

\date{\today}

\label{firstpage}

\begin{abstract}
We present a newly developed software package which implements a wide range of routines frequently used in Weak Gravitational Lensing (WL). With the continuously increasing size of the WL scientific community we feel that easy to use Application Program Interfaces (APIs) for common calculations are a necessity to ensure efficiency and coordination across different working groups. Coupled with existing open source codes, such as \ttt{CAMB}\citep{CAMB} and \ttt{Gadget2}\citep{Gadget2}, \LT brings together a cosmic shear simulation pipeline which, complemented with a variety of WL feature measurement tools and parameter sampling routines, provides easy access to the numerics for theoretical studies of WL as well as for experiment forecasts. Being implemented in {\sc python}\citep{python}, \LT takes full advantage of a range of state--of--the art techniques developed by the large and growing open--source software community \citep{scipy,pandas,astropy,scikit-learn,emcee}. We made the \LT code available on the Python Package Index and published its documentation on \url{http://lenstools.readthedocs.io}

\end{abstract}

\begin{keyword}
Weak Gravitational Lensing \sep Simulations
\PACS 98.80.-k \sep 95.36.+x \sep 95.30.Sf \sep 98.62.Sb
\end{keyword}

\maketitle

%%%%%%%%%%%%%%%%%%%%%%%%%% INTRO %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Introduction}
%
Cosmology is entering a data driven era. After the Cosmic Microwave Background (CMB) \citep{WMAP,PlanckXVI2013} provided strong experimental evidences of cosmological theories, a variety of different probes have been proposed to unveil the secret of the cosmos. Weak Gravitational Lensing uses the correlation between image distortions of background sources by Large Scale Structure (LSS) to infer cosmological parameter values \citep{WLprimer}. Because WL probes are sensitive to late universe physics, where the density fluctuations are in the non--linear regime, quadratic features such as two--point correlation functions might miss some of the cosmological information. In the theoretical study of more complicated WL features (see for example \citep{3pcf1,bispectrum1,moments1,peaks1} for a non comprehensive list) simulation pipelines play a vital role, as in general these features cannot be predicted analytically from cosmological parameters. In this work we present a flexible, customizable and easy to deploy WL simulation pipeline that bridges the gap between simulations of shear fields, feature measurement from simulated images and cosmological parameter estimation. The paper is organized as follows: first we give an overview of the shear field simulation routines and present their runtime and memory usage benchmarks. We then outline the \LT image analysis capabilities as well as the parameter estimation routines. We finally present a summary of our work and outline our conclusions. We complement our work with some illustrative coding examples that show how to operationally use \LT for some of the former tasks.

%%%%%%%%%%%%%%%%%%% Shear simulations %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Shear simulations}
\label{shearsim}

\subsection{Formalism}
%
In this paragraph we give an overview of the \LT shear field simulation pipeline. This consists in a series of routines that, starting from a $w$CDM cosmological model specified by the cosmological parameters
\begin{equation}
\bb{p}=(h,\Omega_m,\Omega_\Lambda,\Omega_b,w_0,w_a,n_s,\sigma_8)
\end{equation}
%
produces random realizations of shear fields $\pmb{\gamma}(\pmb{\theta})$ in cosmology $\bb{p}$. Here $\pmb{\theta}=(\theta_x,\theta_y)$ is the angle on sky as seen from the observer. Given a background source (such as a galaxy) at redshift $z_s$, the dark matter density fluctuations $\delta(\bb{x},z)$ between the observer and the source will cause its apparent shape to be distorted due to the gravitational lensing effect, as predicted by General Relativity \citep{wlreview}. The apparent source ellipticity, assuming the unperturbed shape is a circle, can be estimated in terms of the cosmic shear $\pmb{\gamma}$ defined in equation (\ref{convsheardef}). Non circular shapes can be modeled with redshift--dependent shape noise terms \citep{wlreview}, the treatment of which goes beyond the scope of this paper. The multi--lens--plane algorithm \citep{RayTracingHartlap,Ray1,Ray2,Ray3} is a popular technique to compute light ray deflections across the path $z\in[z_s,0]$ and hence to compute the apparent source shape distortion. The mass distribution between the source and the observer is approximated as a finite set of two dimensional lenses perpendicular to the line of sight, of thickness $\Delta$ and a surface density $\sigma$ given by

\begin{equation}
\label{surfacedensity}
\sigma(\bb{x},z) = \frac{3H_0^2\Omega_m\chi(z)}{2c^2a(z)}\int_\Delta d\chi'\delta\left(\bb{x},z(\chi')\right)
\end{equation}
%
where $\chi$ is the lens comoving distance and $a=1/(1+z)$ the scale factor. A light ray crossing a lens at redshift $z$ at a transverse position $\bb{x}$ will be deflected by a small angle $\pmb{\alpha}$ which can be shown to be the gradient of the 2D gravitational potential $\phi$ (see again \citep{RayTracingHartlap})
%
\begin{equation}
\label{poisson}
\nabla^2_\bb{x} \phi(\bb{x},z) = 2\sigma(\bb{x},z)
\end{equation}

\begin{equation}
\pmb{\alpha}(\bb{x},z) = \nabla_\bb{x} \phi(\bb{x},z)
\end{equation}

\begin{equation}
\bb{T}(\bb{x},z) = \nabla_\bb{x}\nabla_\bb{x}^T \phi(\bb{x},z)
\end{equation}
%
where we indicated $\bb{T}$ as the gradient of the deflection, which will be called distortion tensor throughout the rest of the paper. An example of a lens plane computed with the \LT pipeline according to equations (\ref{surfacedensity}),(\ref{poisson}) is shown in Figure \ref{lensplanefig}.

\begin{figure*}
\begin{center}
\includegraphics[scale=0.35]{Figures/lens_plane_density.eps}
\includegraphics[scale=0.35]{Figures/lens_plane_potential.eps}
\caption{This figure shows a lens plane, computed with a gridding procedure based on equations (\ref{surfacedensity}),(\ref{poisson}). This lens plane has been generated from a $N$--body simulation of size $L_b=480\,{\rm Mpc}/h$ with $N_p=1024^3$ particles at $z_s=2$. The plane resolution is $512\,\,$pixels per side. We show both the surface density $\sigma$ (left) and the lensing potential $\phi$ (right). Because the solution of the Poisson equation (\ref{poisson}) is computed using FFTs, there is an underlying assumption about periodic boundary conditions in the lensing potential reconstruction, which can introduce edge effects as can be seen in the corners of the potential plot. Because of these edge effects, it is a good idea to choose the field of view for the ray--tracing to be smaller than the potential field by about $\sim$ 50 pixels per side.}
\label{lensplanefig}
\end{center}
\end{figure*}
%
The trajectory of a light ray $\bb{x}(z)$ follows the geodesic equation

\begin{equation}
\label{geodesic3D}
\frac{d^2\bb{x}}{d\chi^2} = -\frac{2}{c^2}\nabla_\bb{x_\perp}\Phi(\bb{x},z)
\end{equation}
%
which can be translated into a second order differential equation for the light ray angular position $\pmb{\beta}(z)=\bb{x}_\perp(z)/\chi(z)$ as seen from the observer. Following \citep{RayTracingHartlap}, the trajectory of each light ray originating at $\pmb{\beta}(0)=\pmb{\theta}$ can be calculated solving numerically a discretized version of (\ref{geodesic3D}), assuming a finite number of lenses placed at redshifts $\{z_k\}$:
%

\begin{equation}
\label{geodesic2D}
\pmb{\beta}_{k} = \pmb{\theta} + \sum_{i=1}^k\delta\pmb{\beta}_i
\end{equation}

\begin{equation}
\label{deflections}
\delta\pmb{\beta}_{k+1} = (B_k-1)\delta\pmb{\beta}_k + C_k\pmb{\alpha}_k \,\, ; \,\, \delta\pmb{\beta}_0=0
\end{equation}

\begin{equation}
\label{jacobian}
\bb{A}_{k} = \mathds{1}_{2\times2} + \sum_{i=1}^k\delta\bb{A}_i
\end{equation}

\begin{equation}
\label{sheartensorproduct}
\delta\bb{A}_{k+1} = (B_k-1)\delta\bb{A}_k + C_k\bb{T}_k\bb{A}_k \,\, ; \,\, \delta\bb{A}_0=0
\end{equation}
%
where $\bb{A}$ is the Jacobian matrix of the trajectory $\pmb{\beta}$ with respect to the initial light ray position $\pmb{\theta}$, $\bb{A}_k=\nabla_{\pmb{\theta}}\pmb{\beta}_k(\pmb{\theta})$. The factors $B_k,C_k$ depend on the geometry of the lens system

\begin{equation}
B_k = \frac{\chi_{k}}{\chi_{k+1}}\left(1 + \frac{\chi_{k+1}-\chi_{k}}{\chi_{k}-\chi_{k-1}}\right) \,\,\,\, ; \,\,\,\, C_k = \frac{\chi_{k}}{\chi_{k+1}} - 1 \\ \\
\end{equation}
%
where we use the subscript $k$ to indicate the redshift $z_k$ of the $k$--th lens for notational simplicity. After tracing the evolution of $\bb{A}$ from the observer to the source at $z_s$, we are able to evaluate the cosmic shear $\pmb{\gamma}$ and convergence $\kappa$ at $z_s$ looking at the components of $\bb{A}$
\begin{equation}
\label{convsheardef}
\bb{A}(\pmb{\theta},z_s) =
\begin{pmatrix}
1-\kappa(\pmb{\theta}) - \gamma_1(\pmb{\theta}) & -\gamma_2(\pmb{\theta}) \\
-\gamma_2(\pmb{\theta}) & 1-\kappa(\pmb{\theta}) + \gamma_1(\pmb{\theta})
\end{pmatrix}
\end{equation}
%
The solution of equation (\ref{geodesic3D}) via the multi--lens--plane algorithm yields a single realization of the WL fields $(\kappa,\pmb{\gamma})$. Multiple random realizations of these fields can be obtained by altering the lens system before ray--tracing using the randomization technique described in \citep{Petri16}.

The iterative solution of equations (\ref{geodesic2D})--(\ref{sheartensorproduct}) requires knowledge of the density fluctuation $\delta(\bb{x},z)$, from which the lens surface density $\sigma$ and gravitational potential $\phi$ can be inferred through equations (\ref{surfacedensity}),(\ref{poisson}). $\delta$ can be calculated running numerical simulations, such as $N$--body simulations \citep{Gadget2,HACC} or hydrodynamical simulations \citep{Flash}. In this paragraph we focus mainly on $N$--body simulations, in which the matter distribution in the universe is approximated as a set of $N_p$ particles of mass $M_p$, which move in their self--generated gravitational field. For this purpose we use the publicly available code \ttt{Gadget2}\citep{Gadget2}, although alternatives can be adopted (see \citep{HACC} for example). Once the $N$--body simulations are run, \LT provides a {\sc python} implementation of the multi--lens--plane algorithm \citep{RayTracingHartlap} described above, which takes care of projecting the density fluctuation on two dimensional lenses as in (\ref{surfacedensity}), solving the Poisson equation as in (\ref{poisson}), and computing the light ray deflections as in (\ref{geodesic2D})--(\ref{sheartensorproduct}). An overview of the pipeline operations, from the cosmological parameter specifications to the final shear map products, is outlined in Figure \ref{pipescheme}. We make the claim that the way the \LT code is organized makes it portable between different research groups that rely on different choices, rather than \ttt{Gadget2}, for running $N$--body simulations. The transition between snapshots and lens planes is handled by instances of the \ttt{NbodySnapshot} class, which implements the algorithms based on equations (\ref{poisson}). \ttt{NbodySnapshot} can be sub--classed to implement the necessary input routines from the $N$--body snapshots. \LT comes with one of such possibilities, the \ttt{Gadget2Snapshot} class, that handles input from snapshots in the \ttt{Gadget2} binary format. Other subclass types that allow input from other formats can be coded by the user with minimal effort.

\begin{figure*}
\includegraphics[scale=0.65]{Figures/flow.eps}
\caption{Workflow of the \LT WL shear simulation operations, from the specifications of the cosmological parameters $\bb{p}$ to the finished image products. The diagram has to be read from left to right: the arrows originating from the grey nodes mean that the executable is run with the target node as an input. The produced output is passed down the pipeline on the right. Matter power spectra at high redshift are computed with \ttt{CAMB} and are used to generate the initial conditions for the $N$--body simulations (for which we use the \ttt{NGen-IC} add--on to the \ttt{Gadget2} code). These initial conditions, each with a different random seed, are then evolved in time with \ttt{Gadget2}. After the $N$--body snapshots are written to disk, \LT slices them into two dimensional lens planes. The slicing is done by an executable named \ttt{lenstools.planes}, which is a convenient wrapper for the operations implemented by the \ttt{NbodySnapshot} class. The ray--tracing operations are carried on by an executable named \ttt{lenstools.raytracing}, which conveniently wraps the operations implemented by the \ttt{RayTracer} class. From the final positions of the light rays the convergence $\kappa$ and the cosmic shear $\pmb{\gamma}$ can be inferred. Details on how to use the \ttt{lenstools.planes} and \ttt{lenstools.raytracing} executables, that come with \LT, can be found in the \LT documentation at the URL \url{http://lenstools.readthedocs.io/en/latest/pipeline.html}.}
\label{pipescheme}
\end{figure*}

\subsection{Pipeline code structure}
%
In this paragraph we describe how the \LT pipeline code is organized. The simulation products are placed in a directory tree structure designed for easy resource access. The directory tree is mirrored in two locations, a so called \textit{Home} location, which holds all the book--keeping information such as small data files (such as \ttt{CAMB} matter power spectra) and configuration files, and a \textit{Storage} location which holds the simulation products (\ttt{Gadget2} snapshots, lens planes and finished shear maps). The reason for this is that while the \textit{Home} location does not require much disk space, the \textit{Storage} location can reach disk sizes of several Terabytes. We found it convenient to keep the two locations separated to simplify sharing data among machines in a cluster, and help with portability issues among different clusters.
A batch of simulations is handled by a single instance of a \ttt{SimulationBatch} object (which holds both the \textit{Home} and \textit{Storage} parts). The first level in the tree corresponds to a choice of values for cosmological parameters $\bb{p}$: each node on this level of the tree is an instance of the \ttt{SimulationModel} class. The second level in the tree specifies the size and resolution of the $N$--body box, namely the box size $L_b$ and the number of particles $N_p$: each node on the second level corresponds to a \ttt{SimulationCollection} object. Inside a simulation collection, we are free to choose different random realizations of the initial conditions, which will then be evolved in time by the $N$--body code. Each such realization lives on a node which is one level deeper in the tree, and is encoded in a \ttt{SimulationIC} object. The deepest level in the tree contains the two dimensional slices of the $N$--body simulation boxes. Each node on this level is an instance of the \ttt{SimulationPlanes} class. Once the lens planes are generated, the ray--tracing operations can be performed. In principle we can use lens planes that live under the same \ttt{SimulationCollection}, but belong to different \ttt{SimulationIC} nodes, to produce either single redshift shear images (each ensemble of images lives in a \ttt{SimulationMaps} object) or shear catalogs of $N_g$ sources, in the form of a table in which each of the $N_g$ rows is a tuple $(x_g,y_g,z_g,\gamma_{1,g},\gamma_{2,g})$. Each ensemble of catalogs corresponds to an instance of the \ttt{SimulationCatalog} class. Note that, because they combine lens planes with different initial random seed (see \citep{Petri16}), both \ttt{SimulationMaps} and \ttt{SimulationCatalog} nodes live on the same level of the directory tree, one level below \ttt{SimulationCollection}. An example on how to create a {\sc python} script to lay down such a directory tree is available in {\sc IPython} notebook format\footnote{\url{http://nbviewer.jupyter.org/github/apetri/LensTools/blob/master/notebooks/dirtree.ipynb}}. A comprehensive guide on how to deploy the \LT simulation pipeline on a computer cluster can be found in the code documentation \citep{lenstoolsdocs}.

\subsection{Performance}
%
We summarize the runtime and memory usage benchmarks of the \LT shear simulation pipeline. The tests were run on the XSEDE Stampede computer cluster\footnote{\url{https://portal.xsede.org/tacc-stampede}}. Table \ref{benchmarktable} shows a summary of the ray--tracing operations performed by \LT, indicating the complexity and runtime of each operation for a selected test case. At the lens plane generation stage we can clearly see that the two bottlenecks in the flow are the read operations from $N$--body snapshot files and the Poisson equation solving via FFT. The number of tasks $N_t$ used to read in a single snapshot can be optimized if the parallel input performance is ideal (i.e independent on $N_t$), as is an optimal value of $N_t(N_p,L_p)$ that minimizes the combined input, gridding and \ttt{MPI} communication operations, which have a combined complexity

\begin{equation}
t_{\rm in+grid+MPI} = A_1\frac{N_p}{N_t} + A_2L_p\log{N_t}
\end{equation}
%
The optimal $N_t$ depends on the number of particles $N_p$ and the lens plane resolution (in pixels) $L_p$. In principle this bottleneck can be removed if the capability of generating lens planes is embedded into the $N$--body code, which could avoid saving the intermediate 3D snapshots to disk. \LT allows such a possibility by creating a channel of communication between \ttt{Gadget2} and the plane application using named pipes. For this option to be viable, \ttt{Gadget2} and the plane computation must run on the same node.

The Poisson solver has a complexity of $O(L_p\log{L_p})$ which is dominated by FFT performance. Although we make use of the \ttt{numpy} FFT pack \citep{scipy} to perform such operations, other alternatives are also possible (such as FFTW \citep{FFTW05}). The \LT code modularity makes it very easy to switch between different implementations of the FFT algorithm, both coming from external libraries or coded up by the user.
The bottleneck of the ray--tracing operations consists in the calculations of the ray deflections in equation (\ref{deflections}) and the distortion tensor products (\ref{sheartensorproduct}). Although the complexity of these operations is already optimal, improvements on the runtime can be made by using specialized libraries to handle matrix products. \ttt{numpy} can link to the most up--to--date version of specialized libraries such as \ttt{LAPACK} \citep{lapack} and \ttt{Intel MKL} \citep{intel-mkl}. Some versions of \ttt{numpy} even support automatic offload of linear algebra operations to \ttt{Intel XEON Phi} co--processors \citep{xeonphi}.

We tracked the memory usage of the lens plane and ray--tracing operations. This is an important step, since {\sc python} has some subtleties when dealing with large memory applications. Memory allocated by a {\sc python} process cannot be released manually as in {\sc C}, but is managed by the built--in garbage collector instead. Figure \ref{memoryfig} shows the peak memory usage during the lens plane generation and ray--tracing operations. We can see that, for the test case outlined in Table \ref{benchmarktable}, memory consumption stabilizes around 1.3\,GB per task for the lens planes and 1.8\,GB per task during the ray--tracing. These considerations make the \LT pipeline suitable for deployment on computer clusters with $\gtrsim$2\,GB memory per core, such as the one we used. These numbers refer to the test case described in Table \ref{benchmarktable}. Producing higher resolution lens planes and WL maps will in general require more memory. A rough estimate for the memory scaling with resolution can be made noting that the dominant contribution to the memory usage for the lens planes comes from the input from the $N$--body simulations and should hence scale with the number of particles $N_p$. For the ray--tracing, on the other hand, the dominant contribution to the memory usage comes from the lenses and should hence scale as the lens pixel resolution $L_p$, also because the resolution of the WL maps $N_r$ needs to be smaller than $L_p$.

\begin{table*}
\begin{center}
\begin{tabular}{l|c|c|c}
\toprule
{Step} &            Complexity &            Test case &           Runtime \\ \hline \hline
\midrule
\multicolumn{4}{c}{\textbf{Lens generation}} \\ \hline
Snapshot input & $O(N_p/N_t)$  & $N_p=512^3$, $N_t=16$  & 2.10\,s  \\
Gridding        & $O(N_p/N_t)$   & $N_p=512^3$, $N_t=16$  & 0.20\,s \\
\ttt{MPI} Communication  & $O(L_p\log{N_t})$   & $N_t=16$, $L_p=4096^2$  & 0.76\,s   \\
Poisson solver (FFT)           & $O(L_p\log{L_p})$ & $L_p=4096^2$  &  2.78\,s    \\
Lens output           & $O(L_p)$ & $L_p=4096^2$   & 0.04\,s  \\ \hline \hline

\multicolumn{4}{c}{\textbf{Ray tracing}} \\ \hline
Lens input &  $O(L_p)$ & $L_p=4096^2$ & 0.32\,s \\
Random lens shift &  $O(L_p)$ & $L_p=4096^2$ & 0.15\,s \\
Deflection calculation        &  $O(N_r)$ & $N_r=2048^2$   & 1.54\,s  \\
Shear tensor product               &  $O(N_r)$ & $N_r=2048^2$   &  1.29\,s \\ \hline \hline

\bottomrule
\end{tabular}
\caption{Summary of the ray--tracing benchmarks: each $N$--body snapshot is divided in $N_t$ files, which are read in parallel and contain a total of $N_p$ particles (perfect input performance is assumed in the complexity analysis). After the gridding procedure (\ref{surfacedensity}) is performed by each task, the total sufrace density (computed for a plane of $L_p$ pixels) is collected by the master task, which then proceeds in solving the Poission equation (\ref{poisson}) via Fast Fourier Transforms and saves the output to disk. In a subsequent step, the lens potential files are read from disk, and the geodesic equations (\ref{geodesic2D}) are solved for $N_r$ different starting positions $\pmb{\theta}$ that allow to reconstruct the shear and convergence fields $\pmb{\gamma},\kappa$. The numbers refer to tests conducted on the XSEDE Stampede cluster. Parallel operations are implemented with \ttt{mpi4py} \citep{mpi4py}, a {\sc python} wrapper of the \ttt{MPI} library \citep{MPI}.}
\label{benchmarktable}
\end{center}
\end{table*}

\begin{figure}
\begin{center}
\includegraphics[scale=0.3]{Figures/memory_usage.eps}
\caption{Peak memory usage for the lens plane generation (black) and ray--tracing (red) as a function of runtime $t$ for the test case indicated in Table \ref{benchmarktable}. The vertical lines are drawn in correspondence of the completion of a lens plane calculation (black) and a lens crossing (red).}
\label{memoryfig}
\end{center}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Image analysis}
%
In this section we describe some useful routines that \LT provides for analyzing simulated convergence and shear fields. The \ttt{ConvergenceMap} and \ttt{ShearMap} classes implement several operations that can be performed on two dimensional $\kappa,\pmb{\gamma}$ images (for a complete list look at the \LT documentation \citep{lenstoolsdocs}). Both classes allow flexible I/O formats from files through the \ttt{load} method (the FITS format \citep{cfitsio} is a popular choice, but not the only one possible; user custom format can be easily dealt within \LT). Efficient routines are available for smoothing the maps with Gaussian kernels (optimal smoothing complexity--wise can be performed via FFT for kernel sizes bigger than $\sim 10$ pixel, otherwise real--space techniques are preferrable. \LT allows for both possibilities.), measuring the pixel PDF, counting the local maxima and measuring their position, measuring topological descriptors such as Minkowski Functionals \citep{MatsubaraMink}. An example of such operations is shown in Figure \ref{convergencefig}.

\begin{figure*}
\begin{center}
\includegraphics[scale=0.35]{Figures/convergence_visualize.eps}
\includegraphics[scale=0.3]{Figures/convergence_stats.eps}
\caption{One of the convergence maps produced with the \LT shear simulation pipeline. The map has been generated assuming a uniform background source distribution at $z_s=2$ and has an angular size of $3.5^\circ$ and a resolution of 2048 pixels per side, which correspond to a pixel resolution of $0.1^\prime$. The black crosses in the left panel identify local maxima (\textit{peaks}) in the $\kappa$ field with a significance of at least 2$\sigma$. The right panel shows the PDF of the $\kappa$ field (red) and its peak histogram (black). The code to produce this figure is available in {\sc IPython} notebook format at \url{http://nbviewer.jupyter.org/github/apetri/LensTools/blob/master/notebooks/image.ipynb}.}
\label{convergencefig}
\end{center}
\end{figure*}
%
In addition to real space statistics such as the ones outlined before, \LT provides access to quadratic Fourier statistics such as power spectra. The convergence angular power spectrum $P^{\kappa\kappa}(\ell)$ is defined as
\begin{equation}
\label{kappapsdefinition}
\langle\tilde{\kappa}(\pmb{\ell})\tilde{\kappa}(\pmb{\ell}^\prime)\rangle = (2\pi)^2\delta_D(\pmb{\ell}+\pmb{\ell}^\prime)P^{\kappa\kappa}(\ell)
\end{equation}
%
where we indicate the Fourier transform of $\kappa(\pmb{\theta})$ as $\tilde{\kappa}(\pmb{\ell})$. The shear field can be decomposed into its $E$ and $B$ components according to

\begin{equation}
\label{ebmodeeqs}
\begin{matrix}
E(\pmb{\ell}) = \frac{(\ell_x^2-\ell_y^2)\tilde{\gamma}^1(\pmb{\ell})+2\ell_x\ell_y\tilde{\gamma}^2(\pmb{\ell})}{\ell_x^2+\ell_y^2}  \\ \\
B(\pmb{\ell}) = \frac{-2\ell_x\ell_y\tilde{\gamma}^1(\pmb{\ell})+(\ell_x^2-\ell_y^2)\tilde{\gamma}^2(\pmb{\ell})}{\ell_x^2+\ell_y^2}
\end{matrix}
\end{equation}
%
Because of the nature of the density perturbations that cause background source lensing, the $E$ component dominates in the weak lensing limit, because $E=O(\phi)$ and $B=O(\phi^2)$. The shear $E$--mode is the convergence $\tilde{\kappa}$ (in Fourier space), hence $P^{EE}(\ell)=P^{\kappa\kappa}(\ell)$. Figure \ref{ebmodefig} shows the shear $E$ and $B$ modes power spectra measured from one realization of the shear field.

\begin{figure}
\includegraphics[scale=0.3]{Figures/eb_modes.eps}
\caption{$E$ and $B$ mode power spectra of one of the shear maps generated with the \LT simulation pipeline. We show the $E$--mode power spectrum $P^{EE}$ (red) and the $B$--mode power spectrum (blue), computed using equation (\ref{ebmodeeqs}) We also show an analytical prediction of the $\kappa$ power spectrum obtained with the public code \ttt{Nicaea} (dashed red line). There is a discrepancy between the simulations and the analytical results at high $\ell$ which has to do with the finite resolution of the $N$-body simulations and of the WL maps. This finite resolution causes the $\kappa$ fields to lack power on small scales.}
\label{ebmodefig}
\end{figure}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Cosmology constraints}

\subsection{Formalism}
%
In this section we outline the basic routines that \LT provides for constraining cosmology. Let $\bb{p}$ be the parameters to constrain (for example $\bb{p}=(\Omega_m,w,\sigma_8)$) and let $\bb{d}$ be a WL feature that contains information on $\bb{p}$. An example of $\bb{d}$ can be the $\kappa$ power spectrum $P^{\kappa\kappa}(\ell)$ defined in (\ref{kappapsdefinition}). Given a feature measurement $\bbh{d}$, we are interested in calculating the likelihood $\mathcal{L}(\bb{p}\vert\bbh{d})$ of the parameters, given the measurement. Using the Bayes theorem we can express the parameter likelihood as
\begin{equation}
\label{parameterlikelihood}
\mathcal{L}(\bb{p}\vert\bbh{d}) = \mathcal{N}_\mathcal{L}\mathcal{L}(\bbh{d}\vert\bb{p})\Pi(\bb{p})
\end{equation}
%
where $\Pi(\bb{p})$ is the prior on the parameters, $\mathcal{N}_\mathcal{L}$ is a $\bb{p}$--independent normalization constant and $\mathcal{L}(\bbh{d}\vert\bb{p})$ is the feature likelihood. A popular choice for the feature likelihood is a normal distribution with mean $\bb{d}(\bb{p})$ and covariance $\bb{C}$, $\mathcal{L}(\bbh{d}\vert\bb{p}) = \exp{\left[-\chi^2\left(\bbh{d}\vert\bb{d}(\bb{p}),\bb{C}\right)/2\right]}$ with

\begin{equation}
\label{chi2definition}
\chi^2\left(\bbh{d}\vert\bb{d}(\bb{p}),\bb{C}\right) = (\bbh{d}-\bb{d}(\bb{p}))^T\bb{C}^{-1}(\bbh{d}-\bb{d}(\bb{p}))
\end{equation}
%
\LT provides efficient routines for computing (\ref{chi2definition}) at arbitrary points $\bb{p}$ in parameter space. We argue that efficient evaluation of $\chi^2$ is not only useful when data likelihoods are Gaussian, but also gives access to more advanced likelihood sampling methods such as Approximate Bayesian Computation \citep{cosmoabc}.
Running the simulation pipeline described in \S~\ref{shearsim} for a variety of cosmological models $\{\bb{p}_i\}$, $i=1...N_M$, gives access to a discrete set of features $\{\bb{d}(\bb{p}_i)\}$ evaluated at $\bb{p}_i$. We are able to evaluate equation (\ref{chi2definition}) at an arbitrary point $\bb{p}$ in parameter space using a Radial Basis Function (RBF) interpolation scheme
\begin{equation}
\label{rbf}
d^{RBF}_i(\bb{p}) = \sum_{j=1}^{N_M}\lambda_{ij} f\left(\left\vert\bb{p}-\bb{p}_j\right\vert;R\right)
\end{equation}
%
where $f$ is an isotropic smoothing kernel of scale $R$ \footnote{For the examples shown in this work we chose a multiquadric kernel $f(x;R)=\sqrt{1+\frac{x^2}{R^2}}$ with $R$ chosen as the mean distance between the simulated points $\bb{p}_k$} and the weights $\lambda_{ij}$ can be determined from the known simulated features $\bb{d}(\bb{p}_j)$ with a matrix inversion
\begin{equation}
\lambda_{ij} = \sum_{k=1}^{N_M}d_i(\bb{p}_k)\left[f\left(\left\vert\bb{p}_k-\bb{p}_j\right\vert;R\right)\right]^{-1}
\end{equation}
%
Once the weights are determined from the simulated features, equation (\ref{rbf}) allows for a fast vectorized evaluation of the parameter likelihood from equations (\ref{parameterlikelihood}),(\ref{chi2definition}) once an assumption for the ($\bb{p}$--independent) covariance matrix $\bb{C}$ is made. A possible choice is estimating the covariance $\bb{C}$ from the shear simulations themselves \footnote{In the case in which the covariance matrix is estimated from simulations, the estimator for its inverse $\bbh{C}^{-1}$ is biased \citep{Taylor12}. \LT sampling routines use the unbiased estimator $\frac{N_r-N_b-2}{N_r-1}\bbh{C}^{-1}$ where $N_r$ is the number of realizations used to estimate the $N_b\times N_b$ feature covariance}. Having access to an efficient routine for computing the parameter likelihood $\mathcal{L}(\bb{p}\vert\bbh{d})$ allow access to a variety of parameter sampling techniques. In this section we give examples of three different parameter sampling techniques supported in \LT:
\begin{enumerate}

\item Likelihood Grid Evaluation: if the dimensionality of the parameter space is not too big, the parameter likelihood in equation (\ref{parameterlikelihood}) can be evaluated on a regularly spaced grid of points. Since each point can be treated independently, this procedure is easily parallelizable. \LT provides a parallel implementation of the likelihood grid evaluation based on the \ttt{MPI} protocol \citep{MPI}. Access to the values of the likelihood on a regularly spaced grid makes the determination of confidence intervals straightforward. Such an approach has been used before in the literature \citep{cfhtpeaks,cfhtmink}.

\item MCMC sampling of the parameter space: the efficient \LT likelihood evaluation routines are specifically designed to be compatible with widely used {\sc python} packages such as \ttt{emcee} \citep{emcee} and \ttt{pymc} \citep{pymc} that specialize in generating parameter samples with the Markov Chain Monte Carlo (MCMC) technique \citep{mcmc}. \ttt{emcee} supports parallel MCMC sampling via \ttt{mpi4py}.

\item Fisher Matrix approximation: for the sake of simplicity, sometimes it is convenient to approximate the parameter likelihood as a Gaussian centered around its maximum. If we have a reasonable guess for the likelihood peak location $\bb{p}_0$ (in the case where the likelihood is single--modal), the parameter covariance matrix can be approximated as
\begin{equation}
{\rm Cov}(p_\alpha,p_\beta) = -\left(\left.\frac{\partial\ln\mathcal{L}(\bb{p}\vert\bbh{d})}{\partial p_\alpha\partial p_\beta}\right\vert_{\bb{p}_0}\right)^{-1}
\end{equation}
%
The partial derivatives of the likelihood with respect to the parameters are easy to evaluate with finite differences.
\end{enumerate}

\subsection{Code structure}

We give a brief overview of the \LT object types that handle parameter constraints operations. Feature emulators in \LT are row--oriented data structures, in which each of the $N_M$ rows contains a tuple of cosmological (and/or nuisance) parameters $\bb{p}_i$ and the simulated feature at $\bb{p}_i$. The base class that handles row--oriented data in \LT is the \ttt{Ensemble} class. \ttt{Ensemble} inherits from \ttt{pandas.DataFrame} \citep{pandas} and provides additional routines for feature measurement from simulations (both serial and parallel in the number of $\kappa$ maps with \ttt{mpi4py} \citep{mpi4py}), Principal Component Analysis, statistics bootstrapping. As a sub--class of \ttt{pandas.DataFrame}, \ttt{Ensemble} supports I/O and queries from local and remote SQL databases. Parameter space sampling operations in \LT are handled by instances of the \ttt{Emulator} class, which inherits from \ttt{Ensemble} and provides access to vectorized RBF feature interpolation between parameters (which can be used to build emulators), $\chi^2$ evaluation, and the parameter sampling techniques outlined above. Examples of parameter sampling routines are shown in Figure \ref{samplingfig}.

\begin{figure}
\begin{center}
\includegraphics[scale=0.4]{Figures/parameter_sampling.eps}
\caption{Cosmological constraints based on the sampling of the parameter space $(\Omega_m,\sigma_8)$ with three different techniques, assuming a flat parameter prior $\Pi(\bb{p})$. We use a Likelihood Grid Evaluation of parameter likelihood (red), a Fisher matrix approximation (blue) and a MCMC sampling using the \ttt{emcee} package (black dots). The parameter space considered is the doublet $(\Omega_m,\sigma_8)$, and the feature used is the $\kappa$ power spectrum $P^{\kappa\kappa}(\ell)$. The $1\sigma$ confidence level contours are shown. The RBF interpolation is based on $N_M=100$ simulated models and the feature covariance matrix $\bb{C}$ has been estimated from $N_r=1024$ random realizations of the feature vector in a fiducial model with $(\Omega_m,\sigma_8)=(0.26,0.8)$. The code to produce this figure is available in {\sc IPython} notebook format at \url{http://nbviewer.jupyter.org/github/apetri/LensTools/blob/master/notebooks/sampling.ipynb}.}
\label{samplingfig}
\end{center}
\end{figure}

%%%%%%%%%%%%%%%%%% CONCLUSION %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{Conclusion}
%
In this work we presented the \LT computing package, which is a collection of tools targeted to theoretical studies of WL. The package includes a WL shear simulation pipeline and is complemented with a variety of image analysis tools and parameter sampling routines. The simulation pipeline combines different existing codes to simulate cosmological volumes (\ttt{CAMB},\ttt{Gadget2}) and provides a {\sc python} implementation of the multi--lens--plane algorithm. \LT is flexible in terms of snapshot file formats, making the use of different $N$--body simulation codes (see \citep{HACC} for example) possible with minimal additional efforts. This makes \LT portable between different research groups in the WL community. \LT makes use of the \ttt{numpy} array as its primary data structure for numerical calculations, making it very convenient to combine with popular algorithmic packages for data selection (\ttt{pandas} \citep{pandas}), astronomical tools (\ttt{astropy} \citep{astropy}), MCMC sampling (\ttt{pymc} \citep{pymc}, \ttt{emcee} \citep{emcee}), advanced statistical analysis and machine learning (\ttt{scikit-learn} \citep{scikit-learn}). Because of these reasons, we believe that \LT will become a valuable asset to the WL community, in particular to groups that already have experience with $N$--body simulations and want to study Weak Lensing.

%%%%%%%%%%%%%%%%%%%%%%%%%% ACKNOWLEDGMENTS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Acknowledgements}
We thank Zolt\`an Haiman, Jia Liu, Jose M. Zorrilla and Morgan May for invaluable support and useful discussions. The simulations in this work were performed at the NSF Extreme Science and Engineering Discovery Environment (XSEDE), supported by grant number ACI-1053575, at the Yeti computing cluster at Columbia University, and at the New York Center for Computational Sciences, a cooperative effort between Brookhaven National Laboratory and Stony Brook University, supported in part by the State of New York. This work was supported in part by the U.S. Department of Energy under Contract Nos. DE-AC02-98CH10886 and DESC0012704, and by the NSF Grant No. AST-1210877 (to Z.H.) and by the Research Opportunities and Approaches to Data Science (ROADS) program at the Institute for Data Sciences and Engineering at Columbia University.

\bibliography{ref}

\label{lastpage}
\end{document}


%
% personal commentary:
%        DRAFT DRAFT DRAFT
%        - KFALL
%
\section{\shdr{Delays and Links}{delay.cc}{sec:delays}}

Delays represent the time required for a packet to
traverse a link.
A special form of this object (``dynamic link'')
also captures the possibility of a link failure.
The amount of time required for a packet to traverse a link
is defined to be $s/b + d$ where $s$ is the packet size
(as recorded in its IP header), $b$ is the speed of the link
in bits/sec, and $d$ is the link delay in seconds.
The implementation of link delays is closely associated with
the blocking procedures described for Queues in
Section\ref{sec:qblock}.

\subsection{\shdr{The LinkDelay Class}{delay.cc}{sec:delayclass}}

The \code{LinkDelay} class is derived from a
\code{Connector} base class.
The following definitions are provided in \code{delay.cc}
for the \code{DelayLink} object:
\begin{small}
\begin{verbatim}
        class LinkDelay : public Connector {
         public:
                LinkDelay();
                void recv(Packet* p, Handler*);
		void send(Packet* p, Handler*);
		double delay();    /* line latency on this link */
		double bandwidth(); /* bandwidth on this link */
		double txtime(Packet* p); /* time to send pkt p on this link */
         protected:
                double bandwidth_; /* bandwidth of underlying link (bits/sec) */
                double delay_;     /* line latency */
                Event intr_;
        };

\end{verbatim}
\end{small}

The \code{recv} function overrides the \code{Connector} base class version.
It is defined as follows:
\begin{small}
\begin{verbatim}
        void LinkDelay::recv(Packet* p, Handler* h)
        {
        	double txt = txtime(p);
        	Scheduler& s = Scheduler::instance();
        	if (dynamic_) {
        		Event* e = (Event*)p;
        		e->time_ = s.clock() + txt + delay_;
        		itq_->enque(p);
        		schedule_next();
        	} else {
        		s.schedule(target_, p, txt + delay_);
        	}
        	/*XXX only need one intr_ since upstream object should
        	 * block until it's handler is called
        	 *
        	 * This only holds if the link is not dynamic.  If it is, then
        	 * the link itself will hold the packet, and call the upstream
        	 * object at the appropriate time.  This second interrupt is
        	 * called inTransit_, and is invoked through schedule_next()
        	 */
        	s.schedule(h, &intr_, txt);
        }

\end{verbatim}
\end{small}
For ``non-dynamic'' links,
this method operates by receiving a packet and scheduling two
events.
Assume these two events are called $E_1$ and $E_2$, and that
event $E_1$ is scheduled to occur before $E_2$.
$E_1$ is scheduled to occur when the upstream node attached to this
delay element has completed sending the current packet
(which takes time equal to the packet size divided by the link bandwidth).
$E_1$ is usually associated with a \code{Queue} object, and will
cause it to (possibly) become unblocked (see section \ref{sec:qblock}).
$E_2$ represents the packet arrival event at the downstream neighbor
of the delay element.
Event $E_2$ occurs a number of seconds later than $E_1$ equal to the
link delay.

\subsection{\shdr{The Otcl Link Class}{ns-link.tcl}{sec:nslink}}

The \code{Link} class is one of a handful of classes implemented
entirely in Otcl.  It has a subclass called \code{SimpleLink}
used frequently in the simulator.
The OTcl \code{SimpleLink} class uses the C++ \code{LinkDelay} class
to simulate packet delivery delays as described above.
The \code{Link} class is defined as follows:
\begin{small}
\begin{verbatim}
        Class Link
        Link instproc init { src dst } {
                $self next
                $self instvar trace_ fromNode_ toNode_
                set fromNode_ $src
                set toNode_ $dst
                set trace_ ""
        }

        Link instproc head {} {
                $self instvar head_
                return $head_
        }

        Link instproc queue {} {
                $self instvar queue_
                return $queue_
        }

        Link instproc link {} {
                $self instvar link_
                return $link_
        }
\end{verbatim}
\end{small}
This class really just holds enough state to describe what nodes the two
endpoints of the link are attached to and also hold references to
C++ \code{Queue} and \code{LinkDelay} objects which are used by classes
derived from \code{Link}, such as \code{SimpleLink}.
The \code{head\_} member indicates the first object to receive
a packet that traverses down the link.
Orinarily this would be the \code{Queue} object, but
sometimes it is some form of trace object (see Section\ref{sec:trace}).

\subsection{\shdr{The Otcl SimpleLink Class}{ns-link.tcl}{sec:nslink}}

The Otcl class \code{SimpleLink} implements a simple point-to-point
link with an associated queue and
delay.\footnote{The current
version also includes an object to examine the
network layer ``ttl'' field and discard packets if the
field reaches zero.}
It is derived from the base Otcl class \code{Link} as follows:
\begin{small}
\begin{verbatim}
        Class SimpleLink -superclass Link
        SimpleLink instproc init { src dst bw delay q { lltype "DelayLink" } } {
                $self next $src $dst
                $self instvar link_ queue_ head_ toNode_ ttl_
                ...
                set queue_ $q
                set link_ [new Delay/Link]
                $link_ set bandwidth_ $bw
                $link_ set delay_ $delay

                $queue_ target $link_
                $link_ target [$toNode_ entry]

                ...
                # XXX
                # put the ttl checker after the delay
                # so we don't have to worry about accounting
                # for ttl-drops within the trace and/or monitor
                # fabric
                #
                set ttl_ [new TTLChecker]
                $ttl_ target [$link_ target]
                $link_ target $ttl_
        }
\end{verbatim}
\end{small}
The ``init'' proc listed here is the Otcl analogue of a C++
constructor.
Thus, when a \code{SimpleLink} object is created,
new \code{Delay/Link} and \code{TTLChecker} objects are
also created.
Note that a \code{Queue} object must have already been created.

There are two additional methods implemented (in OTcl) as part
of the \code{SimpleLink} class: \code{trace} and \code{init-monitor}.
These functions are further described in the section about tracing
(see Section\ref{sec:trace}).