    \filetitle{filter}{Kalman smoother and estimator of out-of-likelihood parameters}{model/filter}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
[M,Outp,V,Delta,PE,SCov] = filter(M,Inp,Range,...)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\item
  \texttt{M} {[} model {]} - Solved model object.
\item
  \texttt{Inp} {[} struct \textbar{} cell {]} - Input database or
  datapack from which the measurement variables will be taken.
\item
  \texttt{Range} {[} numeric {]} - Filter date range.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\item
  \texttt{M} {[} model {]} - Model object with updates of std devs (if
  \texttt{'relative='} is true) and/or updates of out-of-likelihood
  parameters (if \texttt{'outoflik='} is non-empty).
\item
  \texttt{Outp} {[} struct \textbar{} cell {]} - Output struct with
  smoother or prediction data.
\item
  \texttt{V} {[} numeric {]} - Estimated variance scale factor if the
  \texttt{'relative='} options is true; otherwise \texttt{V} is 1.
\item
  \texttt{Delta} {[} struct {]} - Database with estimates of
  out-of-likelihood parameters.
\item
  \texttt{PE} {[} struct {]} - Database with prediction errors for
  measurement variables.
\item
  \texttt{SCov} {[} numeric {]} - Sample covariance matrix of smoothed
  shocks; the covariance matrix is computed using shock estimates in
  periods that are included in the option \texttt{'objrange='} and, at
  the same time, contain at least one observation of measurement
  variables.
\end{itemize}

\paragraph{Options}\label{options}

\begin{itemize}
\item
  \texttt{'ahead='} {[} numeric \textbar{} \emph{\texttt{1}} {]} -
  Predictions will be computed this number of period ahead.
\item
  \texttt{'chkFmse='} {[} \texttt{true} \textbar{} \emph{\texttt{false}}
  {]} - Check the condition number of the forecast MSE matrix in each
  step of the Kalman filter, and return immediately if the matrix is
  ill-conditioned; see also the option \texttt{'fmseCondTol='}.
\item
  \texttt{'condition='} {[} char \textbar{} cellstr \textbar{}
  \emph{empty} {]} - List of conditioning measurement variables.
  Condition time t\textbar{}t-1 prediction errors (that enter the
  likelihood function) on time t observations of these measurement
  variables.
\item
  \texttt{'deviation='} {[} \texttt{true} \textbar{}
  \emph{\texttt{false}} {]} - Treat input and output data as deviations
  from balanced-growth path.
\item
  \texttt{'dtrends='} {[} \emph{\texttt{@auto}} \textbar{} \texttt{true}
  \textbar{} \texttt{false} {]} - Measurement data contain deterministic
  trends.
\item
  \texttt{'data='} {[} \texttt{'predict'} \textbar{}
  \emph{\texttt{'smooth'}} \textbar{} \texttt{'predict,smooth'} {]} -
  Return smoother data or prediction data or both.
\item
  \texttt{'fmseCondTol='} {[} \emph{\texttt{eps()}} \textbar{} numeric
  {]} - Tolerance for the FMSE condition number test; not used unless
  \texttt{'chkFmse=' true}.
\item
  \texttt{'initCond='} {[} \texttt{'fixed'} \textbar{}
  \texttt{'optimal'} \textbar{} \emph{\texttt{'stochastic'}} \textbar{}
  struct {]} - Method or data to initialise the Kalman filter;
  user-supplied initial condition must be a mean database or a mean-MSE
  struct.
\item
  \texttt{'lastSmooth='} {[} numeric \textbar{} \emph{\texttt{Inf}} {]}
  - Last date up to which to smooth data backward from the end of the
  range; if \texttt{Inf} smoother will run on the entire range.
\item
  \texttt{'meanOnly='} {[} \texttt{true} \textbar{}
  \emph{\texttt{false}} {]} - Return a plain database with mean data
  only; this option overrides the \texttt{'return*='} options, i.e.
  \texttt{'returnCont='}, \texttt{'returnMse='}, \texttt{'returnStd='}.
\item
  \texttt{'outOfLik='} {[} cellstr \textbar{} empty {]} - List of
  parameters in deterministic trends that will be estimated by
  concentrating them out of the likelihood function.
\item
  \texttt{'objFunc='} {[} \emph{\texttt{'-loglik'}} \textbar{}
  \texttt{'prederr'} {]} - Objective function computed; can be either
  minus the log likelihood function or weighted sum of prediction
  errors.
\item
  \texttt{'objRange='} {[} numeric \textbar{} \emph{\texttt{Inf}} {]} -
  The objective function will be computed on the specified range only;
  \texttt{Inf} means the entire filter range.
\item
  \texttt{'precision='} {[} \emph{\texttt{'double'}} \textbar{}
  \texttt{'single'} {]} - Numeric precision to which output data will be
  stored; all calculations themselves always run to double precision.
\item
  \texttt{'relative='} {[} \emph{\texttt{true}} \textbar{}
  \texttt{false} {]} - Std devs of shocks assigned in the model object
  will be treated as relative std devs, and a common variance scale
  factor will be estimated.
\item
  \texttt{'returnCont='} {[} \texttt{true} \textbar{}
  \emph{\texttt{false}} {]} - Return contributions of prediction errors
  in measurement variables to the estimates of all variables and shocks.
\item
  \texttt{'returnMse='} {[} \emph{\texttt{true}} \textbar{}
  \texttt{false} {]} - Return MSE matrices for predetermined state
  variables; these can be used for settin up initial condition in
  subsequent call to another \texttt{filter} or \texttt{jforecast}.
\item
  \texttt{'returnStd='} {[} \emph{\texttt{true}} \textbar{}
  \texttt{false} {]} - Return database with std devs of model variables.
\item
  \texttt{'weighting='} {[} numeric \textbar{} \emph{empty} {]} -
  Weighting vector or matrix for prediction errors when
  \texttt{'objective=' 'prederr'}; empty means prediction errors are
  weighted equally.
\end{itemize}

\paragraph{Options for models with non-linearised
equations}\label{options-for-models-with-non-linearised-equations}

\begin{itemize}
\item
  \texttt{'nonlinear='} {[} numeric \textbar{} \emph{\texttt{0}} {]} -
  If non-zero the prediction step in the Kalman filter will be run in an
  exact non-linear mode using the same technique as
  \href{model/simulate}{\texttt{model/simulate}}.
\item
  \texttt{'simulate='} {[} cell \textbar{} empty {]} - Options passed in
  to \texttt{simulate} when invoking the non-linear simulation in the
  prediction step; only used when \texttt{nonlinear=} is greater than
  \texttt{0}.
\end{itemize}

\paragraph{Description}\label{description}

The \texttt{'ahead='} and \texttt{'rollback='} options cannot be
combined with one another, or with multiple data sets, or with multiple
parameterisations.

\subparagraph{Initial conditions in time
domain}\label{initial-conditions-in-time-domain}

By default (with \texttt{'initCond=' 'stochastic'}), the Kalman filter
starts from the model-implied asymptotic distribution. You can change
this behaviour by setting the option \texttt{'initCond='} to one of the
following four different values:

\begin{itemize}
\item
  \texttt{'fixed'} -- the filter starts from the model-implied
  asymptotic mean (steady state) but with no initial uncertainty. The
  initial condition is treated as a vector of fixed, non-stochastic,
  numbers.
\item
  \texttt{'optimal'} -- the filter starts from a vector of fixed numbers
  that is estimated optimally (likelihood maximising).
\item
  database (i.e.~struct with fields for individual model variables) -- a
  database through which you supply the mean for all the required
  initial conditions, see help on \href{model/get}{\texttt{model/get}}
  for how to view the list of required initial conditions.
\item
  mean-mse struct (i.e.~struct with fields \texttt{.mean} and
  \texttt{.mse}) -- a struct through which you supply the mean and MSE
  for all the required initial conditions.
\end{itemize}

\subparagraph{Contributions of measurement variables to the estimates of
all
variables}\label{contributions-of-measurement-variables-to-the-estimates-of-all-variables}

Use the option \texttt{'returnCont=' true} to request the decomposition
of measurement variables, transition variables, and shocks into the
contributions of each individual measurement variable. The resulting
output database will include one extra subdatabase called
\texttt{.cont}. In the \texttt{.cont} subdatabase, each time series will
have Ny columns where Ny is the number of measurement variables in the
model. The k-th column will be the contribution of the observations on
the k-th measurement variable.

The contributions are additive for linearised variables, and
multiplicative for log-linearised variables (log-variables). The
difference between the actual path for a particular variable and the sum
of the contributions (or their product in the case of log-varibles) is
due to the effect of constant terms and deterministic trends.

\paragraph{Example}\label{example}


\section{Base model construction}
\subsection{Platform for infectious disease dynamics simulation}

We developed a deterministic compartmental model of COVID-19 transmission using the AuTuMN platform,
publicly available https://github.com/monash-emu/AuTuMN/.
Our repository allows for the rapid and robust creation and stratification of models of infectious disease epidemiology and includes pluggable modules to simulate heterogeneous population mixing, demographic processes, multiple circulating pathogen strains, repeated stratification and other dynamics relevant to infectious disease transmission.
The platform was created to simulate TB dynamics, being an infectious disease whose epidemiology differs markedly by setting, such that considerable flexibility was desirable \cite{RN18}.
We have progressively developed the structures of our platform over recent years, and further adapted it to be sufficiently flexible to permit simulation of other infectious diseases, such as COVID-19. A similar model has been applied to several countries of the Asia-Pacific, with the application to the Philippines (without structure to represent geospatial stratification or contact tracing) previously described.\cite{RN79}

\subsection{Base COVID-19 model}
Using the base framework of an SEIR model (susceptible, exposed, infectious, removed), we split the exposed and infectious compartments into two sequential compartments each (SEEIIR). The two sequential exposed compartments represent the non-infectious and infectious phases of the incubation period, with the latter representing the ``presymptomatic" phase such that infectiousness occurs during three of the six sequential phases. For this reason, ``active" is a more accurate term for the two sequential ``I" compartments and is preferred henceforward. The two infectious compartments represent early and late phases of active disease, during which symptoms occur if the disease episode is symptomatic, and allow explicit representation of notification, case isolation, hospitalisation and admission to ICU. The ``active" compartment also includes some persons who remain asymptomatic throughout their disease episode, such that these compartments do not map directly to either persons who are infectious or those who are symptomatic (Figure \ref{fig:seeiir}).

\begin{figure}[ht]
    \includegraphics[width=\textwidth]{../covid_19/covid_19_seeiir.pdf}
   \title{Unstratified compartmental model structure.}
    \caption{\textbf{Unstratified compartmental model structure.} S = susceptible, E = exposed, I = active, R = recovered/removed. Depth of pink/red shading indicates the infectiousness of the compartment.}
    \label{fig:seeiir}
\end{figure}

The latently infected and infectious presymptomatic periods together comprise the incubation period, with the incubation period and the proportion of this period for which patients are infectious defined by input parameters described below. In general, two sequential compartments can be used to form a gamma-distributed profile of transition to infectiousness following exposure if the progression rates for these two compartments are equal, although in implementing this model the relative sojourn times in the two sequential compartments usually differed. Nevertheless, the profiles implemented are broadly consistent with the empirically observed log-normal distribution of individual incubation periods \cite{RN13}.

The transition from early active to late active represents the point at which patients are detected (for those persons for whom detection does eventually occur) and isolation then occurs from this point forward (i.e. applied during the late disease phase only). This transition point is also intended to represent the point of admission to hospital or transition from hospital ward to intensive care for patients for whom this occurs (see Section \ref{clin}).


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                 %
%  GEANT manual in LaTeX form                                     %
%                                                                 %
%  Version 1.00                                                   %
%                                                                 %
%  Last Mod. 8 June 1993  17:20  MG                               %
%                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentstyle[11pt,fleqn,epsfig,crngeant,bibunits,fontcmds]{cernman}
\newcommand{\Title}{GEANT User's Guide}%           Title for document
\psfigdriver{DVIPS}
\makeindex
\romanfont{times}
\PScommands% Initialize PS boxes
\newmathalphabet*{\mathtt}{cmtt}{m}{n}
\newmathalphabet*{\mathbf}{cmr}{b}{n}
\begin{document}
%  ==================== Front material ============================
\include{geafront}
%  ==================== Body of text ==============================
\pagenumbering{arabic}
\setcounter{page}{1}

%%%%%%   Catalog of Program packages and entries%%%%

\def\Rtnr{Catalog}%Dummy routine name to appear at bottom of page
\include{geantcat}

\let\LARGE\large
\let\Large\large
\let\DL\DLtt

% Here come the different files to be included

%%     TEMP part     %%

\begin{bibunit}[unsrt]
\renewcommand{\bibname}{TEMP Bibliography}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%                                                                 %
%  GEANT manual in LaTeX form                                     %
%                                                                 %
%  Version 1.00                                                   %
%  Last Mod.  8 June 1993  1300   MG                              %
%                                                                 %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\Authors {G.N. Patrick, L.Urban}   \Origin {Same}
\Version{Geant 3.16}\Routid{PHYS341}
\Submitted {26.10.84}  \Revised{19.12.92}
\Makehead{Simulation  of   discrete Bremsstrahlung by electrons}

\section{Subroutines}

\Shubr{GBREME}{}

\Rind{GBREME} generates a photon from Bremsstrahlung of an electron by
treating it as a discrete process.
The secondary photon energy is sampled from a parameterization of
the Bremsstrahlung calculation of Seltzer and Berger
\cite{bib-SEL1} for electron energies below 10 GeV, and from the screened
Bethe-Heitler cross-section above 10 GeV.
Midgal corrections are applied in both cases.
For the angular distribution of the photon, is calls the function
\Rind{GBTETH}.

Input :  via common block \FCind{/GCTRAK/}

Output:  via common block \FCind{/GCKING/}.

\Rind{GBREME} is called automatically from the tracking
routine \Rind{GTELEC} if, and when,
the parent electron reaches its radiation point during the
tracking stage of GEANT.

\Sfunc{GBTETH}{THETA = GBTETH(ENER,PARTM,EFRAC)}

\Rind{GBTETH} calculates the angular distribution for the $e^+ e^-$-pair
in pair production and for the photon in Bremsstrahlung.
In case of Bremsstrahlung it gives the
scaled angle for a photon radiated by an electron (mass {\tt PARTM}) of
energy {\tt ENER}. The energy of photon is {\tt EFRAC} times
the initial energy of the electron.
\Rind{GBTETH} is called by \Rind{GBREME}.

\section {Method}

The Coulomb corrected Bethe-Heitler differential cross-section for
production of a photon of energy $\epsilon E$
($\epsilon = k/E$, where $k$ is the photon energy)
by an electron of primary energy $E$ is given as in
\notHTML{\cite{bib-WILL,bib-BUTC,bib-EGS3}}\HTML{\cite{bib-WILL}, \cite{bib-BUTC}, \cite{bib-EGS3}}:

\begin{equation}
\frac{d\sigma(Z,E,\epsilon)} {d \epsilon} =
      \frac{r_0^2 \alpha Z[Z +\xi(Z)]}{\epsilon}
    \left\{[1+(1 -\epsilon)^2][\Phi_1(\delta )
   - F(Z)] -\frac{2} {3}(1 - \epsilon)[\Phi_2(\delta) - F(Z)]\right\}
\end{equation}
where, $\Phi_i(\delta)$ are the screening functions depending on the screening
variable $\delta$
\[ \begin{array}{ll}

\begin{array}{l}
\delta  =  \frac {\Dstm 136m}{\Dstm Z^{1/3}E} \end{array}
            \frac{\Dstm \epsilon}{\Dstm (1-\epsilon)}
              &        m = \mbox{electron mass} \\
& \\
\noindent \left.\begin{array}{l}
\Phi_1(\delta)  =  20.867 - 3.242\delta +0.625\delta^2  \\
\Phi_2(\delta)  =  20.209 - 1.930\delta - 0.086\delta^2  \\
\end{array}        \right\} & \; \delta\leq 1 \\
\begin{array}{l} \Phi_1(\delta) = \Phi_2(\delta) =
21.12 - 4.184 \ln(\delta+0.952) \end{array}
                    & \; \delta > 1  \\
&\\
\begin{array}{l} F(Z) \end{array}  = \left\{\begin{array}{l}
        4/3 \ln Z          \\
        4/3 \ln Z+4f_c(Z)  \\
        \end{array}\right.
&  \begin{array}{l}
E<0.05 \; GeV      \\
E\geq 0.05 \; GeV  \\
\end{array}  \\
\\
\begin{array}{l}  \xi(Z)=\frac{\Dstm\ln {(1440/Z^{2/3})}}{\Dstm\ln
{(183/Z^{1/3})}-f_c(Z)} \end{array}  & \\
\\
\begin{array}{l} f_c(Z) \mbox{\qquad the Coulomb correction function}
\end{array}  \\
\end{array} \]

%all these arrays of one column are here just to line up the text
\begin{eqnarray*}
f_c(Z) & = & a(1/(1+a)+0.20206-0.0369a+0.0083*a^2-0.002a^3)\\
    a   & = & (\alpha*Z)^2                                     \\
\alpha  & = & 1./137.
\end{eqnarray*}
 The kinematically allowed region for $\epsilon$ is
\begin{equation}
\epsilon_c \equiv \frac{k_c}{E} \leq \epsilon \leq  1- \frac{m}{E}
\equiv \epsilon_1
\end{equation}
where,  $k_c$ is the photon cut-off energy below which the Bremsstrahlung
is treated as a continuous energy loss ({\tt BCUTE} in the program).
The cross-section (1) can be decomposed as
(the normalization is not important !)
\begin{equation}
\frac{d \sigma}{d \epsilon} =  \sum^{2}_{i=1} \alpha_i f_i(\epsilon)
      g_i(\epsilon )
\end{equation}
\begin{flushleft}
$\begin{array}{l@{}l@{\hspace{4cm}}l}
\mbox{where} & & \\
& \alpha_1 = F_{10}\ln \frac{\Tstm \epsilon_1}{\Tstm \epsilon_c}&
  \alpha_2 = F_{20}\frac{\Tstm 3}{\Tstm 8}\
             \frac{\Tstm \epsilon_1^2 -\epsilon_c^2}{\Tstm 1-\epsilon_c}\\
& & \\
& f_1 (\epsilon) = \frac{\Tstm 1}
                 {\Tstm \epsilon \; \ln {(\epsilon_1/ \epsilon_c)}} &
  f_2(\epsilon) = \frac{\Tstm 2 \epsilon}{\Tstm \epsilon_1^2 -\epsilon_c^2}\\
& g_1(\epsilon)=\frac{\Tstm F_1}{\Tstm F_{10}}
                \frac{\Tstm 1-\epsilon}{\Tstm 1-\epsilon_c} M(\epsilon)&
g_2(\epsilon )  = \frac{\Tstm F_2}{\Tstm F_{20}} . C_M (\epsilon )\\
\mbox{and} & & \\
& F_1(\delta) = 3\Phi_1(\delta)-\Phi_2(\delta)-2F(Z)  &
  F_{10}=F_1(\delta_{min})                     \\
& F_2(\delta) = 2\Phi_1(\delta)-2F(Z)                 &
  F_{20} = F_2(\delta_{min})                   \\
\\
& \delta_{min} = \frac{\Tstm 136m}{\Tstm Z^{1/3}E}
                \frac{\Tstm \epsilon_c}{\Tstm 1- \epsilon_c}
\end{array}$
\end{flushleft}
$\delta_{min}$ is the minimum value of the variable $\delta$, and
$C_M(\epsilon)$  is the Midgal correction factor \cite{bib-MIGD}:
\[
C_M (\epsilon)  =\frac{1 + C_0 / \epsilon_1^2}
               {1 + C_0 / \epsilon^2}
\]
 where
\[
C_0 =\frac{nr_0 \lambdar^2 }{\pi}
\]
\begin{DL}{MM}
\item[$n$]           electron density in the medium
\item[$r_0$]         classical electron radius
\item[$\lambdar$]    reduced Compton wavelength of the electron.
\end{DL}
This correction decreases the cross-section for low photon energy.
Using the decomposition (3) the sampling of $\epsilon$ can be done by
(for a set of random numbers: $0\leq r_i\leq 1$)
\begin{OL}
\item  selecting the integer i (1 or 2)\\
$ \begin{array}{ll}
\mbox{if  }r_0 \leq BPAR = \alpha_1/(\alpha_1+\alpha_2), & \mbox{then  } i=1 \\
\mbox{if  } r_0 > BPAR,                                  & \mbox{then  } i=2
\end{array}$
\item
sampling $\epsilon$ from $f_i(\epsilon) $ \\
$\begin{array}{ll}
\epsilon =\epsilon c(\epsilon_1/\epsilon_c)^{r1} & \mbox{when  } i=1 \\
\epsilon  =\sqrt{\epsilon_c^2 + r_1 (\epsilon m^2-\epsilon_c^2)}
                                                 & \mbox{when  } i=2
\end{array}$
\item
calculating the rejection function $g_i(\epsilon)$ and\\
if $ r_2>g_i(\epsilon)$  rejecting $\epsilon$, starting again from 1\\
if $ r_2\leq g_i(\epsilon)$ accepting $\epsilon$
\end{OL}
As in the case of the pair production after sampling $\epsilon$ from
$f_i(\epsilon)$,
we have to check that
\[
\delta  = \frac{136 m} {Z^{1/3}E} \frac{\epsilon} {1 - \epsilon_c}
       \leq \delta_{max}=
       \exp  \left(\frac{21.12-f(Z)} {4.184}\right)  - 0.952
\]
and, if this is not the case, we have to start again from step 1.

The decomposition (3) used here is simpler and more effective
than that used in the earlier versions of GEANT and in {\tt EGS},
and the method outlined above has no convergence problems.

After the successful sampling of $\epsilon$, \Rind{GBREME} generates the
polar angles
of the radiated photon with respect to an axis defined along the parent
electron's momentum.
The energy-angle distribution is given by
Tsai~\notHTML{\cite{bib-TSAI,bib-TSAI-err}}\HTML{\cite{bib-TSAI}, \cite{bib-TSAI-err}}
as following:
\begin{eqnarray}
\frac{d \sigma}{dkd \Omega}
& = & \frac{2 \alpha^{2}e^{2}}{\pi k m^{4}}
      \left\{ \left[ \frac{2y-2}{(1+l)^2}+\frac{12l(1-y)}{(1+l)^4}\right]
      (Z^{2}+Z) \nonumber  \right. \\
&   & \mbox{} + \left. \left[ \frac{2-2y-y^{2}}{(1+l)^2}-
      \frac{4l(1-y)}{(1+l)^4}
      \right]
      \left[ X-2Z^{2}f((\alpha Z)^{2})\right]
      \right\} \nonumber \\
\end{eqnarray}
where $k$ is the photon energy, $E$ is the initial electron energy,
$y = k/E$ and $l = E^{2} \theta^{2}/m^{2}$. This distribution
is quite complicated for sampling and, furthermore, for a variable
$u = E \theta/m$, shows a very weak dependence on $Z$, $E(k)$,
$y = k/E$. Thus, the distribution can be approximated by a function
\begin{equation}
f(u) = C \left( u e^{-au} + d u e^{-3au} \right)
\end{equation}
where
\begin{eqnarray*}
C & = & \frac{9a^{2}}{9 + d} \\
a & = & 0.625 \\
d & = & 27.0 \\
\end{eqnarray*}
The sampling of the function $f(u)$ can be done in the following way
($r_{i}$ are uniformly distributed random numbers
in [0,1]):
\begin{enumerate}
\item Choose between $u e^{-au}$ and $d u e^{-3au}$, \\
      weights: $9/(9+d)$ and $d/(9+d)$. \\
      If $r_{1} < 9/(9+d)$ then $a=b$, \\
      else $b=3a$.
\item Sample $u e^{-bu}$, \\
      $u=-(\ln r_{2} + \ln r_{3})/b$.
\end{enumerate}
The azimuthal angle, $\Phi$, is generated isotropically.
This information is used to calculate the momentum vector of the radiated
photon, to transform it to the GEANT coordinate system and
to store the result into common block \FCind{/GCKING/}. Also, the momentum of the parent electron
is updated.

\subsection{Restrictions}

\begin{enumerate}
\item
Effects due to breakdown of the Born approximation at low energies are ignored.
\item
Target materials composed of compounds or mixtures are treated identically
to elements using the effective atomic number $Z$ calculated in
\Rind{GSMIXT}
(this is not the case when computing the mean free path!).
\end{enumerate}
\putbib[cnasbibl,geabibl]
\end{bibunit}

%  ==================== Index material ============================

\setcounter{page}{1}%                                Reset page counter
\def\Rtnr{Index}%Dummy routine name to appear at bottom of page
\input{\jobname.ind} % index

\end{document}


% Dynamic Programming: Advanced
%
% Slides on dynamic programming considered too advanced for the introductory course.

\subsection{Dynamic Programming (advanced)}
\begin{frame}
  \frametitle{Gap Scoring}
  \begin{itemize}
    \item Simple gap penalty: all gaps score the same
    \begin{itemize}
      \item \emph{one} parameter in the alignment model
    \end{itemize}
    \item Affine gap penalty: opening and extending score differently
    \begin{itemize}
      \item \emph{two} parameters in the alignment model
    \end{itemize}
  \end{itemize}
  \begin{center}
    \includegraphics[width=0.3\textwidth]{images/gap_scores}
  \end{center}
\end{frame}

\begin{frame}
  \frametitle{Banded Alignment}
  \begin{itemize}
    \item Given a maximum scoring alignment, restrict search to a "band" between endpoints
    \item Width of this band = "bandwidth", another parameter in the model (may be floating)
    \item Speeds up search, reduces memory use
  \end{itemize}
  \begin{center}
    \includegraphics[width=0.3\textwidth]{images/banded}
  \end{center}
\end{frame}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% June 19, 2004
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\documentclass[10pt,twoside]{hsqcd}
\usepackage{epsf,amsmath}
\usepackage{graphicx}

\setcounter{page}{1}
\renewcommand{\baselinestretch}{1.15}
\textwidth 13.5cm \textheight 17cm \pagestyle{myheadings}

\begin{document}

\title{Universalities in hadron production and the maximum entropy
principle}

\author{A.Kropivnitskaya, A.Rostovtsev \\
Institute for Theoretical and Experimental Physics \\
Moscow 117218, Russia\\
E-mail: kropiv@itep.ru, rostov@itep.ru }

\maketitle

\begin{abstract}
A shape of statistical momentum distribution of hadrons produced in high
energy particle collisions closely resembles one observed for a broad
variety of phenomena in nature. An attempt was made to understand a
genesis of this distribution beyond the context of each particular
phenomenon.
\end{abstract}

A multi-hadron production in high energy hadronic collisions is a
complicated phenomenon generally modeled by a dynamic system of
hadronizing quarks and gluons with many degrees of freedom and high level
of correlations. The properties of produced hadrons at any given
interaction cannot be predicted. But statistical properties, energy and
momentum averages, correlation functions, and probability density
functions show regular behavior. Thus, statistical methods must be applied
to understand properties of particle production presented by the
experimentally measured  distributions.

A statistical model for computing high energy collisions of protons with
multiple production of particles has been discussed first by E.Fermi in
1950~\cite{Fermi}. Later on, R.Hagedorn has proposed a statistical
thermodynamic model to describe the momentum spectra of particles produced
in $pp-$collisions~\cite{Hagedorn}. This model approximates the
experimentally measured exponential momentum spectra of hadrons with a
Boltzmann-like statistical distribution.
With an advance of high energy collision experiments with high statistic
accumulated the measured momentum spectra are found to deviate from the
exponential form. Namely, at high values of  particle's transverse
momentum~$(P_T)$ the spectrum shows a power-law behaviour. This
observation is interpreted as a proof of the underlying QCD dynamics of
hadronizing partonic system produced in the particle collisions. Though
using QCD one calculates elementary interactions of quarks and gluons at
microscopic level, a complexity of the system generally doesn't allow an
extension of these calculations to predict the observed particle spectra.
In addition, it is observed that the relative rates of different particle
production and parameters of the momentum spectra depend on global
particle's variables like the mass and spin~\cite{LEP, HERA}. Therefore,
it is tempting to reconsider a statistical approach to understand the
particle's transverse momentum spectra in the whole range.

Firstly, we note that the invariant particle production cross-section is
approximated by a damped power-law function of a particle's transverse
momentum:
\begin{equation}
\frac{d\sigma}{dydP_T^2} \sim \frac{1}{(a_0 + P_T)^n}\,,
\label{pdf}
\end{equation}
where $a_0$ and $n$ are parameters of the distribution function. For
simplicity, we consider only particles produced at central rapidity
plateau. These particles do not belong to the fragmentation regions of the
colliding beams.

It is interesting to note, the same form of the damped power-law
distribution describes a variety of phenomena like a velocity spectrum in
turbulent liquid, ion energy distribution in geomagnetic plasma,  a
distribution of the earthquakes as function of it's magnitude (empirical
Gutenberg-Richter law), distributions of the avalanches and landslides,
forest fires and solar flares, rains and winds, distributions related to
the human activities like earnings and settlements sizes, distribution of
inter-trade intervals observed on stock markets, statistical distribution
of a number of sexual partners, etc.
These distributions arise because the same stochastic process is at work,
and this process can be understood beyond the context of each example.
Moreover, one can see that for small values of the distributed variable
the damped power-law expression is reduced to the exponential statistical
distribution. What is a genesis of the power-law and exponential
statistical distributions?

    The least biased method to obtain statistical distributions, which are
realized in the nature was promoted by E.T.Jaynes as Maximum Entropy
Principle~\cite{MEP}. This Principle states that the physical
observable has a distribution, consistent with given constraints which
maximizes the entropy. As an example, the Boltzmann exponential
distribution arises naturally from a maximization of Gibbs-Shannon entropy
under a constraint on an average value of the distributed variable. The
textbook examples of the exponential distributions are the energy of the
molecules of an ideal gas within an isolated volume and a distance between
two neighboring points with $N-$points randomly spread over a limited
interval with length~$L$. Indeed, in both examples the average values of
the distributed variable are well defined. Namely, due to a conservation
of the number of molecules and the total energy in the gas volume their
ratio gives a predefined value of average kinetic energy per molecule of
gas.
Similarly, an average distance between two points is obviously defined by
a ratio of $L/(N-1)$. However, generally speaking, the constraint on the
average value in most cases has a little sense. Therefore, one needs
another suitable form of a constraint which results in the damped
power-law distribution while maximizing the Gibbs-Shannon
entropy~\footnote{For simplicity here we stick to the Gibbs-Shannon
entropy form only. Postulating other forms of entropy, like one used in
non-extensive thermodynamics~\cite{Tsallis} reveals additional conceptual
problems}. This new constraint has to correspond to a new conservation
law. There is no obvious known conservation law which could yield the
damped power-law distribution. Therefore, we propose here a toy model. In
this model each measurement of a physical value~$x_i$ of interest
corresponds to a gain of some bits of information equal to $ln(a_0+x_i)$.
Assume, that an observer is allowed to gain in a set of measurements in
average a limited information only. This assumption corresponds to the
following constraint
\begin{equation}
                                \Sigma P_i ln(a_0 + x_i) = const\,,
\label{constraint}
\end{equation}
where $P_i$ is a probability to observe a value $x_i$. Maximization of
entropy under the constraint~(\ref{constraint}) results in the damped
power-law probability distribution as function of~$x_i$. It is interesting
to note that the above constraint~(\ref{constraint}) is reduced to a
geometric mean value of terms~$(a_0+x_i)$. This makes different events
correlated with a correlation strength increasing for smaller $a_0$
values. A conservation of an average information is lively exemplified in
a world of fractals. An observer doesn't gain new information about the
fractal object structure by changing a scale of resolution. If so, a
meaning of
the $a_0$ parameter is to define a minimal available information.
It is interesting to note, the experimental data
show that for hadron transverse momentum spectra the $a_0$ value is close
to the mass of produced hadron~\cite{HERA}.
In addition to the direct measurements of particle spectra a
particle-particle correlation and a broadening of non-Poisson particle
multiplicity distribution are likely intrinsically related to the
appearance of power-law tails in particle momentum spectra. Therefore, an
analysis of their dynamics for varying interaction energies will allow
further progress. This makes high statistics data accumulated in high
energy particle interactions to be a unique laboratory to study the whole
class of the power-law phenomena.

\section*{Acknowledgements} The authors thank the organizing committtee of
HSQCD'04
conference for warm hospitality. This work was partially
supported by grants RFBR-03-02-17291 and RFBR-04-02-16445.

\begin{thebibliography}{0}

\bibitem{Fermi}  E.Fermi,
Progress of Theoretical Physics, {\bf 5}  (1950) 570.

\bibitem{Hagedorn}
R.Hagedorn, Suppl. Nuovo Cimento, {\bf 3} (1965) 147.

\bibitem{LEP}
P.V. Chliapnikov, O.G. Chikilev, V.A. Uvarov,
Phys. Lett. {\bf B352}, (1995) 461.

\bibitem{HERA} A.Kropivnitskaya, Surveys High
Energ.Phys.{\bf 18}, (2003) 173.

\bibitem{MEP}  E.T.Jeynes, Phys.Rev.,
{\bf 106} (1957) 620.

\bibitem{Tsallis}
C.Tsallis J.Stat.Phys., {\bf 52} (1988) 479.

\end{thebibliography}

\end{document}


\documentclass[conference]{IEEEtran}
\IEEEoverridecommandlockouts
% The preceding line is only needed to identify funding in the first footnote. If that is unneeded, please comment it out.
\usepackage{acronym}

\usepackage{esvect}

\usepackage{listings}

\acrodef{PSO}[PSO]{\emph{particle swarm optimisation algorithm}}
\acrodef{VEPSO}[VEPSO]{\emph{vector evaluated particle swarm optimization}}
\acrodef{NN}[NN]{\emph{neural network}}
\acrodef{CPSO}[CPSO]{\emph{chaotic particle swarm optimisation algorithm}}
\acrodef{RNG}[RNG]{\emph{random number generator}}
\acrodef{CPRNG}[CPRNG]{\emph{chaotic pseudo-random number generator}}
\acrodef{VN}[VN]{\emph{Von Neumann}}
\acrodef{MSE}[MSE]{\emph{mean squared error}}
\usepackage{cite}
\usepackage{amsmath,amssymb,amsfonts}
\usepackage{commath}
\usepackage{algorithmic}
\usepackage{array,multirow,graphicx}
\usepackage{textcomp}
\def\BibTeX{{\rm B\kern-.05em{\sc i\kern-.025em b}\kern-.08em
    T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}
\begin{document}

\title{Multiple Sequence Alignment using Multi-Objective Particle Swarm Optimization\\
}

\author{\IEEEauthorblockN{\\Quinton Weenink}
\IEEEauthorblockA{\textit{dept. Computer Science} \\
\textit{University of Pretoria}\\
Pretoria, South Africa \\
u13176545@tuks.co.za}
}

\maketitle

\section{Introduction}
This paper aims to investigate how to achieve multiple sequence alignment using a multi-objective \ac{PSO}. While many multi-objective optimisation techniques exist this paper will investigate the performance of \ac{VEPSO} to optimise the required objectives.

\section{Background}
Sequence alignment is a way of arranging the sequences to identify regions of similarity. For this research we will be attempting to align the sequences by identifying multiple objectives and attempting to solve them, namely:
\begin{itemize}
	\item maximise the number of vertically aligned characters, while simultaneously
	\item minimising the number of leading spaces inserted.
\end{itemize}

An example of such a sequence is as follows:
\begin{lstlisting}
        a b c d e f
        b b d h g
        c a b f
\end{lstlisting}

A possible solution to the above sequence is:

\begin{lstlisting}
        - a b - c d e f - -
        - - b b - d - - h g
        c a b - - - - f - -
\end{lstlisting}

\subsection{Fitness}
\noindent The following describes how the fitness will be calculated for each of the objectives
\subsubsection{Alignment fitness}
The alignment fitness is calculated by incriminating the fitness each time the same character is aligned. Furthermore the fitness is decreased when different characters are aligned. The result is then negated in order to make both objectives minimisation problems. A solution who's characters are misaligned is considered an in-valid solution for this research.

\subsubsection{Leading space fitness}
The leading space fitness is calculated by adding the number of spaces that the solution requires.

The objective fitness for the above solution would be:
\begin{itemize}
	\item Alignment fitness $= -(1 + 2 + 1 + 1) = -5$
	\item Leading space fitness $= 15$
\end{itemize}
\subsection{Position}
Each particle will have a position that corresponds to each of the characters. For the above sequence that means a position vector of size 15 where $x_j \in [0, 7]$. Positions are floored in order to get the number of spaces to be inserted before the characters.

\section{Original Particle Swarm Optimization}
A \ac{PSO} is a machine learning algorithm where each particle moves around the search space attempting to find the optimal solution with the influence of its neighbouring particles. The position for each particle iteration is calculated as follows.

\begin{equation} \label{eq:pso:update-position}
\vec{x}_{i}^{\,t} = \vec{x}_{i}^{\,t-1} + \vec{v}_{i}^{\,t}
\end{equation}

\noindent Where:
\begin{itemize}
	\item $ \vec{x} $ is the particle position vector in n-dimensions
	\item $ \vec{v} $ is the particle velocity vector, used to calculate the particles next position
	\item $t$ is the time increment
	\item $i$ is the particle number
\end{itemize}
\vspace{5mm}
\noindent For each iteration of the algorithm, a new location of the particle is calculated based on its previous location and velocity vector as seen in (\ref{eq:pso:update-position}). The following describes a the velocity update algorithm.

\begin{equation} \label{eq:pso:update-velocity}
\vec{v}_{i}^{\,t} = w . \vec{v}_{i}^{\,t-1} + c_1 . \vec{r}_{1}^{\,t} . (\vec{x}_{pBest, i} - \vec{x}_{i}^{t-1}) + c_2 . \vec{r}_{2}^{\,t} . (\vec{x}_{nBest, i} - \vec{x}_{i}^{t-1})
\end{equation}

\noindent Where:
\begin{itemize}
	\item $c_1$ is the acceleration coefficient for the cognitive component
	\item $c_2$ is the acceleration coefficient for the social component
	\item $r_1$ and $r_2$ is a vector of random numbers in the range (0, 1)
	\item $\vec{x}_{pBest}$ is the personal best position of that particle
	\item $\vec{x}_{nBest}$ is the best position found in that particles neighbourhood
\end{itemize}
\vspace{5mm}
\noindent Particle's social component, the third term in (\ref{eq:pso:update-velocity}), requires $ nBest $, the best neighbouring particles position. The set of neighbouring particles is determined by the topology used in the \ac{PSO}. Different topologies could influence the performance of the \ac{PSO}.

    \subsection{Social Network Topologies}

    In the above mentioned \ac{PSO} the neighbourhood could be described in a variety of ways. One such method is the $ gBest $ topology, where each particle is a neighbour of every other particle in the network. This means that the particle's $ nBest $ is always the global best particle for the swarm.

    Another social topology called $ lBest $ connects particles in a ring topology, the neighbourhood is specified by a fixed size of $ n_s $. The $ nBest $ in this case is determined by the best particle within $ n_s $ neighbours from the current particle\cite{vanwyk:overfitting-psoffnn}.

\section{Vector Evaluated Particle Swarm Optimization}
In this multi-swarm concept, each objective function is optimised by a swarm of particles using the $gBest$ from another swarm\cite{kian-lim:pso}.

While position for each particle is updated in the same way as an original \ac{PSO} the velocity update function changes. Assuming a multi-objective problem with 2 objectives the following describes the velocity update for each of the swarms.

\begin{equation} \label{eq:vepso:update-velocity}
\begin{array}{l}
S_1.\vec{v}_{i}^{\,t} = w . S_1 . \vec{v}_{i}^{\,t-1} + c_1 . \vec{r}_{1}^{\,t} . (S_1 . \vec{x}_{pBest, i} - S_1 . \vec{x}_{i}^{t-1})\\
\hspace{29mm} + c_2 . \vec{r}_{2}^{\,t} . (S_2 . \vec{x}_{nBest, i} - S_1 . \vec{x}_{i}^{t-1})
\end{array}
\end{equation}

\begin{equation} \label{eq:vepso:update-velocity2}
\begin{array}{l}
S_2.\vec{v}_{i}^{\,t} = w . S_2 . \vec{v}_{i}^{\,t-1} + c_1 . \vec{r}_{1}^{\,t} . (S_2 . \vec{x}_{pBest, i} - S_2 . \vec{x}_{i}^{t-1})\\
\hspace{29mm} + c_2 . \vec{r}_{2}^{\,t} . (S_1 . \vec{x}_{nBest, i} - S_2 . \vec{x}_{i}^{t-1})
\end{array}
\end{equation}

\noindent Where:
\begin{itemize}
    \item $S_1$ and $S_2$ are the values associated with swarm 1 and swarm 2 respectively
	\item There rest remain the same as in the original pso velocity update function (\ref{eq:pso:update-velocity})
\end{itemize}
\vspace{5mm}

\subsection{Swarm Social Network Topologies}

Similarly to the social network topologies mentioned in for the original \ac{PSO} the same topologies apply to \ac{VEPSO}. In the same way the $lBest$ functions for particle social topologies the swarm discussed will share information to the next swarm in the same way.

$pBest$ and $nBest$ positions were only used as a local or global guide if they were non-dominated by a previous solution as well as a valid solution.

\subsection{Objectives}

Each of the swarm's error functions is the resulting fitness of one of the objectives. A good multi-objective optimisation algorithm should be able to solve its own objective while not forgetting about the other swarms objectives.

\subsection{Boundary Constraint Handling Mechanisms}

The the boundary constraint clamping approach used in this research to prevent particles from leaving the search space. If a particle violates a boundary constraint in a specific dimension, then the particles position was clamped to the edge of the constraint. For this research this was specified by the max length of the sequences.

\section{Experiments}
    \subsection{\ac{PSO} configuration}
    In order to achieve viable results, each experiment was run as a mean over 30 samples. $r_1$ and $r_2$ sampled from a uniform distribution $ (0, 1) $ as specified in equation \ref{eq:pso:update-velocity}.

    A memory based \ac{PSO} was used with a star or $ gBest $ social topologie.

    50 particles were used for all functions.

    The constants $ c_1 = c_2 = 1.4 $ as well $ w = 0.7 $ were set as suggested in \cite{vanwyk:overfitting-psoffnn}.

    No $ V_{max} $ was used during this study.

\section{Results}

\begin{table}[htbp]
\caption{Results after 2000 iterations. Means are reported over 30 samples with standard deviations in parenthesis}
\begin{center}
\begin{tabular}{|c|c|c|c|c|}
\hline
& & $ f_1 $ & $ f_2 $ & $ f_3 $\\\hline
\multirow{4}{*}{$ S_1 $} &
\multirow{2}{*}{$ f1 $} &
-4.2666666 & -5.6333333 & -3.93333333\\
&&(0.7272474) & (1.0159833) & (0.2494438)\\
&\multirow{2}{*}{$ f2 $} &
40.6 & 86.5 & 45.7\\
&&(11.842297) & (14.6986393) & (10.030453)\\
\hline
\multirow{4}{*}{$ S_2 $} &
\multirow{2}{*}{$ f1 $} &
-0.866666 & -2.6333333 & -2.1\\
&&(4.74505590) & (3.69218393) & (3.02599845)\\
&\multirow{2}{*}{$ f2 $} &
27.2 & 73.3 & 30.1\\
&&(5.3628350) & (7.6164296) & (4.7423622)\\
\hline
\end{tabular}
\label{tab:glass}
\end{center}
\end{table}

Figures (\ref{fig:f1:fitness}, \ref{fig:f2:fitness}, \ref{fig:f3:fitness}) indicated the objective's fitness over the 2000 iterations that these solutions were trained over. It can be observed that each swarm is attempting to achieve result best for their objective. Each swarm being the best fitness for its particular objective.

One can also observe the objectives of each swarm approaching each other as their results are influenced by the global guides that are effecting their velocity.

\begin{figure}[htbp]
\centerline{\includegraphics[width=90mm]{images/results/f1-fitness.png}}
\caption{$f_1$'s fitness objectives over time }
\label{fig:f1:fitness}
\end{figure}

\begin{figure}[htbp]
\centerline{\includegraphics[width=90mm]{images/results/f2-fitness.png}}
\caption{$f_2$'s fitness objectives over time }
\label{fig:f2:fitness}
\end{figure}

\begin{figure}[htbp]
\centerline{\includegraphics[width=90mm]{images/results/f3-fitness.png}}
\caption{$f_3$'s fitness objectives over time }
\label{fig:f3:fitness}
\end{figure}

\newpage
Figures (\ref{fig:f1:solutions}, \ref{fig:f2:solutions}, \ref{fig:f3:solutions}) plot the solutions according to their swarm (x-axis: Alignment fitness, y-axis: Leading space fitness). For each of the testing sequences one can see that the swarms form what appears to be a Pareto front \cite{kian-lim:pso}. This being the place where the one objective can't improve more without negatively impacting the other.

\begin{figure}[htbp]
\centerline{\includegraphics[width=90mm]{images/results/f1-solutions.png}}
\caption{$f_1$'s non-dominated solutions }
\label{fig:f1:solutions}
\end{figure}

\begin{figure}[htbp]
\centerline{\includegraphics[width=90mm]{images/results/f2-solutions.png}}
\caption{$f_2$'s non-dominated solutions }
\label{fig:f2:solutions}
\end{figure}

\begin{figure}[htbp]
\centerline{\includegraphics[width=90mm]{images/results/f3-solutions.png}}
\caption{$f_3$'s non-dominated solutions }
\label{fig:f3:solutions}
\end{figure}

\newpage
\section{Conclusion}
In conclusion one can observe the ability of \ac{VEPSO}'s in solving multiple sequence alignment problems. \ac{VEPSO} did however struggle with lager sequences like in $f_2$ and one can assume this is due to lack of exploration. The addition of achieve, as discussed in \cite{kian-lim:pso}, would significantly benefit the optimisation as well as allow the addition of multiple solutions resulting in in increased exploration.

\newpage

\nocite{*}
\noindent
\bibliographystyle{IEEEtran}
\indent
\bibliography{IEEEabrv,sbc-template}

\section{Appendix}

$f_1$, sequence provided, where $x_j \in [0, 7]$
\begin{lstlisting}
        a b c d e f
        b b d h g
        c a b f
\end{lstlisting}

$f_2$, longer sequence, where $x_j \in [0, 10]$
\begin{lstlisting}
        a b c d e f a a h
        f b b d h g h h
        c a b f f e a
\end{lstlisting}

$f_3$, a sequence that would benefit from bounds bigger than the number of sequences, where $x_j \in [0, 7]$
\begin{lstlisting}
        a b c d e f
        g h i j k
        a b c d
\end{lstlisting}

\end{document}


\documentclass{stdlocal}
\begin{document}
\section{Previous Work} % (fold)
\label{sec:previous_work}
  In the last few years, the vectorization of PRNGs has not received great attention.
  We will give a brief overview of the major contributions concerning the implementation of vectorized PRNGs.
  The respective mathematical concepts and properties concerning PRNGs are given in section \ref{sec:pseudorandom_number_generators}.

  Because the SIMD routines are a form of data-level parallelism, their application to PRNGs has to exploit the facility of multiple streams for small generators to take full advantage of the whole size of vector registers.
  According to \textcite{fog2015}, there are a number of techniques to initialize multiple, independent pseudorandom streams for vector registers.
  One of it uses multiple instances of the same generator with different seeds.
  This method can lead to overlapping subsequences.
  \citeauthor{fog2015} calculates the probability of this to happen and gives an example of when to ignore a possible overlap.
  Some PRNGs offer a jump-ahead feature which can be used to initialize all instances with only one seed.
  Each stream will then be a non-overlapping subsequence of the same generator with certain length.
  There are families of PRNGs that are described by the same algorithm only differing in their underlying parameters.
  Using different sets of parameters for every instance will generate independent streams.
  If the PRNG has a larger state size, like the standard Mersenne Twister, called MT19937, we do not have to deal with multiple instances of the same generator.
  Instead, multiple streams can be generated by computation of consecutive elements.
  \citeauthor{fog2015} compares these methods and gives a general advice.
  Furthermore, he explains that it will be better to combine different generators for superior statistical properties.
  But this approach has no direct impact on vectorization techniques as we first have to discuss the implementation of the stand-alone generators.
  Combining those will then be a trivial operation that will not be discussed here.

  \textcite{saito2008} presented an alternative algorithm, called SFMT, to the widely used Mersenne Twister optimized for the SSE instruction set which is working with 128-bit data types.
  The authors have been one of the first people to officially vectorize PRNGs.
  They provide a library for academic usage and could successfully show that their generation scheme is much more efficient when using SSE-based instructions \autocite{sfmt}.
  On the other hand, processor architectures are evolving.
  Using the SFMT on modern hardware supporting AVX or even AVX512 will not make full use of the complete vector registers because the algorithm is specialized for the older SSE registers.

  With the \citetitle{intel-mkl}, \citeauthor{intel-mkl} provides a high-performance library for C and C++ exploiting the features and microarchitectures of Intel processors.
  Besides optimized mathematical functions, they also provide several different implementations of vectorized PRNGs ready to be used after linking the library.
  Especially, we get variants of the standard Mersenne Twister MT19937 and the SFMT to name the most common generators.
  Because they develop their own CPUs, Intel is able to create fine-tuned code for nearly every given microarchitecture resulting in a high-quality of their implementations.
  However, the source code of the library is not open-source and cannot be easily adjusted for differing architectures that are not built by Intel.
  Furthermore, the interface to use the routines of the library tend to be complicated and due to their backwards compatibility it is unlikely to change.
  \autocite{intel-mkl}

  \textcite{barash2017} describe how to implement a lot of PRNGs by using the AVX instruction set facilities.
  They, too, provide a library, called \citetitle{rngavxlib}, for academic use with an implementation of the MT19937 even surpassing Intel's implementation \autocite{guskova2016}.
  But the interface of the library is again C-compatible and introduces an overhead in development time while using C++.
  \citeauthor{barash2017} manually vectorize their code through inline assembly statements which makes it unreadable, difficult to understand and not generalizable to other SIMD architectures.

  In \textcite{lemire-pcg} and \textcite{lemire-xorshift}, \citeauthor{lemire-pcg} supplies two open-source code snippets for the vectorization of a permuted congruential generator (PCG) and a Xorshift generator with Intel's AVX and AVX512 instruction set features by using the respective intrinsics in C++.
  Therefore the code is well suited for analyzing the application of SIMD intrinsics to PRNGs and comparing different vectorization schemes.
  His implementations are not tested and not well documented.

  \textcite{vigna-xoroshiro} presents a modern PRNG with good statistical properties which is also known as Xoroshiro128+.
  In his explanation, \citeauthor{vigna-xoroshiro} talks about triggering automatic vectorization by using four independent streams of the named generator.
  He mentions, that it sometimes seems to be difficult to successfully force the compiler to generate vector intrinsics.
  As a consequence, a manual vectorization technique should be much more appropriate as we will show in the next sections.
  \textcite{blacher2018} already uses a vectorized implementation of the Xoroshiro128+ to calculate the pivot of randomized and vectorized quicksort implementation without further testing its performance or quality.

  According to the explanations and descriptions given, we have chosen to vectorize another variant of the MT19937 with the SSE and AVX instruction set because, even if it shows some statistical flaws and has a big state, it is the de facto standard for current applications.
  Additionally, the MT19937 is reliable, has an extremely long period, can be used for multidimensional simulations and will perfectly serve as an academic example on how to vectorize a PRNG.
  Furthermore, we will show how to implement two PRNGs with the SSE and AVX instruction set, namely the Xoroshiro128+ and the Middle Square Weyl Sequence RNG (MSWS).
  The idea is to raise the performance of the Xoroshiro128+ in comparison to an automatic vectorization process and to measure its statistical quality and speed-up.
  The MSWS has not been vectorized, yet.
  It is a non-linear but fast and modern generator for which we have to use a slightly different vectorization technique.
  All this will give us an insight into the vectorization of PRNGs.
% section previous_work (end)
\end{document}


\chapter{Conclusion and Outlook}
This work shows that a source independent approach for the high-level event reconstruction for the LST-1 prototype yields good results,
if sufficiently strict event pre-selection is performend.

Necessary preprocessing was performend using two different versions of the official low-level analysis pipeline \texttt{lstchain}, which is still
in development, and the different results were compared.
In this work, the high-level reconstruction is based on the random forest algorithm and its implementation in the \texttt{aict-tools} package.
In order to use the \texttt{aict-tools}, some changes to the data had to be done complementing the progressing development of the \texttt{aict-tools}.
By the time of writing many CTA conventions are now being suppored by the \texttt{aict-tools} and development continues.

Simulations of diffuse protons, diffuse gamma-rays and gamma-rays from a point-like source were used for the training of the reconstruction models and
the angular resolution was calculated using point-like gamma-rays.
Background separation, energy estimation and origin reconstruction based on the disp method perform well and the best angular resolution is
achieved for energies between $\SI{1}{\tera\electronvolt}$ and $\SI{10}{\tera\electronvolt}$.
The trained models were used to analyze observations of the Crab Nebula and Markarian 421 done by the LST-1.
In both cases a detection with a significance of $\num{27.7} \sigma$ for the Crab Nebula and $\num{43.7} \sigma$ for Markarian 421 was achieved.

During the course of this work, members of the LST Collaboration discovered, that the optical efficiency of the camera is lower than
the value assumed for the simulations.
To compensate for this, they introduced a constant scaling factor for the optical efficiency into the preprocessing.
This improved the results overall which is shown as part of this work.
If a less strict event pre-selection is used which includes more low energy events, the performance of the individual reconstruction models gets only slightly worse.
However this does not apply for the observational results for the Crab Nebula and Markarian 421, instead a much larger negative impact can be observed.

A steep decrease of the reconstruction performance is visible for energies below $\SI{0.3}{\tera\electronvolt}$.
Because of this, the observational results for less strict event pre-selection might improve if the size of the on and off regions $\theta_\text{max}^2$ and the
prediction threshold $t_\gamma$ get chosen differently depending on the reconstructed energy of the observed particle.
As the mismatch of the optical efficiency is fixed now, those two parameters should also be optimized based on simulations instead of the observational data
for the Crab Nebula.
This would make the results for different observations more comparable.

As the LST-1 still was not fully functional in January, no observations of the Crab Nebula using wobble have been done by the time of writing.
Such observations would likely yield better results, because wobble mode enables better background estimation as described in \autoref{sec:wobble}.
Further optimization of the event pre-selection is also advised.
Overall the continued development of the whole analysis pipeline and new observations will help to improve on the results presented in this work.


\section{Conclusions and future work}

\paragraph{Testing physical space}
You, gentle reader, may wonder why of all the theorems to be proved
given this set up we pick the one above. In some sense it's hardly
central to quantum mechanics. We see it as central in the sense that
it firmly establishes a notion of physical space arising from a notion
of the equivalence of behavior. Relating bisimulation to a metric is a
big step forward, but one is faced with interpreting the relationship
of that metric space to something more physical. Quantum mechanical
notions of ``physical'' space are still far from intuitive, but by
relating this idea of distance as testing to calculations that predict
physical circumstances we are making a not insignificant step forward
toward an understanding of the physical space we inhabit as
essentially dynamic.

\paragraph{Effectivity and simulation}
One of the observations we have yet to make is that the entire program
spelled out here is effective. We have built various interpreters for
the reflective calculus at work in this interpretation. In principle,
then, we can simulate quantum mechanics on a computer. The place where
the simulation may lose fidelity is the infinitely branching summation
for the annihilator.

In this connection i also want to point out that the evaluation style
calculation of the inner product puts the non-determinism of the
summation right at the heart of measurement. This suggests that
Milner's original reduction-based formulation of the dynamics of his
calculi in terms of sums was not just notationally suggestive of a
notion of measure-and-continue but captured some significant part of a
physical intuition.

\paragraph{Quantum continuations}
In light of this last observation i want to point out that the
connection to the observation regarding iterated experiments and
continuations made in section \ref{sec:quantum_continuation}. An
account of continuation is necessary to provide a truly compositional
story of physical processes involving quantum operations. As noted, in
a real lab, when a measurement is made the observation can be made to
feed into another device that then makes another measurement
conditioned on the results of the first. This is not a mere nicety; it
is a pragmatic requirement of a theory that provides a computational
basis for the design of large-scale processes (i.e., consisting of
many steps, or having a high degree of branching, or enjoying some
other complexity characteristic) involving quantum operations. It
shows up in descriptions of quantum cryptographic protocols
\cite{DBLP:conf/popl/GayN05} \cite{DBLP:conf/lics/AbramskyC04} and
computational processes that mix classical and quantum computations
\cite{DBLP:conf/tlca/SelingerV05}
\cite{DBLP:conf/flops/Selinger04}. This suggests that there might be
advantages to a more uniform account of dynamics that encompasses both
quantum and computational paradigms.

\paragraph{Quantum logic}
In this connection, we also note that by virtue of having the
Hennessy-Milner construction, we can pull the construction through the
interpretation of QM. This gives us a natural candidate for a quantum
logic that enjoys an extremely tight connection with it's domain of
interpretation, making the construction much less ad hoc (rather it is
the image of functor!).

\paragraph{Quantum probabiity}
i have questions about the basis of the interpretation of inner
product as probability amplitude. In particular, using which
axiomatization of probability theory does the notion of probability
amplitude earn the right to be so dubbed? In other words, where is the
proof that the operation for calculating a probability amplitude (and
then squaring) satisfies the axioms of what it means to calculate a
probability? Even if such a proof exists (i have yet to find it in the
literature), i wonder if it might not be possible to turn things on
their heads. Can we view the calculation of the probability amplitude
as an axiomatization of probability? If so, then the definition we
give for calculating probability amplitude may provide the basis for
an \emph{effective} theory of probability.

\paragraph{Quantum vs ``biological'' information}
Finally, i want to conclude with a more philosophical observation. At
a recent workshop in which QM was a predominant topic i noticed
something about quantum information. The speaker was giving a riveting
discussion of axiomatic QM and showing how properties of ``no
cloning'' and ``no deleting'' emerged as consequences of the
axiomatization. Theorems of this form are necessary to give us a sense
of confidence that our axioms characterize the physical theory. What
struck me, though, was that if quantum information is neither erasable
nor replicable it is markedly different from \emph{life}. Two of the
things we know about life is that

\begin{itemize}
  \item it ends;
  \item to gain some measure of persistence, to transcend it's
    finitude it is imminently copyable.
\end{itemize}

Both of these qualities are summarized succinctly in the aphorism: all
flesh is grass. For me these two kinds of ``information'' -- call them
quantum and biological -- are end points on a spectrum of strategies
for persistence. At one end, we have those curious entities that enjoy
uniqueness and permanence; at the other, we have those who in the face
of a certain end and an uncertain present make a go of passing
something on. To me one of the more remarkable aspects of the latter
strategy is that in the presence of noise (and certain features of
copying) we get a kind of dynamism, a chance for improvement against a
given persistent condition.

% subsection other_calculi_other_bisimulations_and_geometry_as_behavior (end)


\section{Anisotropy Quantification}
\label{sec:anisotropy_code}

See the function \textit{quantify\_anisotropy} in Appendix \ref{sec:omega_code}
on how the anisotropies are calculated.


\section{Introduction}

Bitcoin, the most widely known and most valuable cryptocurrency, uses
a graph-based ledger model built on the concept of \emph{\UTXO{}s} (\emph{unspent
  transaction outputs})~\cite{formal-model-of-bitcoin-transactions,Zahnentferner18-UTxO}. Individual \emph{transactions} consist of a list of \emph{inputs} and a list of \emph{outputs}, where outputs represent a specific \emph{value} (of a cryptocurrency) that is available to be spent by inputs of subsequent transactions. Each output can be spent by (i.e., connect to) exactly one input. Moreover, we don't admit cycles in these connections, and hence we can regard a collection of transactions spending from each other as a directed acyclic graph, where a transaction with $m$ inputs and $n$ outputs is represented by a node in the graph with $m$ edges in and $n$ edges out.
The sum of the values consumed by a transaction's inputs must equal the sum of the values provided by its outputs, thus value is conserved.

Whether an output can be consumed by an input is determined by a function $\nu$ attached to the output, which we call the output's \emph{validator}. A transaction input proves its eligibility to spent an output by providing a \emph{redeemer} value $\rho$, such that \(\nu(\rho) = \true\) --- redeemers are often called \emph{witnesses} in Bitcoin. In the simplest case, the redeemer is a cryptographic hash of the spending transaction signed by an authorised spender's private key, which is checked by the validator, which embeds the corresponding public key. More sophisticated protocols are possible by using more complex validator functions and redeemers --- see \cite{bitml} for a high-level model of what is possible with the functionality provided by Bitcoin.

The benefit of this graph-based approach to a cryptocurrency ledger is that it plays well with the concurrent and distributed nature of blockchains. In particular, it forgoes any notion of shared mutable state, which is known to lead to highly complex semantics in the face of concurrent and distributed computations involving that shared mutable state.

Nevertheless, the \UTXO{} model, generally, and Bitcoin, specifically, has been criticised for the limited expressiveness of programmability achieved by the validator concept. In particular, Ethereum's \emph{account-based ledger} and the associated notion of \emph{contract accounts} has been motivated by the desire to overcome those limitations. Unfortunately, it does so by introducing a notion of shared mutable state, which significantly complicates the semantics of contract code. In particular, contract authors need to understand the subtleties of this semantics or risk introducing security issues (such as the vulnerability to recursive contract invocations that led to the infamous DAO attack~\cite{DAO-attack}).

\paragraph{Contributions.}

The contribution of the present paper is to propose an extension to the basic \UTXO{} ledger model, which
\begin{inparaenum}[(a)]
\item provably increases expressiveness, while simultaneously
\item preserving the dataflow properties of the \UTXO{} graph; in particular, it forgoes introducing any notion of shared mutable state.
\end{inparaenum}
More specifically, we make the following contributions:
%
\begin{itemize}
\item We propose the \emph{\EUTXO{} model}, informally in Section~\ref{sec:informal-eutxo} and formally in Section~\ref{sec:formal-model}.
\item We demonstrate that the \EUTXO{} model supports the implementation of a specific form of state machines (\emph{Constraint Emitting Machines}, or \CEM{}s), which the basic \UTXO{} model does not support, in Section~\ref{sec:expressiveness}.
\item We provide a formalisation of both the \EUTXO{} model
  and Constraint Emitting Machines. We prove a weak bisimulation
  between the two using the Agda proof
  assistant\site{https://github.com/\GitUser/formal-utxo/tree/\AgdaCommit}, building on
  previous work by Melkonian et al.~\cite{formal-eutxo}.
\end{itemize}

Section~\ref{sec:related} summarises related work.

The \EUTXO{} model will be used in the ledger of \Cardano{}, a major blockchain
system currently being developed by IOHK.
It also provides the foundation of Cardano's smart contract platform
\emph{Plutus}\site{https://github.com/input-output-hk/plutus}, which includes a small
functional programming language \emph{\Plutus{}} which is used to implement \script{}s.
Although a technical description of Cardano itself is beyond the scope of this paper,
one can try out the Plutus Platform in an online playground.\site{https://prod.playground.plutus.iohkdev.io/}

Other future work includes a formal comparison of \EUTXO{} with Ethereum's account-based model.

\iftoggle{anonymous}{
\paragraph{Anonymisation.}

For the purposes of submission, we have anonymised the following names:
\begin{itemize}
\item \Cardano{}: a major blockchain system.
\item \Plutus{}: a small functional programming language.
\end{itemize}
}{}


% !TEX root =  paper.tex

\section{Related Work}

We review some of the most successful segmentation methods that have been applied to large-scale EM images in connectomics.
%Fig.~\ref{fig:pipeline} shows the results of a typical connectomics segmentation pipeline.

\paragraph{Pixel-based methods.}
A large amount of connectomics research considers the problem of extracting segmentation information at the pixel (i.e., voxel) level from the raw EM images. Some early techniques apply computationally expensive graph partitioning algorithms with a single node per pixel~\cite{andres2012globally}. However, these methods do not scale to terabyte datasets. More recent methods train classifiers to predict membrane probabilities per image slice either using 2D~\cite{ciresan2012deep,jain2010boundary,kaynig2015large,seymour2016rhoananet,amelio_segmentation} or 3D CNNs~\cite{lee2015recursive,ronneberger2015u,turaga2010convolutional}.

Oftentimes these networks produce probabilities for the affinity between two voxels (i.e., the probability that adjacent voxels belong to the same neuron). The MALIS cost function is specifically designed for generating affinities that produce good segmentations~\cite{briggman2009maximin}. More recently, flood-filling networks produce segmentations by training an end-to-end neural network that goes from EM images directly to label volumes~\cite{januszewski2016flood}. These networks produce impressive accuracies but at a high computational cost.

\paragraph{Region-based methods.}
%Lately, many recent advancements in segmentation use artificial neural networks since machine-learned features outperform hand-designed ones~\cite{bogovic2013learned}.
%A significant amount of research focuses on the training and optimization of these deep neural networks~\cite{chatfield2014return,maas2013rectifier,nesterov1983method}.
Several pixel-based approaches generate probabilities that neighboring pixels belong to the same neuron.
Often a watershed algorithm will then cluster pixels into super-pixels~\cite{zlateski2015image}.
Many methods build on top of these region-based strategies and train random-forest classifiers to produce the final segmentations~\cite{seymour2016rhoananet,nunez2014graph,10.1371/journal.pone.0125825,parag2017anisotropic,zlateski2015image}.

\paragraph{Error-correction methods.}
Some recent research builds on top of these region-based methods to correct errors in the segmentation either using human proofreading~\cite{haehn2017guided,haehn2014design,mojo2} or fully automatically~\cite{rolnick2017morphological,error_correction_using_CNN}.
However, to our knowledge, our method is the first to extract a 3D graph from pixel-based segmentations for a true top-down error correction approach. This allows us to enforce domain-specific biology constraints and use efficient graph partitioning algorithms. Many segmentation and clustering algorithms use graph partitioning techniques~\cite{andres2012globally} or normalized cuts for traditional image segmentation~\cite{kappes2016higher,shi2000normalized,tatiraju2008image}.
Even though graph partitioning is an NP-Hard problem~\cite{demaine2006correlation} there are several useful multicut heuristics that provide good approximations with reasonable computational costs~\cite{horvnakova2017analysis}. We use the method of Keuper et al.~\cite{keuper2015efficient} to partition the extracted 3D graph into the final neural reconstruction.


\section{Operations on Specifications}
\label{OperationsOnSpecifications}

The specification building operators \kw{+} and \kw{rename} in
module expressions can also be invoked interactively.
%As CEC is planned to be an experimental system for specification (program) development
%it offers several operations for combining specifications. The operations include the
%renaming of operator and sort names as well as the union of two specifications.

With these primitive operations more complex ones, such as passing actual parameters to
specification modules can be derived, cf. chapter \ref{exampleSession}.
%The parameter passing is
%demonstrated by the example of sorted lists over an arbitrary ordered set, that is
%replaced by the set of natural numbers \\
%(\refArrow file \file{cec/KBMANUAL/combining\_example} in this distribution).

\subsection{Renaming of Operator and Sort Names}

\begin{command}[\com{renameSpec}{\kw{[}\comArg{OldSort1} \kw{<-} \comArg{NewSort1}, \ldots ,
\comArg{OldSortN} \kw{<-} \comArg{NewSortN},\\ \hspace*{6.2em}
\comArg{OldOperator1} \kw{<-} \comArg{NewOperator1}, \ldots ,
\comArg{OldOperatorM} \kw{<-} \comArg{NewOperatorM}\/\kw{]}}]
renames the current specification according to the given lists of sort associations
and operator associations. Only injective renamings of operators are allowed.
Sorts may be renamed arbitrarily. Sorts and operators which are not mentioned remain
unchanged.
\end{command}

%The system tries to carry over the  previous termination proof to the renamed
%specification. This is not always possible, as the collapsing of previously distinct
%operator symbols might not be compatible with precedences or abstract interpretations.
%In such a case, all rules will be turned back into equations.

\subsection{Combining of Specifications}

The combine operator forms the union of two specifications if they can be combined,
that is if their termination orderings can proved to be compatible. If so, the two orderings
are combined. \cec{kns} and \cec{poly}\nt{N} are assumed to be incompatible.\bigskip

\begin{command}[\com{combineSpecs}{\comArg{StateName1}\ad\comArg{StateName2}\ad\comArg{CombinedSpec}}]
The specifications  stored (\refArrow \comRef{store})
in specification variables \comArg{StateName1} and \comArg{StateName2}
will  be combined, if possible, by forming
the union of the signature, axioms and pragmas.
If \comArg{CombinedSpec} $\neq$ \cec{user}, the result will be stored in
\comArg{CombinedSpec}, and the current specification will not be affected.
Otherwise, the combined specification becomes the new current
specification.
\comRef{combineSpecs} requires the compatibility of the termination
orderings of the involved specifications:
\begin{itemize}
\item
Termination orderings of the same type may be combinable, also \kw{kns} together with
\kw{neqkns}, yields \kw{kns} for the combined specification.
In the case of \kw{kns} and \kw{neqkns}
the combination fails if the operator precedences are contradictory,
e.g.\ $f > g$ in the first specification but $g \geq f$ in the second
specifcation. Also, different stati for operators are not compatible.
\item
\kw{poly}\nt{N} and \kw{poly}\nt{M} can be combined
into \kw{poly}\nt{N} if $N \geq M$. In addition, operators
which occur in both operand specifications must have
the same interpretation polynomials in the first M components.
\end{itemize}
\end{command}


\section{Part 3.a Likelihood maximization}

For this part, I could not calculate the log-likelihood for the distributions. At the same time, to continue on,
the satellite distribution for the given mass bin can be shown to change as the mass increases. This is shown in Figures
\ref{fig:m11}, \ref{fig:m12}, \ref{fig:m13}, \ref{fig:m14}, and \ref{fig:m15}. To complete this task, what I should have
done would have been to construct, based on the given $n(x)$, the log-likelihood for where most of the satellites would
be located in $x$ dependent on $a$,$b$,and $c$. The maximum likelihood would be where the peak of the satellite distribution
is located on these figures.

The file of the functions used for this exercise is:

\lstinputlisting{three_a.py}

The result of this function is given by:

%\lstinputlisting{3a.txt}

\begin{figure}[h!]
  \centering
  \includegraphics[width=0.9\linewidth]{./plots/satgals_m11.png}
  \caption{Radial distribution of satellites in m11.txt}
  \label{fig:m11}
\end{figure}

\begin{figure}[h!]
  \centering
  \includegraphics[width=0.9\linewidth]{./plots/satgals_m12.png}
  \caption{Radial distribution of satellites in m12.txt}
  \label{fig:m12}
\end{figure}

\begin{figure}[h!]
  \centering
  \includegraphics[width=0.9\linewidth]{./plots/satgals_m13.png}
  \caption{Radial distribution of satellites in m13.txt}
  \label{fig:m13}
\end{figure}

\begin{figure}[h!]
  \centering
  \includegraphics[width=0.9\linewidth]{./plots/satgals_m14.png}
  \caption{Radial distribution of satellites in m14.txt}
  \label{fig:m14}
\end{figure}

\begin{figure}[h!]
  \centering
  \includegraphics[width=0.9\linewidth]{./plots/satgals_m15.png}
  \caption{Radial distribution of satellites in m15.txt}
  \label{fig:m15}
\end{figure}

\section{Part 3.b Function Fitting}

The file of the functions used for this exercise is:

\lstinputlisting{three_b.py}

As in part a), for this part I did not successfully create an interpolator for a,b,and c based off the halo mass bin.


\section{Search for EW-$ZZjj$}

\subsection{MD discriminant}

To further separate the EW-$ZZjj$ component from QCD-$ZZjj$, a MD based on Gradient Boosted Decision Tree (BDT) algorithm~\cite{Coadou_BDT}
is trained with simulated events via TMVA framework\cite{Speckmayer_2010}.
Training is performed between EW (signal) and QCD (background) processes.
Twelve event kinematic variables sensitive to the characteristics of the EW signal are used as input features in training.
Table~\ref{tab:bdt_features} lists those input variables with the order of their importance in BDT response provided by TMVA tool.
One can see the jet-related information provides larger sensitivity.
Then the MD distributions in both SR and QCD CR region are used for statistical fit.

\begin{table}[h]
\begin{center}
\renewcommand\arraystretch{1.8}
\begin{spacing}{0.8}
\begin{tabular}{p{1cm}|p{2cm}|p{8cm}}
\hline
\hline
Rank & Variables                    & Description 			\\ \hline
1    & \mjj                         & Dijet invariant mass 		\\ \hline
2    & $p_{T}^{j1}$                 & \pT of the leading jet		\\ \hline
3    & $p_{T}^{j2}$                 & \pT of the sub-leading jet	\\ \hline
4    & $\frac{p_{T}(ZZjj)}{H_{T}(ZZjj)}$  & \pT of the $ZZjj$ system divided by the scalar \pT sum of Z bosons and two jets \\ \hline
5    & $y_{j1} \times y_{j2}$       & Product of jet rapidities		\\ \hline
6    & \dyjj                        & Rapidity difference between two jets \\ \hline
7    & $Y_{Z2}^{*}$                 & Rapidity of the second Z boson \\ \hline
8    & $Y_{Z1}^{*}$                 & Rapidity of the Z boson reconstructed from the lepton pair with the mass closer to the Z boson mass \\ \hline
9    & $p_{T}^{ZZ}$                 & \pT of 4l system \\ \hline
10   & $m_{ZZ}$                     & Invariant mass of 4l system \\ \hline
11   & $p_{T}^{Z1}$                 & \pT of the Z boson reconstructed from the lepton pair with the mass closer to the Z boson mass \\ \hline
12   & $p_{T}^{\ell 3}$             & \pT of the third lepton \\ \hline
\hline
\hline
\end{tabular}
\end{spacing}
\caption{Input features for the training of MD. }
\label{tab:bdt_features}
\end{center}
\end{table}

\iffalse
\subsection{Statistical procedure}

To examine the compatibility between data and the signal-plus-background hypothesis,
a test statistic is based on the profile likelihood ratio method.
%The likelihood function is the product of all the Poisson probability density functions built in individual MD bins given as:
The binned likelihood function is given as
\begin{equation}
	\mathcal{L}(\mu,\sigma) = \prod_{i}^\mathrm{bins} \mathcal{L}_{\mathrm{poiss}}(N_{\mathrm{data}}\,|\,\mu s(\theta)+b(\theta))_{i} \times \mathcal{L}_{\text{gauss}}(\theta)_{i}
\end{equation}
where the Poisson term presents the statistical fluctuations of the data
and a Gaussian term models the pdf of auxiliary measurement to constrain the systematics.
$\mu$ denotes the signal strength of EW-$ZZjj$ process, computed as the ratio between measured (expected) cross section to the SM prediction.
$\theta$ presents the nuisance parameter, which is the set of parameters that parameterize the effect of systematic uncertainties described in section~\ref{sec:systematics} following the Gaussian distribution.
$N_{data}$ is the number of selected data events, while the $s(\theta)$ is the expected signal yield and $b(\theta)$ is the expected background yield as the function of nuisance parameters.

The test statistic $q_{\mu}$ is defined as:
\begin{equation}
	q_\mu = -2 \ln \left( \dfrac{\mathcal{L}(\mu,\hat{\hat{\theta}}_{\mu})}{\mathcal{L}(\hat{\mu},\hat{\theta})} \right)
\end{equation}
in which $\mathcal{L}(\hat{\mu},\hat{\theta})$ is the unconditional likelihood with respect to both $\mu$ and $\theta$,
and $\mathcal{L}(\mu,\hat{\hat{\theta}}_{\mu})$ is the conditional likelihood for a constant $\mu$.
Signal-like data distributions are more likely to have a low test-statistic ($q_\mu$ close to 0)
while the contributions of background-like data have a larger $q_\mu$.
Under the background-only hypothesis, the compatibility of the observed (Asimov) data with the prediction
is calculated to obtain the observed (expected) significance respectively.
\fi

\subsection{Fitting procedure}

A profile likelihood fit, as described in chapter~\ref{sec:statisticalfit}, is performed on MD discriminant to extract the EW-$ZZjj$ signal from backgrounds.
%The fit combined the observed and expected measurements from both \llll and \llvv channel from EW-ZZ production to gain more statistic.
The binning of MD distributions in SR is optimized to maximize the sensitivity for detecting EW signal.
The normalization of QCD-$ZZjj$ production ($\mu_{QCD}^{llll}$) in \llll channel is determined by data from simultaneously fit in SR and QCD CR as described in section~\ref{sec:background}.
The signal strength of EW-$ZZjj$ production ($\mu_{EW}$) is taken as parameter of interest and floated in the fit.
The effects of the uncertainties related to normalizations and shapes described previously in section~\ref{sec:systematics}
of background processes in the MD distribution are all taken into account.

In most case, a common nuisance parameter is used for each source of systematic in all bins and all categories.
The statistical uncertainties for simulated samples are uncorrelated among all bins, and the background uncertainties only applied to their corresponding backgrounds.
%For combination between two channels, the theoretical uncertainties between \llll and \llvv are uncorrelated due to different fiducial volumes definition.
Furthermore, to be more conservative, the generator modelling uncertainty for QCD-$ZZjj$ production mentioned in section~\ref{sec:systematics}
is separated to be two nuisance parameters in low and high MD region.

\subsection{Result of fit}

The statistical fit is performed both in individual \llll channel, as well as the combination between \llll and \llvv channel to gain more statistic.
The results of statistical fit in \llll final state,
and the one in combined channel are presented in table~\ref{tab:fit_result}.
The \llvv analysis will not be described in this dissertation, but more details can refer to~\cite{ATLAS:2019vrv}.
To drive expected results, the observed data is used for QCD CR to extract normalization factor of QCD component ($\mu_{QCD}^{\llll}$),
while in SR, asimov data built from background prediction and signal model with SM assumed cross section is used.

\begin{table}[!htbp]
\begin{center}
\begin{tabular}{c|c|c|c}
\hline
                 & $\mu_{\mathrm{EW}}$ &  $\mu^{\llll}_{\mathrm{QCD}}$   &  Significance Obs. (Exp.) \\
\hline
\llll          & $1.54 \pm 0.42$     &  $0.95 \pm 0.22$                  &  5.48 (3.90) $\sigma$     \\
\hline
Combination of \llll and \llvv         & $1.35 \pm 0.34$     &  $0.96 \pm 0.22$                  &  5.52 (4.30) $\sigma$     \\
\hline
\end{tabular}
\end{center}
\caption{
Observed \muEW and \muQCD, as well as the observed and expected significance from the individual \llll channel.% and the combined fits.
The full set of systematic uncertainties are included.
}
\label{tab:fit_result}
\end{table}

As a conclusion, in \llll channel, the background-only hypothesis is rejected at 5.5$\sigma$ (3.9$\sigma$) for observed (expected) data,
which leads to the observation of EW-$ZZjj$ production.

Figure~\ref{fig:fit_MD} shows the post-fit MD distributions for \llll events after performing a combined fit in SR (left) and QCD CR (right).
The EW-$ZZjj$ cross section measured in \llll channel is extracted to be $0.94 \pm 0.26$~fb.
\begin{figure}[!htbp]
\begin{center}
\includegraphics[width=0.42\textwidth]{figures/VBSZZ/fit/BDT_4l_SR_postFit.pdf}
\includegraphics[width=0.42\textwidth]{figures/VBSZZ/fit/BDT_4l_QCD_CR_postFit.pdf}
\end{center}
\caption{Observed and post-fit expected multivariate discriminant distributions after the statistical fit in the \llll SR (left) and QCD CR (right).
        The error bands include the experimental and theoretical uncertainties,
        as well as the uncertainties in \muEW and \muQCD.
        The error bars on the data points show the statistical uncertainty on data.
        }
\label{fig:fit_MD}
\end{figure}

Figure~\ref{fig:scaled_mjj} shows the $\mjj$ distribution in SR (left) and QCD CR (right),
where the normalization of EW and QCD processes are scaled according to their observed value in table~\ref{tab:fit_result}.
High $\mjj$ region is more sensitive for EW-$ZZjj$ events detection from this figure.
Figure~\ref{fig:scaled_mzz} shows the spectrum of invariant mass of \llll system ($\mzz$) in SR
also with the normalization of EW and QCD processes scaled.

\begin{figure}[!htbp]
\begin{center}
\includegraphics[width=0.32\textwidth]{figures/VBSZZ/fit/MJJ_4l_SR.pdf}
\includegraphics[width=0.32\textwidth]{figures/VBSZZ/fit/MJJ_4l_QCD_CR.pdf}
\end{center}
\caption{Observed and post-fit expected \mjj distributions in SR (left) and QCD CR (right).
        The error bands include the expected experimental and theoretical uncertainties.
        The error bars on the data points show the statistical uncertainty.
        The contributions from the QCD and EW production of $ZZjj$ events are scaled by 0.96 and 1.35, respectively,
        corresponding to the observed normalization factors in the statistical fit.
        The last bin includes the overflow events.
        }
\label{fig:scaled_mjj}
\end{figure}

\begin{figure}[!htbp]
\begin{center}
\includegraphics[width=0.4\textwidth]{figures/VBSZZ/fit/MZZ_4l_SR.pdf}
\end{center}
\caption{Observed and post-fit expected $\mzz$ spectrum in SR.
        The error bands include the expected experimental and theoretical uncertainties.
        The error bars on the data points show the statistical uncertainty.
        The contributions from the QCD and EW production of $ZZjj$ events are scaled by 0.96 and 1.35, respectively,
        corresponding to the observed normalization factors in the statistical fit.
        The last bin includes the overflow events.
        }
\label{fig:scaled_mzz}
\end{figure}

Figure~\ref{fig:event_display} is the display of one event candidate of EW-$ZZjj$ production in $2e2\mu$ final state with two jets in forward and backward region.
\begin{figure}[!htbp]
\begin{center}
\includegraphics[width=1.0\textwidth]{figures/VBSZZ/fit/resize_340368_454611985_v3.pdf}
\end{center}
\caption{Display of an event candidate of EW-$ZZjj$ production in $2e2\mu$ channel in last MD bin (0.875 < MD < 1.0).
         The invariant mass of the di-jet (four-lepton) system is 2228 (605)~\gev. }
\label{fig:event_display}
\end{figure}


\chapter{}{Literature Review}{Literature Review}

Zero-shot learning (ZSL) approaches can be broadly classified into two categories based on the unseen class information the model has during the training process. In the inductive ZSL framework, we have access to to labeled image data from the seen classes as well as auxiliary information (i.e., semantic attributes/descriptions) about both, seen and unseen classes during the training phase. In the transductive ZSL framework, we have access to auxiliary information (i.e., semantic attributes/descriptions) about both, seen and unseen classes during the training phase as in the case of the inductive ZSL framework. The major difference is that in the case of transductive ZSL, we have access to labeled image data from the seen classes \textit{and} unlabeled image data from the unseen classes during the training phase which is a departure from inductive ZSL. Within both frameworks, one can make an distinction based on the type of setting used to evaluate the model during testing. i.e. the conventional ZSL setting and generalized ZSL setting as described in Chapter 1. In this section we review work on both the inductive and transductive ZSL frameworks and place the proposed approach within the ZSL taxonomy.

\par
\medskip

Preliminary work in inductive ZSL uses a two-stage approach to infer unseen class labels. In the first stage, the attributes of an image are predicted. In the next stage, the class label is inferred by searching for the class label with the most similar set of attributes. Lampert et al~\cite{DAP} introduced the Directed Attribute Prediction (DAP) and Indirect Attribute Prediction (IAP) models  which use the aforementioned two-stage approach. In DAP~\cite{DAP}, a probabilistic attribute classifier is first learned. The class posteriors are then computed and class labels predicted via a maximum a posteriori (MAP) estimate. In IAP~\cite{DAP}, a multi-class classifier is first used to predict the class posterior. The probability of each class is then used to compute the attribute posteriors of an image. While the DAP and IAP frameworks have historically been some of the most widely cited ZSL methods in the literature, they suffer from the problem of domain shift~\cite{domainshift} where the intermediate functions learned from the auxiliary information without any adaptation to the target domain introduce an unknown bias.

\par
\medskip

Subsequent ZSL frameworks attempt to learn a compatibility function from image feature space to the semantic or auxiliary space. These frameworks can be further categorized based on the type of compatibility function that they learn. The first set of methods learn linear compatibility functions whereas the next set of methods learn non-linear compatibility functions.

\par
\medskip

\noindent
\textbf{Linear Compatibility.} Attribute Label Embedding (ALE)~\cite{ale} learns a bi-linear compatibility function between the image space and auxiliary space using a weighted approximate ranking objective. ALE improves upon DAP \cite{DAP} significantly since it can use multiple sources of auxiliary information such as word embeddings and class taxonomies. ALE also overcomes the drawbacks of the two-step process used in DAP by directly predicting the class label without the need for an intermediate step. The Deep Visual-Semantic Embedding (DEVISE) model~\cite{devise} also learns a linear mapping between the image space and semantic space and has been shown to perform well on the large-scale ImageNet data set. The Structured Joint Embedding (SJE) scheme~\cite{SJE} uses an unregularized structured SVM to learn the compatibility function coupled with the Stochastic Gradient Descent (SGD) algorithm for optimization. The Embarrassingly Simple Zero-Shot Learning (ESZSL) scheme~\cite{ESZSL} uses an additional regularization term to suppress noise in the auxiliary space.

\par
\medskip

\noindent
\textbf{Non-Linear Compatibility.}  The Latent Embedding (LATEM) scheme~\cite{LATEM} extends linear compatibility approaches by learning multiple mappings thereby finding a piece-wise linear compatibility function using every image-class pair. LATEM shows improved accuracy over the state-of-art the linear compatibility-based SJE scheme~\cite{SJE}. The Cross-Modal Transfer (CMT) scheme~\cite{CMT} uses a neural network with two hidden layers to learn non-linear projections from image space to Word2Vec~\cite{w2v} space. CMT exploits only information from word embeddings and does not use other sources of auxiliary information such as class attributes used by other methods.
%%% SMB: INCLUDE THE FOLLOWING REFERENCE AND CITATION FOR CMT:
%%% R. Socher, M. Ganjoo, C.D. Manning and A.Y. Ng, Zero-Shot Learning Through Cross-Modal Transfer, Proc. NIPS 2013.

\par
\medskip

Drawing from linear and non-linear compatibility approaches, hybrid models~\cite{gbu} learn a joint embedding of both the image and semantic features into a combined intermediate space. The Semantic Similarity Embedding (SSE) scheme~\cite{sse} uses a max-margin framework to jointly optimize domain data and semantic data. The Convex Combination of Semantic Embeddings (CONSE) scheme~\cite{conse} is inspired by DEVISE~\cite{devise} and maps images into the semantic embedding space via convex combination of the class label embedding vectors without the need for additional training. Synthesized Classifiers (SYNC) \cite{sync} introduces a set of “phantom” object classes whose coordinates exist in both the semantic space and the model space which are then optimized using labeled data such that the synthesized real object classifiers achieve optimal discriminating performance. Wang et al.~\cite{gcn} use a Graph Convolution Network (GCN) and the GLoVe text embedding model~\cite{glove} to generate a knowledge graph embedding. The knowledge graph embedding exploits both, semantic embeddings and domain relationships to predict the object classifiers.
%%% SMB: INCLUDE THE FOLLOWING REFERENCE AND CITATION FOR CONSE:
%%% M. Norouzi, T. Mikolov, S. Bengio, Y. Singer, J. Shlens, A. Frome, G.S. Corrado, and J. Dean, Zero-Shot Learning by Convex Combination of Semantic Embeddings, Proc. ICLR, 2014.

\par
\medskip

Recently, there has been a rise in the use of generative models for ZSL that represent each class as a probability distribution. Generative Framework for Zero-Shot Learning (GFZSL)~\cite{gfzsl} models the class-conditional distributions of seen as well as unseen classes using a multi-variate Gaussian distribution. Generative models such as GFZSL exhibit a significant performance boost in the transductive ZSL setting. The Feature Generating Network (FGN)~\cite{fgn} introduces a novel Generative Adversarial Network (GAN) that synthesizes Convolutional Neural Network (CNN) features conditioned on class-level semantic information, offering a direct shortcut  from a semantic descriptor of a class to a class-conditional feature distribution. The Leveraging the Invariant Side GAN (LisGAN) approach~\cite{LISGAN} generates unseen features from random noise functions which are conditioned by the semantic descriptions. They train a conditional Wasserstein GAN in which the generator synthesizes fake unseen features from noises and the discriminator distinguishes the fake from real via a minimax game. The recent approach based on leveraging the semantic relationships between the seen and unseen object categories termed as LsrGAN~\cite{lsrGAN} performs explicit knowledge transfer by incorporating a novel Semantic Regularized Loss (SR-Loss) function. A Tensorflow-based combination of a Variable Auto-Encoder (VAE) and GAN, termed as TF-vaegan~\cite{tf-vegan}, introduces a feedback loop from a semantic embedding decoder, that iteratively refines the generated features during both the training and feature synthesis stages. The synthesized features together with their corresponding latent embeddings from the decoder are then transformed into discriminative features and exploited during classification to reduce the ambiguities amongst the categories. The TF-vaegan framework~\cite{tf-vegan} is currently regarded as the benchmark in inductive and transductive ZSL settings on the AWA2~\cite{awa}, CUB~\cite{cub}, and SUN~\cite{sun} data sets on the zero-shot learning and generalized zero-shot learning settings.

\par
\medskip

The proposed framework draws from the two-stage approach used by the DAP and IAP approaches~\cite{DAP} but the problem being addressed is substantially different from the standard ZSL problem that the aforementioned ZSL frameworks are designed for. Note that the proposed approach aims to predict a large number of unseen classes from few seen classes, and consequently, we have far less training data compared to the conventional ZSL settings. Hence, generative-adversarial-based and compatibility learning-based ZSL frameworks which require a lot of training data would be expected to perform poorly in this situation. The proposed ZSL framework represents a first step towards a novel ZSL problem formulation that strives to understand how classification accuracy measures change with change in number and type of seen classes. The proposed framework also allows for selection of an optimal number and type of seen classes based on an expected overall classification accuracy measure which aids in the training data collection process.

\newpage