\section{Tracibility and Correctness of Implementation}~\label{sec:correct}

\todo{Discuss the implementation of labels, ACSL contracts, and proofs}

The SBV backend generates one function per expression of the reified AST (abstract syntax tree). Thus the main task consists in generating a contract for the expression, which can be done by induction on the syntax. This is very similar to writing a pretty printer for expressions (corresponding to the token $\langle funStream \rangle$ in the grammar), except for some tricks, which have to be taken into account, which are often due to the absence of implementation of some features described in ACSL.

\begin{itemize}
	\item No let bindings can be written in ACSL contracts.
	\subitem It was decided to do not support let bindings during ACSL contract generation, thus making a code containing let bindings unprovable. Nevertheless, the programmer can use a functional style, changing the let bindings into function, at the cost of having arguments in order to use variables that should be in scope for let bindings.
	\item No mathematical functions are available (sin, cos, tan, sqrt, ...).
	\subitem This is quite normal, given that on embedded systems, the access to math.h is not guaranteed. Thus we decided to transform each trigonometrical function into a call to an external function, called the same way (or sinf if working on floats), and it's up to the user to provide an implementation of the function (or to choose the standard implementation in math.h).
	\item No global invariants are implemented in WP plugin.
	\subitem We split the dereferencing of the pointer in an external function (called ident, which is equivalent to the identity), which allows to have simple enough contracts to be able to prove the assertions, and given that the splitting does not change the semantics of the program (it is similar to an beta-expansion).
	\item No bitwise operator.
	\subitem SBV library was changed in that purpose, and bitwise operators have been deprecated\footnote{https://github.com/LeventErkok/sbv/issues/177}.
\end{itemize}

As requested by the DO-178B norm regarding the safety of critical flight avionics, a traceability has to be ensured between the origin haskell source code file and the generated C code produced. For that purpose, a labeling system was implemented, similar to the trace instruction from Debug.Trace, but prints its label in the C source file instead of stdout. Moreover, the fact that the expressions could be huge after reification (all functions are beta expanded in expressions), it caused problems with frama-c which could not parse contracts of 500000 characters long.

For this purpose, three features were implemented.
\begin{itemize}
	\item The first is a prover mode, which will optimize the code in order to make it easier to prove (by splitting absolute values in half for example).
	\item The second is a pretty printer of the AST into a dot file, which then generates a graph of what a C file is supposed to do (this graph is equivalent to the ACSL contract, so it is easier to see a problem in the contract by looking at the AST graph).
	\item The last are magic characters in labels, which can affect the generated C code when the compilation is in mode prover. The only one implemented is the character \texttt{?} which causes the backend to cut the AST at this node and create an other one. This is done by using the identity as external function which is shown in the Figure~\ref{fig:splitting label} for the following simple Copilot code :
\end{itemize}
\begin{lstlisting}[frame=single,language=haskell]
	import qualified Copilot.Compile.SBV as S

	alt :: Stream Bool
	alt = (label "?splitting" $ not $ externB "externvar" Nothing)

	spec :: Spec
	spec = do
	trigger "trigger" (alt) []

	main = do
	reify spec >>= S.proofACSL S.defaultParams
\end{lstlisting}
\begin{figure}[!htb]
	\vspace*{2cm}
	\includegraphics[width=42mm]{images/label/main.ps}
	\includegraphics[width=42mm]{images/label/splitted.ps}
	\includegraphics[width=42mm]{images/label/splitted2.ps}
	\caption{The splitting process. The AST on the left is split into two smaller ASTs, that then generate two separate functions into two separate files, making it easier to prove the contracts for them.}
	\label{fig:splitting label}
\end{figure}

\subsection{How to create beautiful dot graphs}
Dot code is generated automatically in each C source file. You just have to extract it and then generate the graph usit dot.
The best thing to do, is to use the following C script in \ref{fig:scriptdot} :
\begin{figure}[!htb]
\begin{lstlisting}[language=C]
#include <cstdio>
#include <cstdlib>
#include <string.h>
#include <ctype.h>

#define taille 1000000

char s[taille+1];

int main(int argc, char ** argv)
{
   if (argc <= 1) printf("error not enough arguments\n");
   else if (argc == 2)
   {
      int state = 0;
      FILE* f = fopen(argv[1],"r+");
      if (f == NULL) {printf ("Error opening file"); exit(-1);}
      while(fgets(s,taille, f) != NULL)
      {
         if(strstr(s, "DotEnd*/") != NULL) state = 2;
         if(state == 1)
         {
            if(strstr(s,"?????") != NULL)
            {
               strtok(argv[1],"/");
               printf("%s%s",strtok (NULL, "/"),s+5);
            }
            else printf("%s",s);
         }
         if(strstr(s, "/*DotBegin") != NULL) state = 1;
      }
      fclose(f);
      f = NULL;
   }
   else if (argc > 2) printf("error too many arguments\n");

   return 0;
}

\end{lstlisting}
\caption{The C script that does everything}
\label{fig:scriptdot}
\end{figure}

To prove anything with ACSL, you should do the following :
\begin{enumerate}
	\item First compile with SBV with the function proofACSL (this will generate C source code in copilot-sbv-codegen).
	\item Go in the directory copilot-sbv-codegen and then do make fwp. This will create two log files, one called logfwp, and another compact one named logfwpcompact.
	\item You can automatize the whole process with the make file, by looking at the examples for ACSL.
\end{enumerate}

\begin{figure}[!htb]
	\begin{lstlisting}[language=C]
sandbox:
	m4 main.hs > tmp.hs
	cabal exec -- ghc tmp.hs -o main
	./main

sindent:
	m4 main.hs > tmp.hs
	cabal exec -- ghc -fno-code -ddump-parsed tmp.hs

clean:
	rm *~
	rm tmp.hi
	rm tmp.o
	rm -rf copilot-c99-codegen
	rm main

dot:
	gcc scriptdot.cpp -o a
	for i in copilot-sbv-codegen/*; do if [ "$${i}" != "$${i%.c}" ]; \
	then ./a $$i > $$i.gv; dot -Tps $$i.gv -o $$i.ps; fi; done

acsl:
	m4 main.hs > tmp.hs
	cabal exec -- ghc tmp.hs -o main
	./main
	make dot
	make fwp -C copilot-sbv-codegen

compile:
	m4 -DproofACSL=compile main.hs > tmp.hs
	cabal exec -- ghc tmp.hs -o main
	./main
	make all -C copilot-sbv-codegen

\end{lstlisting}
	\caption{The makefile that does everything.}
	\label{fig:makefile}
\end{figure}


\section{\sc Notable \\ Projects}
	\textbf{Implementation} of the SimpNet convolutional neural network architecture in Python \& Tensorflow.

	\textbf{Implementation} of a real-time face recognition system based on the maximum likelihood approximate nearest neighbor method in Tensorflow \& OpenCV.

	\textbf{Implementation} \& optimization of a fast naive image template matching in CUDA.

	\textbf{Implementation} of the AlexNet convolutional neural network architecture in Tensorflow.

	\textbf{Implementation} of 1D \& 2D kernel density estimation \& visualization in Python.

	\textbf{Implementation} of a pure functional programming language interpreter in Racket.

	\textbf{Implementation} of a wireless network using nRF24L01 and Arduino UNO.

	\textbf{Implementation} of the histogram equalization algorithm on the STMF32F407VGT6 with ARMv4T architecture using ARM assembly.

	\textbf{Co-design} of a smart parking system in VHDL and C.

	\textbf{Implementation} of a weightless neural network library in Python. (ongoing)

More available on \url{https://github.com/hexpheus}.

\endinput


\chapter{Implementation of methods}
\label{cha:implementation_methods}

As many pattern recognition techniques were originally not designed to cope with large amounts of irrelevant features, combining them with FS techniques has become a necessity in many applications (Guyon and Elisseeff, 2003; Liu and Motoda, 1998; Liu and Yu, 2005). The objectives of feature selection are manifold, the most important ones being: (a) to avoid overfitting and improve model performance, i.e. prediction performance in the case of supervised classification and better cluster detection in the case of clustering, (b) to provide faster and more cost-effective models and (c) to gain a deeper insight into the underlying processes that generated the data. However, the advantages of feature selection techniques come at a certain price, as the search for a subset of relevant features introduces an additional layer of complexity in the modelling task. Instead of just optimizing the parameters of the model for the full feature subset, we now need to find the optimal model parameters for the optimal feature subset, as there is no guarantee that the optimal parameters for the full feature set are equally optimal for the optimal feature subset (Daelemans et al., 2003). As a result, the search in the model hypothesis space is augmented by another dimension: the one of finding the optimal subset of relevant features. Feature selection techniques differ from each other in the way they incorporate this search in the added space of feature subsets in the model selection.

In the context of classification, feature selection techniques can be organized into three categories, depending on how they combine the feature selection search with the construction of the classification model: filter methods, wrapper methods and embedded methods. Table 1 provides a common taxonomy of feature selection methods, showing for each technique the most prominent advantages and disadvantages, as well as some examples of the most influential techniques.

----------------------------------------------------------

\section{Feature Selection methods} % (fold)
\label{sec:feature_selection}

\subsection{PCA - Principal Component Analysis} % (fold)
\label{sec:pca}

\begin{lstlisting}[language=Python]
# Calculate the variance explained by principle components
print("Variance_of_each_component:", pca.explained_variance_ratio_)
print("Total_Variance:Explained:",
        round(sum(list(pca.explained_variance_ratio_))*100, 2))
\end{lstlisting}

\begin{figure}[h]
    \centering
    \includegraphics[width=0.8\textwidth,height=0.06\textheight]{Chapters/Figures/pca_2_comp_variance.png}
    \caption{Variance of the best 2 components and their sum}
    \label{fig:total_variance}
\end{figure}

\begin{figure}[h]
    \centering
    \includegraphics[width=0.75\textwidth,height=0.3\textheight]{Chapters/Figures/number_of_components.png}
    \caption{Cumulative Explained Variance\\ as a function of the Number of Components}
    \label{fig:variance_graph}
\end{figure}

\subsection{SOM - Self Organizing Maps} % (fold)
\label{sec:som}

\subsection{LDA - Linear Driscriminant Analysis} % (fold)
\label{sec:lda}

\section{Feature Extraction methods} % (fold)
\label{sec:feature_extraction}

\subsection{Univariate Filtering} % (fold)
\label{sec:inserting_tables}

\subsubsection{Chi-Square test} % (fold)
\label{sec:inserting_tables}

\subsubsection{ANOVA F-test} % (fold)
\label{sec:inserting_tables}

\subsection{Multivariate Filtering} % (fold)
\label{sec:inserting_tables}

\subsubsection{CFS – Correlation Based Feature Selection} % (fold)
\label{sec:inserting_tables}

\subsubsection{MRMR – Minimum Redundancy Maximum Relevance} % (fold)
\label{sec:inserting_tables}


\documentclass[envcountset]{beamer}
%
% Choose how your presentation looks.
%
% For more themes, color themes and font themes, see:
% http://deic.uab.es/~iblanes/beamer_gallery/index_by_theme.html

\mode<presentation>
{
  \usetheme{default}      % or try Darmstadt, Madrid, Warsaw, ...
  \usecolortheme{default} % or try albatross, beaver, crane, ...
  \usefonttheme{}  % or try serif, structurebold, ...
  \setbeamertemplate{navigation symbols}{}
  \setbeamertemplate{caption}[numbered]
  \setbeamertemplate{footline}[frame number]
}
\usepackage{units}
\usepackage{mathtools}
\usepackage[english]{babel}
\usepackage[utf8x]{inputenc}
\usepackage{lmodern}
\usepackage{graphicx}
\usepackage{amssymb,amsmath,amsthm, amsmath}  % use mathematical symbols
\usepackage{bbold}
\usepackage{framed}
\usepackage{comment}
\usepackage[%
	font={small, sf},
	labelfont=bf,
	format=hang,
	format=plain,
	margin=0pt,
	width=0.8\textwidth,
]{caption}
\usepackage[list=true]{subcaption}

\usepackage{tikz}
\usetikzlibrary{matrix, arrows,positioning, fit, automata, calc}
\usepackage[absolute,overlay]{textpos}
\theoremstyle{definition}
\newtheorem{mydef}{Def.}[theorem]

\newenvironment{proenv}{\only{\setbeamercolor{local structure}{fg=green}}}{}

\title[PH Distributions]{Phase-type distributions}
\subtitle{}
\author{Fernando Zepeda}
\institute{TU Berlin - Lufthansa Systems}
\date{\today}

%source macro for the images

\setbeamercolor{framesource}{fg=gray}
\setbeamerfont{framesource}{size=\tiny}
\newcommand{\source}[1]{\begin{textblock*}{4cm}(8.7cm,8.6cm)

    \begin{beamercolorbox}[ht=0.5cm,right]{framesource}
        \usebeamerfont{framesource}\usebeamercolor[fg]{framesource} Source: {#1}
    \end{beamercolorbox}
\end{textblock*}}
%----------------------------------------

\begin{document}

\begin{frame}[noframenumbering, plain]
  \titlepage

\end{frame}

% Uncomment these lines for an automatically generated outline.
\begin{frame}{Outline}
  \tableofcontents
\end{frame}

\section{Introduction}

\begin{frame}{Introduction}

Some data do not follow necessarily the form of any standard distribution (unconventional forms):
	\begin{itemize}
	\item Probability distributions may posses any shape
	\item Rugged and multimodal data distributions
	\end{itemize}

\vspace{0.3cm}

The \textbf{phase-type (PH)} family of distributions have some useful properties in fitting this kind of data
	\begin{itemize}
	\item They are dense in the field of all positive-valued distributions $[ 0,\infty)$
	\item Thus, they can be used to approximate any positive-valued distribution Bladt. et al. (2003)
	\item Markovian properties make them analytically flexible
	\end{itemize}

\end{frame}
\begin{frame}{Motivation}

\textbf{Goal}:
Estimate
\begin{itemize}
\item Buying probabilities (and ticketing probabilities)
\item Inter-arrival time of bookings
\end{itemize}

by fitting PH distributions to them given booking historical data

\end{frame}

\section{Markov Chains}

\begin{frame}{Markov Chains}

Let $\mathbf{S}$ denote a countable set of states, and let $\{X(t)\}_{t\geq0}^{\infty}$ be a stochastic process with state space $\mathbf{S}$

	\begin{itemize}
	\item Let $n \in \mathbb{N}$ be the size of $\mathbf{S}$
	\end{itemize}

$\{X(t)\}_{t\geq0}^{\infty}$ is a continuous-time Markov chain \textbf{(CTMC)}, if it is characterized by the \textit{Markov property}:
\\[10pt]
\centering
\begin{framed}
	$Prob(X(t_{k+1})=  x_{k+1} | X(t_k) = x_k, \ldots, X(t_0) = x_0) =$
	\\[4pt]
	$Prob(X(t_{k+1}) = x_{k+1} | X(t_k) = x_k)$
\end{framed}

for any $t_{k+1} \geq t_{k} \geq \ldots \geq t_1 \geq t_0 \geq 0 $ and $x_l \in \mathbf{S}$
\end{frame}

\begin{frame}{}

The transition between states $x$ are defined by the values $p_t(i,j)$ which are contained in a matrix with transition probabilities $\mathbf{P_t}$
\\[10pt]
\begin{framed}
The state probabilities at time $t$ are denoted by
\\[5pt]
\centering
$p_t(j) = Prob(X(t) = j), j \in \mathbf{S}$ with $\sum_j p_t(j)=1$
\\[10pt]
$\pi(0) = [ p_0(1), p_0(2), \ldots, p_0(n)]$: initial probability vector
\end{framed}
\centering
\flushleft
Random times between states transitions $V_1, V_2, \ldots, V_n$ are exponentially distributed random variables with parameter $\lambda(i)$ for state $i$
\flushleft
$V_i$: holding time in state $i$ with distribution function:
$$Prob(V_i \leq t) = 1 - e^{-\lambda(i)t}, t \geq 0$$
\end{frame}

\begin{frame}{}

The probabilistic behavior if the CTMC can be summarize in terms of its \underline{infinitesimal generator}: a $n\times n$ matrix $\mathbf{Q}$ with components:

$$
\mathbf{Q(i,j)} =
\begin{cases}
  -\lambda(i)           & \text{if}\ i = j	,\\
  \phantom{-}\lambda(i,j) & \text{if}\ i \neq j .

\end{cases}
$$

\begin{framed}
\begin{itemize}
\item Since $\lambda(i) \geq 0$, it follows $\mathbf{Q}(i,i) \leq 0$: \textit{diagonal elements are non-positive}
\item Transition to $j$ feasible in state $i$ then $\mathbf{Q}(i,j) > 0$, otherwise $\mathbf{Q}(i,j) = 0$: \textit{non-diagonal elements must be non-negative}
\item $\sum_j\mathbf{Q}(i,j) = 0$
\end{itemize}
\end{framed}

\end{frame}

\begin{frame}{Absorbing Markov Chains}
\begin{figure}
\centering
\subcaptionbox[CTMC]{%
		\centering
		CTMC with transient states only%
      		\label{subfig:CTMC}%
}[
	0.40\textwidth
]
{
	\begin{tikzpicture}[node distance = 3cm, ->, >=latex, auto, every edge/.append style={thick}]
	\node[state] (1) {$1$};
	\node[state] (2)[right of =1]{$2$};
	\path (1) edge[bend left] node{$4$} (2)

	  	(2) edge[bend left] node{$2$} (1);
\end{tikzpicture}

}
\hspace{0.1\textwidth}
\subcaptionbox[Short Subcaption]{%
		\centering
		CTMC with one absorbing state%
      		\label{subfig:sublabel2}%
}[
	0.40\textwidth
]
{
\begin{tikzpicture}[node distance = 2.5cm, ->, >=latex, auto, every edge/.append style={thick}]
	\node[state](1) {$1$};
	\node[state](2)[right of =1]{$2$};
	\coordinate(Middle) at ($(1)!0.5!(2)$);
	\node[state, accepting](3)[below of =Middle]{$3$};

	\path  (1) edge[bend left] node{$4$} (2)
		  (2) edge[bend left] node{$2$} (1)
		  (2) edge[bend left] node{$\nicefrac{1}{2}$} (3)
		  (1) edge[bend right, left] node{$\nicefrac{1}{2}$} (3);
\end{tikzpicture}
}

\caption[Short Caption]{\centering Different Markov chains and their transition representation}
\label{fig:label}
\end{figure}
\end{frame}

\begin{frame}{Absorbing Markov Chains}
	\framesubtitle{Some definitions}

A state $j$ is reachable from another $i$ when we have

$$p_t(i,j) = Prob(X(t+u) = j | X(u) = i) > 0$$
for some $t$

\begin{mydef}
States $i$ and $j$ can communicate with each other if $i$ is reachable from $j$ and viceversa.
\end{mydef}
\vspace{5pt}
Let $\mathbf{C} \subset \mathbf{S}$. If all states in $\mathbf{C}$ communicate, we call it communicating set

\begin{mydef}
A subset $\mathbf{C}$ of the state space $\mathbf{S}$ is said to be closed if $\mathbf{P}(i,j) = 0$ for any $i \in \mathbf{C}, j \not\in \mathbf{C}$.
\end{mydef}

\end{frame}

\begin{frame}
\hspace{5pt} If $\mathbf{C}$ consists of a single state, say $i$, then $i$ is said to be an absorbing state.
\begin{framed}
It holds for $i$ that $\mathbf{P}(i,i) = 1$: \textit{a process can never leave a closed set after entering it}
\end{framed}
\begin{mydef}
A state $i \in \mathbf{S}$ is transient, if the probability of returning to $i$ after leaving it is less than 1.
\end{mydef}

\end{frame}

\section{Phase-type distributions}

\begin{frame}{PH distributions and absorbing Markov chains}

If every state in a Markov chain is either absorbing or transient, then the Markov chain is called \textit{absorbing Markov chain}

\begin{framed}
	Particularly interesting case: absorbing Markov chains with one single absorbing state
\end{framed}

\underline{Assumption}: state space $\mathbf{S}$ of the continuous time absorbing Markov process $\{X(t)\}_{t\geq0}^{\infty}$ is finite and contains the set of \textbf{transient} states $\mathbf{S}_T = \{1, \ldots, n\}$ and a single absorbing state $n+1$

\end{frame}

\begin{frame}[fragile]
\Large

\[ \mathbf{Q} =
\begin{tikzpicture}[baseline=-0.5]

\matrix[matrix of math nodes, left delimiter = (, right delimiter = ) ](m){
\mathbf{D_0} & \mathbf{d_1}\\
\mathbf{0} & 0\\
};
\node<2>[label={\tiny $n\times 1$},fit=(m-1-2), rounded corners, draw=blue, scale=.6]{};
\node<2>[label={\tiny $n\times n$},fit=(m-1-1), draw=red, scale=.6]{};
\node<2>[label=below:{\tiny $1\times n$},fit=(m-2-1), draw=green, scale=.6]{};
\node<2>[label=below:{\tiny $1\times 1$},fit=(m-2-2), draw=violet, scale=.6]{};
\end{tikzpicture}
\]

\normalsize
is the generator matrix, where:

\begin{itemize}
\item $\mathbf{D_0}$: describes transitions between transient states
\item $\mathbf{d_1}$: transition from transient states to the absorbing state
\item row vector $\mathbf{0}$: since no transition from the absorbing state to the transient states
\end{itemize}

\end{frame}

\begin{frame}
Since the states are transient, implies:

$$\lim_{t\to\infty} Prob(X(t)<n+1)=0,$$

i.e. the absorption occurs with probability 1.

\vspace{5pt}

$\mathbf{D_0}$ is invertible and matrix $(-\mathbf{D_0})^{-1}$ is the fundamental matrix of the absorbing CTMC
\begin{framed}
\begin{block}{Def. Phase-type distribution (PHD)}

A PHD is defined as the distribution of the lifetime $X$ or the time to absorption from the set of transient states $\mathbf{S_T}$ of an absorbing continuous Markov process $\{X(t)\}_{t\geq0}^{\infty}$

\end{block}
\end{framed}
\end{frame}

\begin{frame}{Phase Type Distributions (PHD)}

The Markov process starts in an arbitrary state from $\mathbf{S}=\mathbf{S_T} \cup \mathbf{S_A}$, so the vector
$$\pi=[\pi(1),\ldots,\pi(n)]$$

represents the inital probabilites for transient states and $\pi(n+1)$ gives the probability for a direct start in the absorbing (assumption: $\pi(n+1)=0$)
\end{frame}

\begin{frame}{Phase Type Distributions (PHD)}
\framesubtitle{Simple example}

$\mathbf{S}=\mathbf{S_T} \cup \mathbf{S_A} = \{1,2\} \cup \{3\}$

\begin{minipage}[c]{0.45\textwidth}
	\begin{minipage}[c]{1.\textwidth}
	\begin{equation*}
	\mathbf{Q} =
	\left(
	\begin{array}{cc|c}
	-4.5 & 4 & 0.5 \\
	2 & -2.5 &0.5\\
	\hline
	0 &0 &0 \\
	\end{array}
	\right)
	%%
	\end{equation*}
	\end{minipage}
	\vfill
	\begin{minipage}{1.\textwidth}
	\flushright
	\begin{equation*}
	\pi=[\frac{1}{2},  \frac{1}{2}]
	\end{equation*}
	\end{minipage}
\end{minipage}
\hfill
\begin{minipage}[c]{0.45\textwidth}
\begin{tikzpicture}[node distance = 2.5cm, ->, >=latex, auto, every edge/.append style={thick}]
	\node[state, initial, initial text={$\nicefrac{1}{2}$}, initial where=above](1) {$1$};
	\node[state, initial, initial text={$\nicefrac{1}{2}$},initial where=above](2)[right of =1]{$2$};
	\coordinate(Middle) at ($(1)!0.5!(2)$);
	\node[state, accepting](3)[below of =Middle]{$3$};

	\path  (1) edge[bend left] node{$4$} (2)
		  (2) edge[bend left] node{$2$} (1)
		  (2) edge[bend left] node{$\nicefrac{1}{2}$} (3)
		  (1) edge[bend right, left] node{$\nicefrac{1}{2}$} (3);
\end{tikzpicture}
\end{minipage}

\end{frame}

\begin{frame}{PHD Representation}

We can say now that the random variable $X$ describing the time until absorption is of phase-type with the vector-matrix tuple representation:

$$(\pi, \mathbf{D_0})$$

since the vectors $\mathbf{d_1}$ and the value $\pi(n+1)$ are implicitly given by the matrix $\mathbf{D_0}$

\vspace{10pt}
The size of the PHD is the size of the sub-generator matrix $\mathbf{D_0}$, $n$

\end{frame}

\begin{frame}{Analytical Properties of PHD}
$$F(x) = \mathbb{1} - \pi e^{\mathbf{D_0}x}\mathbb{1}, \text{ for }  x \geq 0 \text{ (c.d.f.)}$$

$$f(x) = \pi e^{\mathbf{D_0}x}\mathbf{d_1}, \text{ for }  x \geq 0 \text{   (p.d.f.)}$$

$$\mu_i=E[X^i]=i ! \pi \mathbf{(-D_0^{-1})}^i \mathbb{1}, \text{ PHD $i$th moment } $$

\vspace{10pt}

Note 1: $\mathbf{M} = \mathbf{-D_0^{-1}}$ is the moment matrix
$$\rightarrow \mu_i=E[X^i]=i ! \pi \mathbf{\mathbf{M}}^i \mathbb{1}$$

Note 2: $\mathbf{-D_0^{-1}}(i,j)=\mathbf{M}(i,j)$ is the expected total time spent in phase $j$ before absorption, given that initial phase is $i$

\end{frame}

\section{Classes}
\begin{frame}{PH Classes}

Based on the structure of the underlying Markov chain several classes of PH distributions can be distinguished

	\begin{itemize}
	\item Structure of PH representation often has an impact on its application
	\item Some structures allow more efficient solutions
	\end{itemize}

Most important distinction:

	\begin{enumerate}
	\item\textcolor<2,3>{blue}{Acyclic phase-type distributions (APH)}
		\onslide<3>{
		\begin{itemize}
		\item Hyper-Erlang distributions
		\item Hyper-Exponential distributions
		\end{itemize}}
	\item General phase-type distributions
	\end{enumerate}

\end{frame}
\begin{frame}{Hyper-Erlang distributions (HEr PHD)}

\begin{figure}
\centering
\begin{tikzpicture}[minimum size=1pt, node distance = 2cm, ->, >=latex, auto, every edge/.append style={thick}]
	\node[state, initial, initial text={$\beta_1$}](1) {$$};
	\node[state](2)[right of =1]{$$};
	\node[state](3)[right of =2]{$$};
	\node[state, initial, initial text={$\beta_2$}](4)[below of =1]{$$};
	\node[state](5)[right of =4]{$$};
	\coordinate(medio) at ($(5)!0.9!(3)$);
		\coordinate(Middle) at ($(1)!0.5!(4)$);
	\node[state, accepting](6)[right of =medio]{$$};

	\path  (1) edge node{$\lambda_1$} (2)
		  (2) edge node{$\lambda_1$} (3)
		  (3) edge[above] node{$\lambda_1$} (6)
		  (4) edge[below] node{$\lambda_2$} (5)
		  (5) edge[below] node{$\lambda_2$} (6);
\end{tikzpicture}
\caption{Hyper-Erlang distribution with two branches $(\beta, m, \mathbf{b}, \lambda)$ = $([\beta_1, \beta_2], 2, [3,2], [\lambda_1, \lambda_2])$ }
\end{figure}

Size of the Hyper-Erlang distribution: $n = \mathbf{b}\mathbb{1}$

\vspace{5pt}
\underline{Special case}:
$(\beta, m, \mathbf{b}, \lambda) = ([\beta_1=1], 1, \mathbf{b}, \lambda_1)$
$\rightarrow$ Erlang distr.

\end{frame}
\begin{frame}{Hyper-Exponential distributions (HEx PHD)}
\begin{figure}
\centering
\begin{tikzpicture}[minimum size=1pt, node distance = 2cm, ->, >=latex, auto, every edge/.append style={thick}]
	\node[state, initial, initial text={$\beta_1$},  initial where=below](1) {$$};
	\node[state, initial, initial text={$\beta_2$}, initial where=below](2)[right of =1]{$$};
	\node[state, initial, initial text={$\beta_3$}, initial where=below](3)[right of =2]{$$};
	\node[state, initial, initial text={$\beta_4$}, initial where=below](4)[right of =3]{$$};
	\node[state, accepting](5)[right of =4]{$$};

	\path  (1) edge[bend left] node{$\lambda_1$} (5)
		  (2) edge[bend left, right] node[above,pos=0.1] {$\lambda_2$} (5)
		  (3) edge[bend right] node[above, pos=0.2]{$\lambda_3$} (5)
		  (4) edge node{$\lambda_4$} (5);
\end{tikzpicture}
\caption{Hyper-Exponential distribution with four branches $(\beta, n, \lambda)$ = $([\beta_1, \beta_2, \beta_3, \beta_4], 4, [\lambda_1, \lambda_2, \lambda_3, \lambda_4])$}
\end{figure}

\underline{Special case}: $(\beta, n, \lambda) = ([\beta=1], 1, \lambda_1) \rightarrow$ Exponential distr.

\vspace{5pt}
Note: HEx are subclass of HEr when $\mathbf{b}=\mathbb{1} \text{ and } \mathbf{m}=n$

\end{frame}

\section{Canonical Representations}

\begin{frame}{Canonical Representation for APH distributions}
\framesubtitle{CF-1}

\begin{mydef}
The Canonical Form 1 (CF-1 form) is a bi-diagonal representation $(\beta, \Lambda)$, where $\beta$ initial probabilities and rates $\lambda_i$ in $\Lambda$ are in increasing order: $\lambda_1 \leq \lambda_2 \leq \ldots \leq \lambda_n$
\end{mydef}

\begin{figure}
\centering
\begin{tikzpicture}[minimum size=1pt, node distance = 2cm, ->, >=latex, auto, every edge/.append style={thick}]
	\node[state, initial, initial text={$\beta_1$},  initial where=below](1) {$$};
	\node[state, initial, initial text={$\beta_2$}, initial where=below](2)[right of =1]{$$};
	\node[state, initial, initial text={$\beta_3$}, initial where=below](3)[right of =2]{$$};
	\node[state, initial, initial text={$\beta_4$}, initial where=below](4)[right of =3]{$$};
	\node[state, accepting](5)[right of =4]{$$};

	\path  (1) edge node{$\lambda_1$} (2)
		  (2) edge node{$\lambda_2$} (3)
		  (3) edge node{$\lambda_3$} (4)
		  (4) edge node{$\lambda_4$} (5);
\end{tikzpicture}
\caption{APH distribution in CF-1 form}
\end{figure}

\end{frame}

\begin{frame}{Canonical Representation for APH distributions}

\begin{framed}
Every acyclic phase type distribution (APH) with a Markovian representation of size $n$ has an unique CF-1 representation of the same size.
\end{framed}
\begin{itemize}
\item A general APH has $n-1$ initial probabilities and $n^2$ entries in the sub-generator matrix $\mathbf{D_0}$, i.e. the number of parameters is $n^2+n-1$

\item In the CF-1 form $\mathbf{D_0}$ is an upper bi-diagonal matrix with $\lambda_{i,i}=-\lambda_{i+1,i}$, therefore the CF-1 form has $2n-1$ parameters
\end{itemize}
\end{frame}

\begin{frame}{PH-Fitting Tools}
Two general classes of algorithms
\begin{itemize}
\item Analytical: direct computation of parameters
	\begin{itemize}
	\item Moment Matching: $\mu_1, \mu_2, \mu_3$
	\end{itemize}
\item Statistical: maximum likelihood method (iterative process)
	\begin{itemize}
	\item G-FIT: Hyper-Erlang distributions
	\item PhFit: Acyclic PHD CF-1 form
	\end{itemize}
\end{itemize}

\end{frame}

\begin{frame}{Summary}

\begin{itemize}
\item PHD can be used to approximate any positive-valued distribution
\item They represent the distribution of the time to absorption of an absorbing continuous time Markov process $\{X(t)\}_{t\geq0}^{\infty}$
\item They are defined by the tuple $(\pi, \mathbf{D_0})$
\item Two big families: Hyper-Erlang and Hyper-Exponential distributions
\item Acyclic phase-type distributions deserve an special attention
\item They can be reduced in a canonical form in order to estimate their parameters $(2n-1)$
\item Fitting techniques
\end{itemize}

\end{frame}

\begin{frame}[noframenumbering, plain]{References}

\begin{thebibliography}{2}
\footnotesize{
\setbeamertemplate{bibliography item}[book]
\bibitem{A} Bucholtz, Kriege \& Felko (2014). Input Modeling with Phase-Type Distributions and Markov Models: Theory and Applications
\setbeamertemplate{bibliography item}[article]
\bibitem{A} Cumani, A (1982). On the canonical representation of homogeneous Markov processes modelling failure-time distributions microelectronics and reliability 22. 582-602.
\setbeamertemplate{bibliography item}[article]
\bibitem{B} Reinicke, P \& Horváth G. (2012). Phase-type distributions for realistic modeling in discrete-event simulation. \textit{In Proceedings of the 5th International ICST Conference on Simulation Tools and Techniques} (pp.283-290). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering).}

\end{thebibliography}

\end{frame}

\end{document}


%You can leave alone everything before Line 84.
\documentclass{article}
\usepackage{url,amsfonts, amsmath, amssymb, amsthm}
\usepackage{pict2e}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}

% Page layout
\setlength{\textheight}{8.75in}
\setlength{\columnsep}{2.0pc}
\setlength{\textwidth}{6.5in}
\setlength{\topmargin}{0in}
\setlength{\headheight}{0.0in}
\setlength{\headsep}{0.0in}
\setlength{\oddsidemargin}{0in}
\setlength{\evensidemargin}{0in}
\setlength{\parindent}{1pc}
\newcommand{\shortbar}{\begin{center}\rule{5ex}{0.1pt}\end{center}}
%\renewcommand{\baselinestretch}{1.1}
% Macros for course info
\newcommand{\courseNumber}{CMSC 657}
\newcommand{\courseTitle}{Introduction to Quantum Information Processing}
\newcommand{\semester}{Fall 2019}
% Theorem-like structures are numbered within SECTION units
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{statement}[theorem]{Statement}
\newtheorem{conjecture}[theorem]{Conjecture}
\newtheorem{fact}{Fact}
%definition style
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{example}{Example}
%\newtheorem{problem}{Problem}
\newtheorem{exercise}{Exercise}
%\newtheorem{algorithm}{Algorithm}
%remark style
\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}
\newtheorem{reduction}[theorem]{Reduction}
%\newtheorem{question}[theorem]{Question}
\newtheorem{question}{Question}
%\newtheorem{claim}[theorem]{Claim}
%
% Proof-making commands and environments
\newcommand{\beginproof}{\medskip\noindent{\bf Proof.~}}
\newcommand{\beginproofof}[1]{\medskip\noindent{\bf Proof of #1.~}}
\newcommand{\finishproof}{\hspace{0.2ex}\rule{1ex}{1ex}}
\newenvironment{problem}[1]{\medskip\noindent{\bf Problem #1.~}}{\shortbar}
\newenvironment{solution}[1]{\medskip\noindent{\bf Solution #1.~}}{\shortbar}
%====header======
%\newcommand{\solutions}[3]{
%\renewcommand{\thesolution}{{\large #2}.\arabic{problem}}
%\vspace{-2ex}
%\begin{center}
%{\small  \courseNumber, \courseTitle
%\hfill {\large \bf {Due: #1} }\\
%\semester, University of Maryland \hfill
%{\em Date: #3}}\\
%\vspace{-1ex}
%\hrulefill\\
%\vspace{4ex}
%{\Large #2}\\
%\vspace{2ex}
%\end{center}
%\shortbar
%\vspace{3ex}
%}

\newcommand{\points}[1]{\textit{(#1 points)}}
\newcommand{\bonus}[1]{\textit{(Bonus: #1 points)}}

%\input{Qcircuit}

\newcommand{\E}{\mathbb{E}}
\usepackage{colonequals}
%\newcommand{\<}{\langle}
%\renewcommand{\>}{\rangle}

\begin{document}
	%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
	%\solutions{September 26th, 2019}{Assignment 2}{\today}
	%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
	%
	% Begin the solution for each problem by
	% \begin{solution}{Problem Number} and ends it with \end{solution}
	%
	% the solution for Problem 1v
	\begin{center}
		\huge Finding curvature in RL objective surfaces
	\end{center}

	\begin{center}
		\large Ben Black
	\end{center}

	\section{Outline}

	The first step to understanding curvature in the objective surface is to compute the Hessian of that surface around a point. The first section derives a formula to estimate that local Hessian. The formalism follows the derivation of the policy gradient very closely.

	The second step is to create an algorithm that computes the Hessian linear operation (i.e. can compute a Hessian-vector dot product) efficiently. In particular, it should allow for dense, batched estimation on a GPU.

	The third step is to identify the maximally curved directions in the space, by computing the minimum and maximum eigenvalues and their corresponding eigenvectors.

	\section{Policy Hessian Theorem}

	Notation taken from \url{https://sites.google.com/view/icml17deeprl}

	In particular, $J$ is the objective function, $\theta$ are the parameters of the RL policy, $\tau$ is all the states and actions in an episode trajectory, $S_i$ is the state at timestep $i$, $a_i$ is the action at step $i$.
	%$H$, $P$, and $\nabla$ stand for the usual Hessian, probabilty, and gradient, respectively.

	Def: The policy hessian is
	$$
	\begin{aligned}
	H_\theta(J(\theta)) &= \nabla_\theta \nabla_\theta J(\theta)
	\end{aligned}
	$$

	In order to derive an efficient algorithm we will use the following two lemmas:

	\subsection{calculus trick lemma}
	$$\nabla_x P(x) = P(x) \nabla_x \log(P(x))$$
	proof:
	$$P(x) \nabla_x \log(P(x)) = P(x) \frac{1}{P(x)}\nabla_x P(x) = \nabla_x P(x)$$
	\subsection{Policy gradient lemma}
	$$\nabla_\theta \log(P(\tau)) = \sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))$$
	$$
	\begin{aligned}
	\nabla_\theta \log(P(\tau)) &= \nabla_\theta(\log(P(S_0))) + \sum_{t=0}^{t_\tau} \nabla_\theta(\log(P(a_t|S_t))) + \nabla_\theta(\log(P(S_{t+1}|a_t,S_t))) \\
	& \text{buf the transition function does not depend on theta, so the gradient is zero}\\
	&=  \sum_{t=0}^{t_\tau} \nabla_\theta(\log(P(a_t|S_t))) \\
	&=  \sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi(a_t,S_t)) \\
	\end{aligned}$$

	\subsection{Full derivation of formula}
	Def: The policy hessian is
	$$
	\begin{aligned}
	H_\theta(J(\theta)) &= \nabla_\theta \nabla_\theta J(\theta) \\
	&= \nabla_\theta \nabla_\theta \mathbb{E}_{\tau \sim \pi_\theta} R(\tau) \\
	&= \nabla_\theta \nabla_\theta \sum_\tau P_\theta(\tau) R(\tau) \\
	&= \text{applying calculus trick lemma} \\
	&= \nabla_\theta  \sum_\tau P_\theta(\tau)\nabla_\theta \log(P_\theta(\tau))  R(\tau) \\
	&=  \sum_\tau ((\nabla_\theta P_\theta(\tau))\nabla_\theta \log(P_\theta(\tau))  + P_\theta(\tau) \nabla_\theta \nabla_\theta \log(P_\theta(\tau))) R(\tau) \\
	&= \text{applying calculus trick lemma} \\
	&=  \sum_\tau ( P_\theta(\tau) \nabla_\theta \log( P_\theta(\tau))\nabla_\theta \log(P_\theta(\tau))  + P_\theta(\tau) \nabla_\theta \nabla_\theta \log(P_\theta(\tau))) R(\tau) \\
	&=  \sum_\tau P_\theta(\tau) ((\nabla_\theta \log( P_\theta(\tau)))^2  + \nabla_\theta \nabla_\theta \log(P_\theta(\tau))) R(\tau) \\
	&= \mathbb{E}_{\tau \sim \pi_\theta} ((\nabla_\theta \log( P_\theta(\tau)))^2  + \nabla_\theta \nabla_\theta \log(P_\theta(\tau))) R(\tau) \\
	&\text{policy gradient lemma} \\
	&= \mathbb{E}_{\tau \sim \pi_\theta} \left( \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right)^2  + \nabla_\theta \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right) \right) R(\tau) \\
	&= \mathbb{E}_{\tau \sim \pi_\theta} \left( \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right)^2  +  \left(\sum_{t=0}^{t_\tau} \nabla_\theta \nabla_\theta \log(\pi_\theta(s,a))\right) \right) R(\tau) \\		\end{aligned}	$$

	Reformatting the function to a useful format for estimation:
	$$
	\begin{aligned}
	& \ \ \mathbb{E}_{\tau \sim \pi_\theta} \left( \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right)^2  + \nabla_\theta \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right) \right)R(\tau)   \\
	&= \mathbb{E}_{\tau \sim \pi_\theta} \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right)\left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))R(\tau)\right)  + \nabla_\theta \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))R(\tau)\right)  \\
	\end{aligned}
	$$

	This formula will be used to create the policy hessian vector dot product algorithm below.

	\section{Policy Hessian vector dot product}
	Now, like all hessians over neural networks, it is large and inefficient to compute. Except worse because it is over an expectation of possibly sparse samples.

	So instead, we can learn something about the Hessian by computing the Hessian vector dot product i.e. for vector $v$ of the same shape as the parameter vector $\theta$, compute
	$$H(J(\theta))v$$

	Given the rearranged version of the policy gradient above we have
	$$
	\begin{aligned}
	H(J(\theta))v	&= \mathbb{E}_{\tau \sim \pi_\theta} \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))\right)\left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))vR(\tau)\right)  + \nabla_\theta \left(\sum_{t=0}^{t_\tau} \nabla_\theta \log(\pi_\theta(s,a))vR(\tau)\right)  \\
	\end{aligned}
	$$
	Which gives a natural, efficient batchable (within episodes) evaluation algorithm

	\begin{algorithm}
		\caption{Policy Hessian vector dot product}
		\begin{algorithmic}[1]
			\Procedure{DotHJ}{$\theta,v$}
			\State $\text{acc} = 0 \cdot \theta$
			\For{$\tau \sim \pi_\theta$}
				\State gradacc $= 0 \cdot \theta$
				\State gradvecacc $= 0$
				\State heshvecacc $= 0 \cdot \theta$
				\For{Batched $S,a,\text{ret}$ in $\tau$}
					\State logprobs $ = \log(\pi_\theta(a,S))$
					\State grad $ = \nabla_\theta \sum_b^\text{batch}\text{logprobs}_b$
					\State gradvec $ = (\nabla_\theta \sum_b^\text{batch} \text{logprobs}_b\text{ret}_b)) \cdot v$
					\State heshvec $ = \nabla_\theta \text{gradvec}$ \Comment{During implementation, make sure to compute gradient using autograds's create\_graph=true so this line is computed correctly}
					\State gradacc $+=$ grad
					\State gradvecacc $+=$ gradvec
					\State heshvecacc $+=$ heshvec
				\EndFor{}
				\State acc $ +=$ gradvecacc $\cdot$ gradacc $ + $ heshvecacc
			\EndFor{}
			\Return acc / num\_episodes
			\EndProcedure

		\end{algorithmic}
	\end{algorithm}

	\section{Eigenvalue/Eigenvector estimation}

	The next step is to estimate the eigenvectors and eigenvalues of the Hessian using the hessian vector dot product algorithm above. Note that Hessian is a linear operation that we can compute. Since we can compute this linear operation, standard linear algebra methods of finding the eigenvectors and eigenvalues of linear operations can be used

	The ARPACK library\cite{arpack} implements a number of efficient methods to solve linear algebra problems on arbitrary linear operations without a full matrix. In particular it can find estimates of the eigenvectors and eigenvalues of a linear operation using the Implicitly Restarted Lanczos Method, without having to compute all the entries of the matrix defining the linear operation. Scipy's python based \texttt{eigsh} interface offers a convenient way of accessing ARPACK functionality. Also note that
	\cite{visualloss}'s code \footnote{\url{https://github.com/tomgoldstein/loss-landscape/blob/master/hess_vec_prod.py}} was helpful to develop this method of finding the eigenvectors and eigenvalues.

	Note that this method does not guarantee an unbiased estimate of the eigenvector, in particular since we only have estimates of the actual linear operation under study. So the results should be treated with some care.

%	\section{Information matrix}
%
%
%	\subsection{Optimization Surface Visualization in Machine Learning}
%
%	Visualizing the surface of anything you're optimizing is a common practice, and has clear benefits. This has historically not been done in machine learning, because creating visualizations that are actually representative of surfaces over $10^5$ or more dimensions turns out to be very challenging. This was first done by \cite{visualloss}, with additional metrics to show the validity of these results in \cite{ronny2019generalization}. These concepts were first adapted from classifier loss surfaces to reward surfaces in reinforcement learning in \cite{mitpolicylandscape}, though we introduce extensions to these methods.
%
%	\section{Experiments}
%
%	We seek to understand how variations of the parameters of the policy affect the resulting reward. To understand this, we will be making a visualization of the surface where each set of parameters gives some specific reward. The challenge of this sort of visualization is that the neural network that defines the policy has a huge number of parameters (at least 1e5). Despite this challenge, prior work has successfully employed visualizations to understand the parameter space of neural networks by carefully choosing a projection of this high dimensional space onto a 2 dimensional space.
%
%	The first work to do so successfully was \cite{visualloss} which projected along filter normalized random directions. Filter normalization is a technique where the impact a parameter has on the function is assumed to be relative to the filter it is in. So it is normalized by the average magnitude of the parameters that make up the filter. In particular, take $\theta_{i,j}$ to be the $j$th filter of the $i$th layer of neural network with parameters $\theta$. Now we produce a random Guassian direction vectors $d{i,j}$ with the same dimensions as $\theta_{i,j}$. To apply filter normalization, the filters of these random direction vectors will be made the same magnitude as the filters in theta as follows:
%
%	$$ d^\prime_{i,j} = \frac{d_{i,j}}{\|d_{i,j}\|}\|\theta_{i,j}\|$$
%
%	Using this technique \cite{visualloss} were able to identify the smoothness of neural network surfaces on supervised learning tasks using 3d visualizations. Later this same method was used to visualize the volume of the optimal space of a policy\cite{ronny2019generalization}. Another work \cite{mitpolicylandscape} applied this technique to reinforcement learning, projecting the space onto the gradient direction on one axis and a random direction along another axis to act as a sanity check. The plots shown use this method to generate the projections.
%
%	We found that filter normalized random directions are not as suitable for the sort of neural networks typically used in RL because some filters are much more important than others in determining the outcome of the policy. So instead, we had to use a more sophisticated information based normalization technique which makes fewer assumptions about the structure of the function.
%
%	Recall that in order to properly visualize the local space of the surface, we must not only look at the impact of a handful of parameters on the function, but all of them. In order to do that, they should be equally weighted. The importance of a weight on the result of the function can be fully described by the Fisher information of the parameters $\theta$ on the outcome $X$.
%
%	$ F_{ij} = \mathbb{E}\Big[\frac{\partial}{\partial \theta_i}\log(f(X|\theta)) \frac{\partial}{\partial \theta_j}\log(f(X|\theta))\Big]$
%	The objective $f$ is hard to compute and differentiate, so we estimate it by choosing a similar, but distinct information matrix that looks at the information between the parameters and the action probabilities generated by the parameterized policy $\pi_\theta$.
%	$ F_{ij} = \mathbb{E}\Big[\frac{\partial}{\partial \theta_i}\log(\pi(a|\theta)) \frac{\partial}{\partial \theta_j}\log(\pi(a|\theta))\Big]$
%	In order to argue the similarity of the two matrices, we need to assume that actions generally affect the objective to a similar degree in expectation. Typical policies have a small number of actions, so this assumption is reasonable.
%
%	The importance of a weight can be estimated just from the diagonal element of this matrix \cite{kirk2016ewc}.
%
%	Which leaves
%
%	$\text{importance}_{\theta_i} = \mathbb{E}\Big[\frac{\partial}{\partial \theta_i}\log(\pi(a|\theta)) \frac{\partial}{\partial \theta_i}\log(\pi(a|\theta))\Big]$
%
%	And then this value, rather than the magnitude of the weights, can be the value that is filter-normalized.
%

%
%@article{kirk2016ewc,
%	author    = {James Kirkpatrick and
%		Razvan Pascanu and
%		Neil C. Rabinowitz and
%		Joel Veness and
%		Guillaume Desjardins and
%		Andrei A. Rusu and
%		Kieran Milan and
%		John Quan and
%		Tiago Ramalho and
%		Agnieszka Grabska{-}Barwinska and
%		Demis Hassabis and
%		Claudia Clopath and
%		Dharshan Kumaran and
%		Raia Hadsell},
%	title     = {Overcoming catastrophic forgetting in neural networks},
%	journal   = {CoRR},
%	volume    = {abs/1612.00796},
%	year      = {2016},
%	url       = {http://arxiv.org/abs/1612.00796},
%	archivePrefix = {arXiv},
%	eprint    = {1612.00796},
%	timestamp = {Mon, 13 Aug 2018 16:46:13 +0200},
%	biburl    = {https://dblp.org/rec/journals/corr/KirkpatrickPRVD16.bib},
%	bibsource = {dblp computer science bibliography, https://dblp.org}
%}
%
%@article{ronny2019generalization,
%	author    = {W. Ronny Huang and
%		Zeyad Emam and
%		Micah Goldblum and
%		Liam Fowl and
%		Justin K. Terry and
%		Furong Huang and
%		Tom Goldstein},
%	title     = {Understanding Generalization through Visualizations},
%	journal   = {CoRR},
%	volume    = {abs/1906.03291},
%	year      = {2019},
%	url       = {http://arxiv.org/abs/1906.03291},
%	archivePrefix = {arXiv},
%	eprint    = {1906.03291},
%	timestamp = {Fri, 14 Jun 2019 09:38:24 +0200},
%	biburl    = {https://dblp.org/rec/journals/corr/abs-1906-03291.bib},
%	bibsource = {dblp computer science bibliography, https://dblp.org}
%}

%
	\bibliographystyle{plain}

	\bibliography{main}{}
%
%
\end{document}


\begin{figure}
  \centering
  \includestandalone[width=.5\textwidth]{model_diagram}
  \caption{Model Architecture}  \label{fig:model}
\end{figure}

\subsection{Naive Baseline}

In our most naïve model, we begin by processing the financial headlines into TF-IDF vectors.
Given the multi-level nature of the aspects, we design a hierarchical logistic regression model.
We use a local classifier per parent node approach described by~\cite{Silla:2011tp}. For the sentiment task, we fit a linear regression model on the TF-IDF vectors.

\subsection{Pre-Trained BERT}

We then experiment by fitting a pre-trained BERT model on the FIQA task.
We use the 12-layer BERT\textsubscript{BASE} base model described in~\cite{DBLP:journals/corr/abs-1810-04805}.
Discarding the pooling layer outputs, we take the outputs of the last hidden state and place a single dropout layer and dense layer on top.
We then place a linear regression sentiment head and a hierarchical aspect classification head on top.
See figure~\ref{fig:model} for the full model architecture.
We fit the model using an Adam optimizer and train both heads together.
We use categorical cross entropy loss for the aspects, and mean squared error for the sentiments, taking the sum as the total loss.

\subsection{Domain-Adapted DistilBERT}

For our third model, we adapt the neural baseline to the financial domain.
To do this, we use a large dataset of publicly available financial news articles prepared by~\cite{ding2014using}.
The dataset includes 109,110 news articles from Reuters and 450,341 from Bloomberg.
The headlines span October 2006 to November 2013.

After training a tokenizer on this dataset, we normalize the data.
Normalization steps include converting to lowercase, removing control characters, and applying normalization Form D (NFD) Unicode normalization.
We pre-tokenize on white space and punctuation.
Using a vocab size of 25,000, we train the tokenizer using the WordPiece sub-word segmentation algorithm~\cite{DBLP:journals/corr/WuSCLNMKCGMKSJL16}.

Our tokenizer splits the financial news article dataset into documents of 256 tokens long.
Beginning with a pre-trained DistilBERT model~\citep{DBLP:journals/corr/abs-1910-01108}, we fine-tune our model on the tokenized Reuters dataset using a masked-language modeling (MLM) objective.
We train for 6 epochs.

We follow a similar architecture to our pre-trained BERT model, replacing the BERT model with our finance domain-adapted DistilBERT model.


\chapter{List of Symbols}

%Throughout this dissertation, italicized variables indicate scalars, italicized
% and bold--faced variables indicate vectors, and bold--faced variables are matrices or tensors. The sole exception to this latter are the matrices and tensors indicated with Greek symbols, which are bold--faced and italicized.
\vspace{-0.5cm}

\begin{longtable}{p{0.15\textwidth} p{0.8\textwidth}}
\hline
%\toprule
%  \multicolumn{2}{l}{\rule{0pt}{1pt} \bf Alphanumeric symbols} \\
% \\[-12pt] \midrule
%\rule{0pt}{1pt} \\[-12pt]
%$ \mathbf{1}$ & Unit cell matrix of rank 3 \\
%$ a$ & Length of the first unit cell vector \\
%$ \bm a $ & First unit cell vector \\
%$ b$ & Length of the second unit cell vector \\
%$ \bm b$ & Second unit cell vector \\
%$ \mathbf{B}$ & Second--order  stiffness tensor / elasticity tensor under
% loading\\
%$ c$ & Length of the third unit cell vector \\
%$ \bm c $ & Third unit cell vector \\
%$ \bm d_{ijn} $ & See definition in Eq.\ (\ref{Eq216}) \\
%$ \Delta t$ & Timestep in an MD simulation \\
${\ddagger}$ & Transition state\\
$ \Delta E^{\ddagger}$ &  Electronic energy barrier\\
$ \Delta H^{\ddagger}$ &  Enthalpy barrier\\
$ \Delta G^{\ddagger}$ &  Free energy barrier\\
$ B_0 $ & Bulk modulus\\
$ E $ & Electronic energy\\
$ E_{ZPE}$ & Zero--point vibrational energy \\
$ F $ & Helmholtz free energy\\
$ G $ & Gibbs free energy\\
$\hat{H}$ & Hamiltonian operator\\
$ H $ & Enthalpy\\
$\it{h}$ & Planck constant\\
$ \it{k} $ & Rate coefficient unimolecular reaction\\
$ \it{k}_{B} $ & Boltzmann constant\\
$ N $ & Number of particles\\
$ N_{A} $ & Avogadro constant \\
$ N_{dof} $ & Number of degrees of freedom \\
$ p $ & Pressure\\
$ Q $ & Overall partition function\\
$ Q_{trans} $ & Translational partition function \\
$ Q_{rot,ext} $ & Rotational partition function \\
$ Q_{vib} $ & Vibrational partition function \\
$ \it{q_X} $ & Partition function of species \it{X}\\
$ R $ & Universal gas constant \\
$ r_{ij} $ & Distance between atoms i and j\\
$ S $ & Entropy\\
$ T $ & Temperature \\
$ U $ & Internal energy\\
$ V $ & Volume\\
$ V_0 $ & Equilibrium Volume\\
$ \varepsilon_0 $ & Electronic energy \\
$ \nu_{i}$ & Vibrational frequency \\
$ \rho $ & Density\\ \hline

\end{longtable}

%\begin{longtable}{p{0.15\textwidth} p{0.8\textwidth}}
%\toprule
% \multicolumn{2}{l}{\rule{0pt}{1pt} \bf Greek symbols} \\
% \\[-12pt] \midrule
%\rule{0pt}{1pt} \\[-12pt]
%$ \beta$ & Isothermal compressibility of water \\
%$ \bm \varepsilon$ &  Infinitesimal Lagrangian strain tensor \\
%$ \delta_{ij}$ & Kronecker delta \\
%$ | \phi_N \rangle $ & Nuclear wavefunction in a molecular system \\
%$ | \phi_{N,nm} \rangle $ & $nm^\text{th}$ nuclear wavefunction in a molecular
% system \\
%$ \gamma_P$ & Damping coefficient $=1/\tau_P$ \\
%$ \bm \eta$ & Finite Lagrangian strain tensor \\
%$ \mu $ & Chemical potential \\
%$ \bm \mu$ & Rescaling matrix in the Berendsen barostat (see Eq.\
% (\ref{Eq220})) \\
%$ | \psi_e \rangle$ & Electronic wavefunction in a molecular system \\
%$ | \psi_{e,n} \rangle$ & $n^\text{th}$ electronic wavefunction in a molecular
% system \\
%$| \Psi_n \rangle$ & Eigenfunction (general) \\
%$ \bm \sigma$ & Cauchy stress matrix \\
%$ \bm \sigma_{a}$ & Anisotropic or deviatoric stress matrix \\
%$ \bm \sigma_i$ & Instantaneous Cauchy stress matrix \\
%$ \bm \sigma_{i,a}$ & Instantaneous anisotropic or deviatoric stress matrix \\
%$ \tau_T$ & Thermostat relaxation time \\
%$ \tau_P$ & Barostat relaxation time \\
%$ \bm \Xi $ & Virial tensor \\  \bottomrule
%\end{longtable}


\chapter{Introduction}\label{chap1}
\thispagestyle{plain}

% ====SECTION 1 ================================================================
\section{Motivation}\label{motivation}
Estimating the volume of glaciers is becoming an increasingly important topic, due to the role this plays in climate change. Melting glaciers have in fact an impact in sea level rise \citep{Zemp2017}, availability of fresh drinking water \citep{Kaser2010}) and even energy supplied by hydro electric power plants\citep{Terrier2011}. A reliable estimate of glaciers volume is key in order to properly compute the entity of these and other problems.

Computing the volume of glaciers is a trivial task once in possession of the data needed for it: the glacier surface and its distributed ice thickness. These data however are not easily obtainable due to the amount of effort needed to collect them.

A very big effort in this regard has been carried out with the creation of the Randolph Glacier Inventory (RGI) \citep{RGI2014}, which is a data-set containing a globally complete inventory of glaciers and their surface shapes. From these surface shapes it is easy to compute the surface area of the glaciers.

Ice thickness observations are collected in the Glacier Thickness Database (GlaThiDa) \citep{GlaThiDa2014}, which is the first global effort in collecting all the available ice thickness data in a single database. These observations however, when compared with the glaciers contained in the RGI, are very sparse (see \ref{glathida}). Out of the over 215,000 glaciers included in RGI, only 771 have thickness observations and, for most glaciers, there are not enough measurements to have a complete thickness distribution along the glacier surface.

Modeling glaciers is the only alternative to obtain an estimate of their volumes and is where a big effort from the scientific glaciology community is directed.

%The chapter Introduction leads the reader into the subject matter
%of the thesis. It is sometimes called
%Statement/Formulation/Definition/Presentation of the Problem. It may start with
%a so-called Motivation. It also contains the State of Knowledge or State
%of Research which is based on a literature survey (see section \ref{1sec:2}).
%Further, it contains the Scientific Questions and/or the Goals that are
%addressed in the main part of the thesis (see section \ref{1sec:3}). Finally,
%it provides an Outline of the science thesis (see end of section \ref{1sec:3}).

% ==== SECTION 2 ===============================================================
\section{State of Research}\label{research}
Over the years a number of different approaches have been used to model glaciers and estimate their volumes.
The simplest ones only attempt to compute these volumes from the relationship between the glacier surface and its volume as first stated by \citet{bahr1997}. Using the Buckingham Pi
theorem they found that the relationship between the surface area $S$ and the volume of a glacier $V$ is $V \propto S^{\gamma}$ and they estimated $\gamma=1.375$ for these glaciers and $\gamma=1.25$ for ice caps. Similar approaches also take into consideration other other parameters such as glacier length, width, elevation range \citep{Grinsted2013}, and surface slope. These type of methods are called ``scaling approaches'' and they yield to a volume estimate for each glacier but not to the distributed ice thickness. Due to the small requirements for their implementations they are very effective at computing the global volume of glaciers.

A more sophisticated approach to estimate the volume of glaciers is to reconstruct their three dimensional shape with a model, and thus calculate the volume. Traditionally this is done by making physical assumptions about the glacier dynamics and solving equations deriving from these assumptions to estimate the glacier shape. Numerical methods are then used to obtain a discretization of these equations which are then solved on grid-cell maps by computers. This set of numerically solved equations are what composes the model.
For glaciers this means reconstructing their bed shapes from their surface values, hence inverting the bed shape from the surface measurements using the constructed model.

%The equations needed to create the glacier map depend on parameters which are often unknown. This creates the inverse problem of having to estimate these parameters from observed data. In the case of estimating the glacier ice thickness to compute its volume, the problem would then be to tune the model parameters of the numerically solved equations, to best reproduce the observed ice thickness.
%These equations depend on parameters which need to be tuned in order for the equations to replicate the available observations. The tuning of these parameters is known as the inverse problem which needs to be solved in order for the  In the case of the    This can be done starting from te known values of the glaciers surface such as the glacier shape, available in the RGI, and the topography of the glacier, obtainable from a digital elevation model. This leads to a grid-map containing the glacier surface shape and the elevation of each grid point. Using these and potentially other values such as the glacier climate, one could create an algorithm based on physical assumptions or statistical data, to estimate the bed shape of the glacier. able to numerically compute the glacier bed shape using either a physical or statistical numerical   and extrapolate the volume from the results of this simulation. Usually physical assumptions must be made to be able to reproduce the glacier bed from its surface ice distribution.
First steps in this direction were made by \citet{Nye1965} in the study of  idealized glaciers. He pointed out that the distributed ice thickness could be inferred by the glacier surface slope using estimates of basal shear stress. From there many others developed this idea, such as \citet{haeberli1995} who made the assumption of a known constant basal shear stress, parameterized from the glacier elevation range, to estimate basic glaciological characteristics for glaciers in the Alps.

From these basic concepts, many more approaches have emerged in the years after, taking for example the mass conservation into account \citep{rasmussen_1988}, basal slipperiness together with bedrock topography \citep{Gudmundsson2001}, information such as mass balance and surface velocities (e.g. \citealt{gantayat2014}, \citealt{brinkerhoff2016}) and even more complex ones using forward models of ice flow \citep{vanPelt2013}. Also some non physical methods such as artificial neural networks \citep{Clarke2009}, have been adopted all to tackle the problem of estimating glaciers ice thickness.

All these achievements have been possible thanks to the advancement in research in the glaciological field and to the steadily increase in computational power and its availability. Models which could only be numerically computed for single glaciers, started expanding their range of possibilities, and \citet{Huss2012} released the first glacier model used to model all known glaciers and ice caps worldwide. Since then many more global scale glacier models able to predict ice thickness distribution have been released.

Meanwhile the GlaThiDa database was being compiled. This is a a collection of in-situ observations of glaciers ice thickness, created by joining together in a single database all the available ice thickness observations from worldwide glaciers. In particular it contains a table listing point-wise ice thickness measurements and their geographical coordinates. Even though these data are far from being enough to obtain ice volume estimates for all glaciers in the world, they are very useful for assessing glacier models performances.

The compilation of GlaThiDa together with the increase number of models being adopted by the glaciology community led to the first Ice Thickness Models Intercomparison eXperiment (ITMIX) \citep{Farinotti2017}, which is the first attempt to put together all those models and asses their accuracy on the base of known ice thickness observations available in the GlaThiDa, as well as comparing the models performances with each other.

The experiment focused on 17 glacier models which were given the task to estimate the ice thickness distribution of at least some of 15 glaciers, 3 ice caps and 3 synthetically grown glaciers. In order to predict the ice thickness the models were not allowed to use ice thickness observations for calibration. The output of the models was then compared with data available from the GlaThiDa and with the other models. A composite solution from all the participating models achieved a local ice thickness difference in the order of $10 \pm 24\%$ of the mean ice thickness. In general however the spread of the solutions obtained by the single models often exceeded the local ice thickness, concluding that an ensemble approach could mitigate errors and would reduce the random nature of single models output.

The models participating in the ITMIX were divided into categories depending on the different approaches used to model the ice thickness. The categories are: (1) those taking ice thickness inversion approaches as a minimization problem, (2) those based on mass conservation,  (3) approaches depending on a basal shear stress parametrization, (4) approaches based on surface velocities and (5) other approaches. 15 out of the 17 explored models in the ITMIX, belong to the first 4 categories, which are all categories for methods relaying on physical assumptions to model numerically some of the physical processes happening in a glacier and predict the ice thickness from there. Only 2 models, belonging to category (5) are models not based on physical assumptions and methods, but rather on statistical methods. Of those two the one from \citet{Clarke2009} is based on artificial neural networks, a branch of machine learning. It uses the input data from artificially grown glaciers to make predictions about real glaciers ice thickness, learning from the input data of the artificially grown ones. At the time of Clarke's publication in fact the GalThiDa had not yet been compiled, and there were not enough ice thickness observations available, to train a machine learning model from real glacier data.

Machine learning is a branch of computer science which uses different algorithms to give a computer the ability to learn from a set of samples. The concept of artificial intelligence, which is the super field of machine learning, was first proposed by \citet{Turing1950}.

Many different machine learning algorithms have been developed since 1950 and some even before. Although the algorithms have been around for a long period of time, they only saw wide spread application starting from the late 1990, as computation power and resources largely improved. Since then however the field has seen a huge development and many of its applications have become part of our day to day life. Things such as voice recognition, chat bots, autonomous driving and online products suggestions, are all a products of machine learning algorithms being developed and trained to solve these specific problems.

The basic idea behind machine learning is to allow a computer program to generate a model capable of solving a task, without explicit instructions on how to solve it. In the case of the glacier thickness it would mean to feed the algorithm a data sample including observed glacier ice thicknesses, to learn form possible patterns within this sample. After the learning process is finished one could use the trained model (as algorithms which underwent the learning process are called) to make predictions about glaciers with no thickness observations, or to fill the holes for those with too few ones.

Machine learning however has yet to make a big impact in the glaciological community and most of the models used in the ITMIX, and the ones which achieved better accuracy, are physically based models. In a later experiment, \citet{Farinotti2019} (F19) developed an ensemble model, using 5 of the physically based models form the first ITMIX, to better predict ice thickness distributions of all glaciers in the world. The results from this experiment can be used as good benchmark to compare models based on machine learning algorithms.

The importance glaciers play in climate change and climate modeling, the availability of ice thickness observations, the excitement behind machine learning in almost every possible field, and the lack of previous work done in this sense by the glaciological modeling community, are the driving motivation factors for this thesis.

%Based on the literature survey, the writer draws a picture of the existing
%knowledge in a specific field and points to open questions. Hence, after this
%survey the Introduction will ultimately culminate in the formulation of specific
%scientific questions/goals addressed in the thesis (see section \ref{1sec:3}).

%To cite a certain source (e.g., a paper) use the citation commands
%\verb|\citet| and \verb|\citep| of the \verb|natbib|
%package.\footnote{
%\url{http://www.ctan.org/tex-archive/macros/latex/contrib/natbib/natnotes.pdf}}
%Together with the bibliography style \verb|ametsoc.bst|, which is
%included in the \LaTeX{} manuscript template for AMS journals\footnote{
%\url{http://www.ametsoc.org/PUBS/journals/AMS_Latex_V3.0.tar.gz}}, \verb|natbib|
%produces citations in the author-date format together with a list of
%references that fulfill the AMS citation standard.\footnote{
%\url{http://www.ametsoc.org/PUBS/journals/author_reference_guide.pdf}}
%
%As an example, you can cite papers like \citet{hann66Aag} and \citet{scha93Aag}
%which have to be specified in your BibTeX database file (in this case it is
%\verb|mybibfile.bib|). More than one article of the same author can be cited
%like here: \citet{hoin85Aag,hoin90Aag} studied foehn winds.
%
%You may want to split your review of the literature into several sections.
%Further, use paragraphs to structure your introduction. If you like to cite
%papers in brackets (\emph{passive citations}) you can do this
%as in the following sentence: Gap flows have been studied in the Strait
%of Gibraltar \citep{scor52Aag,dorm95Aag}, in the French Rh\^one Valley
%\citep{pett82Aag}, near Hokkaid\=o in Japan \citep{arak69Aag}, near
%Unimak Island in the Aleutian Chain \citep{pan-99Aag}, and in the Howe
%Sound of British Columbia \citep{jack94Aag,jack94Bag}. Citation of a
%Dissertation: The gap flow in the Wipp Valley has been studied by
%\citet{gohm03Aag}. Citation of a conference paper: \citet{gohm06Aag}
%investigated the boundary layer structure in the Inn Valley. Citation of an
%online document: The AMS provides a guideline for preparing citations and
%references \citep{ams-09Aag}.

% ==== SECTION 3 ===============================================================
\section{Goals and Outline}\label{goals}
In this thesis three different machine learning algorithms have been chosen to  create models able to predict glacier ice thickness. The algorithms have all been trained on the same data-set, the GlaThiDa which contains ice thickness observations for glaciers worldwide.

To train the algorithm a set of attributes (often called features), mainly geometrical, have been generated using the Open Global Glacier Model (OGGM) \citep{OGGM2019}. This attributes are all computable by the only mean of a digital elevation model for the considered glacier, and the outline of the glacier given by the RGI.

The accuracy of the models have been estimated comparing their predictions with the observations from the GlaThiDa as well as with themselves.

An analysis of the most influential attributes in making predictions has been carried out.

Finally the obtained models have been used to predict the volume of all the alpine glaciers and compare their results with those of the \citet{Farinotti2019} (F19).

This thesis then tries to establish:
\begin{itemize}
\item[(1)] How accurate machine learning algorithms are at predicting glaciers ice thickness.
\item[(2)] How three different machine learning algorithms compare with each other in estimating glaciers ice thickness.
\item[(3)] Using those algorithm to estimate the total volume of glaciers in the alps, how these models compare to the model from the \cite{Farinotti2019} (F19).
\end{itemize}

%The so-called SMART
%criteria\footnote{\url{http://en.wikipedia.org/wiki/SMART_criteria}} might be
%used as a guideline to define reasonable goals. Here, the acronym SMART
%describes the properties of ``good'' goals: \underline{s}pecific,
%\underline{m}easurable, \underline{a}ttainable, \underline{r}elevant,
%\underline{t}ime-bound.
%
%The introduction may also describe briefly the methodology chosen and the
%materials (e.g., data, instruments, etc.) used. However, a detailed description
%will follow in the main part (see chapter \ref{chap2}).
%
%Finally you should present an \emph{outline} of your science thesis. Explain
%what the reader will find in the following chapters. For example, chapter
%\ref{chap2} describes the methodology. The results are presented
%in chapter \ref{chap3}. A discussion is provided in chapter \ref{disc} and the
%conclusions are drawn in chapter \ref{concl}.


\documentclass{beamer}
\usefonttheme{professionalfonts}
\mode<presentation>
{
	\usetheme{Warsaw}
	\setbeamercovered{transparent}
}
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage{gensymb}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{cancel}
\usepackage{soul}
\usepackage{xcolor}
\usepackage{mathtools}
\usepackage{hyperref}
\usefonttheme{serif}
\newcommand{\sequiv}{\stackrel{\mathclap{\normalfont\mbox{s}}}{\equiv}}
\title[BRINMANOLA]{A Brief Introduction to the Use of Mathematical Notation as a Language}
\author[VALEFOR, VARIK]{VALEFOR, VARIK \textless varikvalefor@aol.com \!\!\textgreater}
\begin{document}
	\maketitle
	\begin{frame}{Preface: The Curse of Knowledge}
		Curse of knowledge
		\begin{itemize}
			\item Do not remember being a newbie very well
			\item ``What do newbies not know?''
			\item ``What \textit{do} newbies know?''
			\item Confusion should be voiced
		\end{itemize}
\end{frame}
	\begin{frame}{About the Speaker}
		VARIK VALEFOR
			\begin{itemize}
				\item Intelligence analyst
				\begin{itemize}
				\item \textbf{\textit{Not}} a computer programmer!!!
				\item Can only answer a few questions re: work
			\end{itemize}
				\item Diverse set of skills
			\begin{itemize}
				\item Mathematics!
				\begin{itemize}
					\item Can't add numbers or do other low-level things
					\item Dove into calculus in the sixth grade
					\item Concepts understood and applied well
					\item Computers made for a reason
				\end{itemize}
				\item Computer programming
				\begin{itemize}
					\item Obsoletes low-to-mid-level mathematical tasks
				\end{itemize}
				\item Hosting kick-ass parties near Thanksgiving
				\begin{itemize}
					\item Egg splatter
					\item Roomba
					\item Battery explosion
				\end{itemize}
			\end{itemize}
		\end{itemize}
	\end{frame}

	\section{What Is Mathematical Notation?}
		\begin{frame}
			\begin{center}
				What Is Mathematical Notation?
			\end{center}
		\end{frame}
		\subsection{Overview}
			\begin{frame}{What Is Mathematical Notation?}
				Mathematical notation
				\begin{itemize}
					\item Used internally by mathematicians
					\item Different from the ``human languages''
					\begin{itemize}
						\item Simplicity: No adjectives, etc.
						\item Not directly spoken
					\end{itemize}
					\item \textit{Not limited to numbers}
					\begin{itemize}
						\item Neither is mathematics
					\end{itemize}
				\end{itemize}
			\end{frame}
		\subsection{Advantages of Mathematical Notation}
			\begin{frame}{Advantages of Mathematical Notation}
				Extreme precision\\
				\begin{itemize}
					\item Homonyms not a thing here
				\end{itemize}
				Extreme terseness\\
				Variables and functions locally defined
				\begin{itemize}
					\item English formally uses global definitions in dictionaries
				\end{itemize}
				Facilitates analytical thought
				\begin{itemize}
					\item Extremely dry
					\item No loaded words
					\item Arguments easily followed and contested
				\end{itemize}
				Humour still possible
				\begin{itemize}
					\item Surrealism
					\item Weird verbosity
					\item ``No shit, SHERLOCK.''
				\end{itemize}
			\end{frame}
			\begin{frame}{Example of a Joke}
				\[
					\frac{\textrm{d}}{\textrm{d}x} \frac{1}{x} =
          \frac{\cancel{\textrm{d}}}{\cancel{\textrm{d}}x} \frac{1}{x} =
          \frac{}{x} \frac{1}{x} =
          -\frac{1}{x^2}.
				\]
			\end{frame}
		\subsection{Disadvantages of Mathematical Notation}
			\begin{frame}{Disadvantages of Mathematical Notation}
				Not directly spoken
				\begin{itemize}
					\item Converted to ``human language''
				\end{itemize}
				Not very readable for the ``uninitiated''
				\begin{itemize}
					\item Maybe this video can be of use!
				\end{itemize}
				No global definitions
				\begin{itemize}
					\item Definitions \textit{\textbf{must}} be read
				\end{itemize}
				Typos can be \textit{really} bad
			\end{frame}
	\section{Grammar of Mathematical Notation}
		\begin{frame}
			\begin{center}
				The Grammar of Mathematical Notation
			\end{center}
		\end{frame}
			\subsection{Atoms}
				\begin{frame}{Nouns}
					Literals
					\begin{itemize}
						\item $2$, $3.14$, etc.
					\end{itemize}
					Variables
					\begin{itemize}
						\item $\pi$, $e$, $\textit{asshat}$, etc.
						\item Multi-character variable names are valid!
					\end{itemize}
					Functions and operators
					\begin{itemize}
						\item $\sqrt{2}$, $x\left(y\right)$,
            and so on, and so on, and so on, and so on...
					\end{itemize}
					Anything contained within parentheses\\
					Any expression
				\end{frame}
				\begin{frame}{Verbs}
					``Indicators''
					\begin{itemize}
						\item Logical connectives and declarations
						\item ``$=$'', ``$\implies$'', etc.
					\end{itemize}
					Functions
					\begin{itemize}
						\item Yes!
						\item Act as nouns \textit{or} verbs
						\item Example: ``$x\left(y\right).$'' $\sequiv$
              ``$x\left(y\right) \textrm{ IS TRUE}.$''
					\end{itemize}
				\end{frame}
			\subsection{Sentences}
				\begin{frame}{Sentences in Mathematical Notation}
					Sentences
					\begin{itemize}
						\item Contain verb and noun
						\item Must express some sort of idea
						\item ``Can this be literally translated to a proper English sentence?''
					\end{itemize}
				\end{frame}
				\begin{frame}{Examples of Sentences}
					\[
						a := 0! = 1.
					\]
					\[
						\forall z \in \mathbb{N}, \ \left(z + 1\right) \in \mathbb{Z}.
					\]
					\[
						p\left(enis\right).\footnote{$p$ must be a binary function.}
					\]
				\end{frame}
				\begin{frame}{Examples of Non-Sentences}
					\[
						5 + 2
					\]
					\[
						\sqrt{5318008}
					\]
					\[
						90210
					\]
				\end{frame}
		\subsection{Punctuation}
			\begin{frame}{Ending Punctuation}
				Full stops
				\begin{itemize}
					\item Used to end all statements
				\end{itemize}
				Question marks
				\begin{itemize}
					\item \textbf{Not accepted by all men}
					\item Used to indicate uncertainty
					\item Used in place of noun
				\end{itemize}
				$\neg\left(\textrm{Exclamation marks}\right)$
				\begin{itemize}
					\item Used for factorials
					\item Other usage would be a bit confusing
				\end{itemize}
			\end{frame}
			\begin{frame}{Questions}
				Questions in mathematical notation
				\begin{itemize}
					\item Not completely accepted
					\item VARIK wrote this portion of a style guide
					\item Example: ``$X \implies Y?$''
					\item Example: ``$? \ \implies Y.$''
				\end{itemize}
			\end{frame}
	\section{Overview of Common Symbols}
		\begin{frame}
			\begin{center}
				An Overview of Commonly-Used Symbols
			\end{center}
		\end{frame}
		\subsection{Logical Symbols}
			\begin{frame}{Arrows}
				$\implies$ --- implication\\
				\begin{itemize}
					\item ``$A \implies B$.'' $\sequiv$ ``IF $A$, THEN $B$.''
				\end{itemize}
				$\Longleftarrow$  --- ``only if''\\
				\begin{itemize}
					\item ``$A \Longleftarrow B.$'' $\sequiv$ ``IF $B$, THEN $B$.''
				\end{itemize}
				$\iff$ --- ``if and only if'' or ``iff''
				\begin{itemize}
					\item $\left(A \implies B\right) \land \left(B \implies A\right)$
				\end{itemize}
			\end{frame}
			\begin{frame}{Other Logical Symbols}
				``$\exists$'' $\sequiv$ ``there exists''\\
				``$\ni, \ \textrm{s.t.}$'' $\sequiv$ ``such that''\\
				``$\therefore$'' $\sequiv$ ``therefore''\\
				``$\because$'' $\sequiv$ ``because''\\
				``$\land$'' $\sequiv$ ``and''\\
				``$\lor$'' $\sequiv$ ``or''\\
				``$\veebar$'' $\sequiv$ ``xor''\\
				``$\neg$'' $\sequiv$ ``not''
				\begin{itemize}
					\item Example: $\neg\left(1 + 2 = 1 - 2\right)$
				\end{itemize}
				``$\forall$'' $\sequiv$ ``for all''
				\begin{itemize}
					\item Example: $\forall x \in \mathbb{C}, \ x + 1 \neq x.$
				\end{itemize}
			\end{frame}
		\subsection{Sequences and Similar Things}
			\begin{frame}{Sequence/Set Stuff}
				$\in$ --- element of\\
				\begin{itemize}
					\item NOT semantically equivalent to ``$=$''!
					\item Equality statements bidirectional
					\item $\forall a, \ \forall b, \ a = b \iff b = a.$
				\end{itemize}
				$\subset$, $\subseteq$ --- subset\\
				$\#$ --- cardinality
				\begin{itemize}
					\item Example: $\#A = \textrm{ANSWER}
          \left(\textrm{``How many elements does $A$ contain?''}\right)$
				\end{itemize}
				$A_n$ --- element number $n$ of sequence $A$\\
				$\cup$ --- set union\\
				\begin{itemize}
					\item Example: $A \cup B$
				\end{itemize}
				$\cap$ --- set intersection
				\begin{itemize}
					\item Example: $A \cap B$
				\end{itemize}
				$\setminus$ or $-$ --- set difference
				\begin{itemize}
					\item Example: $A \setminus B$ or $A - B$
				\end{itemize}
			\end{frame}
		\subsection{General Stuff}
			\begin{frame}{The Equality Operator}
				Equality operator
				\begin{itemize}
					\item Misunderstood \textit{far} too often
					\item Both sides equal
					\item Means same thing when reversed
					\begin{itemize}
						\item $\left(a + b = c\right) \iff \left(c = b + a\right)$
					\end{itemize}
					\item Improper usage
					\begin{itemize}
						\item $\textrm{SUM}\left(1,2,3,4,5\right)$
						\item $\neg\left(1 + 2 = 3 + 3 = 6 + 4 = 10 + 5 = 15\right).$
					\end{itemize}
				\end{itemize}
			\end{frame}
	\section{Relatively Advanced Stuff}
		\begin{frame}
			\begin{center}
				Relatively Advanced Stuff
			\end{center}
		\end{frame}
		\subsection{Functions}
			\begin{frame}{What Are Functions?}
				Functions
				\begin{itemize}
					\item Map input to output
					\item Can be thought of as ``performing an action''
					\item Extremely versatile
					\item Operate on numbers, Boolean values, sets...
				\end{itemize}
			\end{frame}
			\begin{frame}{Examples}
				\[
					\textrm{colour} : \textrm{OBJ} \mapsto \textrm{SEQ}.
				\]
				\[
					\forall o, \ \textrm{colour}\left(o\right) = \textrm{COLOUR OF OBJECT $o$}.
				\]
			\end{frame}
		\subsection{Piecewise Function Definitions}
			\begin{frame}{Overview}
				Piecewise definitions
				\begin{itemize}
					\item Declare functions
					\item Specifically outline function outputs
					\item Support any sort of input
					\item Equivalent to \texttt{if} statements in CS
				\end{itemize}
			\end{frame}
			\begin{frame}{Example}
				\[
					\forall x \in \mathbb{R}, \
					\left|x\right| :=
					\left\{
						\begin{array}{rl}
							x	&	x \geq 0\\
							-x & x < 0
						\end{array}
					\right..
				\]
			\end{frame}
		\subsection{Sets and Sequences}
			\begin{frame}{Sets}
				Sets
				\begin{itemize}
					\item ``Unordered lists''
					\item Denoted by $\left\{\textrm{curly brackets}\right\}$
					\item Can contain zero elements... or be uncountably infinite
					\item Useful whenever dealing with multiple things
					\item Can contain quite literally anything
					\begin{itemize}
						\item Even things which have never existed!
						\item Even things which never \textit{will} exist!!!
					\end{itemize}
					\item Can be clearly defined
					\begin{itemize}
						\item Example: ``$A = \left\{\textrm{MACAULAY CULKIN},\
							\textrm{SADDAM HUSSEIN}\right\}$''
					\end{itemize}
					\item Or...
				\end{itemize}
			\end{frame}
			\begin{frame}{Sequences}
				Sequences
				\begin{itemize}
					\item Ordered sets
					\item Denoted by $\left(\textrm{parentheses}\right)$ or
						$\left\langle\textrm{angle brackets}\right\rangle$
					\item Access of individual sets via $A_n$ useful
				\end{itemize}
			\end{frame}
			\begin{frame}{Set-Builder Notation}
				Set-builder notation
				\begin{itemize}
					\item Equal to set which matches conditions
					\item Can return null set
					\item Extremely useful when working with infinite sets
					\item Same thing used for sequences
				\end{itemize}
			\end{frame}
			\begin{frame}{Example of Set-Builder Notation}
				\[
					x^2 = 1 \iff
					x \in \left\{y : y^2 = 1\right\} \iff
					x \in \left\{-1,\ 1\right\}.
				\]
				However...
				\[
					A :=
					\left\{
						\textrm{APPLE} :
						\textrm{COLOUR}\left(\textrm{APPLE}\right) =
						\textrm{GREEN}
					\right\}.
				\]
				Also...
				\[
					P :=
					\left\langle
						a \in \left(\mathbb{N} - \left\{0,1\right\}\right) :
						a \textrm{ IS PRIME}
					\right\rangle.
				\]
			\end{frame}
	\section{Use Cases}
		\subsection{Logical Arguments}
			\begin{frame}{Why Use this Notation in Arguments?}
				Easily dissected
				\begin{itemize}
					\item Facilitates checking for errors
					\item Arguing against becomes easy
				\end{itemize}
			\end{frame}
			\begin{frame}{Example}
				Where $a$ denotes a man, \quad $\forall a, \
				\left(k\left(a\right) \iff \textrm{$a$ IS KNOWLEDGABLE}\right) \land
				\left(c\left(a\right) \iff a \textrm{ IS CORRUPT}\right) \land
				\left(p\left(a\right) \iff a \textrm{ IS POWERFUL}\right)$,
				\[
					\forall a, \ k\left(a\right) \implies p\left(a\right).
				\]
				\[
					\forall a, \ p\left(a\right) \implies c\left(a\right).
				\]
				\[
					\therefore \forall a, \ k\left(a\right) \implies c\left(a\right).
				\]
			\end{frame}
		\subsection{General Communication}
			\begin{frame}{Why Use this Notation in General Communication?}
				Mathematical notation...
				\begin{itemize}
					\item Reduces confusion
					\item Saves time
					\item \textit{May} encourage thinking about stuff objectively
					\item Is easily translated to other languages
					\begin{itemize}
						\item Only definitions need be translated
					\end{itemize}
				\end{itemize}
			\end{frame}
			\begin{frame}{Example}
				Refer to the example for the use of mathematical notation in
        logical arguments.  Both things end up looking fairly similar.
			\end{frame}
	\section{Conclusion}
		\subsection{Conclusion}
			\begin{frame}{Conk'd!}
				NOT COMPREHENSIVE!!!\\
				At least one of these things should apply:
				\begin{itemize}
					\item Spiked reader's \st{drink} interest in mathematics
					\item Encouraged better structuring of arguments
				\end{itemize}
				Recommendations welcomed
			\end{frame}
\end{document}