\chapter{Introduction}
\label{chapter1}

\graphicspath{{Chapter1/figs/}}

Data Analytics (DA) intend to seek through a dataset for interesting relationships and information and to effectively present them as insights \cite{TukeyJohnW.andWilk1966DataOverview}. More specifically, Exploratory Data Analysis (EDA) is a systematic way to investigate relevant information from multiple perspectives and it is not fixed to a set of techniques \cite{Yu2003ExploratoryAnalysis}. Commonly, EDA user tasks can involved actions for search or query information in terms of trends, outliers, distributions, correlations, among others \cite{Munzner2014VisualizationDesign}. Statistics and Visual Analytics (VA) are valuable techniques to tackle an EDA process but they present some limitations when data is larger and high-dimensional.

By the other hand, Machine Learning (ML) comprises a set of techniques or algorithms enabling computers to learn from experience and automatically to improve their performance \cite{Michie1968MemoLearning} without being explicitly programmed \cite{Koza1996}. When it is tedious or even impossible to detect patterns in larger and high-dimensional datasets, ML provides mechanisms to explore alternate routes for understanding data \cite{Yu2003ExploratoryAnalysis}. In unsupervised ML, the goal of the Dimensionality Reduction (DR) and Clustering algorithms is to group instances based on a similarity metric. Hopefully, by splitting the problem will be useful for users to extract insights.

It is important to note that ML algorithms are typically black-boxes working autonomously. The implication of this fact in an EDA process is that the user is responsible for checking its results and decide if these fit to the schema, or mental model \cite{Grolemund2014AAnalysis}, representing the real-world phenomenon contained in the data. In this sense, ML have two big problems:

\begin{enumerate}
\item Lack of user feedback. To be an iterative process, training ML models can require a lot of execution time for achieving acceptable results and, in the worst case, these can be completely useless. If the user is a domain-expert, their knowledge about the problem or the data could greatly improve the performance of the ML model in less amount of time.
\item Interpretability of black-box ML models. More sophisticated models can fit better to complex data but generally losing interpretability. When user is involved in the ML process, model performance could not be the unique requirement to be fulfilled. While the ML objective might be to reduce error, the domain-expert user purpose is to derive insights \cite{Lipton2017}. The practice of handing over human judgment to the computer when user does not understand how this is working, it is similar to blindly accept that two datasets are comparable when having the same measures of central tendency.
\end{enumerate}

To tackle this problems, two new sub-fields of research have been proposed to help users to better interact and understand ML models, and by this opening the black-box: Interactive ML and Interpretable ML (also known as Explainable Artificial Intelligence). In this work, a selection of the most noteworthy papers of these sub-fields are presented and some guidelines for designing better and more user-centric ML systems are derived. Subsequently, this review focuses on interactive EDA tools that use DR and Clustering algorithms.

From the previous basis, MLExplore.js, a web interactive tool for exploring high-dimensional tabular data is developed extending the concept that ML models can be used for gaining data understanding as long as the user is able to feed them by interacting with the hyper-parameters and interpret their results in terms of the original attributes. Additional interaction elements involve the ability for changing the set of attributes and instances used for training the models. MLExplore.js is designed for domain-expert users and the exploration is focused on cluster-oriented DR task sequences \cite{Brehmer2014VisualizingSequences}: verify clusters, name clusters and match cluster and classes. For supporting this exploration, two implementations of the t-SNE \cite{VanDerMaaten2008} and K-Means \cite{Lloyd1982LeastPCM} algorithms running in the browser are integrated. Opportunities for a browser-based environment include shareability, interactivity and on-device computation \cite{Smilkov2019TensorFlow.js:Beyond}. The functionalities of the tool include load a dataset, navigate the data, perform attribute selection, train the models changing the hyper-parameters in an iterative way and validate the results in terms of the original attributes.

\section{Motivation Examples}
\label{section1.1}

No extending the discussion about different statistical or VA alternatives for analyzing high-dimensional data, two idioms for visualizing it are parallel coordinates plot and scatter plot matrix. Both idioms are useful for determining correlations between attribute pairs and identifying outliers but, particularly the parallel coordinates, presents a limitation when analyzing non-neighbor attributes in terms of their distribution among the x-axis. These idioms can be extended by including user interaction for coordinated highlighting across sub-views, for the case of the scatter plot matrix, or attribute reordering, seeking to reduce the parallel coordinates plot limitation previously presented.

Figures \ref{fig:iris-parallel} and \ref{fig:iris-scatterplot} shows these idioms applied to one the most widespread datasets in ML community, the Fisher's Iris dataset \cite{FisherIris}. This dataset has 150 instances and 5 attributes, one of them representing the iris species. In both idioms, the main insight from this dataset consisting of the class separability in two of the attributes (PetalLengthCm, PetalWidthCm) can be easily appreciated.

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.7\textwidth]{iris-parallel.png}
 \caption{Parallel coordinates for the Iris dataset.}
 \label{fig:iris-parallel}
\end{figure}

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.9\textwidth]{iris-scatterplot.png}
 \caption{Scatter plot matrix for the Iris dataset.}
 \label{fig:iris-scatterplot}
\end{figure}

In the real-world, there are larger and more complex datasets than the Iris having hundreds of attributes and thousands or millions of instances. For most of these cases, using the previous idioms is ineffective given the user cognition constraints for retaining many details, even when they are all shown at the same time, in addition to the screen size constraint. Next subsections describe two datasets considered relevant for this study and a possible exploratory analysis path supported by ML DR and Clustering algorithms.

\subsection{The FIFA 19 Complete Player Dataset}
\label{subsection1.1.1}

The FIFA dataset is currently one of the most popular datasets in Kaggle\footnote{https://www.kaggle.com/datasets (accessed on April 21th, 2019)}. It contains more than 18.000 player records of the popular video game, information representing the real state of players belonging to a wide variety of soccer clubs around the world. For each player there are 89 categorical and ordered attributes including age, nationality, overall scoring, potential, club, value, wage, preferred foot, international reputation, weak foot, skill moves, work rate, among many others. A possible user task could be to identify players with similar physical and game attributes to subsequently determine, for instance, the best conditions for training, trading options and match planning.

Figure \ref{fig:fifa-navio} summarizes this dataset by using Navio \cite{Guerra-Gomez2018Navio:Datasets} which is an interactive tool for exploring and navigating larger and high-dimensional datasets. More details about Navio are given in Section \ref{section3.2}. This is a good starting point for data understanding, nevertheless, it is not enough for fulfilling with the user task previously stated where attributes need to be aggregated in order to transversely identify similar players.

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.9\textwidth]{fifa-navio.png}
 \caption{Visualization of the FIFA dataset in Navio.}
 \label{fig:fifa-navio}
\end{figure}

In this scenario is where ML can be useful. After validate data quality and transform attributes, a DR model is trained, for instance t-SNE, for producing an embedding that enables the identification of clusters of similar players. In addition, the position categorical attribute is used to encode the color in the embedding. The result is shown in Figure \ref{fig:fifa-tsne}. From this perspective, some analysis questions can be asked:

\begin{itemize}
\item Do clusters visually identified match with the position of the player? Do this match with other attribute in the dataset?
\item What characterize the players of the upper-left cluster and difference them from the players in the central cluster?
\item If goalkeepers (orange cluster) are excluded from the analysis, does the new t-SNE model produce the same three remaining clusters?
\item If physical attributes like height and weight are also excluded, is the player distribution in the embedding affected?
\end{itemize}

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.7\textwidth]{fifa-tsne.png}
 \caption{DR embedding for the FIFA dataset using the t-SNE algorithm.}
 \label{fig:fifa-tsne}
\end{figure}

The first and second questions could asked with a VA tool where user can interact with the color encoding for drawing the embedding, select instances of interest and detail them. Subsequent questions can be asked by additional experimentation where dataset is filtered or reduced in attributes and additional models are trained in a iterative way. If the user is a ML-expert, they will not have problems to perform these analysis in the framework of their preference, otherwise, a tool for interact and interpret the model space and results is valuable.

\subsection{The SALURBAL Dataset}
\label{subsection1.1.2}

SALURBAL is a five-year project aimed to study how urban environments and urban policies impact the health of city residents throughout Latin America \footnote{https://drexel.edu/lac/salurbal/overview/ (accessed on April 22th, 2019)}. For this purpose, datasets of Latin American cities and sub-cities containing urban form, landscape, street design and transportation metrics has been collected to subsequently determine which patterns emerge as mechanism for supporting the decision-making by domain-experts in fields like transportation and urban health. The sub-cities dataset has 1.433 instances and 83 attributes, where all attributes are sequential except for name of the sub-city, type, city, country, presence of BRT, subway and aerial tram being categorical. The Figure \ref{fig:salurbal-navio} summarizes this dataset by using Navio, enabling user to derive important instances by sorting and filtering interactions as explained in  in Section \ref{section3.2}.

Patterns can be builded as clusters, where sub-cities in the same cluster have similar metrics. In this sense, public policies already successful in one sub-city could have high probability to have success in other sub-city belonging to the same cluster.

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.9\textwidth]{salurbal-navio.png}
 \caption{Visualization of the SALURBAL dataset in Navio.}
 \label{fig:salurbal-navio}
\end{figure}

After validate data quality and transform attributes, a K-Means model is trained for producing groups of similar sub-cities that can be analyzed more easily. The model produces a cluster membership for each instance and a set of centroids but not much explanations about how these have been built. An alternative for gaining understanding is to observe the distribution of the original-space attributes, but since there are many of them, using t-SNE for visualizing the cluster distribution and validating their separability or overlapping is a reasonable choice. The result is shown in Figure \ref{fig:salurbal-kmeans-tsne}. Nevertheless, similar to the FIFA dataset, important questions need to be addressed:

\begin{itemize}
%\item Do the clusters of similar sub-cities match with the 2-dimensional distribution from the DR embedding?
\item What characterize the sub-cities of the blue cluster and difference them from the sub-cities in the red cluster?
\item Why the green cluster have only two sub-cities? Do they are so different among Latin American sub-cities? Does 4 clusters not represent a convenient data grouping?
\item These models are obtained including all sub-cities metrics for training. Would more make sense if models for DR and Clustering are trained by groups of attributes like Street Design and Urban Landscape?
\end{itemize}

\begin{figure}[ht]
 \centering
 \includegraphics[width=0.7\textwidth]{salurbal-kmeans-tsne.png}
 \caption{DR embedding + Clustering for the SALURBAL dataset using the t-SNE and K-Means algorithms.}
 \label{fig:salurbal-kmeans-tsne}
\end{figure}

As stated in previous subsection, Chapter \ref{chapter2} focuses on describing some state of the art principles and tools for answering these questions.

\section{Document Structure}
\label{section1.2}

This document is organized as follows: Chapter \ref{chapter2} shows the state of the art for the Interactive ML and Interpretable ML sub-fields and some guidelines for designing better and more user-centric ML systems. This chapter also describes proposals for interact and interpret DR and Clustering algorithms. Chapter \ref{chapter3} presents MLExplore.js, a web interactive tool for exploring high-dimensional tabular data by interacting and interpreting t-SNE and K-Means. Chapter \ref{chapter4} explain the usage of the tool based on one real-world case study. Finally, Chapter \ref{chapter5} concludes the work and presents the challenges and opportunities for the future.


\chapter{Calculation commands and scripting in Warteschlangensimulator}

\renewcommand{\thepage}{\arabic{page}}
\setcounter{page}{1}

\textbf{Calculation commands} can be used in the simulator e.g.\ to determine time periods
(such as processing times) or to specify in which branching direction a client should be directed.

\textbf{Scripts} can be used both for the determination of branch directions and for the evaluation
of simulation results and for running of parameter series. The Warteschlangensimulator uses
Javascript and Java as languages.

\section{Create expressions}

To the right of all input fields into which calculations commands can be entered
always the following button is displayed:
\fbox{\includegraphics[width=0.25cm]{wand.png}}
By using this button the \textbf{Edit expression} dialog
(see figure \ref{fig:ExpressionBuilder}) can be accessed.
The dialog contains a complete list of all the commands available
in the current context, and makes it easy to put together
more complex commands and expressions.

\begin{figure}[ht]
	\caption{Edit expression dialog}
	\centerline{\includegraphics[width=14cm]{DialogExpressionBuilder.png}}
	\label{fig:ExpressionBuilder}
\end{figure}


\newpage
\subsection{Phase functions}
\label{sec:phase}
This section contains a description of all implemented medium scattering models, which
are also known as \emph{phase functions}. These are very similar in principle to surface
scattering models (or \emph{BSDF}s), and essentially describe where light travels after
hitting a particle within the medium.

The most commonly used models for smoke, fog, and other homogeneous media
are isotropic scattering (\pluginref{isotropic}) and the Henyey-Greenstein
phase function (\pluginref{hg}). Mitsuba also supports \emph{anisotropic}
media, where the behavior of the medium changes depending on the direction
of light propagation (e.g. in volumetric representations of fabric). These
are the Kajiya-Kay (\pluginref{kkay}) and Micro-flake (\pluginref{microflake})
models.

Finally, there is also a phase function for simulating scattering in
planetary atmospheres (\pluginref{rayleigh}).


\newcommand{\FS}{
  \begin{figure}
    \includegraphics[width=6cm]{fig/fs.png}
    \caption{First Species Counterpoint}
    \Description{First Species Counterpoint}
    \label{fig:fs}
  \end{figure}
}

\newcommand{\MotionCadence}{
  \begin{figure}
    \includegraphics[width=7cm]{fig/motion.png}
    \hspace{1.2cm}
    \includegraphics[width=3.33cm]{fig/cadence.png}
    \begin{flushleft}
    \begin{small}
      \hspace{1.2cm} contrary \ \ \ \ \ \ \ \ \,parallel \ \ \ \ \ \ \
      \ \ \ \,similar \ \ \ \ \ \ \ \ \ \ oblique
      %\hspace{1.3cm} paralell-5th \ \ \ similar-8th
    \end{small}
    \end{flushleft}
    \caption{Motion (left) and Cadence (right)}
    \Description{Motion (left) and Cadence (right)}
    \label{fig:motioncadence}
  \end{figure}
}

\renewcommand{\SS}{
  \begin{figure}
    \includegraphics[width=7cm]{fig/ss.png}
    \caption{Second Species Counterpoint}
    \Description{Second Species Counterpoint}
    \label{fig:ss}
  \end{figure}
}

\section{Counterpoint}
\label{sec:cp}

Counterpoint is a technique for combining multiple lines of melodies.
Composing counterpoint is like arranging a song for choir:
we start with a \emph{cantus firmus}, which serves as the base melody,
and compose a counterpoint line above or below the cantus firmus.
When doing this, we must make sure that the whole music sounds
harmonically pleasing, and that the individual melodic lines are
distinguishable to the listener.

In this section, we present an implementation of species counterpoint,
based on the formulation given by \citet{fux-cp}.
The idea is to represent ``good'' counterpoint  as a dependent record,
whose fields encode the musical content as well as proofs that the
counterpoint follows certain rules.
Due to the space limitation, we only show some essential parts of
our implementation; see our anonymized supplement
\texttt{Counterpoint.agda} for full details.

\subsection{First Species Counterpoint}
\label{sec:cp:fs}

\FS

First species counterpoint is the simplest variant of counterpoint.
In first species, we set one note against each note in the cantus firmus,
which is required to start with a tonic (the first note of a scale,
such as C in C major) and consist only of whole notes.
Figure~\ref{fig:fs} shows an example of first species counterpoint.
The lower line is the cantus firmus, which we excerpt from a German
song called \emph{Froschgesang} (Frog's song).
The upper line is the counterpoint composed by the second author.

\subsubsection{Representing Music}

In our implementation, we represent each bar as a pitch-interval pair
\texttt{(p ,  i)}, where \texttt{p} is the pitch of a cantus firmus note,
and \texttt{i} is the interval between \texttt{p} and the corresponding
counterpoint note.
We then represent a sequence of bars as a list of pitch-interval pairs,
but with one proviso: we separate the first and last bars from the
middle bars.
Therefore, the Frog's song in Figure~\ref{fig:fs} is represented as a
compound of the following three elements:

\begin{alltt}
first : PitchInterval -- defined as Pitch \(\times\) Interval
first = (c 5 , per8)

middle : List PitchInterval
middle = (d 5 , maj6) :: (e 5 , min6) :: (f 5 , maj3) ::
         (e 5 , min3) :: (d 5 , maj6) :: []

last : PitchInterval
last = (c 5 , per8)
\end{alltt}

\subsubsection{Representing Rules}

The reason behind our three-part representation of music is that
different parts are subject to different rules, as we detail below.

\paragraph{Beginning}

The beginning of the music should express perfection.
As we saw in Section~\ref{sec:music}, there are three intervals that
are classified as perfect: the 1st (usually called the \emph{unison}),
5th, and 8th.
The first interval of the music must then be one of these intervals.
In our formalization, we implement this rule as the
\texttt{checkBeginning} function, which reports an error
\texttt{not158 i} when the first interval \texttt{i} is an invalid one.
Since Agda does not have exceptions, we turn the error into a
\texttt{Maybe} value by wrapping it around the \texttt{just} constructor.

\begin{alltt}
data BeginningError : Set where
  not158 : PitchInterval \(\rightarrow\) BeginningError

checkBeginning : PitchInterval \(\rightarrow\) Maybe BeginningError
checkBeginning pi@(_ , i) =
  if ((i == per1) \(\vee\) (i == per5) \(\vee\) (i == per8))
  then nothing
  else just (not158 pi)
\end{alltt}

\paragraph{Intervals}

The middle bars of the music should maintain consonance of intervals
and independence of melodic lines.
As we saw in Figure~\ref{fig:interval}, consonant intervals include
the unison, 3rd, 5th, 6th, and 8th.
Among these, the unison is clearly an obstacle to distinguishing
between the two lines of music.
Therefore, the middle bars must consist only of the latter four intervals.
We encode this rule as the \texttt{checkIntervals} function, which
returns a list of errors corresponding to the occurrences of dissonant
intervals and unisons.

\begin{alltt}
data IntervalError : Set where
  dissonant : Interval \(\rightarrow\) IntervalError
  unison    : Pitch    \(\rightarrow\) IntervalError

intervalCheck : PitchInterval \(\rightarrow\) Maybe IntervalError
intervalCheck (p , i) with isConsonant i | isUnison i
intervalCheck (p , i) | false | _    = just (dissonant i)
intervalCheck (p , i) | _     | true = just (unison p)
intervalCheck (p , i) | _     | _    = nothing

checkIntervals : List PitchInterval \(\rightarrow\) List IntervalError
checkIntervals = mapMaybe intervalCheck
\end{alltt}

\paragraph{Motion}

\MotionCadence

The independence of melodic lines is also affected by \emph{motion},
i.e., the way one interval moves to another interval.
As Figure~\ref{fig:motioncadence} left shows, there are four kinds of motion:
\emph{contrary} (two lines go in different directions),
\emph{parallel} (two lines go in the same direction by the same
distance), \emph{similar} (two lines go in the same direction by
different distances), and \emph{oblique} (one line plays the same note).
It is known that approaching a perfect interval by parallel or similar
motion tends to fuse the melodic lines into one.
To rule out such motion, we define the \texttt{checkMotion}
function, which inspects all pairs of two adjacent intervals (using
auxiliary functions \texttt{motion} and \texttt{pairs}) and returns a
list of erroneous patterns.

\begin{alltt}
data MotionError : Set where
  parallel : PitchInterval \(\rightarrow\) PitchInterval \(\rightarrow\) MotionError
  similar  : PitchInterval \(\rightarrow\) PitchInterval \(\rightarrow\) MotionError

motionCheck : PitchInterval \(\rightarrow\) PitchInterval \(\rightarrow\) Maybe MotionError
motionCheck i1 i2 with motion i1 i2 | isPerfect (proj\(\sb{2}\) i2)
motionCheck i1 i2 | contrary | \_     = nothing
motionCheck i1 i2 | oblique  | \_     = nothing
motionCheck i1 i2 | parallel | false = nothing
motionCheck i1 i2 | parallel | true  = just (parallel i1 i2)
motionCheck i1 i2 | similar  | false = nothing
motionCheck i1 i2 | similar  | true  = just (similar i1 i2)

checkMotion : List PitchInterval \(\rightarrow\) List MotionError
checkMotion = mapMaybe (uncurry motionCheck) \(\circ\) pairs
\end{alltt}

\paragraph{Ending}

The ending of the music should express relaxation.
Since the unison and 8th are considered most stable, the music
must end with either of these intervals.
The last interval should also be approached by a \emph{cadence},
a progression that gives rise to a sense of resolution
(Figure~\ref{fig:motioncadence} right).
This in turn suggests that a valid ending requires the middle bars
to be non-empty.
We encode these rules as the \texttt{checkEnding} function, which,
upon finding an invalid ending, reports one of the three possible
errors.

\begin{alltt}
data EndingError : Set where
  not18    : PitchInterval      \(\rightarrow\) EndingError
  not27    : PitchInterval      \(\rightarrow\) EndingError
  tooShort : List PitchInterval \(\rightarrow\) EndingError

endingCheck : PitchInterval \(\rightarrow\) PitchInterval \(\rightarrow\) Maybe EndingError
endingCheck pi1@(pitch p , i) (pitch q , interval 0)  =
  if ((p + 1 \(\equiv\sp{\mathtt{b}}\) q) \(\wedge\) (i == min3)) then nothing else just (not27 pi1)
endingCheck pi1@(pitch p , i) (pitch q , interval 12) =
  if ((q + 2 \(\equiv\sp{\mathtt{b}}\) p) \(\wedge\) (i == maj6) \(\vee\) (p + 1 \(\equiv\sp{\mathtt{b}}\) q) \(\wedge\) (i == min10))
  then nothing
  else just (not27 pi1)
endingCheck pi1               pi2                     =
  just (not18 pi2)

checkEnding : List PitchInterval \(\rightarrow\) PitchInterval \(\rightarrow\) Maybe EndingError
checkEnding []        \_ = just (tooShort [])
checkEnding (p :: []) q = endingCheck p q
checkEnding (p :: ps) q = checkEnding ps q
\end{alltt}

\subsubsection{Putting Things Together}

Using the encoding of music and rules we have seen so far, we define
\texttt{FirstSpecies}, a record type inhabited by correct first species
counterpoint.
The first three fields of this record type represent the first, middle,
and last bars, respectively.
The last four fields stand for the proofs that the music satisfies all
the required properties.

\begin{alltt}
record FirstSpecies : Set where
  constructor firstSpecies
  field
    firstBar    : PitchInterval
    middleBars  : List PitchInterval
    lastBar     : PitchInterval
    beginningOk : checkBeginning firstBar \(\equiv\) nothing
    intervalsOk : checkIntervals middleBars \(\equiv\) []
    motionOk    : checkMotion (firstBar :: middleBars) \(\equiv\) []
    endingOk    : checkEnding middleBars lastBar \(\equiv\) nothing
\end{alltt}

With this record type, we can show that the counterpoint in
Figure~\ref{fig:fs} is correct, by constructing the following proof term:

\begin{alltt}
fs : FirstSpecies
fs = firstSpecies first middle last refl refl refl refl
\end{alltt}

\subsection{Second Species Counterpoint}
\label{sec:cp:ss}

\SS

We next discuss a more complex variant of counterpoint, called the
second species.
In second species, we set \emph{two} half notes against every cantus
firmus note.
This gives rise to the distinction between \emph{strong} beats (the
first interval in a bar) and \emph{weak beats} (the second interval).
Figure~\ref{fig:ss} is an example of two-against-one counterpoint,
again composed for the Frog's song.

\subsubsection{Representing Music}

In second species, the first and last bars may have a different structure
from the middle bars.
More specifically, it is encouraged to begin the counterpoint line with
a half rest and end with a whole note.
Therefore, in our implementation, we reuse the \texttt{PitchInterval}
type for the first and last bars, and define a new type
\texttt{PitchInterval2} for the middle bars.

\begin{alltt}
first2 : PitchInterval
first2 = (c 5 , per5)

middle2 : List PitchInterval2 -- defined as Pitch \(\times\) Interval \(\times\) Interval
middle2 =
  (d 5 , min3 , per5) :: (e 5 , min3 , min6) :: (f 5 , maj3 , aug4) ::
  (e 5 , min6 , min3) :: (d 5 , min3 , maj6) :: []

last2 : PitchInterval
last2 = (c 5 , per8)
\end{alltt}

\subsubsection{Representing Rules}

The rules for second species counterpoint can be obtained by
tweaking those for first species and adding a few new ones.
Here we go through the rules without showing the corresponding
Agda functions, as they are largely similar to what we defined for
first species.

\paragraph{Beginning}

The beginning of the music may be either the 5th or 8th, but
\emph{not} the unison, as it prevents the listener from recognizing
the beginning of the counterpoint line.
% We restrict the first interval by defining a new error
% \texttt{BeginningError2} and a function \texttt{checkBeginning2}
% that works analogously to \texttt{checkBeginning} for first species.

\paragraph{Strong Beats}

Strong beats in the middle bars are constrained by the same rules
as in first species: they must all be consonant, non-unison intervals.
% We encode this restriction as the \texttt{checkStrongBeats}
% function, which, as the \texttt{checkIntervals} function for first
% species, returns a list of found errors.

\paragraph{Weak Beats}

Weak beats are allowed to be dissonant if they are created by a
\emph{passing tone}, i.e., a note in the middle of two step-wise
motions in the same direction (as in bars 4-5 of Figure~\ref{fig:ss}).
They may also be the unison if they are left by step in the opposite
direction from their approach (as in bars 5-6 of Figure~\ref{fig:ss}).
% We implement these rules as the \texttt{checkWeakBeats} function,
% which checks the validity of every three successive beats in the
% middle bars.

\paragraph{Motion}

Parallel and similar motion towards a perfect interval is prohibited
across bars, that is, between the weak beat in one bar and the
strong beat in the next bar.
% We enforce this rule using the \texttt{checkMotion2} function,
% which uses \texttt{expandPitchIntervals2} to convert a list of
% \texttt{PitchInterval2} into a list of \texttt{PitchInterval}, and
% checks every pair of weak beat and strong beat intervals.

\paragraph{Ending}

The last interval must be the unison or 8th, preceded by an
appropriate interval that constitutes a cadence structure.

\subsubsection{Putting Things Together}

Now we define \texttt{SecondSpecies}, a record type inhabited by
correct second species counterpoint.
As in \texttt{FirstSpecies}, we have three fields holding the musical
content, followed by five fields carrying the proofs of the required
properties discussed above\footnote{
  The auxiliary function \texttt{secondPitch} in \texttt{weakBeatsOk}
  extracts the counterpoint note of a given interval, and
  \texttt{expandPitchIntervals2} turns a list of \texttt{PitchInterval2}s
  into a list of \texttt{PitchInterval}s.
}.

\begin{alltt}
record SecondSpecies : Set where
  constructor secondSpecies
  field
    firstBar      : PitchInterval
    middleBars    : List PitchInterval2
    lastBar       : PitchInterval
    beginningOk   : checkBeginning2 firstBar \(\equiv\) nothing
    strongBeatsOk : checkStrongBeats middleBars \(\equiv\) []
    weakBeatsOk   : checkWeakBeats middleBars (secondPitch lastBar) \(\equiv\) []
    motionOk      : checkMotion2 (firstBar ::
                                  (expandPitchIntervals2 middleBars)) \(\equiv\) []
    endingOk      : checkEnding2 middleBars lastBar \(\equiv\) nothing
\end{alltt}

Using \texttt{SecondSpecies}, we can show that the second species
counterpoint in Figure~\ref{fig:ss} is correct.

\begin{alltt}
ss : SecondSpecies
ss = SecondSpecies first2 middle2 last2 refl refl refl refl refl
\end{alltt}

\subsection{Comparison with Previous Work}
\label{sec:cp:comp}

The type-theoretical formalization of counterpoint has first been
attempted by \citet{szamozvancev-welltyped}.
They encode the rule for intervals by declaring a type class
\texttt{ValidHarmInterval} that classifies valid intervals, and then
specifying which instances are invalid using GHC's support for
custom type errors.
They also rule out invalid motion using a similar technique.

The idea of producing custom musical type errors underlies our
implementation as well.
The difference is that we have no means to leverage the Agda
type checker due to the lack of such a language feature, which
we consider to be a downside of using Agda for musical purposes.

More recently, \citet{cong-cp} give an implementation of first
species counterpoint using dependent types in Agda.
Their idea is to represent correct counterpoint as a list-like
datatype, where the base cases enforce a valid ending and the
inductive case guarantees correct uses of intervals and motion
(by means of implicit arguments representing proofs).

Our representation of counterpoint improves on Cong and Leo's
in two ways.
First, while they rely on the Agda type checker to report errors,
we use our own type checker (implemented as the
\texttt{checkXXX} functions) and type errors (defined as the
\texttt{XXXError} datatypes).
This allows us to produce more user-friendly error messages.
To see the difference, suppose we have replaced the second interval
of Figure~\ref{fig:fs} by an octave (8th) of D, breaking the rule
that perfect intervals cannot be approached by parallel motion.
In our implementation, this causes an error in the \texttt{motionOk}
field, with the following message:

\begin{alltt}
(parallel (c 5 , per8) (d 5 , per8) :: []) != [] of type (List MotionError)
\end{alltt}

\noindent In contrast, Cong and Leo would just report the existence
of an unsolved metavariable, meaning that Agda failed to construct
a proof required by the counterpoint constructor.
Thus, the user would see the following message in the Agda buffer:

\begin{alltt}
\_13 : motionOk (c 5 , per8) (d 5 , per8)
\end{alltt}

\noindent For a non-expert user, it may be difficult to connect
this message to the corresponding musical error.

The second improvement from \citet{cong-cp} is that, instead of
hard-coding all the rules into a datatype designed specifically for
counterpoint, we define each rule as a separate function and
combine them using a record type.
This makes it easy to switch between strict and relaxed rule sets:
we just need to add or remove certain fields.
It also allows us to reuse some of the rules in a context other than
counterpoint, as we will see in Section~\ref{sec:harmony:complete}.


\renewcommand{\legdnwonoptinitcare}{Efficient batch}
\renewcommand{\legdnwtwoptinitnormal}{Random batch}

\subsubsection{CIFAR10}

\textbf{Two Layer, opti=adagrad, init=uniform,batch size=1xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar10_carebatch_TwoLayer_10B_0E_adagrad_unif.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar10_TwoLayer_10B_0E_adagrad_unif.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

\textbf{Two Layer, opti=adagrad, init=he uniform,batch size=1xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar10_carebatch_TwoLayer_10B_0E_adagrad_he_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar10_TwoLayer_10B_0E_adagrad_he_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

\textbf{Two Layer, opti=adagrad, init=unif,batch size=2xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar10_carebatch_TwoLayer_20B_0E_adagrad_unif.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar10_TwoLayer_20B_0E_adagrad_unif.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%MNIST%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{MNIST}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adagrad, init=glorot uniform, batch size=4xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_carebatch_TwoLayer_40B_0E_adagrad_gl_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_TwoLayer_40B_0E_adagrad_gl_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adamax, init=he uniform, batch size=2xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_carebatch_TwoLayer_20B_0E_adamax_he_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_TwoLayer_20B_0E_adamax_he_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adagrad, init=zero, batch size=8xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_carebatch_TwoLayer_80B_0E_adagrad_zero.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_TwoLayer_80B_0E_adagrad_zero.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%MNIST%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{CIFAR100}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adagrad, init=glorot uniform, batch size=4xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_carebatch_TwoLayer_40B_0E_adagrad_gl_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/mnist_TwoLayer_40B_0E_adagrad_gl_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%CIFAR100%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{CIFAR100}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adagrad, init=he uniform, batch size=1xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_carebatch_TwoLayer_100B_0E_adagrad_he_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_TwoLayer_100B_0E_adagrad_he_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adagrad, init=he uniform, batch size=2xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_carebatch_TwoLayer_200B_0E_adagrad_he_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_TwoLayer_200B_0E_adagrad_he_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adagrad, init=uniform, batch size=1xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_carebatch_TwoLayer_100B_0E_adagrad_unif.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_TwoLayer_100B_0E_adagrad_unif.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adamx , init=gl uniform, batch size=1xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_carebatch_ThreeLayer_100B_0E_adamax_gl_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/cifar100_ThreeLayer_100B_0E_adamax_gl_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%STL10%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsubsection{STL10}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adamax, init=he uniform, batch size=1xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/stl10_carebatch_FiveLayer_10B_0E_adamax_gl_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/stl10_FiveLayer_10B_0E_adamax_gl_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adamax, init=he uniform, batch size=2xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/stl10_carebatch_TwoLayer_20B_0E_adamax_he_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/stl10_TwoLayer_20B_0E_adamax_he_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%NEW %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\textbf{Two Layer, opti=adamax, init=gl uniform, batch size=2xc}

\pgfplotstableclear{\dnwonoptinitcare}
\pgfplotstableclear{\dnwtwoptinitnormal}

\pgfplotstableread[col sep = comma]{ResultsNormal/stl10_carebatch_TwoLayer_20B_0E_adamax_gl_u.csv}\dnwonoptinitcare
\pgfplotstableread[col sep = comma]{ResultsNormal/stl10_TwoLayer_20B_0E_adamax_gl_u.csv}\dnwtwoptinitnormal

\renewcommand{\initerr}{Error for initial epochs}
\renewcommand{\initacc}{Accuracy for initial epcohs}
\renewcommand{\lateerr}{Error for later epochs}
\renewcommand{\lateacc}{Accuracy for later epochs}
\renewcommand{\boxerr}{Error variation}
\renewcommand{\boxacc}{Accuracy variation}

\renewcommand{\initfigtitle}{Different batch results for starting 15 epochs }
\renewcommand{\initcaption}{The initial epochs seen shows that 2x batch size(2x batch size=2*number of classes) started with the lowest error(a) as well as best accuracy(b), surprisingly 8x batch size started quite well inspite of less network updates in an epoch.
	Then till 15 epochs it is mixed results and no batch size cannot be singled out as a best performer.}

\renewcommand{\latefigtitle}{Different batch results for later epochs}
\renewcommand{\latecaption}{8x started well but in late epochs it could not sustain the start, Another good perfromance is shown by 10x which has reached quite good accuracy but its error become high and entered overfitting regime, 2x performance in error as well as accuracy side remain quite robust.}

\renewcommand{\boxerraccfigtitle}{accuracy and error plot for full training epochs}
\renewcommand{\boxerracccaption}{4x perform quite well as error remain with in small range, but on high side, 16x and 18x have low error regime as their minimum error but their variance range is on higher side, 10x shows quite good accuracy and has its maximum value is maximum among all other batch sizes, while its error had high variance than 2x. 2x is consistent in both low error with less vairance as well as accuracy. Low error makes 2x a robust classifier.}

\renewcommand{\trtscaption}{2x shows quite good performance}
\renewcommand	{\ymaxlim}{0.99}

\input{efficient_init_epoch.tex}
%\input{efficient_train_test.tex}
\input{efficient_box_plots.tex}
\input{efficient_late_epochs.tex}


\documentclass[10pt,DIV16,a4paper,abstract=true,twoside=semi,openright]{scrreprt}
\usepackage[english]{babel}
\usepackage[numbers, sort&compress]{natbib}
\usepackage{isabelle,isabellesym}
\usepackage{booktabs}
\usepackage{paralist}
\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{xspace}
\usepackage{xcolor}
\usepackage{hyperref}

\usepackage{enumitem} % Mess about with itemize, enumerate, description styles
\newcommand\mydescriptionlabel[1]{\hspace{\leftmargini}\textbf{#1}}
\newenvironment{where}{%
  \let\descriptionlabel\mydescriptionlabel
  \description[itemsep=0em, font=\normalfont]
}{%
  \enddescription
}

\pagestyle{headings}
\isabellestyle{default}
\setcounter{tocdepth}{1}
\newcommand{\ie}{i.\,e.\xspace}
\newcommand{\eg}{e.\,g.\xspace}
\newcommand{\thy}{\isabellecontext}
\renewcommand{\isamarkupsection}[1]{%
  \begingroup%
  \def\isacharunderscore{\textunderscore}%
  \section{#1 (\thy)}%
  \endgroup%
}

\newcommand{\orcidID}[1]{} % temp. hack

\newcommand{\repeatisanl}[1]
{\ifnum#1=0\else\isanewline\repeatisanl{\numexpr#1-1}\fi}
\newcommand{\snip}[4]{\repeatisanl#2#4\repeatisanl#3}

 \newcommand{\DefineSnippet}[2]{%
   \expandafter\newcommand\csname snippet--#1\endcsname{%
     \begin{quote}
     \begin{isabelle}
     #2
     \end{isabelle}
     \end{quote}}}
 \newcommand{\Snippet}[1]{%
   \ifcsname snippet--#1\endcsname{\csname snippet--#1\endcsname}%
   \else+++++++ERROR: Snippet ``#1 not defined''+++++++ \fi}

\title{A Formal Model of Extended Finite State Machines}%
\author{%
\begin{minipage}{.8\textwidth}
  \centering
      Michael~Foster\footnotemark[1]\orcidID{0000-0001-8233-9873}%
      \qquad\qquad%
      Achim~D.~Brucker\footnotemark[2]\orcidID{0000-0002-6355-1200}%
      \\%
      Ramsay~G.~Taylor\footnotemark[1]\orcidID{0000-0002-4036-7590}%
      \qquad\qquad%
      John~Derrick\footnotemark[1]\orcidID{0000-0002-6631-8914}%
     \end{minipage}
}

\publishers{%
  \footnotemark[1]~Department of Computer Science, The University of Sheffield, Sheffield, UK\texorpdfstring{\\}{, }%
  \texttt{\{%
  	\href{mailto:jmafoster1@sheffield.ac.uk}{jmafoster1},%
  	\href{mailto:r.g.taylor@sheffield.ac.uk}{r.g.taylor},%
  	\href{mailto:j.derrick@sheffield.ac.uk}{j.derrick}%
    \}@sheffield.ac.uk}\\[2em]%
  \footnotemark[2]~%
  Department of Computer Science, University of Exeter, Exeter, UK\texorpdfstring{\\}{, }%
  \href{mailto:a.brucker@exeter.ac.uk}{\texttt{a.brucker@exeter.ac.uk}}%
}

\begin{document}
\maketitle
\begin{abstract}
  In this AFP entry, we provide a formalisation of extended finite state machines (EFSMs) where models are represented as finite sets of transitions between states. EFSMs execute traces to produce observable outputs. We also define various simulation and equality metrics for EFSMs in terms of traces and prove their strengths in relation to each other. Another key contribution is a framework of function definitions such that LTL properties can be phrased over EFSMs. Finally, we provide a simple example case study in the form of a drinks machine.
  \begin{quote}
    \bigskip
    \noindent{\textbf{Keywords:} Extended Finite State Machines, Automata, Linear Temporal Logic}
  \end{quote}
\end{abstract}

\tableofcontents
\cleardoublepage

\chapter{Introduction}
This AFP entry formalises extended finite state machines (EFSMs) as defined in \cite{foster2018}. Here, models maintain both a \emph{control flow state} and a \emph{data state}, which takes the form of a set of \emph{registers} to which values may be assigned. Transitions may take additional input parameters, and may impose guard conditions on the values of both inputs and registers. Additionally, transitions may produce observable outputs and update the data state by evaluating arithmetic functions over inputs and registers.

As defined in \cite{foster2018}, an EFSM is a tuple, $(S, s_0, T)$ where
\begin{where}
  \item [$S$] is a finite non-empty set of states.
  \item [$s_0 \in S$]is the initial state.
  \item [$T$] is the transition matrix $T:(S \times S) \to \mathcal{P}(L \times \mathbb{N} \times G \times F \times U)$ with rows representing origin states and columns representing destination states.
\end{where}
In $T$
\begin{where}
  \item [$L$] is a finite set of transition labels
  \item [$\mathbb{N}$] gives the transition \emph{arity} (the number of input parameters), which may be zero.
  \item [$G$] is a finite set of Boolean guard functions $G:(I \times R) \to \mathbb{B}$.
  \item [$F$] is a finite set of \emph{output functions} $F:(I \times R) \to O$.
  \item [$U$] is a finite set of \emph{update functions} $U:(I \times R) \to R$.
\end{where}
In $G$, $F$, and $U$
\begin{where}
  \item [$I$] is a list $[i_0, i_1, \ldots, i_{m-1}]$ of values representing the inputs of a transition, which is empty if the arity is zero.
  \item [$R$] is a mapping from variables $[r_0, r_1, \ldots]$, representing each register of the machine, to their values.
  \item [$O$] is a list $[o_0, o_1, \ldots, o_{n-1}]$ of values, which may be empty, representing the outputs of a transition.
\end{where}

EFSM transitions have five components: label, arity, guards, outputs, and updates. Transition labels are strings, and the arities natural numbers. Guards have a defined type of \emph{guard expression} (\texttt{gexp}) and the outputs and updates are defined using \emph{arithmetic expressions} (\texttt{aexp}). Outputs are simply a list of expressions to be evaluated. Updates are a list of pairs with the first element being the index of the register to be updated, and the second element being an arithmetic expression to be evaluated.

\begin{figure}
  \centering
  \includegraphics[height=\textheight]{session_graph}
  \caption{The Dependency Graph of the Isabelle Theories.\label{fig:session-graph}}
\end{figure}
The rest of this document is automatically generated from the
formalization in Isabelle/HOL, i.e., all content is checked by
Isabelle.  Overall, the structure of this document follows the
theory dependencies (see \autoref{fig:session-graph}):

\nocite{foster2018}

\clearpage
% \chapter{Theories}

\input{session}

{\small
  \bibliographystyle{abbrvnat}
  \bibliography{root}
}
\end{document}
\endinput
%%% Local Variables:
%%% mode: latex
%%% TeX-master: t
%%% End:


\section{First experiments}
\label{sec:exp1}

In the first set of experiments, we aim to compare our model with the errors made by measuring a well known object, shown in Figure \ref{fig:c5-cad}. The target was made with a \acs{CNC} machine with a precision of $0.1 \, mm$ with respect to all the measures reported in Figure \ref{fig:c5-cad-misure}. Our goal was to reach this measurement precision. \\

The point cloud obtained with our algorithm is shown in Figure \ref{fig:c5-profile_pixel}: the points are plotted in the world reference system, after that we have projected the laser spots, detected in the image.

\begin{figure}
  \begin{minipage}[c]{.44\textwidth}
    \centering
    \includegraphics[width=\textwidth]{./images/analysis/t1/cad.png}
    \caption{CAD of the target \\ object}
    \label{fig:c5-cad}
  \end{minipage}
  \hfill
  \begin{minipage}[c]{.55\textwidth}
    \centering
    \includegraphics[width=\textwidth]{./images/analysis/t1/pixel_profile_cut.jpg}
    \caption{Laser spots, complete profile}
    \label{fig:c5-profile_pixel}
  \end{minipage}
\end{figure}
\vfill
\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{./images/analysis/t1/cad_misure.png}
  \caption{Measures of the target object}
  \label{fig:c5-cad-misure}
\end{figure}
\vfill
\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{./images/analysis/t1/pixel_profile_det.jpg}
  \caption{Details of the laser spots}
  \label{fig:c5-profile_pixel_det}
\end{figure}
\vfill
\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{./images/analysis/t1/peak_distrib.jpg}
  \caption{Distribution of the energy along a row of the image. Vertical lines indicate the 3 windows used.}
  \label{fig:c5-peak_distrib}
\end{figure}
\vfill
\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{./images/analysis/t1/com6_cmp.jpg}
  \caption{Comparison of \textit{Center of Mass} applied with extended widow sizes}
  \label{fig:c5-com_cmp_6}
\end{figure}
\vfill
\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{./images/analysis/t1/br6_cmp.jpg}
  \caption{Comparison of \textit{Blais and Rioux} applied with extended widow sizes}
  \label{fig:c5-br_cmp_6}
\end{figure}
\begin{figure}
  \centering
  \includegraphics[width=\textwidth]{./images/analysis/t1/fir6_cmp.jpg}
  \caption{Comparison of \textit{FIR} applied with extended widow sizes}
  \label{fig:c5-fir_cmp_6}
\end{figure}
\clearpage

If we magnify the data, as shown in Figure \ref{fig:c5-profile_pixel_det}, we can observe that the acquired profile is very thick. This effect is due to the noise of the laser beam (e.g. speckle), that causes small variations of the positions of the peaks, along a line of the image. Thus, we expected that the use of sub-pixel filters, should reduce the thickness of the beam. All the model described in Section \ref{sec:laser-peaks}, were introduced considering only few pixel around the candidate to be the peak. So we tried to apply them, using windows of $3$, $5$ and $7$ pixel were possible. \\

Unfortunately, the results made us wrong. Observing the live images, we noticed that the width of the Gaussian was on average equal to $20$ pixels, as the row shown in Figure \ref{fig:c5-peak_distrib}: the used interval are too small with respect to signal amplitude. Hence, we decided to increase the window to $11$ and $16$ pixels where possible. In Figure \ref{fig:c5-com_cmp_6} we can see the improvement of the detail shown in Figure \ref{fig:c5-profile_pixel_det}, obtained using the \textit{center of mass}. Therefore, we concluded that, in order to get a sufficiently precise smoothing, it is necessary that the window is bigger enough to contain the whole laser spot. To complete our analysis, we tried to further increase the size of the window. This tests did not lead to significant improvements over the windows of $11$ or $16$ pixels. On the contrary in some cases, for example in noisy areas of the images, or because the background noise, the use of a larger window deteriorated the peak recognition. \\
We reached the same results analysing the remaining filters, as shown in Figures \ref{fig:c5-br_cmp_6} and \ref{fig:c5-fir_cmp_6}. From these observations, we concluded that the windows sizes have to be comparable with the Gaussian: if they are too small, the profile will be less precise; if they are too big, we risk to add noise.

Regarding the remaining filters (Gaussian, Linear and Parabolic), we have found some difficulties extending them, in order to increase the number of points involved in: as we have already said, $3$ pixels are too few to obtain a good approximation of the laser beam. However, this operation has proved to be meaningless. The three models are based on weighted linear models, such that they work properly only if three pixel are used. Change these weights, changes the meaning of the model itself. The only one that has been extended (the Parabolic) did not reach the results reached with the \textit{\acs{COM}}, the \textit{Blaise\&Rioux} or with the \textit{FIR} filter. Thus, we decided to ignore them from our analysis.

A second observation in support of this choice is due to the saturation effect. Sometimes could happen that some consecutive pixels have the same (maximum) value, caused by changing in the illumination of the working place, out of focus lenses or lasers, or caused by translucid or labertian surfaces. If we use only $3$ or $4$ pixels, it is impossible for these filters to properly identify the peak the the laser spot. \\

At this point, we were able to evaluate some measures. To simplify the task, we decided to consider a few well-recognizable points of the profile, i.e. the points used in Figure \ref{fig:c5-cad-misure} to add quotes. Thus, we used a simple 3D corner detector, and the measures were computed as differences between the points, after the alignment of the detected profile to the model. \\
Before achieving remarkable results, we found some difficulties. The most important, is related to calibration. As we said in Section \ref{sec:teo-calibration}, it is fundamental that the grid of calibration points covers the entire \acs{FOV} of the camera. Despite that, the points have to be always on focus. If for any reasons this condition is not true, some of these points could affect the quality of the calibration itself. Thus, we manually reduced the grid until we reached a tiny type I error. \\

Finally, we had got the first results. In Figure \ref{fig:c5-match} we shown the match computed before estimate the measures, while in Tables \ref{tab:c5-r1-teo} and \ref{tab:c5-r1-real} we reported an example of how we estimated the measurement error (theoretically and with empirical data), in particular these results are obtained using the \textit{center of mass} with a window of $16$ pixels. Note that, in order to evaluate column \textit{point error}, we assumed that the event that a point falls in $p = (x, y)$ can be modelled as the union of two i.i.d. events, followed by:
  \begin{equation}
    \sigma_p = \sqrt{\sigma_x^2 + \sigma_y^2}
    \label{eq:exp:var-prop}
  \end{equation}
  \begin{figure}[t!]
    \centering
    \includegraphics[width=\textwidth]{./images/analysis/t1/example_cut.jpg}
    \caption{Example of match of the profile extracted using \textit{center of mass} with window of $16$ pixel. The model is plotted in blue, while the extracted profile is plotted in red.}
    \label{fig:c5-match}
  \end{figure}
  \input{./src/chapters/ch5-Analysis/tab-res1-teo.tex}
  \input{./src/chapters/ch5-Analysis/tab-res1-real.tex}

In Table \ref{tab:c5-r1-all}, instead, we have summarized the results obtained with all the filters considered. As we can see, they are significantly higher than the $0.1 \, mm$ that we wanted to reach. However, some considerations can already be made.
  \input{./src/chapters/ch5-Analysis/tab-res1-all.tex}

First of all, it is clear that all the proposed filters increase the precision of the spot localization. Then, it seems that the assumptions made in Section \ref{sec:laser-peaks} about the performances of the filters, are not complete: from a mathematical point of view, the \textit{center of mass} is better; however, in real case, the \textit{Blais\&Rioux} and the \textit{FIR} gave better results.

The second thing to consider, is the quality of the optical bench:
  \begin{itemize}
    \item The laser wasn't good enough, it was too thick and affected by noise. We did several attempts to properly set the properties of the camera, in order to reduce the amount of light collected by the sensor, i.e. reduce the width of the collected laser beam. Nevertheless, the thickness of the resulting profile is about $0.1 \, mm$.
    \item The model wasn't well defined. Many parameters, such as the error of the Scheimpfulg angles, or the error made positioning the laser projector, weren't known.
  \end{itemize}
Regarding this last point, we understood that the parameters not estimated by the model, can be defined in two different ways: on the one hand they strongly depends by the system itself, and by how good it is build; on the other, they allow to evaluate how good it must be made to reach the precision proposed by the model. Thus, in this first experiment, the model was tuned to reach the results empirically obtained, instead that the contrary. \\

All the imprecisions and the found problems, suggested us to use a more precise and well built systems to evaluate the correctness of our model.


% !TEX encoding = UTF-8 Unicode
\documentclass[a4paper]{article}

\usepackage{url}

\usepackage{graphicx}
\graphicspath{{../slike}}

\begin{document}

\title{Intrinsically disordered proteins (\textit{IDPs}) in \textit{SARS-CoV-2} interactome {\large (manuscript)}}

\author{Lazar Vasović {\small (student)}\\Gordana Pavlović-Lažetić {\small (professor)}}

\maketitle

\abstract{This manuscript discusses the properties of proteins and their relations in the interactomes of the selected subsets of \textit{SARS-CoV-2} proteome -- membrane, non-structural proteins, and, finally, full proteome. Protein disorder and respective node degrees in the protein interaction networks were singled out as the features of interest. Viral interactomes were also combined with that of human lung tissue so as to establish which connections in the resulting viral-host interactome are interesting and how they are linked to protein disorder. Results, which include several scatter plots and correlation analysis, indicate that hubs (highly connected nodes) and their neighbors tend to be on average more ordered than proteins with a small number of connections and their neighbors. This is in contrast to the similar studies conducted on eukaryotic interactomes and possibly opens the path to a new direction of research on viral interactomes.}

\tableofcontents

\newpage

\section{Introduction}
\label{intro}

\textbf{Biological systems} are typically viewed as networks of mutually interacting \textbf{basic building blocks} \cite{junker, guzzi}. Units that operate independently are rare, so \textbf{interactions} are a key part of any biological system. In fact, no system can be viewed as a simple sum of elements. This is true both for small cells and huge ecosystems, and the only significant difference between them is the type of basic blocks. Example microblocks (small basic blocks) are genes, proteins, and metabolites, while individual organisms and generally species are considered macroblocks (big basic blocks).

In order to conduct \textbf{computational analysis} of biological systems, it is necessary to choose an appropriate \textbf{mathematical model}. One natural model of interacting blocks is a \textbf{graph}. The use of graph formalism enables the effective representation of the real-life entities and the complex and mutual interrelationships among those entities. Basic blocks are represented as graph \textbf{nodes} and interactions as graph \textbf{edges}. Of course, this representation has its flaws, mostly because it oversimplifies some complex non-binary interactions. However, it is still quite a useful abstraction, since it opens the path to effective computational analysis.

\textbf{Protein interaction networks (\textit{PINs})}, also known as protein-protein interaction networks (\textit{PPINs} or \textit{PPI} networks), are an important class of biological systems. Each their node represents a \textbf{protein}, while \textbf{undirected} edges encapsulate binary physical interaction between two proteins, also known as \textbf{protein-protein interactions (\textit{PPIs})}. As a part of larger \textit{omics} studies, \textit{PINs} are also known as \textbf{interactomes} and graph nodes as \textbf{interactors}. The importance of these networks lies in the fact that protein molecules are the workhorses of the cell, performing and controlling almost all activities in an organism. For example, highly connected proteins (hubs) within a network are most probably vital molecules and are in many cases more essential for survival than proteins with lower connectivity. Related \textit{PINs} can also be aligned with each other as a way of identifying substructures that have been preserved throughout evolution. Densely connected parts of the network may represent potential protein complexes, which is an additional benefit of using graph formalism.

One particularly interesting class of proteins are \textbf{intrinsically disordered proteins (\textit{IDPs})}. These proteins lack a fixed and well-defined three-dimensional structure and are highly abundant in nature \cite{uversky}. Their conformational flexibility (intrinsic disorder) may be present in a molten globular form, in a form of a random coil, or a pre-molten globule. Only a few proteins are fully disordered in their native state, so \textit{IDPs} also include semi-disordered proteins, whose flexible parts are called \textbf{intrinsically disordered regions (\textit{IDRs})}. \textit{IDPs} and \textit{IDRs} often facilitate protein-protein interactions, being flexible linkers and binding to multiple partners, thus \textit{IDPs} frequently serve as hubs in complex \textit{PPI} networks.

In the last years, many authors have researched the topic of interactomics and intrinsically disordered proteins. Much has been revealed about the biological importance of \textit{IDPs}/\textit{IDRs}, as well as their \textbf{markedly big} interactomes \cite{teilum, cumberworth}. Most studies in the field of bioinformatics and computational analysis focused on exploring \textit{PINs} of eukaryotic organisms such as human (\textit{Homo sapiens}) and baker's yeast (\textit{Saccharomyces cerevisiae}). There is an agreement between these papers that hubs, which are highly connected nodes, are \textbf{on average more disordered} than ends, which are proteins with a small number of connections \cite{haynes, dosztanyi, singh, patil, hu}.

Contrary to the related work, this manuscript focuses on analyzing the interactomes of the selected subsets of \textbf{\textit{SARS-CoV-2} proteome}. \textit{SARS-CoV-2} is a virus, so its proteins could possibly act differently than those of eukaryotes. Protein \textbf{disorder degree} and respective \textbf{node degrees} in the \textit{PINs} were singled out as the features of interest. Additionally, since merged \textbf{viral-host interactomes} had been found as useful in establishing which interactions are biologically interesting \cite{gao}, viral interactomes were combined with that of human lung tissue. This made it possible to discuss if links in the viral-host interplay are related to protein disorder. The methodology is described in section \ref{meth}, while results, which are based on several scatter plots and correlation analysis, are shown in section \ref{res}.

\section{Materials and methods}
\label{meth}

The paper aimed to examine several \textit{SARS-CoV-2} interactomes, all constructed from \textbf{publicly available data}. Four different graphs were chosen as the starting point, with their core properties shown in table \ref{inter}.

\begin{table}[h!]
  \centering
  \caption{Analyzed interactomes and their properties}
  \resizebox{.9\textwidth}{!}{%
  \begin{tabular}{c | c c | c c}
     & Main protein(s) & Source database & Nodes & Edges \\ \hline
    M\_IntAct & membrane (M) & IntAct & 198 & 294 \\
    M\_iRef & membrane (M) & iRefIndex & 211 & 451 \\
    Nsp\_iRef & non-structural (NSPs) & iRefIndex & 311 & 1066 \\
    SARS2\_iRef & full proteome & iRefIndex & 1419 & 8722 \\
  \end{tabular}}
  \label{inter}
\end{table}

As an example, the interactome of non-structural proteins (\textit{NSPs}) is shown in figure \ref{nsp}. Nodes are \textbf{colored} by origin -- human are blue, while viral are orange. Node \textbf{size} is proportional to its degree in the network.

\begin{figure}[h!]
  \centering
  \includegraphics[scale=0.45]{Nsp_iRef.png}
  \caption{Nsp\_iRef, interactome of non-structural proteins}
  \label{nsp}
\end{figure}

Database \textbf{\textit{IntAct}}\footnotemark \footnotetext{IntAct Molecular Interaction Database: \url{https://www.ebi.ac.uk/intact/home}} was initially chosen as one of the most popular in the field of protein-protein interaction warehousing. It, however, doesn't include separate data for the \textit{NSPs} of the considered \textit{SARS-CoV-2}\footnotemark\footnotetext{SARS-CoV-2: \url{https://www.ncbi.nlm.nih.gov/Structure/SARS-CoV-2.html}}, which makes it inappropriate for conducting full-scale analysis. Instead, \textbf{\textit{iRefIndex}}\footnotemark \footnotetext{iRefIndex PSICQUIC WS: \url{https://irefindex.vib.be/}} was chosen as the only significant database that meets the coverage criterion. Both databases were queried by protein \textit{IDs} and the resulting lists of \textit{PPIs} were used for building the networks. It can be noted that \textit{iRefIndex} produces \textbf{noticeably denser} networks in comparison to \textit{IntAct}, i.e. it contains many additional interactions. Another interesting fact is that the graphs containing non-structural proteins (\textit{Nsp} and \textit{SARS2\_iRef}) are much \textbf{more connected} than those without them (\textit{M}).

Two additional interactomes were taken into consideration so as to analyze the dynamics of viral-host interplay. They are shown in table \ref{addit}.

\begin{table}[h!]
  \centering
  \caption{Additional interactomes and their properties}
  \resizebox{.9\textwidth}{!}{%
  \begin{tabular}{c | c c | c c}
     & Main proteins & Source database & Nodes & Edges \\ \hline
    SARS\_iRef & only viral proteins & iRefIndex & 30 & 42 \\
    lungs\_HuRI & human lung tissue & NDEx & 175 & 162 \\
  \end{tabular}}
  \label{addit}
\end{table}

The first graph, containing only \textbf{viral proteins}, is shown in figure \ref{viral}.

\begin{figure}[h!]
  \centering
  \includegraphics[scale=0.55]{SARS_iRef.png}
  \caption{SARS\_iRef, interactome with only viral proteins}
  \label{viral}
\end{figure}

Nodes are colored by their type. Light blue denotes structural proteins, which are spike (\textit{S}), envelope (\textit{E}), membrane (\textit{M}), and nucleocapsid (\textit{N}). Orange stands for non-structural proteins, which are \textit{Nsp1} to \textit{Nsp16} (not all are present in \textit{iRefIndex}). Dark blue represents accessory proteins, which are \textit{ORFs}. Once again, size is proportional to node degree.

The second graph, which represents the interactome of the \textbf{human lung tissue}, was taken from a reference map of the human binary protein interactome known as \textbf{\textit{HuRI}} (\textit{Human Reference Interactome}) \cite{luck}. The network itself was downloaded from the \textbf{\textit{NDEx}}\footnotemark \footnotetext{NDEx: \url{https://www.ndexbio.org/#/network/289c6fe3-5fb9-11e9-9f06-0ac135e8bacf}} database. It was chosen because \textit{SARS-CoV-2} is a respiratory virus and is shown in figure \ref{lungs}. It is made of two distinct types of nodes -- hubs (green) and ends (blue).

\begin{figure}[h!]
  \centering
  \includegraphics[scale=0.65]{pluca_HuRI.png}
  \caption{lungs\_HuRI, interactome of human lung tissue}
  \label{lungs}
\end{figure}

The additional interactomes were combined and the nature of the links in the merged network was examined. Discussion is given in section \ref{res}.

Since the idea was to analyze the protein disorder in the studied interactomes, the next step was to choose appropriate \textbf{disorder predictors}. Amino acid composition\footnotemark, \footnotetext{Composition Profiler: \url{http://www.cprofiler.org/help.html}} \textit{PONDR}\footnotemark \footnotetext{Predictor of Natural Disordered Regions (PONDR): \url{http://www.pondr.com/}} family, and charge-hydropathy (\textit{CH})\footnotemark[6] were the methods of choice. Number of short (at least five residues in length) and long (more than thirty residues in length) disordered regions, as well as the average disorder of neighbors ("the other side of interactions") were additionally considered. Specialized databases like \textit{DisProt} and \textit{D²P²} lack many \textit{SARS-CoV-2} proteins, so they were disregarded.

\textbf{Correlation analysis} was the main method used. The relationship between node degree and disorder degree of the respective protein and its neighbors according to all the listed predictors was represented on the plane, as a series of \textbf{scatter plots}. Node degree was put on the horizontal axis, while the vertical axis was reserved for the output of disorder predictors. Concrete examples and discussion are given in section \ref{res}, dedicated to the results. The entire processing, which ranges from retrieving the data to the final analyses, was done as a \textbf{\textit{Python}}\footnotemark \footnotetext{Python programming language: \url{https://www.python.org/}} script.

\section{Results}
\label{res}

\newpage
\addcontentsline{toc}{section}{References}
\appendix
\bibliography{IDP_in_SARS-CoV-2}
\bibliographystyle{unsrt}

\end{document}