\chapter{sub3term Example of Substrate Resistance Extraction}
\section{Introduction}
\label{STintro}
In this example, we will be studying a configuration of 3 substrate terminals.
We will see how Space is used to compute the 3D substrate resistances.
See also the "Space Substrate Resistance Extraction User's Manual".
\\[1 ex]
The layout looks as follows (such a figure can be printed using \io{getepslay}, see
\manualpage{getepslay}):

\begin{figure}[h]
\centerline{\epsfig{figure=sub3term/sub3term.eps, width=8cm}}
\end{figure}

\section{Files}
This tutorial is located in the directory \CACDTOP{demo/sub3term}.
Initially, it contains the following files:
\begin{filelist}
\item[README] A file containing information about the demo.
\item[sub3term.gds] The layout of the sub3term design.
\item[elem.s] The technology file for Space. See Section \ref{STtech}.
\item[param.p] The parameter settings file for Space. See Section \ref{STparam}.
\item[script.sh] A file containing the commands for executing all
steps of the demo in sequence.
\end{filelist}

\section{Technology File}
\label{STtech}
The technology file that we will use is the file \io{elem.s}, see listing below.

\small \begin{Verbatim}[frame=single]
listing of file elem.s
   3    #
   4    # space element definition file for metal substrate terminals
   5    #
   6
   7    colors :
   8        cmf   blue
   9        @sub  pink
  10
  11    conductors :
  12      # name     : condition : mask : resistivity : type
  13        cond_cmf : cmf       : cmf  : 0.0         : m
  14
  15    contacts :
  16      # name     : condition : lay1 lay2 : resistivity
  17        cont_cmf : cmf       : cmf  @sub : 0.0
  18
  19    sublayers :
  20      # name       conductivity  top
  21        epi        6.7           0.0
  22        substrate  2.0e3        -7.0
  23
  24    #EOF
\end{Verbatim}
\normalsize
Please note that the masks to be used in this file have already been defined
in the \io{maskdata} file in the process library \CACDTOP{lib/process}.
In this example, we will only use the first metal mask (\io{cmf}).

When we look to the file, we first see on line \io{7} the \io{colors} specification.
The color of the \io{cmf} mask is chosen to be \io{blue}.
The color for the \io{@sub} mesh is chosen to be \io{pink}.
Note that this info is only used by the visualization tool \io{Xspace}.

Starting on line \io{11} we find the \io{conductors} definitions.
This is an important section, because without this we can not specify
element pins and contacts.
The conductor value is for this example unimportant,
because we don't extract interconnect resistances.

Starting on line \io{15} we find the \io{contacts} definitions.
This is also an important section,
because here is the connection between first metal and the substrate specified.
These nodes are also called substrate terminals.
However, substrate terminals can also be bulk regions or pins of capacitances.

For 3D substrate resistance extraction, we need only one additional section.
Starting on line \io{19} we find the \io{sublayers} definitions.
These definitions specify for each substrate layer the conductivity (in $S / m$)
and the starting position (in $\mu m$).
Note that the last layer (on line \io{22}) extends to minus infinity.

\section{Parameter File}
\label{STparam}
The parameter file \io{param.p} is controlling the extraction, see listing below.

\small \begin{Verbatim}[frame=single]
listing of file param.p
  3
  4  BEGIN sub3d
  5  be_mode         0c   # piecewise constant collocation
  6  max_be_area     1.0  # max. size of interior elements in sq. microns
  7  edge_be_ratio   0.05 # max. size edge elem. / max size inter. elem.
  8  edge_be_split   0.2  # split fraction for edge elements
  9  be_window     inf    # infinite window, all resistances
 10  END sub3d
 11
 12  disp.save_prepass_image   on
\end{Verbatim}
\normalsize
In the above listing, we see a \io{sub3d} block from line \io{4} to line \io{10}.
These parameters are only used for controlling the substrate 3D extraction.
The \io{max\_be\_area} (in $\mu m^2$) and \io{be\_window} (width in $\mu m$) parameter
needs always to be specified.
On line \io{5}, you find also the \io{be\_mode} parameter.
However, this boundary element mode is the default.
On line \io{9}, you see that the \io{be\_window} parameter is chosen to be infinity.
That means, that all substrate terminal nodes are in the same strip,
and are all connected with each other.
See for an explanation of the parameters also the "Space Substrate Extraction User's Manual".

On line \io{12}, you find a parameter for the visualization tool \io{Xspace}.
It means, that the image of the prepass may not be cleared
when starting drawing the image of the last pass.
See for an explanation also the "Xspace User's Manual".

\section{Running the Extractor}
The file \io{script.sh} is a batch file for running all the commands for this example.
First, it changes the current working directory '.' into a project directory:
\small
\begin{Verbatim}
% mkpr -p scmos_n -l 0.1 .
\end{Verbatim}
\normalsize
We use a lambda value (option \io{-l}) of $0.1 \mu m$.
For the CMOS technology, we use the \io{scmos\_n} process from the technology library.
We use the mask names as defined in the \io{maskdata} file of the library.
But, we are not using the default technology
file \io{space.def.s} and parameter file \io{space.def.p} of the library.
Thus, the local technology file \io{elem.s} needs to be compiled with \io{tecc} to the format
(a \io{.t} file) which is used by the extractor:
\small
\begin{Verbatim}
% tecc elem.s
\end{Verbatim}
\normalsize
Second, the layout description is put into the project database.
The layout is supplied in a GDS2 file, which can be converted to
internal database format with the \io{cgi} program:
\small
\begin{Verbatim}
% cgi sub3term.gds
\end{Verbatim}
\normalsize
Now, we can extract a circuit description for the layout of the \io{sub3term} cell, as follows:
\small
\begin{Verbatim}
% space3d -vB -E elem.t -P param.p sub3term
\end{Verbatim}
\normalsize
We use the verbose option (\io{-v}) to see what the extractor program is doing.
Second, we use option \io{-B} to perform a 3D substrate resistance extraction.
The substrate resistances are calculated, using a 3D mesh method (BEM).
The calculated resistance values are assigned between substrate terminals and between the substrate
terminals and the substrate plane (node SUBSTR).
\\[1 ex]
The extracted circuit can be inspected using the \io{xspice} program.
We use option \io{-a} to get alpha-numeric node names.
\small
\begin{Verbatim}
% xspice -a sub3term
\end{Verbatim}
\normalsize
The output of the SPICE circuit is listed below.

\small \begin{Verbatim}[frame=single]
   1    sub3term
   2
   3    * Generated by: xspice 2.39 25-Jan-2006
   4    * Date: 21-Jun-06 14:56:15 GMT
   5    * Path: /users/simon/sub3term
   6    * Language: SPICE
   7
   8    * circuit sub3term c b a
   9    r1 c a 1.265872meg
  10    r2 c b 679.1891k
  11    r3 c SUBSTR 75.86724k
  12    r4 a b 679.1891k
  13    r5 a SUBSTR 75.86724k
  14    r6 b SUBSTR 49.44378k
  15    * end sub3term
\end{Verbatim}
\normalsize
Below, you find a drawing of the listing of the extracted circuit.

\begin{figure}[h]
\centerline{\epsfig{figure=sub3term/circuit.eps, width=9cm}}
\end{figure}

Note that there are substrate resistances between all nodes of the circuit.
This happens, because the chosen \io{be\_window} has a width of infinity.
Each node has also a substrate resistance to the substrate plane (node \io{SUBSTR}).

\section{Running the Visualization Tool}
When you want to see the boundary element mesh,
which is used for the substrate terminal regions,
you must use the visualization tool \io{Xspace}.
It runs the \io{space3d} extractor in the background and requests for graphical output.
The extractor writes this graphical output to a "display.out" file, which \io{Xspace} is reading.
The advantage of the output file is, that you can playback it again without running the extractor.
And it makes zooming and panning on the image very easy.
For additional information, consult the "Xspace User's Manual".

Note that you can playback only the file "display.out" and that \io{Xspace} needs the color
information of the used technology file.
To show something, you need to set items in the "Display" menu.
Note that you can only show items, which are written to the file "display.out".
To start a playback, use hotkey 'a' (or click on "extract again" in the "Extract" menu).

Now, we shall extract a circuit description for the layout of the \io{sub3term} cell,
using the visualization tool as follows:
\small
\begin{Verbatim}
% Xspace -vB -E elem.t -P param.p sub3term
\end{Verbatim}
\normalsize
You don't need to set the cell name in the "Database" menu and
also don't need to set the extract options in the "Options" menu,
because that is already specified on the command line.

\noindent
You need only to set some display options.
Go to the "Display" menu and click on button "DrawBEMesh".
To run the extractor, go to the "Extract" menu and click on "extract" (or use hotkey 'e').

\noindent
You get the following picture:
\begin{figure}[h]
\centerline{\epsfig{figure=sub3term/xspace1.eps, width=12cm}}
\end{figure}

\noindent
You see three substrate terminals, which are meshed into a number of area's.
The complete area of terminal "a" and "c" is $1.0 \mu m^2$.
The area of terminal "b" is $2.0 \mu m^2$.
\\[1 ex]
When you look to the space parameter file "param.p",
you see that the maximum size of an interior element may be $1.0 \mu m^2$
(parameter \io{max\_be\_area}).
The maximum size of an edge element, however, may be $0.05 * 1.0 \mu m^2$
(parameter \io{edge\_be\_ratio}).
When we edit the file "param.p" and change the \io{edge\_be\_ratio} to its default value of $1.0$
(or make the line to be a comment line by placing a '\#' before it),
then we shall see what happens.
\\[1 ex]
We can now run the visualization tool again.
Or better, when you have put it in the background, hit the extract key 'e' again.
\\[1 ex]
You get the following picture:
\newpage
\begin{figure}[h]
\centerline{\epsfig{figure=sub3term/xspace2.eps, width=10cm}}
\end{figure}

\noindent
And indeed, we get another mesh for the 3 substrate terminals.
\\[1 ex]
Below you see what happens, if we don't change the \io{edge\_be\_ratio},
but the \io{edge\_be\_split} parameter instead (set this fraction to its default value of $0.5$).
You get the following picture:
\begin{figure}[h]
\centerline{\epsfig{figure=sub3term/xspace3.eps, width=10cm}}
\end{figure}

\noindent
The accuracy of the extraction result is depended of the mesh you chose.
And, how finer the mesh, how more computation time is needed to calculate the result.


\chapter{Evaluation}

\section{Measures of Success}

The success of the project can be measured by comparing the project results with objectives. We will also review the design choices in constructing the tool. The review will highlight the practical outcomes from using argumentation to explain scheduling.
\linespace
Arguably, the most important outcome of this project is that the tool is functionality correct. This means the tool is required to explain schedules for feasibility, efficiency and satisfaction with respect to user decisions.
\linespace
One objective is to implement an accessible tool. To measure accessibility, we can refer to the tractability complexity from explanations. The length of explanations either in the number of words or characters may be correlated with understandability. In addition, we conduct a survey targeted towards potential users. Because the understandability of explanations are difficult to measure, we will use open-ended questions. To carefully evaluate explanations, one would need to refer to cognitive science, which is beyond the scope of this project. We will have two groups, one group without access to the tool, another group with access. By using these metrics, we can measure the accessibility of our tool. Moreover, we can measure performance to reflect responsiveness and scalability of the tool. This may be achieved by using profiling utilities to measure performance metrics such as execution time and memory consumption.

\section{Theoretical Results}

We summarise our complexity results from Chapter \ref{implementation}.

\begin{figure}[H]
	\begin{tabular}{lccc}
		\hline
		Algorithm & Computational & Memory & Tractability\\
		\hline
		\textsc{Construct-Feasibility} & $\mathcal{O}(m^2n^2)$ & $\mathcal{O}(m^2n^2)$ &\\
		\textsc{Construct-Efficiency} & $\mathcal{O}(m^2n^2)$ & $\mathcal{O}(m^2n^2)$ &\\
		\textsc{Construct-Satisfaction} & $\mathcal{O}(m^2n)$ & $\mathcal{O}(m^2n^2)$ &\\
		\textsc{Compute-Unattacked} & $\mathcal{O}(m^2n^2)$ & $\mathcal{O}(mn)$ &\\
		\textsc{Compute-Partial-Conflicts} & $\mathcal{O}(mn)$ & $\mathcal{O}(mn)$ &\\
		\textsc{Explain-Stability} & $\mathcal{O}(m^2n^2)$ & $\mathcal{O}(m^2n^2)$ &\\
		\textsc{Explain-Feasibility} & $\mathcal{O}(mn^2)$ & $\mathcal{O}(mn)$ & $\mathcal{O}(mn)$\\
		\textsc{Explain-Efficiency} & $\mathcal{O}(mn^2\log(mn^2))$ & $\mathcal{O}(mn^2)$ & $\mathcal{O}(mn^2)$\\
		\textsc{Explain-Satisfaction} & $\mathcal{O}(mn)$ & $\mathcal{O}(mn)$ & $\mathcal{O}(mn)$\\
		\textsc{Full-Precomputation-Explain} & $\mathcal{O}(m^2n^2\log(mn^2))$ & $\mathcal{O}(m^2n^2)$ & $\mathcal{O}(mn^2)$\\
		\textsc{Partial-Precomputation-Explain} & $\mathcal{O}(m^2n^2\log(mn^2))$ & $\mathcal{O}(mn^2)$ & $\mathcal{O}(mn^2)$\\
		\hline
	\end{tabular}
	\caption{Computational, memory and tractability complexity of algorithms using argumentation}
\end{figure}

Using an naive explanation approach only improves the computational complexity of verifying the feasibility property, from $\mathcal{O}(m^2n^2)$ to $\mathcal{O}(mn)$.

\section{Practical Results}

The tool's functionality is demonstrated in the user documentation guide in Appendix \ref{userguide}. We will compare different implementation methods and see how fit for purpose our tool is through a survey.

\subsection{Comparison of Explanation Generation Methods}

We will compare two algorithmic approaches to AAF construction with and the naive approach without argumentation. The algorithms are implemented, then profiled for elapsed time and maximum allocated memory. The time and memory are measured with the Python's \texttt{cProfile} and \texttt{memory-profiler} modules respectively. The tool was executed on Department of Computing's virtual machines, with the specification of dual-core CPU at 2GHz with 2GiB RAM.
\linespace
Time comparison:
\begin{itemize}
	\item Elapsed time measurements are noisy.
	\item For less than 100 jobs, all approaches have approximately equal timings.
	\item From profiling, the tool takes 0.4 seconds on average to startup.
	\item Partial precomputation is 3\% faster than full precomputation on average, excluding startup time.
	\item Naive is 18\% faster than partial precomputation on average, excluding startup time.
	\item Both graphs hints at quadratic complexity.
\end{itemize}

Memory comparison:
\begin{itemize}
	\item For less than 40 jobs, all approaches have approximately equal memory usage.
	\item From profiling, the tool uses 52MiB on average to startup.
	\item For a large number of jobs, partial-precomputation is 7 times more efficient than full precomputation excluding startup memory.
	\item Naive and partial-precomputation have less than 1\% memory usage difference.
	\item Both graphs hints at quadratic complexity.
\end{itemize}

\newpage

\begin{figure}[H]
	\centering
	\includegraphics[scale=0.7]{figures/precomputation_cpu_small}
	\caption{Elapsed time comparison where $m=10$ and $0\leq n\leq 100$ with 1000 samples.}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[scale=0.7]{figures/precomputation_cpu_big}
	\caption{Elapsed time comparison where $m=10$ and $0\leq n\leq 800$ with 8000 samples.}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[scale=0.7]{figures/precomputation_memory_small}
	\caption{Maximum allocated memory comparison where $m=10$ and $0\leq n\leq 100$ with 1000 samples.}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[scale=0.7]{figures/precomputation_memory_big}
	\caption{Maximum allocated memory comparison where $m=10$ and $0\leq n\leq 800$ with 8000 samples.}
\end{figure}

\subsection{Survey}

Used two sample groups, one group without access to the tool, and one group with access. In both groups, we used the same questionnaire, which can be found in the appendix. Each question was designed with purpose. We try not to use too many machines and jobs to ensure that the questions could be done without too much effort. Note we do not ask questions about feasibility because its validation is trivial.

\begin{itemize}
	\item Question 1 shows a schedule which is efficient, but not optimal. The tool does not explain for optimality, so users can see there are cases in which the tool may not be helpful.
	\item Question 2 shows a schedule with more machines and jobs, to test whether users can find improvements in a non-trivial schedule. It is possible to optimise the schedule in one single exchange, but this may not be obvious.
	\item Question 3 shows an application of fixed decisions.
	\item Question 4 explores pairs of jobs, which can be modelled using fixed decisions. The question explores how intuition can be formed using additional constraints.
	\item Question 5 explores replanning of a schedule by insertion of a new job. Clearly, trivially inserting a new job does not result in a optimal schedule. It is possible to arrange the schedule in one single exchange before insertion to achieve an optimal updated schedule.
\end{itemize}

Although these five questions do not cover the tool's complete functionality, they cover questions and seek explanations that require intuition, with or without the tool. This is important in assessing how users can operate the tool in non-trivial cases.

\begin{figure}[H]
	\label{notooltable}
	\begin{center}
		\begin{tabular}{c | c c c c}
			Question & Incorrect & Incomplete & Complex & Correct \\
			\hline
			1 & 0 & 2 & 0 & 8 \\
			2 & 2 & 1 & 2 & 5 \\
			3 & 3 & 0 & 0 & 7 \\
			4 & 2 & 5 & 0 & 3 \\
			5 & 2 & 1 & 3 & 4 \\
		\end{tabular}
	\end{center}
	\caption{Aggregated results from Section \ref{notool}}
\end{figure}

In this survey, we asked a wide demographic, from GCSE students to graduates, including people who do have little knowledge of computer science or mathematics. From Figure \ref{notooltable}, we see that the majority of respondents are able to get most questions correct. Some responses are considered complex, when their answers are technically correct, but give more steps than necessary. This may hint that some intuitive ways to optimise a schedule may not minimise the number of job exchanges. Some responses are considered incomplete, where only the question was partially answered. We can see in more complex schedules that users give more complex answers. Excluding outliers, the questionnaire took 15 minutes on average to complete.

\subsection{Feedback}

During development of the tool, I asked my colleagues and friends for advice about the usability of my tool.

\begin{itemize}
	\item In early versions of the tool, the GUI did not arrange textboxes and buttons into groups such as problem, schedule and explanation. Instead, all buttons and their functionality were exposed as an menu. This was confusing to new users because it was not evident which textboxes were input or output.
	\item The paper \cite{aes} represented machine job pair assignments as a pairs of integer indices. The tool internally uses indices to compute explanations, however exposing both machines and job as integer indices was confusing. Therefore, jobs are now represented alphabetically.
	\item The tool uses regular expressions to validate user input. This was too strict, and users often forgot to input well-formatted white-space. To solve this, the tool automatically corrects missing or surplus white-space.
\end{itemize}


\documentclass[colorlinks=true,pdfstartview=FitV,linkcolor=blue,
            citecolor=red,urlcolor=magenta]{ligodoc}

\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{longtable}
\usepackage{rotating}
\usepackage[usenames,dvipsnames]{color}
\usepackage{fancyhdr}
\usepackage{subfigure}
\usepackage{hyperref}
\usepackage{mathtools}
\usepackage{listings}
\usepackage{minted}

\ligodccnumber{T}{17}{00198}{}{v1}% \ligodistribution{AIC, ISC}

\setlength\parindent{24pt}

\title{Online Detector Characterization using Neural Networks}

\author{Roxana Popescu}

\begin{document}

\tableofcontents

\section{Abstract}

\indent

\par Data obtained from LIGO has noise that comes from many sources. To be able to better distinguish signals from the noise, it is important to characterize the type of noise observed. Machine learning algorithms can be used to look for patterns within the data and to classify the data into distinct categories. We are using clustering algorithms, such as kmeans, to identify earthquakes in seismic noise data. To test how well the clustering algorithms work, we compare the clusters to the times when earthquake waves reach the detector site. This comparison will be used to evaluate how well a neural network determines earthquakes compared to the clustering algorithms.

\section{Introduction}

\indent

\par The data obained from LIGO contains noise that comes from many sources. In order to be able to better distinguish signals from the noise, it is important to characterize the type of noise observed. Machine learning algorithms can be used to look for patterns within the data and to classify or cluster the data into different categories.

\par There are many sensors near the LIGO detectors that measure sources of noise. For example, there are several stations at each LIGO detector that measure seismic noise in different frequency channels in each of the X,Y, and Z directions. Within the data, there are different types of seismic noise such as earthquakes and anthropogenic noise.

\par In order to sort data, machine learning algorithms can use one of two approaches: classification or clustering. Classification algorithms search the data and sort the data into already defined categories. Clustering algorithms look for relationships within the data to create categories into which the data is then sorted. Classification algorithms are part of supervised learning since the computer determines the structure of the data from data that is already provided. Clustering algorithms are part of unsupervised learning since the computer determines the structure of the data without any previous information. Clustering algorithms can be used to characterize the noise by identifying common characteristics within the noise and the clustering algorithms can further help with classification. \cite{Citation1}

\par Neural networks can be used to find relationships between the inputed data by using hidden layers of connections within the data. Neural networks consist of layers of units called artifical neurons, which activate based on whether inputs into the unit meet a certain threshold. \cite{Citation1}

\par The aim of this project was to characterize different sources of noise from LIGO using machine learning algorithms. First we tested clustering algorithms on seismic data, and then implemented a neural network to characterize the seismic noise data.

\section{Clustering Algorithms}

\subsection{K-means Clustering}

\indent

\par The k-means clustering algorithm creates clusters by separating data points into k number of groups. The value of k is inputted into the algorithm. The clusters are determined by minimizing the inertia, or the within-cluster sum-of-squares. The inertia is a measure of how coherent the clusters are. By minimizing the intertia, the algorithm tries to minimize the difference between the mean value of a cluster and the values of points in the cluster. If a set of n samples x are inputted, the algorithm divides the samples into k clusters that are referred to as  C. Each cluster is described by its mean \(u_j\), or centroid. The interia of a cluster is caluclated by the following expression:

\[\sum_{i=0}^{n} \min_{\mu_j \in  C}(\|x_j-\mu_i\|^2)\]

\par The inertia is not normalized, but lower values are better and zero is the optimum value. The inertia assumes that the clusters are convex and isotropic, and would not work well to cluster irregular or elongated clusters. \cite{Citation2}\cite{Citation3}

\subsection{DBSCAN Clustering}

\indent

\par The DBSCAN clustering algorithm creates clusters out of areas in the data of higher density. Unlike kmeans, it does not consider clusters to have any particular shapes, and the algorithm determines the number of clusters based on inputted parameters. Core samples are points that are in areas of high densities. The algorithm creates clusters around core samples so that the clusters consist of core samples, and non-core samples that are close to the core samples. The core samples are determined by two input parameters, the minimum samples and a specified distance, $\varepsilon$. A point is in the $\varepsilon$-neighborhood if the distance d from a point p to a point q is within a radius of of $\varepsilon$. High density areas have the minimum sample of values within the $\varepsilon$-neighborhood. By increasing the number of minimum samples, and decreasing the distance, $\varepsilon$, a cluster's density is increased. \cite{Citation2}\cite{Citation4}

\subsection{Agglomerative Clustering}

\indent

\par Agglomerative clustering is a type of hierarchical clustering algorithm. Hierarchical clustering builds clusters by merging and splitting clusters many times. Agglomerative clustering works by initially giving each data point its own cluster and then merging the clusters until the inputed number of clusters is reached. \cite{Citation2}

\subsection{Birch Clustering}

\indent

\par Birch clustering stands for balanced iterative reducing and clustering hierarchies. It is a hierarchical clustering algorithm which builds clusterings by merging and splitting clusters many times. \cite{Citation5}

\subsection{Evaluating Clustering Algorithms}

\subsubsection{Calinsky Harabaz Index}

\indent

\par The Calinsky-Harabaz index is a method used to evaluate how well clustering algorithms work, that does not require input of external data. The Calinsky-Harabaz score is calculated by finding the ratio of the between-clusters dispersion mean to the within-cluster dispersion mean. This ratio is calculated as follows:

\[s(k) = \frac{Tr(B_k)}{Tr(W_k)}\times\frac{N-k}{k-1}\]

\par Where k is the number of clusters, \(B_k\) is the between group dispersion matrix, \(W_k\) is the within group dispersion matrix and N is the number of data points. \(W_k\) and  \(B_k\) are defined by:

\[W_k = \sum_{q=1}^{k} \sum_{x \in C_q} (x-c_q)(x-c_q)^T\]

\[B_k = \sum_{q} n_q (c_q-c)(c_q-c)^T\]

\par Where \(C_q\) is the number of set points in cluster q, \(c_q\) is the center of cluster q, c is the center of the clusters, and \(n_q\) is the number of points in cluster q. \cite{Citation2}

\subsubsection{Comparing Clusters to Earthquake Times}

\indent

\par Another way to evaluate how well the clustering algorithms work is to add up the cluster labels that occur five  minutes before and after an earthquake Rayleigh wave arrives, to add up the total amount of cluster labels, and for each individual cluster to divide the number of cluster labels that appear near the earthquake by the total number of cluster labels. For each cluster k, the earthquake comparison score, E(k) can be determined by:

\[E(k) = \frac{N_e}{N_t}\]

Where \(N_e\) is the number of cluster labels five minutes before and after an earthquake, and \(N_t\) is the total number of cluster labels. If a cluster corresponds to the presence of an earthquake then it will have a high percentage of its cluster labels present near an earthquake.

\section{Clustering Results}

\indent

\par I compared how well the clusters determined by clustering algorithms correspond to earthquakes that show up as peaks in the data. I read in seismic data taken from three seismometers. I read in the earthquake band channels from the data and then clustered the channels using kmeans, and dbscan. The script counts the cluster labels five mintues before and after the time when earthquake Rayleigh waves arrive at the site as well as the total number of cluster labels. For each individual cluster, the earthquake comparison score is calculated. Only earthquakes with ground displacement greater than 65 percentile are considered. This score is used to determine how well a cluster corresponds to an earthquake.

\par I have used seismic data from the Hanford observatory from March 2017. I have clustered the data from the earthquake channels using kmeans, and  dbscan. I've also used the Calinsky-Harabaz index to evaluate how well the clustering works. Tables \ref{Table 1} and \ref{Table 2} show the clustering results for the kmeans and dbscan algorithms respectively.

\begin{table}[h!]
\centering
 \begin{tabular}{| m{3.5cm} m{3.5cm} m{3.5cm} m{3.5cm}|}
 \hline
 Number of Clusters & Calinsky-Harabaz Score & Cluster of Earthquake Score & Earthquake Score\\ [0.5ex]
 \hline\hline
 2 & 40192 & 1 & 0.5\\
 \hline
 3 & 37288 & 1 & 0.48\\
 \hline
 4 & 43960 & 2 & 0.31\\
 \hline
 5 & 44225 & 2 & 0.34\\
 \hline
 6 & 45618 & 2 & 0.33\\
 \hline
 7 & 46338 & 2 & 0.33\\
 \hline
 8 & 46349 & 1 & 0.44\\
 \hline
 9 & 46190 & 3 & 0.59\\
 \hline
 10 & 45323 & 6 & 0.75\\
 \hline
 Average & 43943 & N/A & 0.45\\
 \hline
 \end{tabular}
 \caption{Results of kmeans clustering}
 \label{Table 1}
\end{table}

\begin{table}[h!]
\centering
 \begin{tabular}{|m{2cm} m{2.25cm} m{2.25cm} m{2.25cm} m{2.25cm} m{2.25cm}|}
 \hline
 Epsilon Value & Minimum Samples & Number of Clusters & Calinsky-Harabaz Score & Cluster of Earthquake Score & Earthquake Score\\ [0.5ex]
 \hline\hline
 1 & 15 & 1 & 14 & -1 & 0.01\\
 \hline
 2 & 10 & 15  & 5 & -1 & 0.01\\
 \hline
 2 & 15 & 5 & 6 & -1 & 0.01\\
 \hline
 2 & 20 & 1 & 14 & -1 & 0.01\\
 \hline
 2 & 25 & 1 & 14 & -1 & 0.01\\
 \hline
 2 & 30 & 1 & 14 & -1 & 0.01\\
 \hline
 3 & 15 & 6 & 123 & -1 & 0.01\\
 \hline
 4 & 15 & 8 & 194 & -1 & 0.01\\
 \hline
 \end{tabular}
 \caption{Results of DBSCAN clustering}
 \label{Table 2}
\end{table}

\par The dbscan clustering algorithm does not seem to work well because it places the majority of its points in the noise cluster (indicated as -1)  which results in most of the points near earthquakes being classified as noise.

\par A plot of the data clustered by kmeans is shown in Figure \ref{fig:image1} and Figure \ref{fig:image2}. The lines that indicate the time that the Rayleigh waves from the earthquake reach the sight do not align with the peaks of the earthquakes on the plot. As a result, a new criteria was defined to evaluate how well the clusters align with earthquake predictions. The peaks in the data are taken to be earthquakes and the peak earthquake score  counts the cluster labels five mintues before and after the center of the ppeaks. This score is used to determine how well a cluster corresponds to an earthquake as determined from peaks in the data. Tables \ref{Table 3}, \ref{Table 4}, \ref{Table 5}, and \ref{Table 6} show the clustering results for the kmeans, dbscan, agglomerative clustering, and birch algorithms, respectively.A plot of the data clustered by kmeans, and compared with peaks in the data, is shown in Figure \ref{fig:image3} and Figure \ref{fig:image4}.

\begin{table}[h!]
\centering
 \begin{tabular}{| m{3.5cm} m{3.5cm} m{3.5cm} m{3.5cm}|}
 \hline
 Number of Clusters & Calinsky-Harabaz Score & Cluster of Earthquake Score & Earthquake Score\\ [0.5ex]
 \hline\hline
 2 & 40172 & 1 & 0.03\\
 \hline
 3 & 37282 & 1 & 0.04\\
 \hline
 4 & 43960 & 1 & 0.07\\
 \hline
 5 & 44225 & 4 & 0.08\\
 \hline
 6 & 45616 & 3 & 0.08\\
 \hline
 7 & 46338 & 3 & 0.08\\
 \hline
 8 & 46349 & 7 & 0.11\\
 \hline
 9 & 46095 & 1 & 0.11\\
 \hline
 10 & 46747 & 6 & 0.13\\
 \hline
 Average & 46747 & N/A & 0.08\\
 \hline
 \end{tabular}
 \caption{Results of kmeans clustering (Earthquake score determined by peaks)}
 \label{Table 3}
\end{table}

\begin{table}[h!]
\centering
 \begin{tabular}{|m{2cm} m{2.25cm} m{2.25cm} m{2.25cm} m{2.25cm} m{2.25cm}|}
 \hline
 Epsilon Value & Minimum Samples & Number of Clusters & Calinsky-Harabaz Score & Cluster of Earthquake Score & Earthquake Score\\ [0.5ex]
 \hline\hline
 1 & 15 & 1 & 14 & -1 & 0.0125\\
 \hline
 2 & 10 & 15  & 5 & -1 & 0.0126\\
 \hline
 2 & 15 & 5 & 6 & -1 & 0.0125\\
 \hline
 2 & 20 & 1 & 14 & -1 & 0.0125\\
 \hline
 2 & 25 & 1 & 14 & -1 & 0.0125\\
 \hline
 2 & 30 & 1 & 14 & -1 & 0.0125\\
 \hline
 3 & 15 & 6 & 123 & -1 & 0.0141\\
 \hline
 4 & 15 & 6 & 194 & -1 & 0.0159\\
 \hline
 5 & 15 & 8 & 373 & -1 & 0.0176\\
 \hline
 \end{tabular}
 \caption{Results of DBSCAN clustering (Earthquake score determined by peaks)}
 \label{Table 4}
\end{table}

\begin{table}[h!]
\centering
 \begin{tabular}{| m{3.5cm} m{3.5cm} m{3.5cm} m{3.5cm}|}
 \hline
 Number of Clusters & Calinsky-Harabaz Score & Cluster of Earthquake Score & Earthquake Score\\ [0.5ex]
 \hline\hline
 2 & 37801 & 1 & 0.04\\
 \hline
 3 & 33282 & 0 & 0.04\\
 \hline
 4 & 40010 & 1 & 0.08\\
 \hline
 5 & 40007 & 1 & 0.08\\
 \hline
 6 & 40339 & 1 & 0.08\\
 \hline
 7 & 43396 & 0 & 0.08\\
 \hline
 8 & 42621 & 3 & 0.13\\
 \hline
 9 & 42045 & 8 & 0.13\\
 \hline
 10 & 41586 & 8 & 0.13\\
 \hline
 Average & 46747 & N/A & 0.08\\
 \hline
 \end{tabular}
 \caption{Results of agglomerative clustering (Earthquake score determined by peaks)}
 \label{Table 5}
\end{table}

\begin{table}[h!]
\centering
 \begin{tabular}{| m{3.5cm} m{3.5cm} m{3.5cm} m{3.5cm}|}
 \hline
 Number of Clusters & Calinsky-Harabaz Score & Cluster of Earthquake Score & Earthquake Score\\ [0.5ex]
 \hline\hline
 2 & 37801 & 1 & 0.04\\
 \hline
 3 & 33282 & 0 & 0.04\\
 \hline
 4 & 40010 & 1 & 0.08\\
 \hline
 5 & 40007 & 1 & 0.08\\
 \hline
 6 & 40339 & 1 & 0.08\\
 \hline
 7 & 43396 & 0 & 0.08\\
 \hline
 8 & 42621 & 3 & 0.13\\
 \hline
 9 & 42045 & 8 & 0.13\\
 \hline
 10 & 41586 & 8 & 0.13\\
 \hline
 Average & 46747 & N/A & 0.08\\
 \hline
 \end{tabular}
 \caption{Results of birch clustering (Earthquake score determined by peaks)}
 \label{Table 6}
\end{table}

\par Based on the earthquake score based on peaks, the kmeans clustering algorithm does not appear to agree very well with results.  The agglomerative clustering and birch clustering algorithms give the same results, which are not very different from the kmeans clustering results. None of the clustering algorithms have clusters that clearly identify earthquakes.

\par In order to obtain clusters that better correspond to earthquakes, I added rows to the data that are shifted by time and inputted this timeshifted data into the clustering algorithms. Table \ref{Table 7} shows the average comarpison of different timeshifts for the kmeans clustering algorithm.

\begin{table}[h!]
\centering
 \begin{tabular}{| m{4cm} m{4cm} m{4cm}|}
 \hline
 Timeshift (minutes) & Calinsky-Harabaz Average & Maximum Earthquake Score Average\\ [0.5ex]
 \hline\hline
 0 & 44087 & 0.08\\
 \hline
 10 & 49251 & 0.08\\
 \hline
 30 & 44081 & 0.09\\
 \hline
 60 & 44066 & 0.08\\
 \hline
 \end{tabular}
 \caption{Results of Timeshifted Kmeans Clustering (Earthquake score determined by peaks)}
 \label{Table 7}
\end{table}

\par Shifting the data and applying the kmeans clustering algorithm does not improve the performance of the kmeans clustering algorithm.

\section{Neural Network Results}

\indent

\par The neural network was implemented using the keras package with tensforflow as a backend. Earthquake channel seismic BLRMS data from Hanford during March 2017 was timeshifted by 30 mintues and then read into the neural network. For each point in the data it was indicated whether or not it corresponds to an earthquake. The peaks in the data were used to determine which points correspond to eartquakes. Points that occur 5 minutes before and after a peak in the data. The neural network consists of a sequential model with three dense layers.

\par An accuracy of 0.998 is obtained when using part of the data set to validate the trained neural network. A plot of the neural network is shown in figures \ref{image:fig5} and  \ref{image:fig6}. The vertical lines correspond to peaks in data that are caused by earthquakes. The dotted lines indicate where the neural network detects an earthquake. The neural network does a better job of detecting earthquakes than the clustering algorithms based on comparing how well the neural network detects earthquakes to how the clustering algorithm detects earthquakes. However, more data is needed in order to test the neural network, and to improve the accuray of the neural network.

\section{Acknowledgments}

\indent

\par I would like to thank the LIGO SURF Program as well as  Dr. Rana Adhikari, Dr. TJ Massinger, and Dr. Jess McIver for help and guidance in this project.

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=1.3\textwidth,angle=90]{EQdata_Kmeans_6_.png}
\caption{Plot of data from earthquake channels clustered using kmeans with k=6 (earthquakes indicated)}
\label{fig:image1}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.5]{EQdata2_Kmeans_6_crop.png}
\caption{Figure 1 zoomed in for detail}
\label{fig:image2}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=1.3\textwidth,angle=90]{EQdata_Kmeans_6_2.png}
\caption{Plot of data from earthquake channels clustered using kmeans with k=6 (peaks indicated)}
\label{fig:image3}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.5]{EQdata_Kmeans_6_crop_2.png}
\caption{Figure 3 zoomed in for detail}
\label{fig:image4}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[width=1.3\textwidth,angle=90]{ConvNet-Comparison2.png}
\caption{Plot of data from earthquake channels classified by neural net (peaks indicated)}
\label{fig:image5}
\end{center}
\end{figure}

\begin{figure}[htbp]
\begin{center}
\includegraphics[scale = 0.35]{ConvNet-Comparison2_crop.png}
\caption{Figure 6 zoomed in for detail}
\label{fig:image6}
\end{center}
\end{figure}

\begin{thebibliography}{9}

	\bibitem{Citation1}
	  Aurelien Geron,
	  \emph{Hands-On Machine Learning with Scikit-Learn and TensorFlow}.
	  O'Reilly Media Inc., (2017).

        \bibitem{Citation2}
          \url{http://scikit-learn.org/stable/modules/clustering.html}

        \bibitem{Citation3}
          David Arthur, Sergei Vassilvitskii,
          \emph{k-means++: The Advantages of Careful Seeding}.
          Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, Society of Industrial and Applied Mathematics, (2007).

        \bibitem{Citation4}
          Martin Ester, Hans-Peter Kriegel, Jorg Sander, Xiaowei Xu,
          \emph{A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise}.
          Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, (1996).

        \bibitem{Citation5}
          Tian Zhang, Raghu Ramakrishnan, Miron Liviny
          \emph{BIRCH: An Efficient Data Clustering Method for Very Large Databases}
          Proceedings of the 1996 ACM SIGMOD international conference on Management of data, (1996).

\end{thebibliography}

\appendix
\section{Script for Evaluating Clustering Algorithms}
\begin{minted}{python}
'''
This script reads in seismic noise data from March 2017 and earthquake data.
It shifts the data by time for clustering
It determined earthquake times by looking at peaks in data
It clusters earthquake channels using kmeans and dbscan.
It compares the clusters around the earthquake times to deterime effectiveness of clustering
It plots the data as clustered by kmeans and dbscan
'''

from __future__ import division
from sklearn.cluster import KMeans
from sklearn.cluster import DBSCAN
from sklearn.cluster import AffinityPropagation
from sklearn.cluster import MeanShift,estimate_bandwidth
from sklearn.cluster import spectral_clustering
from sklearn.cluster import AgglomerativeClustering
from sklearn.cluster import Birch
from sklearn import metrics
from sklearn.preprocessing import StandardScaler
import numpy as np
from scipy.io import loadmat
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from matplotlib.pyplot import cm
import scipy.signal as sig
from astropy.time import Time
import collections

plt.rc('text',   usetex = True)
plt.rc('font',   **{'family': 'serif', 'serif': ['Computer Modern']})
plt.rc('axes',   labelsize = 20.0)
plt.rc('axes',   axisbelow = True)
plt.rc('axes.formatter', limits=[-3,4])
plt.rc('legend', fontsize  = 14.0)
plt.rc('xtick',  labelsize = 16.0)
plt.rc('ytick',  labelsize = 16.0)
plt.rc('figure', dpi = 100)

# colors for clusters
colors = np.array(['r', 'g', 'b','y','c','m','darkgreen','plum',
                       'darkblue','pink','orangered','indigo'])
cl          = 6   # number of clusters for kmeans
eps         = 2   # min distance for density for DBscan
min_samples = 15  # min samples for DBscan

#read in data
H1dat = loadmat('Data/' + 'H1_SeismicBLRMS.mat')
#edat  = np.loadtxt('Data/H1_earthquakes.txt')

# read in earthquake channels
cols   = [6,12,18,24,30,36,42,48]      # NEED comment here
vdat   = np.array(H1dat['data'][0])
vchans = np.array(H1dat['chans'][0])
for i in cols:
    add = np.array(H1dat['data'][i])
    vdat = np.vstack((vdat, add))
for i in cols:
    vchans = np.append(vchans,H1dat['chans'][i])
timetuples = vdat.T

# shift the data
vdat2   = vdat
vchans2 = vchans
num     = 10
t_shift = 10 # how many minutes to shift the data by
for i in cols:
    add = np.array(H1dat['data'][i])
    for j in range(1, t_shift+1):
        add_shift = add[j:]
        add_values = np.zeros((j,1))
        add_shift = np.append(add_shift, add_values)
        vdat2 = np.vstack((vdat2, add_shift))
        chan = 'Time_Shift_' + str(j) + '_Min_EQ_Band_' + str(i)
        vchans2 = np.append(vchans2, chan)
print(np.shape(vdat2))
vdat2 = vdat[:,:43200-t_shift]
print(np.shape(vdat2))
timetuples2 = vdat.T
timetuples3 = vdat[0:num].T

 #convert time to gps time
times       = '2017-03-01 00:00:00'
ti           = Time(times,format='iso',scale='utc')
t_start     = int(np.floor(ti.gps/60)*60)
dur_in_days = 30
dur_in_minutes = dur_in_days*24*60
dur         = dur_in_minutes*60
t_end       = t_start + dur
t    = np.arange(t_start, t_end, 60)

# create list of earthquake times from peaks
# find peaks in all three z channel directions
widths  = np.arange(5, 140)   # range of widths in minutes
min_snr = 5
noise_perc = 15
peaks1 = sig.find_peaks_cwt(vdat[2], widths,
                                min_snr = min_snr, noise_perc=noise_perc)
peaks2 = sig.find_peaks_cwt(vdat[5], widths,
                                min_snr = min_snr, noise_perc=noise_perc)
peaks3 = sig.find_peaks_cwt(vdat[8], widths,
                                min_snr = min_snr, noise_perc=noise_perc)

# takes average time for earthquake times from three channels
# that are within dtau minutes of each other
dtau = 3
peak_list = np.array([])
for i in peaks1:
    for j in peaks2:
        for k in peaks3:
            if (abs(i-j) <= dtau and abs(i-k) <= dtau):
                avg = (i+j+k)/3
                peak_list = np.append(peak_list, avg)
EQ_times = np.array([])
for i in peak_list:
    EQ_times = np.append(EQ_times, t[int(i)])

# kmeans clustering loop

Nmin = 2
num = 9
Nmax = Nmin + num
for cl in range(Nmin, Nmax):
    kmeans   = KMeans(n_clusters=cl, random_state=13).fit(timetuples2)
    kpoints  = np.array([])
    xvals    = np.arange(t_start, t_end, 60)
    for t in EQ_times: #for each EQ: collect indices within 30 min of EQ
        tmin = int(t - 10)
        tmax = int(t + 10)
        for j  in range(tmin, tmax):
            val     = abs(xvals - j)
            aval    = np.argmin(val)
            kpoints = np.append(kpoints, aval)
    kpoints   = np.unique(kpoints) # make sure there are no repeating indices
    kclusters = np.array([])
    for i in kpoints:
        #for each index find the corresponding cluster and store them in array
        kclusters = np.append(kclusters, kmeans.labels_[int(i)])
        # kmeans score determined by ratio of points in
        # cluster/points near EQ to  points in cluster/all points
    k_count      = collections.Counter(kclusters).most_common()
    ktot_count   = collections.Counter(kmeans.labels_).most_common()
    k_list_cl    = [x[0] for x in k_count] #cluster number
    k_list       = [x[1] for x in k_count] #occurences of cluster
    ktot_list_cl = [x[0] for x in ktot_count]
    ktot_list    = [x[1] for x in ktot_count]
    k_clusters   = np.array([])
    k_compare    = np.array([])
    k_list2      = np.array([])
    ktot_list2   = np.array([])
    # arrange so that k_clusters k_list2 and k_compare are in the same order
    for i in range(len(k_list_cl)):
        for j in range(len(ktot_list_cl)):
            if k_list_cl[i] == ktot_list_cl[j]:
                k_clusters = np.append(k_clusters,k_list_cl[i])
                compare    = k_list[i]/ktot_list[j]
                k_compare  = np.append(k_compare, compare)
                k_list2    = np.append(k_list2, k_list[i])
                ktot_list2 = np.append(ktot_list2, k_list[i])
    np.set_printoptions(precision=3)
    #print(k_clusters)
    #print(k_compare)
    max_val = max(k_compare)
    max_index = np.argmax(k_compare)
    max_cluster = int(k_clusters[max_index])
    k_cal_score = metrics.calinski_harabaz_score(timetuples, kmeans.labels_)
    #print('K-means ' + str(cl) + ':  C-H score = {:0.6g}'.format(k_cal_score))
    print(str(cl) + ' & {:0.6g}'.format(k_cal_score) + ' & ' + str(max_cluster) + ' & {:0.6g}'.format(max_val))
    print('\\hline')

# dbscan clustering loop

min_samples_list = [10,20,25,30]
eps_list  = [1,2,3,4,5]
#for min_samples in min_samples_list:
for eps in eps_list:
    db = DBSCAN(eps=eps,min_samples=min_samples).fit(timetuples)
    #number of clusters
    n_clusters = len(set(db.labels_)) - (1 if -1 in db.labels_ else 0)
    #add up number of clusters that appear next to each earthquake
    xvals = np.arange(t_start,t_end,60)
    dbpoints = np.array([])
    for t in EQ_times: #for each EQ: collect indices within 5 min of EQ
        tmin = int(t-5*60)
        tmax = int(t+5*60)
        for j  in range(tmin,tmax):
            val = abs(xvals-j)
            aval = np.argmin(val)
            dbpoints  = np.append(dbpoints, aval)
    dbpoints = np.unique(dbpoints)
    dbclusters = np.array([])
    for i in dbpoints: dbclusters = np.append(dbclusters,db.labels_[int(i)]) #for each index find the corresponding cluster and store them in array
    #dbscan score determined by percent of points sorted into one cluster near EQ
    db_count = collections.Counter(dbclusters).most_common()
    dbtot_count = collections.Counter(db.labels_).most_common()
    db_list_cl = [x[0] for x in db_count]
    db_list = [x[1] for x in db_count]
    dbtot_list_cl = [x[0] for x in dbtot_count]
    dbtot_list = [x[1] for x in dbtot_count]
    db_clusters = np.array([])
    db_compare = np.array([])
    db_list2 = np.array([])
    dbtot_list2 = np.array([])
    for i in range(len(db_list_cl)):
        for j in range(len(dbtot_list_cl)):
            if db_list_cl[i] == dbtot_list_cl[j]:
                db_clusters = np.append(db_clusters,db_list_cl[i])
                compare = db_list[i]/dbtot_list[j]
                db_compare = np.append(db_compare, compare)
                db_list2 = np.append(db_list2, db_list[i])
                dbtot_list2 = np.append(dbtot_list2, db_list[i])
    #print(db_clusters)
    #print(db_compare)
    max_val = max(db_compare)
    max_index = np.argmax(db_compare)
    max_cluster = int(db_clusters[max_index])
    db_cal_score = metrics.calinski_harabaz_score(timetuples, db.labels_)
    print(str(eps) + ' & ' +  str(min_samples) + ' & ' +  str(n_clusters) + ' & {:0.6g}'.format(db_cal_score) + ' & ' + str(max_cluster) + ' & {:0.6g}'.format(max_val))
    print('\\hline')

#ag clustering loop

Nmin = 2
num = 9
Nmax = Nmin + num
for cl in range(Nmin, Nmax):
    ag   = AgglomerativeClustering(n_clusters=cl).fit(timetuples)
    agpoints  = np.array([])
    xvals    = np.arange(t_start, t_end, 60)
    for t in EQ_times: #for each EQ: collect indices within 30 min of EQ
        tmin = int(t - 10)
        tmax = int(t + 10)
        for j  in range(tmin, tmax):
            val     = abs(xvals - j)
            aval    = np.argmin(val)
            agpoints = np.append(agpoints, aval)
    agpoints   = np.unique(agpoints) # make sure there are no repeating indices
    agclusters = np.array([])
    for i in agpoints:
        #for each index find the corresponding cluster and store them in array
        agclusters = np.append(agclusters, ag.labels_[int(i)])
    ag_count      = collections.Counter(agclusters).most_common()
    agtot_count   = collections.Counter(ag.labels_).most_common()
    ag_list_cl    = [x[0] for x in ag_count] #cluster number
    ag_list       = [x[1] for x in ag_count] #occurences of cluster
    agtot_list_cl = [x[0] for x in agtot_count]
    agtot_list    = [x[1] for x in agtot_count]
    ag_clusters   = np.array([])
    ag_compare    = np.array([])
    ag_list2      = np.array([])
    agtot_list2   = np.array([])
    # arrange so that k_clusters k_list2 and k_compare are in the same order
    for i in range(len(ag_list_cl)):
        for j in range(len(agtot_list_cl)):
            if ag_list_cl[i] == agtot_list_cl[j]:
                ag_clusters = np.append(ag_clusters,ag_list_cl[i])
                compare    = ag_list[i]/agtot_list[j]
                ag_compare  = np.append(ag_compare, compare)
                ag_list2    = np.append(ag_list2, ag_list[i])
                agtot_list2 = np.append(agtot_list2, ag_list[i])
    np.set_printoptions(precision=3)
    max_val = max(ag_compare)
    max_index = np.argmax(ag_compare)
    max_cluster = int(ag_clusters[max_index])
    ag_cal_score = metrics.calinski_harabaz_score(timetuples, ag.labels_)
    print(str(cl) + ' & {:0.6g}'.format(ag_cal_score) + ' & ' + str(max_cluster) + ' & {:0.6g}'.format(max_val))
    print('\\hline')

#birch clustering loop

Nmin = 2
num = 9
Nmax = Nmin + num
for cl in range(Nmin, Nmax):
    birch   = Birch(n_clusters=cl).fit(timetuples)
    bpoints  = np.array([])
    xvals    = np.arange(t_start, t_end, 60)
    for t in EQ_times: #for each EQ: collect indices within 30 min of EQ
        tmin = int(t - 10)
        tmax = int(t + 10)
        for j  in range(tmin, tmax):
            val     = abs(xvals - j)
            aval    = np.argmin(val)
            bpoints = np.append(bpoints, aval)
    bpoints   = np.unique(bpoints) # make sure there are no repeating indices
    bclusters = np.array([])
    for i in bpoints:
        #for each index find the corresponding cluster and store them in array
        bclusters = np.append(bclusters, birch.labels_[int(i)])
    b_count      = collections.Counter(bclusters).most_common()
    btot_count   = collections.Counter(birch.labels_).most_common()
    b_list_cl    = [x[0] for x in b_count] #cluster number
    b_list       = [x[1] for x in b_count] #occurences of cluster
    btot_list_cl = [x[0] for x in btot_count]
    btot_list    = [x[1] for x in btot_count]
    b_clusters   = np.array([])
    b_compare    = np.array([])
    b_list2      = np.array([])
    btot_list2   = np.array([])
    # arrange so that b_clusters b_list2 and k_compare are in the same order
    for i in range(len(b_list_cl)):
        for j in range(len(btot_list_cl)):
            if b_list_cl[i] == btot_list_cl[j]:
                b_clusters = np.append(b_clusters, b_list_cl[i])
                compare    = b_list[i]/btot_list[j]
                b_compare  = np.append(b_compare, compare)
                b_list2    = np.append(b_list2, b_list[i])
                btot_list2 = np.append(btot_list2, b_list[i])
    np.set_printoptions(precision=3)
    max_val = max(b_compare)
    max_index = np.argmax(b_compare)
    max_cluster = int(b_clusters[max_index])
    b_cal_score = metrics.calinski_harabaz_score(timetuples, birch.labels_)
    print(str(cl) + ' & {:0.6g}'.format(b_cal_score) + ' & ' + str(max_cluster) + ' & {:0.6g}'.format(max_val))
    print('\\hline')
\end{minted}

\section{Script for Neural Network}
\begin{minted}{python}
'''
Reads in data from mat file
Creates a neural network using keras
Plots data with prediction labels in a graph
'''

from keras.models import Sequential
from keras.layers import Dense, Dropout
from keras import optimizers
import numpy as np
import scipy.io as sio
from astropy.time import Time
import collections
import matplotlib
matplotlib.use('Agg')
import matplotlib.pyplot as plt
from matplotlib.pyplot import cm

np.random.seed(7)

# read in data
EQ_data  = sio.loadmat('Data/EQ_info.mat')
vdat     = EQ_data['vdat']
vchans   = EQ_data['vchans']
EQ_times = EQ_data['EQ_times']
EQ_times = EQ_times.reshape(49,1)
X        = EQ_data['X']
points, size  = np.shape(X)
Y        = EQ_data['EQ_labels']
Y        = Y.reshape(43200,)
t        = EQ_data['t']

# print info about data
print('size is ' + str(size))
print('Shape of X is ' + str(np.shape(X)))
print('Shape of Y is ' + str(np.shape(Y)))
print(np.shape(EQ_times))

# neural network
optimizer = optimizers.Adam(lr = 1e-5)
model = Sequential()
model.add(Dense(size, input_shape = (size,), activation = 'elu'))
model.add(Dropout(.1))
model.add(Dense(9, activation = 'elu'))
model.add(Dropout(.1))
model.add(Dense(1, activation = 'softmax'))

# model.output_shape
model.compile(loss = 'binary_crossentropy',
                  optimizer = optimizer,
                  metrics = ['accuracy'])
model.fit(X, Y,
              epochs = 10,
              batch_size = 256,
              validation_split=0.1,
              verbose = 1)

#score = model.evaluate(X,Y)
#print(score)

model.summary()

# prediction values
Y_pred = model.predict(X)
Y_pred2 = Y_pred.T
Y_pred3 = np.array([])
for i in Y_pred2:
    Y_pred3 = np.append(Y_pred3, i)
Y_pred3 = Y_pred3.astype(int)

# Plot of data points
colors = np.array(['r', 'b', 'm', 'g'])
labels = Y.T
labels = labels.astype(int)
num  = size

fig,axes  = plt.subplots(len(vdat[0:num]), figsize=(40,4*len(vdat[0:num])))
for ax, data, chan in zip(axes, vdat[0:num], vchans):
    ax.scatter(t, data,c=colors[Y_pred3],edgecolor='',
               s=3, label=r'$\mathrm{%s}$' % chan.replace('_','\_'))
    ax.set_yscale('log')
    ax.set_ylim(np.median(data)*0.1, max(data)*1.1)
    ax.set_xlabel('GPS Time')
    ax.grid(True, which='both')
    ax.legend()
    for e in range(len(EQ_times)):
        ax.axvline(x = EQ_times[e], color = 'r')

fig.tight_layout()

print("Saving plot...")
fig.savefig('Figures/NeuralNetworkComparison3.pdf')
\end{minted}

\end{document}


\section{Line Follower}
The robot utilize a line follower controller in order to navigate the factory floor. This in combination with \textbf{QR} codes and a path-planning algorithm will give a huge variety in places that the robot can be instructed to go, as long as it's on the grid and denoted with a \textbf{QR} code.

\subsection{Choice of Controller} \label{sec:choice_of_cont}

In order to move the robot to a specified point the use of a line follower controller was decided on. The robot will read in values from a camera \footnote{Refrence to the machine vision part} that publishes the angle that a \textbf{PD} controller uses as an input for calculating the signal it sends to the motors.\\
\indent The choice of a \textbf{PD} controller was made for the improved smoothness over a \textbf{P} controller and the fact that the integrating part in a \textbf{PID} controller would have undesired effects, as illustrated in Fig.\ref{fig:PID_bad}.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{img/Pid_bad.eps}
    \caption{The robot will keep acting on the integrated error even when reaching the line, this might eventually cause overshoots that makes the robot lose the line. (Green: correct path, red: actual path)  }
    \label{fig:PID_bad}
\end{figure}