An attempt to recreate their results is described in Section \ref{sec:armamodel}.

\subsubsection{Computing ARMA fits}\label{arimamod}
Following \cite{Nesti2018emergentfailures}, the ARMA models were fitted to the first month of solar and wind generation series using the \texttt{arima} method in \texttt{R}. This produced a result for all wind series.

Unfortunately, for the solar series, the \texttt{arima} method failed to fit an ARMA model for 104 of the 489 buses, reflecting the hypothesis that these series are poorly modelled by an ARMA series. To recreate the results of \cite{Nesti2018emergentfailures}, it is crucial that a model can be fitted for \emph{every} bus. Therefore, a modified method is required.

By the default, the \texttt{arima} method operates in two steps. First, a so-called Conditional-Sum-of-Squares (CSS) method is used as initial guess for the ARMA coefficients. Then, the coefficient \emph{likelihood} function is maximised with the Nelder-Mead optimisation method, using the CSS result as initial value.

In our case, the CSS method often finds initial values that are \emph{non-stationary}.\todo{define?} A possible reason is that solar generation is exactly zero during nighttime, which lasts for up to 16 consecutive hours.\footnote{In reality, solar generation is never \emph{exactly} zero, even after sunset, because solar panels also receive ambient light. This is not included in the (artificial) dataset.
%Ambient light includes sunlight scattered by the atmosphere (\eg 3 hours of twilight in January), artificial light
%TODO
}
To resolve this issue, we use an optimisation method that works well without specifying an initial value: it should find the \emph{global} maximum independently. The Simulated Annealing (SANN) algorithm included in \texttt{R} is such a method, and was chosen for this analysis.

With SANN as optimisation method, \texttt{arima} was able to fit solar models for 484 out of 489 buses. For the remaining 5 buses, a second technique is applied: to accommodate for the long periods of zero generation, a small amount of uniform noise is superimposed on the generation series. Initially, the noise magnitude is set to 1\% of the solar capacity of the bus. This percentage is increased iteratively, until the modified method is able to fit a solar model. In fact, 1\% noise proved sufficient for the remaining 5 buses in January, and no more than 2\% noise was needed for any bus in the remaining months.

The modified method described in this section was used to fit solar and wind models to each bus of the network, for each month of 2011. Table \ref{tab:arimafitstats} lists the number of buses that can be analysed using the default and modified methods, showing the poor performance of the default for solar series, especially during the months of summer.
\tocite{Belisle 1992, zie \texttt{optim} docs}

\citet{Nesti2018emergentfailures} do not mention such problems (nor techniques to solve them). Because the same dataset, processing, programming language and method was used in this analysis, it is likely that they have also encountered and solved these problems, possibly in the same manner.
\towrite{discuss other possible techniques: warm start, intercept}

\subsubsection{Runtime}
The default implementation has an estimated (single-core) runtime of 101 hours (4.2 days) when all 12 months are analysed, although only 78\% of fits are successful using this method.

The modified version is able to fit all generation series successfully. The use of this more computationally expensive method increases the runtime to an estimated 224 hours (9.4 days). Because of the long runtime, fits were computed in parallel on a 40-core machine\footnote{Specifications: two \textit{Intel Xeon E5 2630L v4} 10-core processors @2GHz with hyperthreading; 64GB RAM. CPU time was used as estimate for single-core runtime.}, provided by the IT department of the Radboud University.\footnote{Thank you!} This reduced the total runtime down to 7 hours. The results are publicly available at the GitHub repository.

\begin{table}[]
    \centering
    \begin{tabular}{c|ccrr|ccrr}
\toprule
\multicolumn{1}{c}{} & \multicolumn{4}{c}{\textsc{Solar}} & \multicolumn{4}{c}{\textsc{Wind}} \\
\midrule
\multicolumn{1}{c}{} &
\multicolumn{1}{c}{\textit{default}} & \multicolumn{1}{c}{\textit{modified}} &
\multicolumn{1}{c}{1\%} & \multicolumn{1}{c}{2\%} &
\multicolumn{1}{c}{\textit{default}} & \multicolumn{1}{c}{\textit{modified}} &
\multicolumn{1}{c}{1\%} & \multicolumn{1}{c}{2\%} \\
\midrule
Jan & \texttt{385} & \texttt{\textbf{484}} & \texttt{\textbf{5}} & \texttt{\textbf{0}} & \texttt{\textbf{488}} & \texttt{488} & \texttt{\textbf{0}} & \texttt{\textbf{0}} \\
Feb & \texttt{364} & \texttt{\textbf{485}} & \texttt{\textbf{4}} & \texttt{\textbf{0}} & \texttt{\textbf{487}} & \texttt{487} & \texttt{\textbf{0}} & \texttt{\textbf{1}} \\
Mar & \texttt{334} & \texttt{\textbf{475}} & \texttt{\textbf{14}} & \texttt{\textbf{0}} & \texttt{\textbf{487}} & \texttt{487} & \texttt{\textbf{1}} & \texttt{\textbf{0}} \\
Apr & \texttt{211} & \texttt{\textbf{477}} & \texttt{\textbf{11}} & \texttt{\textbf{1}} & \texttt{\textbf{488}} & \texttt{488} & \texttt{\textbf{0}} & \texttt{\textbf{0}} \\
May & \texttt{189} & \texttt{\textbf{473}} & \texttt{\textbf{16}} & \texttt{\textbf{0}} & \texttt{\textbf{467}} & \texttt{467} & \texttt{\textbf{21}} & \texttt{\textbf{0}} \\
Jun & \texttt{255} & \texttt{\textbf{481}} & \texttt{\textbf{8}} & \texttt{\textbf{0}} & \texttt{\textbf{480}} & \texttt{480} & \texttt{\textbf{8}} & \texttt{\textbf{0}} \\
Jul & \texttt{292} & \texttt{\textbf{485}} & \texttt{\textbf{4}} & \texttt{\textbf{0}} & \texttt{\textbf{488}} & \texttt{488} & \texttt{\textbf{0}} & \texttt{\textbf{0}} \\
Aug & \texttt{179} & \texttt{\textbf{478}} & \texttt{\textbf{10}} & \texttt{\textbf{1}} & \texttt{\textbf{488}} & \texttt{488} & \texttt{\textbf{0}} & \texttt{\textbf{0}} \\
Sep & \texttt{259} & \texttt{\textbf{472}} & \texttt{\textbf{16}} & \texttt{\textbf{1}} & \texttt{\textbf{487}} & \texttt{487} & \texttt{\textbf{1}} & \texttt{\textbf{0}} \\
Oct & \texttt{263} & \texttt{\textbf{480}} & \texttt{\textbf{9}} & \texttt{\textbf{0}} & \texttt{\textbf{488}} & \texttt{488} & \texttt{\textbf{0}} & \texttt{\textbf{0}} \\
Nov & \texttt{290} & \texttt{\textbf{472}} & \texttt{\textbf{17}} & \texttt{\textbf{0}} & \texttt{\textbf{486}} & \texttt{486} & \texttt{\textbf{2}} & \texttt{\textbf{0}} \\
Dec & \texttt{317} & \texttt{\textbf{482}} & \texttt{\textbf{7}} & \texttt{\textbf{0}} & \texttt{\textbf{486}} & \texttt{486} & \texttt{\textbf{2}} & \texttt{\textbf{0}} \\
\bottomrule
    \end{tabular}%TODO: modified wind bold?
    \caption{Number of buses (out of 489 solar, 488 wind) for which the ARMA model could be fitted using either the \emph{default} implementation of \texttt{arima} in \texttt{R}, or the \emph{modified} version (described in \ref{arimamod}).
    \newline
    When even the modified method yielded no results, uniform noise was added to the series before fitting. Noise magnitude was set to 1\% of generation capacity, or 2\% when needed.
    \newline
    Figures in \textbf{bold} indicate which results are used.
    %All nodes now have usable results.
    }
    \label{tab:arimafitstats}
\end{table}


\declareIM{ppg}{pre-pre-G}{2021-04-30}{AOS using ComCam; Zernikes}{AOS Zernikes}
\completeIM{\thisIM}{2021-05-27 \JIRA{DM}{28667}{}}

Executive Summary:

Demonstrate the processing of wave front images for ComCam from raw to Zernikes.

\textbf{Does not map to P6}

\subsection{Goals of IM}

Demonstrate ability to:

\begin{itemize}
\item Process Defocussed Images and Generate Zernike coefficients
\end{itemize}

\subsection{Prerequisites}
\begin{itemize}
\item PhoSim images with donuts from ComCam ingested into a butler repo.
\item Working version of wavefront estimation pipeline (wep).
\end{itemize}

\subsection{Procedure}
\begin{itemize}
\item Generate and ingest 1 simulated image with each desired piston: ($[0, -1.5, 0, 1.5, 0]$ mm)
  and appropriate
  metadata in each image:
  \begin{itemize}
  \item Boresight and rotator angle
  \item Piston from camera hexapod
  \end{itemize}
\item Run a Gen3 pipeline from the command line that:
  \begin{itemize}
  \item gets data from the butler
  \item runs the ISR
  \item finds and process isolated donuts
  \item estimates the wavefront in terms of annular Zernikes from each pair of donuts
  \item average the resulting Zernikes over the stars in the field.
  \end{itemize}
  \item uses the butler to put the description of the wavefront to disk
\end{itemize}

\subsection{Acceptance Criteria}
\begin{itemize}
\item A member of SITCom must be able to carry out these operations at NCSA.
  This need not be done using RSP; a login shell on \eg \texttt{lsst-devl3} is acceptable, and
  the SITCom member may be required to install and build packages from \texttt{git}.
\item Confirm that the results are as expected.  This should be carried out using a notebook on the RSP,
  and the tester may be required to install the notebook from \texttt{git}.
\end{itemize}

\subsection{Status}

\begin{description}
   \item[2021-07-30]
        \begin{itemize}
                \item Images taken using nublado
                \item Automatic ingestion into gen3 butler in Chile
                \item Transfer to NCSA and ingestion in gen3 butler visible from \gls{RSP}
                \item Calibs (bias, dark) generated in Chile
        \end{itemize}
   \end{description}


\chapter{Correctness of a Transformation}
\label{ap:proof}

\input{proof/start}

\input{proof/proof}


\documentclass[11pt]{article}

\usepackage{amsmath,amsfonts,amssymb}   % basic math
\usepackage{graphicx}  % images
\usepackage{float}     % floating image support
\usepackage{listings}  % code input
\usepackage{color}     % colors
\usepackage[super,comma]{natbib}    % citing
\usepackage[T1]{fontenc}      % fonts
\usepackage[utf8]{inputenc}   % fonts
\usepackage{titling}   % title control
\usepackage{microtype}
\usepackage{bm}

\usepackage[utf8]{inputenc}
\usepackage[margin=.9in]{geometry}
\usepackage[labelfont=bf]{caption}
\usepackage{subcaption}
\usepackage{indentfirst}
\usepackage[affil-it]{authblk}
\usepackage{enumitem}
\usepackage{scrextend}

\usepackage{varioref}
\usepackage{hyperref}
\usepackage{cleveref}
\hypersetup{
    colorlinks,
    citecolor=blue,
    filecolor=black,
    linkcolor=blue,
    urlcolor=blue
}

\bibliographystyle{unsrtnat}
\renewcommand{\bibname}{References}

% standard code colors
\definecolor{dkgreen}{rgb}{0,0.6,0}
\definecolor{gray}{rgb}{0.5,0.5,0.5}
\definecolor{mauve}{rgb}{0.58,0,0.82}

% setup for python language
\lstset{frame=tb,
  language=Python,
  aboveskip=3mm,
  belowskip=3mm,
  showstringspaces=false,
  columns=flexible,
  basicstyle={\small\ttfamily},
  numbers=none,
  numberstyle=\tiny\color{gray},
  keywordstyle=\color{blue},
  commentstyle=\color{gray},
  stringstyle=\color{dkgreen},
  breaklines=true,
  breakatwhitespace=true,
  tabsize=3
}

\title{StokeD: Simulating Stokesian Dynamics}
\date{}
\author{}

\begin{document}
\maketitle
%\tableofcontents

\section{Overview}
The standard Langevin equations for translation and rotational motion for isotropic drag forces:
\begin{subequations}
\begin{align}
    m_i \frac{d\bm{v}_i(t)}{dt} &= \bm{F}_i(t, \bm{r}) - \lambda_i^T \bm{v}_i(t) + \sqrt{2kT\lambda_i^T} \; \bm{\eta}^T(t) \\
    \frac{d[I_i \bm{\omega}_i(t)]}{dt} &= \bm{\tau}_i(t, \bm{r}) - \lambda_i^R \bm{\omega}_i(t) + \sqrt{2kT\lambda_i^R} \; \bm{\eta}^R(t)
\end{align}
\end{subequations}

Generalized to anisotropic drag forces:
\begin{subequations}
\begin{align}
    m_i \frac{d\bm{v}_i(t)}{dt} &= \bm{F}_i(t, \bm{r}) - \Lambda_i^T \bm{v}_i(t) + \zeta^T \sqrt{2kT} \; \bm{\eta}^T(t) \\
\frac{d[I_i \bm{\omega}_i(t)]}{dt} &= \bm{\tau}_i(t, \bm{r}) - \overleftrightarrow{\bm{\Lambda}_i^R} \bm{\omega}_i(t) + \sqrt{2kT}\; \overleftrightarrow{\bm{\zeta}^R}  \bm{\eta}^R(t)
\end{align}
\end{subequations}
where $\overleftrightarrow{\bm{\zeta}}$ satisfies the property $\overleftrightarrow{\bm{\zeta}} \overleftrightarrow{\bm{\zeta}}^T = 2kT\overleftrightarrow{\bm{\Lambda}}$

For the fully coupled hydrodynamic case, denote the grand momentum
\begin{equation}
    \bm{P} = (P_{1x},\; P_{1y},\; P_{1z}, \ldots, P_{Nx},\; P_{Ny},\; P_{Nz},\;
    L_{1x},\; L_{1y},\; L_{1z}, \ldots, L_{Nx},\; L_{Ny},\; L_{Nz})
\end{equation}
and grand velocity
\begin{equation}
    \bm{V} = (v_{1x},\; v_{1y},\; v_{1z}, \ldots, v_{Nx},\; v_{Ny},\; v_{Nz},\;
    \omega_{1x},\; \omega_{1y},\; \omega_{1z}, \ldots, \omega_{Nx},\; \omega_{Ny},\; \omega_{Nz})
\end{equation}

\end{document}


\documentclass[a4paper,11pt]{article}
\usepackage[utf8]{inputenc}
\usepackage[a4paper,margin=1in]{geometry}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{verbatim}
\usepackage{indentfirst}
\usepackage{url}
\usepackage{enumerate}
\usepackage{enumitem}
\usepackage{tikz}
\usepackage{pict2e}
\usepackage{longtable}
\usepackage{algorithm2e}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\begin{document}
	\title{Compiler Project Part 2}
	\author{Jisen Ren, Haoran Wang, Xiao Han}
	\maketitle
	\section{Preliminaries}
	\subsection{Reusable Code}
	Since this project is inherited from the first part, the following parts of the previous code could be reused to save the workload:
	\begin{enumerate}
		\item The modules for lexical and syntax analysis can be reused with little change;
		\item The modules for C++ code generation based on grammar tree can be reused, but for reasons explained later, some modifications are needed to adapt to the second part.
	\end{enumerate}
	\par
	And the module exclusive to project 2 is, to transform the grammar tree from its original to the differentiation form, which will be inserted into the code generation tool.
	\subsection{Cases Observation}
	By observing the test cases and evaluation tool, we discovered the following features which the cases share in common:
	\begin{enumerate}
		\item Unlike project 1, in project 2 the cases all have data type ``float"; there is always exactly one single output and the output tensor is never among the input list; scalar multipliers like $\alpha$, $\beta$ never occur; each kernel has exactly one statement (however, since \texttt{grad\_to} might contain multiple elements, the output is actually a list of statements).
		\item The kernel of each cases is a linear combination of ``indivisible sub-expressions" which are either single tensors or tensors connected by multiplication. Case 10 is special in that it contains one pair of parentheses; the rest of the cases are all simple additions of ``indivisible sub-expressions". Therefore, the main issue is how to deal with these sub-expressions.
		\item In terms of index, cases 6, 8 and 10 contain expressions other than single variables, but the loop ranges can always be judged from the tensor dimensions associated with single variables. In the following context, we aim to find a universal solution for dealing with index expressions, though.
	\end{enumerate}
	\section{Automatic Differentiation}
	Here we propose our design for theoretical model of automatic differentiation. As mentioned above, this section will cover mostly the differentiation scheme for the ``sum of multiplication chains of tensors" expressions.
	\subsection{Mathematical Foundation}
	The differentiation of functions and matrices satisfy the following properties:
	\begin{enumerate}
		\item The differentiation of constant number is zero, $(c)'=0$;
		\item The differentiation of additions of sub-expressions is the sum of their separated differentiation, $(f+g)'=f'+g'$;
		\item The differentiation of multiplications of sub-expression is the sum of all such parts: replace one sub-expression by its differentiation and let the rest remain unchanged, $(u\cdot v\cdot w)' = u'\cdot v\cdot w+u\cdot v'\cdot w+u\cdot v\cdot w'$;
		\item Since matrix multiplication is a linear operator, its differentiation is simply calculated by replacing the matrix with its changing rate, $d(X\cdot Y) = dX\cdot Y + X\cdot dY$.
	\end{enumerate}
	\subsection{Differentiation of Multiply Chains}
	Consider the ``multiplication chain" $O = I_1 I_2 \dots I_n$ or
	\begin{equation*}
		O = \prod_{k=1}^{n}{I_k}
	\end{equation*}
	\par
	And we want to compute the gradient to $G$, or $dG$, then:
	\begin{equation*}
		dG = \sum_{k:I_k=G}{\left[\prod_{j=1}^{k-1}{I_j}\cdot dO \cdot \prod_{j=k+1}^{n}{I_j}\right]}
	\end{equation*}
	\par
	Among each component of summation, the tensor whose name is identical to $G$ is simply replaced by $dO$.
	\subsection{Issues of Index}
	While replacing $G$ with $dO$, its index must be considered since not all indexes are expressed with single variables. Supposing $G$ is a $d$ dimensional vector and appears in the expression as $G[f_0(\mathcal{I}), f_1(\mathcal{I}), \dots, f_{d-1}(\mathcal{I})]$ where $\mathcal{I}$ is the set of index variables (in a text case it might appear more than once), then:
	\begin{equation*}
		dG[i_0,i_1,\dots i_{d-1}] = \sum_{k:I_k=G}{\left[\prod_{j=1}^{k-1}{I_j}\cdot \text{select}\left(\bigwedge_{j=0}^{d-1}{i_j=f_j(\mathcal{I})},dO,0\right) \cdot \prod_{j=k+1}^{n}{I_j}\right]}
	\end{equation*}
	\par
	The issue of index occur when $f_j(\mathcal{I}) \neq i_j$, namely, the index expression is more than a single variable.
	\section{Experimental Procedure}
	\subsection{Grammar Tree Traversal}
	In the grammar described in project 1, $R$ is a summation chain of $E$, and each $E$ is a multiplication chain of $T$ (except for case 10, where such property is only satisfied inside the parentheses). \par
	Combined with the knowledge in previous sections we implement the following traversal scheme: first find out whether a pair of parentheses exist; if it does we set $p$ to the immediate node inside the parentheses; otherwise set $p$ to the root of the right-hand expression. For each child $c$ in $p$, if $c$ is $R$ then call $\text{Transform}(c)$ recursively; if $c$ is $E$ then replace $c$ with $\text{Differentiate}(c)$. \par
	The function Differentiate, then, returns a grammar-tree representation of the sub-expression (whose root is passed as the parameter) having the form of multiplication chain.
	\subsection{Assigning Additional Variables}
	Supposing we aim to compute the gradient to $G$, for each dimension of $G$, if an index expression occurs at least once on this dimension, then: replace this dimension of left side $dG$ with an additional loop variable, and attach an equality constraint to the corresponding right-hand sub-expression. \par
	This method is somewhat inefficient, but sufficient to solve the cases in the project, in it satisfies the requirement that left-side tensors do not contain index expressions, and the method does not make use of the cases' names. Moreover, within the scale of the given cases, the loops can be executed without taking much time. \par
	\subsection{List of Function Parameters}
	After obtaining the processed grammar tree, we determine the function parameters in the following order:
	\begin{enumerate}
		\item The tensor name(s) among the list \texttt{ins} which appear in the right-hand expressions at least once;
		\item The character `d' plus the tensor name in \texttt{outs};
		\item The character `d' plus the tensor name(s) in \texttt{grad\_to}.
	\end{enumerate}
	\subsection{Ideas for Refinement}
	The pursuit for efficiency led us to consider ideas for refinement. It would be better if extraneous loops can be eliminated by computing the inverse operator. For example, the inverse of $p+r$ is $p-r$, when $r$ does not appear in the tensor to be ``graded" over. However, the inversion of integer division and mod operator is confusing, since each of these operators itself is not even reversible. If we regard the combination $(//,\%)$ as an operator which maps an integer to a pair of integers, then this operator is indeed reversible. \par
	However, how should we know that $//$ and $\%$ always appear in pairs without examining the ground truth? If they do appear in pairs, how should we find out that truth efficiently? What about more complicated cases, should they also be considered? These issues can be too complicated to solve. The method based on extra loop variables and equality constraints, however, solves these issues universally. That is why we apply this method in the submitted solution.
	\section{Concrete Example}
	In this section we demonstrate our differentiation scheme using a concrete example. We choose the most complicated example, a combination of matrix multiplication and convolution. \par
	In case 6, the input string \texttt{kernel} is: \texttt{A<2, 8, 5, 5>[n, k, p, q] = B<2, 16, 7, 7>[n, c, p + r, q + s] * C<8, 16, 3, 3>[k, c, r, s];} and we need to compute its gradient to \texttt{B}. The right side itself is an indivisible sub-expression without parentheses, so we begin transformation from the root; from the differentiation rule, if $A = B*C$, then $dB = dA* C$. \par
	In terms of index, since the third and fourth dimensions contain expressions, we assign an extra loop variable to each of these dimensions and attach an equality constraint to the right-side component. The answer is thus: \texttt{dB<2, 16, 7, 7>[n, c, i2, i3] = (i2 == p + r \&\& i3 == q + s ? dA[2, 8, 5, 5]<n, k, p, q> : 0) * C<8, 16, 3, 3>[k, c, r, s];}. \par
	Among all cases, this is the one which needs to execute the loop for most iterations. Even so, a total of 2822400 iterations finish in a matter of milliseconds while running \texttt{test2}.
	\section{Knowledge Applied}
	Here we give a brief summary of the compiler knowledge applied in the project:
	\begin{enumerate}
		\item Lexical Analysis. After reading the string from JSON files, the lexical analysis is first performed to obtain a sequence of tokens;
		\item Syntax Analysis. We apply LR automaton based syntax analysis scheme to the (actually ambiguous) grammar, and resolve conflicts properly to construct the grammar tree.
		\item Syntax Directed Definition and Translation. For each tensor reference as well as the whole expression, we associate semantic actions to build the \texttt{for} loops, the \texttt{if} conditions, as well as the C++ representation of tensor references together with the conditions related to its indexes.
		\item Knowledge Exclusive to Project 2. The ``Differentiation" action is also syntax directed, which is the foundation of grammar tree transformation.
	\end{enumerate}
	\appendix
	\section{Task Distribution}
	All group members have played a significant role in the progress of projects, hence it is suggested that all members be given score equally. \par
	\subsection{Jisen Ren}
	Performed LR based syntax analysis in project 1 and the majority of effort of syntax transformation in project 2; responsible for assembling the report documents.
	\subsection{Zhiyi Cheng, who withdrew the course halfway}
	Read input from JSON files using third-party library; performed lexical analysis to obtain the token sequence.
	\subsection{Haoran Wang}
	Completed the code generation part, in which the established syntax tree is translated into the target C++ code; provided useful and convenient auxiliary functions.
	\subsection{Xiao Han}
	Made an attempt to refine the second project aiming at eliminating redundant loops, by exploiting the principles of ``inverse operators".
\end{document}


% !TEX root = ../thesis.tex
%
\chapter{Introduction}
\label{sec:intro}
Today more and more work processes can be supported, or replaced through intelligent machines. Computer Vision proves itself capable of understanding images and video on a near-human level or even outperforming normal human capabilities in some visual tasks. One reason intelligent machines are absolutely needed today and in the future are the enormous masses of data the \textit{digital natives}, companies or scientists generate and store. Independent of the performance difference between computer and human, it its often impossible to work through catalogs or databases of millions or more images, music recordings, scientific data records et cetera. To automate such tasks does not only relieve from monotonous work but also enables humans to achieve more in less time.\\
At the very latest with the development of analog and digital mass storage systems scientists began to base their research on findings from mass data. No different scientists of the humanities move their work into a digital form. Being able to quickly look up any painting as a digital image or shifting through unique, in some archive locked, manuscripts from the middle ages makes drawing connections and research considerable easier. As we can assume these databases to grow or already have grown over any humans capabilities, digital humanities are a prime example for the application of visual learning and automation.

\section{Problem Statement}
\label{sec:intro:motivation}
For this thesis we approach one of possible applications of Computer Vision in the digital humanities, in particular art history. We assume a historian doing work on a huge set of digitized images $\mathbb{I}$. Those could be scans of medieval writings, architecture sketches or photos of historic paintings. The images are assumed to be not at all annotated meaning we do not have any metadata containing e.g. the creator.

Now the historian wants to compare all images $I = \{i_1, i_2,\dotsc,i_n\} \subset \mathbb{I}$ conjunct in-that they all share some visual pattern $p$, may it be they depict the same person or show all a similar scenery. We want to provide the user of this database a method to automatically generate the set $I$. This should be done giving a image $\tau$ which the historian states for, that it contains $p$. Therefore we need a function $C(i)$ which can decide for a image $i\in\mathbb{I}$ whether it possesses $p$ and therefore is part of the seeked set $I$. The only data we can use to adjust the operator $C$ is the sample image $\tau$.\\Proposing one approach for this $C_\tau$ and a generator function $G(\tau) = C_\tau$ is the problem of this thesis.

\section{Thesis Structure}
\label{sec:intro:structure}
The thesis is structured as follows. Firstly in \textbf{\treft{sec:concepts}} we give a quick overview over the topics and techniques used in this work including a short introduction to the inner workings of \glspl{cnn}. In \textbf{\treft{sec:related}} we list related works that tackle different subtasks of our method. Next in \textbf{\treft{sec:pipeline}} we explain in depth our pipeline and report extensive test results in \textbf{\treft{sec:results}}. Finally in \textbf{\treft{sec:application}} we show how this method can be practically applied in the real world and give a short conclusion and outlook in \textbf{\treft{sec:conclusion}}.


\documentclass[]{article}

\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhead{}
\fancyfoot{}
\fancyhead[C]{Data Compression techniques $\bullet$ Information Theory $\bullet$ Unishox - Guaranteed Compression of Short Unicode Strings}
\fancyfoot[L]{ Licensed under CC 4.0 Int'l Attribution License \copyright  2019 Siara Logics (cc)}
\fancyfoot[R]{\thepage}

%opening
\title{Unishox - Guaranteed Compression of Short Unicode Strings using Entropy, Dictionary and Delta encoding techniques}
\author{Arundale Ramanathan}

\begin{document}

\maketitle

\begin{abstract}
A new hybrid encoding method is proposed, with which short unicode strings could be compressed using context aware pre-mapped codes and delta coding resulting in surprisingly good ratios.

Also shown is how this technique can guarantee compression for any language sentence of minimum 3 words.
\end{abstract}

\section{Summary}

\char"00E7 Compression of Short Unicode Strings of arbitrary lengths have not been addressed sufficiently by lossless entropy encoding methods so far.  Although it appears inconsequential, space occupied by such strings become significant in memory constrained environments such as Arduino Uno and when attempting storage of such independent strings in a database. While block compression is available for databases, retrieval efficiency could be improved if the strings are individually compressed.

\section{Basic Definitions}

In information theory, \emph{entropy encoding} is a lossless data compression scheme that is independent of the specific characteristics of the medium \cite{1}.

One of the main types of entropy coding is about creating and assigning a unique \emph{prefix-free code} to each unique symbol that occurs in the input. These entropy encoders then compress data by replacing each fixed-length input symbol with the corresponding variable-length prefix-free output code word.

According to Shannon's source coding theorem, the optimal code length for a symbol is $-log_bP$, where b is the number of symbols used to make output codes and P is the probability of the input symbol \cite{2}. Therefore, the most common symbols use the shortest codes.

The most popular and most used method (even today) for forming optimal prefix-free discrete codes is Huffman coding \cite{3}.

A \emph{Dictionary coder}, also sometimes known as a substitution coder, is a class of lossless data compression algorithms which operate by searching for matches between the text to be compressed and a set of strings contained in a data structure (called the 'dictionary') maintained by the encoder. When the encoder finds such a match, it substitutes a reference to the string's position in the data structure.

The LZ77 family of encoders use the dictionary encoding technique for compressing data. \cite{4}

\emph{Delta coding} is a technique applied where encoding the difference between the previously encoded symbol or set of symbols is smaller compared to encoding the symbol or the set again.  The duffernce is determined by using the set minus operator or subtraction of values. \cite{5}

In contrast to these encoding methods, there are various other approaches to lossless coding including Run Length Encoding (RLE) and Burrows-Wheeler coding \cite{6}.

\section{Existing techniques - Smaz and shoco}

While technologies such as GZip, Deflate, Zip, LZMA are available for file compression, they do not provide optimal compression for short strings.  Eventhough these methods compress far more than what we are proposing, these methods often expand the original source for short strings because the symbol-code mapping also needs to be attached to aid decompression.

To our knowledge, only two other competing technologies exist - Smaz and shoco.

Smaz is a simple compression library suitable for compressing very short strings \cite{10}. It was developed by Salvatore Sanfilippo and is released under the BSD license.

Shoco is a C library to compress short strings \cite{11}. It was developed by Christian Schramm and is released under the MIT license.

While both are lossless encoding methods, Smaz is dictionary based and Shoco classifies as an entropy coder \cite{11}.

In addition to providing a default frequency table as model, shoco provides an option to re-define the frequency table based on training text \cite{11}.

\section{This research}

We propose a hybrid encoding method which relies on the three encoding techniques \emph{viz.} Entropy encoding, Dictionary coding and Delta encoding methods for compression.

Unlike shoco, we propose a fixed frequency table generated based on the characterestics of English language letter frequency.  We re-use the research carried out by Oxford University \cite{7} and other sources \cite{7} \cite{9} and come out with a unique method that takes advantage of the conventions of the language.

We propose a single model that presently is fixed because of the advantages it offers over the training models of shoco.

The disadvantage with the training model, although it may appear to offer more compression, is that it does not consider the patterns that usually appear during text formation.

We can actually see that this performs better than pre-trained model of shoco (See performance section).

For compressing Unicode symbols, we use Delta encoding because usually the difference between subsequent symbols is quite less.

We also use Delta coding for binary symbols (i.e symbols betewen ASCII 0 and 31 ad symbols between ASCII 128 and 255). However not much compression is expected out of this and in many cases, the input size is expanded.

Unlike smaz and shoco, we assume no \emph{a priori} knowledge about the input text.  However we rely on \emph{a posteriori} knowledge about the research carried out on the language and common patterns of sentence formation and come out with pre-assigned codes for each letter.

\section{Model}

In the ASCII chart, we have 95 printable letters starting from 32 through 126.  For the purpose of arriving at fixed codes for each of these letters, we use two sets of prefix-free codes.

The first set consists of 11 codes, which are: 00, 010, 011, 100, 1010, 1011, 1100, 1101, 1110, 11110, 11111.  The second set consists of 7 codes, which are 0, 10, 110, 11100, 11101, 11110, 11111.  Let us call this vcode (vertical code) and hcode (horizontal code) respectively.

With these two sets of codes, we form several sets of letters as shown in the table below and use some rules based on how patterns appear in short strings, to arrive at frequency table.

\begin{center}
	\begin{tabular}{ | l | l | l | l | l | l | l | l | } \hline
		\textbf{hcode $\rightarrow$} & \textbf{10} & \textbf{0} & \textbf{110} & \textbf{11100} & \textbf{11101} & \textbf{11110} & \textbf{11111} \\ \hline
		\textbf{$\downarrow$ vcode} & \textbf{Set 1} & \textbf{Set 1a} & \textbf{Set 1b} & \textbf{Set 2} & \textbf{Set 3} & \textbf{Set 4} & \textbf{Set 4a} \\ \hline
		\textbf{00} & switch & dist / bin & f / F & switch & . & \& & @ \\ \hline
		\textbf{010} & sp / tb & l / L & y / Y & 9 & , & ; & ? \\ \hline
		\textbf{011} & e / E & c / C & v / V & 0 & - & : & ' \\ \hline
		\textbf{100} & Uni / Cont. & d / D & k / K & 1 & / & $\textless$ & \^{} \\ \hline
		\textbf{1010} & t / T & h / H & q / Q & 2 & = & $\textgreater$ & \# \\ \hline
		\textbf{1011} & a / A & u / U & j / J & 3 & + & * & \_ \\ \hline
		\textbf{1100} & o / O & p / P & x / X & 4 & sp & \textquotedblleft & ! \\ \hline
		\textbf{1101} & i / I & m / M & z / Z & 5 & ( & \{ & \textbackslash \\ \hline
		\textbf{1110} & n / N & b / B & lf / rpt & 6 & ) & \} & $|$ \\ \hline
		\textbf{11110} & s / S & g / G & cr+lf & 7 & \$ & [ & \~{} \\ \hline
		\textbf{11111} & r / R & w / W & term & 8 & \% & ] & ` \\ \hline
	\end{tabular}
\end{center}

\section{Rules}

\subsection{Basic rules}
\begin{itemize}
	\item[$\bullet$] It can be seen that the more frequent symbols are assigned smaller codes.
	\item[$\bullet$] Set 1 is always active when beginning compression.  So the letter e has the code 011, t 1010 and so on.
	\item[$\bullet$] If the letter in Set 1a needs to be encoded, the switch code is used followed by 0 to indicate Set 1a.  So the letter c is encoded as 00011, d as 000100 and so on.
	\item[$\bullet$] Similarly, if the letter in Set 1b needs to be encoded, the switch code is used followed by 110. So k is encoded as 00110100, q as 001101010 and so on. Note that the terminator symbol is encoded as 0011011111.
\end{itemize}

\subsection{Upper case symbols}
\begin{itemize}
	\item[$\bullet$] For encoding uppercase letters in Set 1, the switch symbol is used followed by 10 and the code against the symbol itself.  For example, E is encoded as 0010011.  The same applies to tab character.
	\item[$\bullet$] For encoding uppercase letters in Set 1a, the switch symbol is used, followed by 10, again switch symbol, followed by 0 and then the corresponding code against the letter.  For instance, the letter P is encoded as 00100001100.
	\item[$\bullet$] Similarly, for uppercase letters in Set 1b, the prefix 001000110 is used.  So the symbol X is encoded as 0010001101010.
	\item[$\bullet$] If uppercase letters appear continuously, then the encoder may decide to switch to upper case using the prefix 0010 0010.  After that, the same codes for lower case are used to indicate upper case letters until the code sequence 0010 is used again to return to lower case.
\end{itemize}

\subsection{Numbers and related symbols}
\begin{itemize}
	\item[$\bullet$] Symbols in Set 2 are encoded by first switching to the set by using 00 followed by 11100.  So the symbol 9 is encoded as 0011100010.
	\item[$\bullet$] For Set 2, whenever is switch is made from Set 1, it makes Set 2 active. So subsequent numbers are encoded without the switch symbol, as in 1011 for 3, 1100 for 4 and so on.
	\item[$\bullet$] To return to Set 1, the code 0010 is used.
	\item[$\bullet$] To encode symbols in Set3, the prefix 00 is used followed by 0 and the corresponding code for the symbol.  For example, + is encoded as 0001011.
\end{itemize}

\subsection{Other symbols (Set 4)}
\begin{itemize}
	\item[$\bullet$] The special characters in Set 4 can be encoded by using the prefix 0011110 followed by the corresponding code for the letter.  Example: ; is encoded as 0011110010.
	\item[$\bullet$] The symbols in Set 4a are encoded using the prefix 0011111. Example: ? is encoded as 0011111010.
\end{itemize}

\subsection{Sticky sets}
\begin{itemize}
	\item[$\bullet$] When switching to Set 2, it becomes active and is said to be sticky till Set 1 is made active using the symbol 0010.
	\item[$\bullet$] However, no other set is sticky. Set 1 is default. Set 2 automatically becomes sticky when switched to it by using 0011100 and Upper case letters can be made sticky by using 00100010.
    \item[$\bullet$] Symbols in Set1a, Set1b, Set3, Set 4 and Set 4a are never sticky. Once encoded the previous sticky set becomes active.
\end{itemize}

\subsection{Special symbols}
\begin{itemize}
	\item[$\bullet$] term in Set 1b indicates termination of encoding.  This is used if length of the encoded string is not stored.
	\item[$\bullet$] rpt in Set 1b indicates that the symbol last encoded is to be repeated specified number of times.
	\item[$\bullet$] dict in Set 1a indicates that the specified offset in file and length is to be copied at the current position. This is dictionary based encoding.
	\item[$\bullet$] CRLF in Set 1b is encoded using a single code. It will be expanded as two bytes CR LF.  If only LF is used, such as in Unix like systems, a separate code is used in Set 1b.  Also, in the rare case that only CR appears, the Bin code is provided in Set1a.
\end{itemize}

\subsection{Dual access for Set 3}
\begin{itemize}
	\item[$\bullet$] Set 3 can be accessed both when Set 1 and Set 2 is active.  This is because the symbols occur commonly in both Set 1 and 2.  So it is necessary to have minimum length codes for these.
	\item[$\bullet$] For the same reason, the space symbol appears both in Set 1 and Set 3.
\end{itemize}

\subsection{Repeating letters}
\begin{itemize}
	\item[$\bullet$] If any letter repeats more than 3 times, we use a special code (rpt) shown in Set1b of the model.
	\item[$\bullet$] The encoder first codes the letter using the above codes.  Then the rpt code is used followed by the number of times the letter repeats.
	\item[$\bullet$] The number of times the letter repeats is coded using a special bit sequence as explained in section "Encoding counts" that follows.
\end{itemize}

\subsection{Repeating sections}
\begin{itemize}
	\item[$\bullet$] If a section repeats, another code of Set1a (dict) is used followed by four fields as described next.
    \item[$\bullet$] The second field indicates the length of the section that repeats.
	\item[$\bullet$] The third field indicates the distance of the repeating section.
	\item[$\bullet$] The fourth field is coded only if several sets of texts are encoded.  It is a number indicating the set that the section belongs to, assuming there are several sets of text are being encoded.  If only one set of text is encoded, the distance is to be counted from the current position.
	\item[$\bullet$] The second, third and fourth fields are encoded as explained in the following section "Encoding counts".
\end{itemize}

\subsection{Encoding Counts}
\begin{itemize}
	\item[$\bullet$] For encoding counts such as length and distance, the horizontal codes shown in the model are re-used, each code indicating how many bits will follow to indicate count.
	\item[$\bullet$] If code is 0, 2 bits would follow, that is, count is between 0 and 3.
	\item[$\bullet$] If code is 10, 5 bits would follow, that is, count is between 4 and 35.
	\item[$\bullet$] If code is 110, 7 bits would follow, that is, count is between 36 and 163.
	\item[$\bullet$] If code is 11100, 9 bits would follow, that is, count is between 164 and 675.
	\item[$\bullet$] If code is 11101, 12 bits would follow, that is, count is between 676 and 4771.
	\item[$\bullet$] If code is 11110, 16 bits would follow, that is, count is between 4772 and 70307.
	\item[$\bullet$] If code is 11111, a varint would follow, which means a VarInt that uses a terminating byte would be used to indicate the count.  The format for this code is kept for future expansion as it is not expected that an implementation of Unishox would need this.
	\item[$\bullet$] This is shown in tabular form below
	\item[] \begin{tabular}{ | l | l | l |} \hline
	\textbf{Code} & \textbf{Range} & \textbf{Number of bits} \\ \hline
	0 & 0 to 3 & 2 \\ \hline
	10 & 4 to 35 & 5 \\ \hline
	110 & 36 to 163 & 7 \\ \hline
	11100 & 164 to 675 & 9 \\ \hline
	11101 & 676 to 4771 & 12 \\ \hline
	11110 & 4772 to 70307 & 16 \\ \hline
	11111 & Above 70307 & Variable \\ \hline
\end{tabular}
\end{itemize}

\subsection{Encoding Unicode characters}
\begin{itemize}
	\item[$\bullet$] The code 100 is used as prefix to indicate that a Unicode character is being encoded.
	\item[$\bullet$] First, the unicode number is decoded from the input source depending on how it was encoded, such as UTF-8 or UTF-16 or Wide Character set.
	\item[$\bullet$] For the first unicode character, the number decoded is re-coded to the output as it is.
	\item[$\bullet$] For subsequent unicode characters, only the difference between the previous character is re-coded to the output.  Thus, here, delta coding is used.
	\item[$\bullet$] After the code 100, another set of prefix-free codes are used, according to the following table, depending on the size (in bits) of the difference.
	\item[] \begin{tabular}{ | l | l | l |} \hline
	\textbf{Code} & \textbf{Range} & \textbf{Number of bits} \\ \hline
	0 & 0 to 31 & 5 \\ \hline
	10 & 32 to 4127 & 12 \\ \hline
	110 & 4128 to 20511 & 14 \\ \hline
	1110 & 20512 to 86047 & 16 \\ \hline
	11110 & 86048 to 2183199 & 21 \\ \hline
	11111 & Special code & - \\ \hline
    \end{tabular}
	\item[$\bullet$] The Special code is explained in the next section.
	\item[$\bullet$] After 100, one of the above codes is used, followed by the sign bit.  The sign bit is a single bit.  1 indicates that the number following is negative and 0 indicates that the number following is positive.
	\item[$\bullet$] After the sign bit, the unicode value (or difference) is encoded as a number.  The number of bits used depends on its size, as shown in the above table.
	\item[$\bullet$] After encoding the unicode number, the state returns to Set 1, unless continuous unicode encoding was started.  This is explained in the next section.
\end{itemize}

\subsection{Encoding continuous Unicode characters}
\begin{itemize}
	\item[$\bullet$] Since the prefix 100 may become an overhead when several Unicode are to be encoded, a continuous unicode encoding code is used (0010100).
	\item[$\bullet$] When 0010100 is used, unicode characters are encoded continously using delta encoding, until a non-unicode character is encountered.  When this happens, state is returned to Set 1 using the Special code 11111 110 in the table shown in previous section is used.
	\item[$\bullet$] The Special codes are used only when Unicode characters are coded continuously, to indicate special characters and situations occuring in-between. What follows the Special code 11111 is indicated using the table below:
	\item[] \begin{tabular}{ | l | l |} \hline
	\textbf{Code} & \textbf{Character/Situation} \\ \hline
	0 & Space character \\ \hline
	10 & Repeating section \\ \hline
	110 & End continuous unicode encoding \\ \hline
	11100 & Comma (,) \\ \hline
	11101 & Full stop (.) \\ \hline
	11110 & Carriage return (CR) \\ \hline
	11111 & Line feed (LF) \\ \hline
    \end{tabular}
	\item[$\bullet$] It is found that the above characters appear frequently in between continous Unicode characters and so Special codes are needed to avoid switching back and forth from Set1.
\end{itemize}

\subsection{Encoding punctuations}
\begin{itemize}
	\item[$\bullet$] Some languages, such as Japanese and Chinese use their own punctuation characters.  For example full-stop is indicated using U+3002 which is represented visually as a small circle.
	\item[$\bullet$] So when encountering a Japanese full-stop, the special code for full-stop is used, only in this case, the decoder is expected to decode it as U+3002 instead of '.'.  In general, if the prior unicode character is greater than U+3000, then the special full-stop is decoded.
	\item[$\bullet$] There are other types of full-stops used in other languages.  For example, Hindi uses a kind of pipe symbol to indicate full-stop.  However, to avoid confusion, this is left to delta coding, since it does not make much difference in compression ratio.
\end{itemize}

\subsection{Encoding other binary symbols}
\begin{itemize}
	\item[$\bullet$]Binary symbols are not expected for encoding text, so it is out of scope of this exercise.
	\item[$\bullet$] However, for encoding other binary symbols ranging from ASCII 0 to 31 and ASCII 128 to 255, the prefix code 001000000 is used.
	\item[$\bullet$] It is followed by the ASCII value of the symbol, encoded using the method described in section "Encoding counts" in this document.
	\item[$\bullet$] This method actually expands the original size and is provided for the rare cases where such binary symbols appear.
\end{itemize}

Based on the above rules, the following Frequency table is formed.

\section{Frequency table}

\begin{center}
\begin{tabular}{ | l | l | l | l |} \hline
	\textbf{ASCII Code} & \textbf{Letter} & \textbf{Code} & \textbf{Length} \\ \hline
        32 &   & 010 & 3 \\ \hline
		33 & ! & 00111111100 & 11 \\ \hline
		34 & \textquotedblleft & 00111101100 & 11 \\ \hline
		35 & \# & 00111111010 & 11 \\ \hline
		36 & \$ & 001110111110 & 11 \\ \hline
		37 & \% & 001110111111 & 12 \\ \hline
		38 & \& & 001111000    & 9 \\ \hline
		39 & '  & 0011111011   & 10 \\ \hline
		40 & (  & 00111011101  & 11 \\ \hline
		41 & )  & 00111011110  & 11 \\ \hline
		42 & *  & 00111101011  & 11 \\ \hline
		43 & +  & 00111011011  & 11 \\ \hline
		44 & ,  & 0011101010   & 10 \\ \hline
		45 & -  & 0011101011   & 10 \\ \hline
		46 & .  & 001110100    & 9 \\ \hline
		47 & /  & 0011101100   & 10 \\ \hline
		48 & 0  & 0011100011   & 10 \\ \hline
		49 & 1  & 0011100100   & 10 \\ \hline
		50 & 2  & 00111001010  & 11 \\ \hline
		51 & 3  & 00111001011  & 11 \\ \hline
		52 & 4  & 00111001100  & 11 \\ \hline
		53 & 5  & 00111001101  & 11 \\ \hline
		54 & 6  & 00111001110  & 11 \\ \hline
		55 & 7  & 001110011110 & 12 \\ \hline
		56 & 8  & 001110011111 & 12 \\ \hline
		57 & 9  & 0011100010   & 10 \\ \hline
		58 & :  & 0011110011   & 10 \\ \hline
		59 & ;  & 0011110010   & 10 \\ \hline
		60 & $\textless$ & 0011110100 & 10 \\ \hline
		61 & = & 00111011010 & 11 \\ \hline
		62 & $\textgreater$ & 00111101010 & 11 \\ \hline
		63 & ? & 0011111010 & 10 \\ \hline
		64 & @ & 001111100 & 9 \\ \hline
	\end{tabular}
\end{center}

\begin{center}
	\begin{tabular}{ | l | l | l | l |} \hline
		\textbf{ASCII Code} & \textbf{Letter} & \textbf{Code} & \textbf{Length} \\ \hline
	65 & A & 00101011 & 8 \\ \hline
	66 & B & 00100001110 & 11 \\ \hline
	67 & C & 001000000 & 9 \\ \hline
	68 & D & 0010000011 & 10 \\ \hline
	69 & E & 0010011 & 7 \\ \hline
	70 & F & 00100011000 & 11 \\ \hline
	71 & G & 001000011110 & 12 \\ \hline
	72 & H & 00100001010 & 12 \\ \hline
	73 & I & 00101101 & 8 \\ \hline
	74 & J & 0010001101011 & 13 \\ \hline
	75 & K & 001000110100 & 12 \\ \hline
	76 & L & 0010000010 & 10 \\ \hline
	77 & M & 00100001101 & 11 \\ \hline
	78 & N & 00101110 & 8 \\ \hline
	79 & O & 00101100 & 8 \\ \hline
	80 & P & 00100001100 & 11 \\ \hline
	81 & Q & 0010001101010 & 13 \\ \hline
	82 & R & 001011111 & 9 \\ \hline
	83 & S & 001011110 & 9 \\ \hline
	84 & T & 00101010 & 8 \\ \hline
	85 & U & 00100001011 & 11 \\ \hline
	86 & V & 001000110011 & 12 \\ \hline
	87 & W & 001000011111 & 12 \\ \hline
	88 & X & 0010001101100 & 13 \\ \hline
	89 & Y & 001000110010 & 12 \\ \hline
	90 & Z & 0010001101101 & 13 \\ \hline
	91 & [ & 001111011110 & 12 \\ \hline
	92 & \textbackslash & 00111111101 & 11 \\ \hline
	93 & ] & 001111011111 & 12 \\ \hline
	94 & \^{} & 0011111100 & 10 \\ \hline
	95 & \_ & 00111111011 & 11 \\ \hline
	96 & ` & 001111111111 & 12 \\ \hline
	\end{tabular}
\end{center}

\begin{center}
\begin{tabular}{ | l | l | l | l |} \hline
\textbf{ASCII Code} & \textbf{Letter} & \textbf{Code} & \textbf{Length} \\ \hline
	97 & a & 1011 & 4 \\ \hline
	98 & b & 0001110 & 7 \\ \hline
	99 & c & 000011 & 6 \\ \hline
	100 & d & 000100 & 6 \\ \hline
	101 & e & 011 & 3 \\ \hline
	102 & f & 0011000 & 7 \\ \hline
	103 & g & 00011110 & 8 \\ \hline
	104 & h & 0001010 & 7 \\ \hline
	105 & i & 1101 & 4 \\ \hline
	106 & j & 001101011 & 9 \\ \hline
	107 & k & 00110100 & 8 \\ \hline
	108 & l & 000010 & 6 \\ \hline
	109 & m & 0001101 & 7 \\ \hline
	110 & n & 1110 & 4 \\ \hline
	111 & o & 1100 & 4 \\ \hline
	112 & p & 0001100 & 7 \\ \hline
	113 & q & 001101010 & 9 \\ \hline
	114 & r & 11111 & 5 \\ \hline
	115 & s & 11110 & 5 \\ \hline
	116 & t & 1010 & 4 \\ \hline
	117 & u & 0001011 & 7 \\ \hline
	118 & v & 00110011 & 8 \\ \hline
	119 & w & 00011111 & 8 \\ \hline
	120 & x & 001101100 & 9 \\ \hline
	121 & y & 00110010 & 8 \\ \hline
	122 & z & 001101101 & 9 \\ \hline
	123 & \{ & 1101101 & 7 \\ \hline
	124 & | & 1111110 & 7 \\ \hline
	125 & \} & 1101110 & 7 \\ \hline
	126 & \~{} & 11111110 & 8 \\ \hline
	\end{tabular}
\end{center}

Even after the freqency table is formed, the original model is still needed for encoding Sticky sets as explained in the Rules section.

\section{Implementation}

According to the above Rules and Frequency table, a reference implementation has been developed and made available at https://github.com/siara-cc/Unishox as unishox1.c.  This is released under Apache License 2.0.

Further, Unishox has been used in several open source projects shown below:

\begin{itemize}
	\item[$\bullet$] Unishox Compression Library for Arduino Progmem - https://github.com/siara-cc/Shox96\_Arduino\_Progmem\_lib
	\item[$\bullet$] Unishox Compression Library for Arduino - https://github.com/siara-cc/Shox96\_Arduino\_lib
	\item[$\bullet$] Sqlite3 User Defined Function for Unishox as loadable extension - https://github.com/siara-cc/Unishox\_Sqlite\_UDF
	\item[$\bullet$] Sqlite3 Library for ESP32 - https://github.com/siara-cc/esp32\_arduino\_sqlite3\_lib
	\item[$\bullet$] Sqlite3 Library for ESP8266 - https://github.com/siara-cc/esp\_arduino\_sqlite3\_lib
	\item[$\bullet$] Sqlite3 Library for ESP-IDF - https://github.com/siara-cc/esp32-idf-sqlite3
\end{itemize}

\section{Performance Comparison}

The compression performance of all three techniques - Smaz, shoco and Shox96 were compared for different types of strings and results are tabulated below:

\begin{center}
\begin{tabular}{ | p{0.5\textwidth} | p{0.1\textwidth} | p{0.1\textwidth} | p{0.1\textwidth} | p{0.1\textwidth} |} \hline
\textbf{String} & \textbf{Length} & \textbf{Smaz} & \textbf{shoco} & \textbf{Shox96} \\ \hline
Hello World & 11 & 10 & 8 & 8 \\ \hline
The quick brown fox jumps over the lazy dog & 43 & 30 & 34 & 29 \\ \hline
I would have NEVER said that & 28 & 20 & 20 & 18 \\ \hline
In (1970-89), \$25.9 billion; OPEC bilateral aid [1979-89], \$213 million & 67 & 65 & 52 & 54 \\ \hline
	\end{tabular}
\end{center}

Further - world95.txt - the text file obtained from \emph{The Project Gutenberg Etext of the 1995 CIA World Factbook} was compressed using the three techniques and following are the results:

Original size: 2988577 bytes

After Compression using shoco original model: 2385934 bytes

After Compression using shoco trained using world95.txt: 2088141 bytes

After Compression using Shox96: 1966019 bytes

As for memory requirements, shoco requires over 2k bytes, smaz requires over 1k. But Shox96 requires only around $95 * 3 = 285$ bytes for compressor and decompressor together, ideal for using it with even Arduino Uno.

\section{Proving guaranteed compression}

Guaranteed compression means that the length of compressed text will never exceed the length of the source text.

While it is not possible to prove it for any text, we can prove this for most real life scenarios good enough for using it without fear of expansion of original length.

At first we make the following assumptions for a given sentence in English language:
\begin{itemize}
	\item[$\bullet$] The sentence will start with a capital letter.
	\item[$\bullet$] The sentence will end in period (.).
	\item[$\bullet$] The sentence will have at least 3 words.
	\item[$\bullet$] Special characters other than a-z, A-Z and space will not be more than 2 or 3.
	\item[$\bullet$] Terminator symbol is not needed. That is, length of compressed string in bits will be separately maintained.
\end{itemize}

With the above assumptions, we try to prove guaranteed compression as follows:

\begin{itemize}
	\item[$\bullet$] Since the sentence will have atleast two spaces, it saves 5 + 5 = 10 bits.
	\item[$\bullet$] Since any English word will have a vowel and the average length of code in our frequency table is 4, it will save another 12 bits, unless the vowel 'u' appears in all three words, which is not likely in real life.
\end{itemize}

So, with a saving of atleast 22 bits, we can say it is more than sufficient to offset for any symbol being used, such as Uppercase letter or Special character, provided such letters do not exceed 4, since \emph{the maximum length of any code in our frequency table is only 13}.  So if there are 4 such exceeding codes, it will occupy at most $(13 - 8) * 4 = 20$ bits.

This assumption is towards defining a safe limit and since there will be more savings because of the known general frequency of letters, we can safely assume this guarantee.

For Unicode text, the codes in section "Encoding Unicode characters" have been selected in such a way that the prefix-code overhead is offset by delta coding and the fact that UTF-8 encoding has more redundant code overhead.

\section{Conclusion}

As can be seen from the performance numbers, Unishox performs better than available techniques.  It can also be seen that it performs better for a variety of texts, especially those having a mixture of numbers and special characters.

\section{Further work}

We propose to improve Unishox by including support for compressing binary symbols.  We also propose to develop such models for other languages and types of text such as Programs.

\section{About the Author}

\emph{Arundale Ramanathan} has over 20 years of experience working in the IT industry.  He has worked alternatively in large Corporates, MNCs and Startups, including Viewlocity Asia Pacific Pte. Ltd., IBC Systems Pte. Ltd. and Polaris Software Lab Ltd. He has founded Siara Logics (cc) and Siara Logics (in) and publishing his open source work at https://github.com/siara-cc and https://github.com/siara-in. He has a masters degree in Computer Science from Anna University.  He can be reached at arun@siara.cc.

\begin{thebibliography}{1}

	\bibitem{1} David MacKay. {\em Information Theory, Inference, and Learning Algorithms}, Cambridge University Press, 2003.

	\bibitem{2} Shannon, Claude E. (July 1948). {\em"A Mathematical Theory of Communication"}, Bell System Technical Journal. 27

	\bibitem{3} D. A. Huffman, {\em“A method for the construction of minimum-redundancy codes“}, Proc. IRE, vol. 40, pp. 1098-1101,1952.

 	\bibitem{4} J. Ziv and A. Lempel. A Universal Algorithm for Data Compression. IEEE Transactions on Information Theory, 23(3):337–343, May 1977.

  	\bibitem{5} Wikipedia, {\em Delta Encoding}, https://en.wikipedia.org/wiki/Delta\_encoding, updated July 2019.

	\bibitem{6} M. Burrows and D. Wheeler. A Block-Sorting Lossless Data Compression Algorithm. Research Report 124, Digital Equipment Corporation, Palo Alto, CA, USA, May 1994.

	\bibitem{7} {\em "Statistical Distributions of English Text"}. data-compression.com. Archived from the original on 2017-09-18.

 	\bibitem{8} What is the frequency of the letters of the alphabet in English?, Oxford Dictionary. Oxford University Press. Retrieved 29 December 2012.

	\bibitem{9} Wikipedia, {\em Letter frequency}, https://en.wikipedia.org/wiki/Letter\_frequency, updated December 2018.

 	\bibitem{10} Salvatore Sanfilippo, {\em SMAZ - compression for very small strings}, https://github.com/antirez/smaz, February 2012.

 	\bibitem{11} Christian Schramm, {\em shoco: a fast compressor for short strings}, https://github.com/Ed-von-Schleck/shoco, December 2015.

\end{thebibliography}

\end{document}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% the tex files below do not contain data, data are in
% \_paper\preamble\group_d_data\PARAMS
% All stored as new command.
% To store special estimates/results etc, create sub_folder in group_d_data for special saves
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{figure}[H]
\subsection{Estimation: Estimated Coefficients}
\vspace*{-3mm}
\makebox[\textwidth][c]{
\fcolorbox{white}{white}{
\centering
\begin{minipage}{1.0\textwidth}
% \input{"C:/Users/fan/Documents/Dropbox (UH-ECON)/Project Dissertation/Notes and Paper/Tables/costEsti.tex"}
\input{"../tables/esti/costEsti.tex"}
\small \emph{Notes:} Estimated Fixed Costs represent the average of the sum of pecuniary and non-pecuniary cost. See Section \ref{subsec:estimates} for detail.
% \input{"C:/Users/fan/Documents/Dropbox (UH-ECON)/Project Dissertation/Notes and Paper/Tables/otherEstimates.tex"}
\input{"../tables/esti/otherEstimates.tex"}
\end{minipage}
}}
\end{figure}


\section{Methods}
\label{sec:methods_chip}

\subsection{Data description}
The populations comprising the set of haplotypes from which tag \glspl{SNP} for the hypothetical \gls{SNP} array is selected are distributed across the continent (figure \ref{fig:afr5pop}). They represent 3 novel populations and 2 populations from phase 1 of \gls{1000G}.
\input{fig/afr5pop}

\subsection{Selection of tag SNPs and identification of informative variants for a hypothetical SNP array}

For identification of informative variants on the chip array we utilised a hybrid algorithm with cycles of \gls{LD} based pairwise tagging, and imputation, as previously described.\cite{Hoffmann2011422}

We have developed a multi-population tagging algorithm based on the algorithm TAGster for \gls{WGS} data.\cite{Xu2007} The methods we used for tagging were identical to those used by TAGster; however, by using seeking and indexing approaches we were able to optimise the computational efficiency of the algorithm by an order of magnitude (unpublished data, Carstensen et al.). We briefly outline the tagging algorithm as follows (figure \ref{fig:SN13f1}):

\begin{enumerate}
\item Calculate \gls{LD} (\textit{r}\textsuperscript{2}) between each \gls{SNP} and all other \glspl{SNP} in the flanking 250kbp region for each population separately. \gls{MAF} thresholds are imposed at this stage, and only pairs of \glspl{SNP} where both exceed the \gls{MAF} threshold are included.
\item For each \gls{SNP} not already in the tagging set, a count of \glspl{SNP} in the target set that are in \gls{LD} exceeding a given threshold \textit{r}\textsuperscript{2} with it is generated across the genome and summed across all populations.
\item The most informative \gls{SNP} (the \gls{SNP} with most target \glspl{SNP} in \gls{LD} with it summed across population) is chosen as the tagging \gls{SNP} and added to the set of tagging \glspl{SNP}. If two \glspl{SNP} have the same ranking, then a \gls{SNP} is chosen by one or all of the following parameters in order of preference: 1) presence on existing SNP arrays (count), 2) presence in 1000G and/or \gls{dbSNP} (binary), 3) location within a gene region (binary), 4) binned genotyping score provided by the vendor, 5) rate of heterozygosity across all populations (continuous).
%We do not take the vicinity to existing tag SNPs into account, despite  SNPs in close proximity being able to interfere with each other when assayed.\cite{21535878} We expect the algorithm to avoid this problem, because SNPs in proximity of each other are also likely to be in LD with each other.
\item This tagging SNP and SNPs in LD with it are now removed from the set of target SNPs. This process is carried out separately for each population, so that a separate set of target SNPs is maintained for each population set. However, SNPs in LD with the tagging SNP can still be picked up as tagging SNPs themselves if they independently tag the maximum no. of SNPs in any iteration.
Steps 2-3 are repeated until either a specified number of SNPs or all target SNPs (chosen as SNPs above a specific MAF threshold per-population) are tagged across all population sets, or until a specific number of SNPs is reached, as specified.
\end{enumerate}

Although this method works well across populations, it only carries out pairwise single-marker tagging. Haplotype based, or multi-marker tagging would potentially be more efficient, and select fewer tagging sites. In order to incorporate haplotype based tagging into our model, we use a hybrid method, with cycles of tagging and imputation, as has been described before.\cite{Hoffmann2011422}

\input{fig/SN13f1}

We implement this method by selecting a maximum number of pre-defined tagging \glspl{SNP} in the first cycle. With these tagging \glspl{SNP} we simulate a chip for each population, and imputation is carried out using a reference panel, to identify additional sites in each population that are tagged at an \gls{r2} threshold above 0.80 by the tagging sites. These sites are removed from the target set for each population, and do not contribute to the next cycle, thereby making the process more efficient. To maximise imputation accuracy we use all samples in table \ref{tab:samples} for the reference panel and merge this reference panel with haplotypes from all of the \gls{1000G} samples from Europe, Asia and the Americas not already present in the reference panel. However, for imputation into each population, all samples from the given population are removed from the reference panel. This ‘leave one population out’ approach would produce relatively conservative results for tagging, with more variants being tagged than if a subsample of the population was included in the reference panel. This is a more realistic scenario, as it is not necessary that any given population genotyped on the chip in future would be represented in the reference panel.

Additionally, pre-selected known biologically relevant variants, valuable to the studies planned for consortia can be included on the array, to replace certain tag \glspl{SNP}. We have previously shown that a 1M tagging variants chosen using the described algorithm can produce greater than 0.80 coverage across diverse populations in Africa. Based on this, we plan to carry out approximately 10 cycles to capture 1.2M tagging variants, in order to prioritise variants to include in the design of a 1M chip array. Following this, we will further validate our tagging algorithm among populations with smaller sample sizes that were not included in the development in the chip array, by selecting tagging variants among these and imputing with the reference panel, excluding these populations. We estimate coverage of each population by such a chip array using imputation. Coverage is defined as the proportion of common variation captured at a correlation greater than 0.80 across the genome in a given population with the combined reference panel (excluding the population being evaluated). $r^{2}$, here, is calculated as the correlation between the sequence data and imputed data on a hypothetical 1M chip array for common variation.

%Instead of random choice do 1) chip overlap, 2) quality score (VQSLOD and MVNcall posterior), 3) white/black list

\subsection{Computational description of calculation of pairwise \gls{LD} values}
Haplotype files are block gzipped\cite{Li05012011}, which allows them to be indexed. This allows for rapid calculation of pairwise \gls{LD} between \glspl{SNP} in a window of a certain size with a small amount of memory usage. I calculated the LD with Python3\footnote{https://github.com/team149/tc9/blob/master/ragtagger/calculateLD.py}. The pairwise \gls{LD} values can also be estimated from non-phased genotypes by using the \gls{EM} algorithm.
%https://github.com/team149/tc9/blob/master/WGS/QC/vcf2pca.py
Alternatively the pairwise \gls{LD} values can now be calculated with PLINK1.9\cite{25722852}. These pairwise LD values are then used as input for the tag SNP selection program RagTagger\footnote{https://github.com/team149/tc9/blob/master/ragtagger/ragtagger.py}.

\subsection{Computational description of selection of tag \glspl{SNP} from pairwise \gls{LD} values}
The selection of tag \glspl{SNP} between imputation cycles after calculation of pairwise \gls{LD} values is carried out with RagTagger\footnote{https://github.com/team149/tc9/blob/master/ragtagger/ragtagger.py}, which is a memory and \gls{CPU} efficient rewrite of the software TAGster.\cite{Xu2007} For the work presented here it was set up to be able to parse sets of pre-selected SNPs and SNPs not to be used as tag SNPs. To avoid keeping all the \gls{LD} information in memory the number and identity of tagged SNPs was then calculated from the \gls{LD} files and written to a block gzipped\cite{Li05012011} file, which was indexed on the fly, which allows subsequent random access. Only the \gls{LD} count and the random access position for each potential tag SNP was kept in memory in hash tables to allow O(1) look ups. The dictionary with SNP IDs as keys and LD counts as values is inverted and these SNPs are looped through from most common to least common until exhaused or the upper limit for tag SNPs (overall or for the cycle) is reached.


\section{The Type-1 API} \label{sec:type1query}

The Type-1 interface is radically different than the Type-0 interface.
While the Type-0 interface uses tabling and logical preferences to
return correct answers according to the {\sc inh} proof system, it is
still resolution-based.  However, when the disjunction and negation of
virdual identifiers is added such an approach is no longer possible,
so that the Type-1 query interface is based on computing logican
consistency and entailment.  Since logical entailment of class
expressions can be reduced to consistency, the Type-1 interface is
based on consistency checking of CDF instances that have been
transformed into class expressions.

Consistency checking of class expressions such as those of
Definition~\ref{def:ce} is decidable, but {\em P-space}
complete~\footnote{Assuming a linear encoding of the integers in the
{\em atLeast} and {\em atMost} constructs.  Formally, the CDF prover
is complete for ${\cal ALCQ}$ description logics extended with
relational hierarchies and product classes.}, so that determening
whether a Type-1 instance has a model requires radically different
checking techniques than Type-0 instances.  Query answering for Type-1
instances is performed by using theorem proving techniques.

\subsection{The CDF Theorem Prover}

Specialized theorem-provers are generally implemented to check
consistency of class expressions.  These provers may use based on
structural subsumption techniques (e.g. as used in
CLASSIC~\cite{PMBRB91}, LOOM~\cite{MacB94} and GRAIL~\cite{RBGHNS97});
tableaux construction~\cite{HoPS99}; or stable model
generation~\cite{Swif04} --- in \version{} of CDF a tableaux-style
prover is used.

At a high level, the CDF prover first translates a class expression
$CE$ to a formula $\psi$ in an ontology language according to
Definition~\ref{def:fot}.  It then attempts to construct a model for
$\psi$: if it succeeds $CE$ is consistent, otherwise $CE$ is
inconsistent (since the prover can be shown to be complete).  The CDF
prover has access to the relational and class hierarchies of a CDF
instance during its execution.  As a result, only the principle
classes and relations of an identifier (Definition~\ref{def:redund})
need be entered in class expressions.  Finally, since objects in the
semantics of CDF are indistinguishable from singleton sets, an object
identifier $O$ can be used in any context that a class identifier can
be used.  The prover takes accont of this by enforcing a cardinality
constraint for the set $O$.

The theorem prover of \version{} uses exhaustive backtracking, rather
than the dependency-directed backtracking that is typical of recent
provers such as the DLP prover \cite{}, the FaCT prover \cite{} or the
Racer prover \cite{}.  As a result, the CDF prover may be slow for
certain types of queries relative to these other provers.
Dependency-directed backtracking has not been added to the CDF prover
largely to keep it simple enough to experiment with different
extensions to the types of class expressions it supports~\footnote{In
particular, work is underway on extending the CDF prover to handle
functional attribute chains and concrete domains (see \cite{}).}.  On
the other hand, the CDF prover is relatively efficient on how it
traverses a CDF instance to check consistency.

When a CDF Type-1 instance is checked, the instance is translated into
either a class expression before it can be sent to the CDF prover.
Due to the high worst-case complexity of consistency checking, input
strings to the prover should be kept as small as possible.  The CDF
system accomplishes this by translating information about a given CDF
identifier into a series of local class expressions
(\refsec{sec:lce}), sending a local class expresion to the CDF prover,
then producing and checking other local class expressions as needed.
Since CDF instances differ in philosophy from terminological systems,
they may be expected to be cyclic, so that a given class identifier
may occur in a level $n$ local class expression of itself.

\subsection{The Type-1 API}

\begin{description}

\ourpreditem{checkIdConsistency/1}{cdftp\_chkCon}
In {\tt checkIdConsistency(IdList)} {\tt IdList} is a (list of) class
or object identifier(s) which is taken as a conjunction.  The
predicate succeeds if {\tt IdList} is consistent in the current CDF
instance.

{\bf Exceptions} {\tt Domain Exception: IdList} is not a class
identifier, an object identifier, or a list of class or object
identifiers.

%-----------------------------------------------------------------------------
\mycomment{
/* The algorithm is described in detail in the CDF system paper.  What
we do here is to prove consistency of an identifier by an iterative
process.  Given an identifier {\tt Id}, a local class expression is
constructed for Id, and a consistency check made for that class
expression.  In other words, we prove the consitency of Id by trying
to construct a model in which Id is a non-empty set if it is a cid, or
a non-empty unique set if Id is an oid.

Local class expressions dont contain all information for an
identifier.  Accordingly, in the model constructed for Id we need to
check the *contexts* for each individual in the model, i.e. if an
individual i belongs to classes C1 and C2 in our model we must ensure
that a model can be constructed for both C1 and C2.  The checker thus
traverses through all the contexts in the model and checks them
recursively.

An important issue occurs if a check for an identifier recursively
leads to a context in which the identifier itself is present.  If this
is the case, we succeed, as it can be shown that the identifier is
consistent.  If an identifier Id depends on itself negatively, we
fail, as we cannot be sure of constructing a model in this case.  A
more elaborate algorithm would take into account even and odd loops,
but that seems a little arcane for our purposes.

This code doesn't use XSBs tabling for two reasons.  First, we want to
succeed on positive loops, and second, we only want a single solution
for each consistency check.  In this homespun tabling, information is
entered about whether a context we are traversing has been queried,
and whether it has succeeded or failed if it is complete.  Once a
consistency check succeeds, all of its choice points are cut away.
Success on positive loops is addressed by passing around an ancestor
list and performing an ancestor check at each call to the sat routine.
If the context is in the ancestor list we succeed, otherwise we call
the sat routine (which succeed or fail on table check).  Note that we
do not need to table the ancestor list -- it is not a set of
assumptions, its just used to succeed on loops.  Also, since all code
requires only a single solution for any consistency check do not have
to worry about incomplete tables that are not in the ancestor list.

Various cases.
1) Not called before
2) Called but incomplete
3) Complete, succeed or fail

checkIdConsistency_1 handles case 2).
Cases 1) and 3) are handled by checkIdConsistency_1
*/
}
\ourpredmoditem{consistentWith/2}{cdftp\_chkCon}
In {\tt consistentWith(Id,CE)}, {\tt Id} can either be a class or an
object identifier and {\tt CE} is a class expression.  This predicate
checks whether {\tt CE} is logically consistent with all that is known
about {\tt Id} in the current CDF instance.  {\tt consistentWith/2}
determines whether there is a model of the current CDF instance that
satisfies the expression {\tt Id,CE}.

This predicate assumes that all class and object identifiers in a
given CDF instance are consistent.

{\bf Exceptions}

{\tt Domain Exception: Id} is not a class or object identifier.

{\tt Domain Exception: CE} is not a well-formed class expression.

\ourpredmoditem{allModelsEntails/2}{cdftp\_chkCon}
In {\tt allModelsEntails(Id,CE)}, {\tt Id} is a class or object
identifier and {\tt CE} is a class expression. {\tt
allModelsEntails/2} succeeds if {\tt CE} is entailed by what is known
about {\tt Id} in the current CDF instance.  In other words, {\tt
allModelsEntails/2} determines whether in all models of the current
CDF instance, if an element is in {\tt Id} then it is also in {\tt
CE}.  It does this by checking the inconsistency of {\tt Id,CE}.

This predicate assumes that all class and object identifiers in a
given CDF instance are correct.

{\bf Exceptions}

{\tt Domain Exception: Id} is not a class or object identifier.

{\tt Domain Exception: CE} is not a well-formed class expression.

\ourpredmoditem{localClassExpression/3}{cdftp\_chkCon}
In {\tt localClassExpression(+IdList,+N,-Expr)} {\tt IdList} is a list
of class identifiers, and {\tt N} is a positive integer.  In its
semantics, {\tt IdList} is interpreted as a conjunction of
identifiers, and upon success, {\tt Expr} is a class expression,
unfolded to depth {\tt N}, that describes {\tt IdList} according to
gthe current CDF instance.

{\bf Exceptions}

{\tt Domain Exception: IdList} is not a class identifier or object
identifier, or a list of class or object identifiers.

{\tt Type Exception: N} is not a positive integer.

{\tt Instantiation Exception: Expr} is not a variable.

\ourpredmoditem{check\_lce/2}{cdftp\_chkCon}
In the goal {\tt check\_lce(+IdList,+N)} {\tt IdList} is a list of
class identifiers, and {\tt N} a positive integer.  In its semantics,
{\tt IdList} is interpreted as a conjunction of identifiers, and {\tt
check\_lce(+IdList,+N)} pretty-prints a class expression, unfolded to
depth {\tt N}, that describes {\tt IdList} according to the current
CDF instance.

{\bf Exceptions}

{\tt Domain Exception: IdList} is not a class identifier or object
identifier, or a list of class or object identifiers.

{\tt Type Exception: N} is not a positive integer.

\end{description}


\documentclass[12pt]{amsart}
\usepackage[margin=0.5in]{geometry}
  % see geometry.pdf on how to lay out the page. There's lots.
\usepackage{bsymb}
\usepackage{calculational}
\usepackage{ulem}
\usepackage{hyperref}
\usepackage{unitb}

\newcommand{\REQ}{\text{REQ}}

\begin{document}

\section{Strategy}

\begin{itemize}
  \item
\end{itemize}
\section{Model m0 --- Requests and non-deterministic handling}
  % \input{lock-free-deque/machine_m0.tex}
\begin{machine}{m0}
  \with{sets}
  \newset{\REQ}
  \newevent{add}{add}
  \newevent{handle}{handle}
  \[ \variable{ req : \set [\REQ] } \]
  \[ \param{add}{ r : \REQ } \]
  \[ \param{handle}{ r : \REQ } \]
  \begin{align*}
      \initialization{m0:init0}{ req = \emptyset } \\
      \evguard{add}{m0:guard}{ \neg r \in req } \\
      \evbcmeq{add}{m0:act0}{req}{ req \bunion \{ r \} } \\
      \cschedule{handle}{m0:sch0}{ \neg req = \emptyset } \\
      \evguard{handle}{m0:grd0}{ r \in req } \\
      \evbcmeq{handle}{m0:act0}{req}{ req \setminus \{ r \} }
      \invariant{inv777}{ var \in Req }
      \[ \variable{var : \{ 'req : \REQ \} } ]
  \end{align*}
\definition{Req}{ [ 'req : \REQ ] }
\noindent
\end{machine}
\section{Model m1 --- Version numbers and individual fairness}
  % \input{lock-free-deque/machine_m1.tex}
\begin{machine}{m1}
    \refines{m0}
  \[ \variable{ ver : \Int } \]
  % \[ \indices{handle}{ r0 : \REQ } \]
  \[ \indices{handle}{ v : \Int } \]
  \promote{handle}{r}
  \removecoarse{handle}{m0:sch0}
  \removeguard{handle}{m0:grd0}
  % \removeact{handle}{m0:act0}
  \[\witness{handle}{r}{r \in req}\]
  \[\witness{handle}{v}{v = ver}\]
  \begin{align}
      \initialization{m1:init0}{ ver = 0 } \\
      \cschedule{handle}{m1:sch0}{ r \in req } \\
      \cschedule{handle}{m1:sch1}{ v = ver } \\
      % \evguard{handle}{m1:grd0}{ r = r0 } \\
      \evbcmeq{handle}{m1:act0}{ver}{ ver + 1 }
  \end{align}
\end{machine}
\section{Model m2 --- Specialized events}
  % \input{lock-free-deque/machine_m2}
\begin{machine}{m2}
  \refines{m1}
  \[ \variable{pshL,pshR,popR,popL : \set [\REQ]} \]
  \begin{align}
    \initialization{m2:init0}{pshL = \emptyset} \\
    \initialization{m2:init1}{popL = \emptyset} \\
    \initialization{m2:init2}{pshR = \emptyset} \\
    \initialization{m2:init3}{popR = \emptyset}
  \end{align}
  \subsection{Specialize \emph{handle}}
  % \begin{align*}
    \refiningevent{handle}{handle:popL}{handle\_popL}
    \refiningevent{handle}{handle:popR}{handle\_popR}
    \refiningevent{handle}{handle:pushL}{handle\_pushL}
    \refiningevent{handle}{handle:pushR}{handle\_pushR}
  % \end{align*}
  \splitevent{handle}{handle:popL,handle:popR,handle:pushR,handle:pushL}
  \begin{align}
    \invariant{m2:inv0}{ pshL \bunion
        pshR \bunion
        popL \bunion
        popR
        = req } \\
    \invariant{m2:inv1}{ pshL \binter pshR = \emptyset } \\
    \invariant{m2:inv2}{ pshL \binter popL = \emptyset } \\
    \invariant{m2:inv3}{ pshL \binter popR = \emptyset } \\
    \invariant{m2:inv4}{ pshR \binter popL = \emptyset } \\
    \invariant{m2:inv5}{ pshR \binter popR = \emptyset } \\
    \invariant{m2:inv6}{ popL \binter popR = \emptyset } \\
    \cschedule{handle:pushL}{m2:sch0}{ r \in pshL } \\
    \evbcmeq{handle:pushL}{m2:act0}{pshL}{pshL \setminus \{ r \}} \\
    \cschedule{handle:pushR}{m2:sch0}{ r \in pshR } \\
    \evbcmeq{handle:pushR}{m2:act0}{pshR}{pshR \setminus \{ r \}} \\
    \cschedule{handle:popL}{m2:sch0}{ r \in popL } \\
    \evbcmeq{handle:popL}{m2:act0}{popL}{popL \setminus \{ r \}} \\
    \cschedule{handle:popR}{m2:sch0}{ r \in popR } \\
    \evbcmeq{handle:popR}{m2:act0}{popR}{popR \setminus \{ r \}}
    % \invariant{m2:inv1}{ popL \subseteq req } \\
    % \invariant{m2:inv2}{ pshR \subseteq req } \\
    % \invariant{m2:inv3}{ popR \subseteq req }
  \end{align}
  \[ \variable{ ppd : \set [\REQ] } \]
  \begin{align}
    \initialization{m2:init4}{ ppd = \emptyset } \\
    \evbcmeq{handle:popR}{m2:act1}{ppd}{ ppd \bunion \{r\} } \\
    \evbcmeq{handle:popL}{m2:act1}{ppd}{ ppd \bunion \{r\} }
  \end{align}
  \newevent{return}{return}
  \[ \indices{return}{ r : \REQ } \]
  \begin{align}
    \cschedule{return}{m2:sch0}{ r \in ppd } \\
    \evbcmeq{return}{m2:act0}{ppd}{ ppd \setminus \{ r \} }
  \end{align}
\subsection{Specialize \emph{add}}
    \refiningevent{add}{add:popL}{add\_popL}
    \refiningevent{add}{add:popR}{add\_popR}
    \refiningevent{add}{add:pushL}{add\_pushL}
    \refiningevent{add}{add:pushR}{add\_pushR}
  \splitevent{add}{add:popL,add:popR,add:pushR,add:pushL}
  \begin{align}
    \evbcmeq{add:popL}{m2:act0}{popL}{popL \bunion \{r\}} \\
    \evbcmeq{add:pushL}{m2:act1}{pshL}{pshL \bunion \{r\}} \\
    \evbcmeq{add:popR}{m2:act2}{popR}{popR \bunion \{r\}} \\
    \evbcmeq{add:pushR}{m2:act3}{pshR}{pshR \bunion \{r\}}
  \end{align}
\subsection{Data refinement}
  \removevar{req}
  \removeinit{m0:init0}
  \removeact{handle:popR}{m0:act0}
  \removeact{handle:pushR}{m0:act0}
  \removeact{handle:popL}{m0:act0}
  \removeact{handle:pushL}{m0:act0}
  \removecoarse{handle:popR}{m1:sch0}
  \removecoarse{handle:pushR}{m1:sch0}
  \removecoarse{handle:popL}{m1:sch0}
  \removecoarse{handle:pushL}{m1:sch0}
  \removeguard{add:popR}{m0:guard}
  \removeguard{add:pushR}{m0:guard}
  \removeguard{add:popL}{m0:guard}
  \removeguard{add:pushL}{m0:guard}
  \removeact{add:popR}{m0:act0}
  \removeact{add:pushR}{m0:act0}
  \removeact{add:popL}{m0:act0}
  \removeact{add:pushL}{m0:act0}
\end{machine}
  \newcommand{\OBJ}{\text{OBJ}}
\section{Model m3 --- The Contents}
  % \input{lock-free-deque/machine_m3}
\begin{machine}{m3}
  \refines{m2}
  \with{functions}
  \with{intervals}

  \newset{\OBJ}
  \[ \variable{ p,q : \Int } \]
  \[ \variable{ qe : \Int \pfun \OBJ } \]
  \begin{align}
    \invariant{m3:inv0}{ qe \in \intervalR{p}{q} \tfun \OBJ } \\
    \invariant{m3:inv1}{ p \le q } \\
    \initialization{m3:init0}{ p = 0 \land q = 0} \\
    \initialization{m3:init1}{ qe = \emptyfun }
  \end{align}
  \subsection{Push}
    \[\param{handle:pushL}{obj : \OBJ} \]
    \[\param{handle:pushR}{obj : \OBJ} \]
  \begin{align}
    \evbcmeq{handle:pushL}{m3:act0}{p}{p - 1} \\
    \evbcmeq{handle:pushL}{m3:act1}{qe}{ qe \2| (p \0- 1 \fun obj)} \\
    \evbcmeq{handle:pushR}{m3:act0}{q}{q + 1} \\
    \evbcmeq{handle:pushR}{m3:act1}{qe}{ qe \2| (q \fun obj)}
  \end{align}
  \[ \variable{ res : \REQ \pfun \OBJ } \]
  \[ \variable{ result : \OBJ } \]
  \[ \variable{ emp : \Bool } \]
  \subsection{Pop}
  \begin{align}
    \initialization{m3:init3}{ res = \emptyfun } \\
    \invariant{m3:inv2}{ res \in ppd \pfun \OBJ } \\
    \evbcmeq{return}{m3:act0}{res}{ \{r\} \domsub res} \\
    \evbcmsuch{return}{m3:act1}{result}{ r \in \dom.res \implies result' = res.r } \\
    \evbcmeq{return}{m3:act2}{emp}{ (r \in \dom.res) }
  \end{align}
  \splitevent{handle:popL}{handle:popL:empty,handle:popL:non:empty}
  \splitevent{handle:popR}{handle:popR:empty,handle:popR:non:empty}
  \refiningevent{handle:popL}{handle:popL:empty}{handle\_popL\_empty}
  \refiningevent{handle:popL}{handle:popL:non:empty}{handle\_popL\_non\_empty}
  \refiningevent{handle:popR}{handle:popR:empty}{handle\_popR\_empty}
  \refiningevent{handle:popR}{handle:popR:non:empty}{handle\_popR\_non\_empty}
  \[ \dummy{v : \Int} \]
  \replace{handle:popL}{m3:sch0}{m3:prog0}
  \begin{align*}
    & \progress{m3:prog0}{v = ver}{p = q \lor p < q}
    \refine{m3:prog0}{implication}{}{}
  \end{align*}
  \begin{align}
    \cschedule{handle:popL:non:empty}{m3:sch0}{ p < q } \\
    \cschedule{handle:popL:empty}{m3:sch0}{ p = q } \\
    \evbcmeq{handle:popL:non:empty}{m3:act0}{res}{ res \1| (r \fun qe.p) } \\
    \evbcmeq{handle:popL:non:empty}{m3:act1}{p}{p+1} \\
    \evbcmeq{handle:popL:non:empty}{m3:act2}{qe}{ \{p\} \domsub qe }
  \end{align}
  \replace{handle:popR}{m3:sch0}{m3:prog0}
  \begin{align}
    \cschedule{handle:popR:non:empty}{m3:sch0}{ p < q } \\
    \cschedule{handle:popR:empty}{m3:sch0}{ p = q } \\
    \evbcmeq{handle:popR:non:empty}{m3:act0}{res}{ res \1| (r \fun qe.(q\0-1)) } \\
    \evbcmeq{handle:popR:non:empty}{m3:act1}{q}{q-1} \\
    \evbcmeq{handle:popR:non:empty}{m3:act2}{qe}{ \{q-1\} \domsub qe }
  \end{align}
\end{machine}
\end{document}


\begin{figure}
\begin{center}
    \includegraphics[width=0.9\textwidth]{figs/abc_av_mssfr.pdf}
    \caption{\label{fig:avmsfr}
        $M_*$ and $\ssfr$ dependence of dust attenuation at $1500 \AA$
        ($A_{1500}$; top) and at $5500\AA$ ($A_{V}$ bottom) predicted by the
        \eda~for SIMBA(left), TNG (center), and EAGLE (right). The colormap in each hexbin
        represents the median attenuation for all simulated galaxies in the
        bin (right color bar). We only include bins with more than 10 galaxies.
        For reference, we include in each panel the $M_*-\ssfr$ relation of
        all galaxies from the simulations (black dashed).
        Overall, simulated galaxies with higher $M_*$ have higher dust
        attenuation at constant SSFR --- consistent with the literature.
        Furthermore, since previous works have primarily focused on star-forming
        galaxies, the \eda~provides new insight into the $\ssfr$ dependence of
        dust attenuation: simulated galaxies with higher $\ssfr$ have steeper
        attenuation curves.
    }
\end{center}
\end{figure}

\subsection{The Galaxy -- Dust Connection}
%In the previous section, we presented the attenuation curves predicted by
%the \eda~for quiescent galaxies.
%By comparing them to the attenuation curves of star-forming galaxies, we
%found significant $\ssfr$ dependence in dust attenuation: quiescent galaxies
%have attenuation curves with shallower slopes and lower amplitude than
%star-forming galaxies.
In this section, we examine the connection between dust attenuation and galaxy
properties in more detail.
First, we examine the galaxy-dust connection using the $M_*$ and $\ssfr$
dependent parameterization of our
\eda~prescription (Eqs~\ref{eq:tauv} and~\ref{eq:delta}).
In Table~\ref{tab:posterior}, we list the median values and the 68\%
confidence interval of the inferred \eda~parameter posteriors for the
three simulations.
We find significant $\ssfr$ dependence, $m_{\tau,\ssfr} > 0$, consistent with
what we found in the previous section: quiescent galaxies have lower dust
attenuation than star-forming galaxies.
We also find significant $M_*$ dependence in $\tau_V$: $m_{\tau,M_*} > 0$.
$V$-band dust attenuation is higher for more massive galaxies.
There is, however, little $M_*$ dependence in the slope of the dust
attenuation.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% table of free parameters
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{table}
    \caption{Inferred the Empirical Dust Attenuation Model Parameters}
    \begin{tabular}{lcccccc} \toprule
        & $m_{\tau,M_*}$ & $m_{\tau,\ssfr}$ & $c_\tau$ & $m_{\delta,M_*}$ & $m_{\delta,\ssfr}$ & $c_\delta$ \\[3pt] \hline\hline
        SIMBA   & $1.27\substack{+0.46\\-0.46}$ &
        $1.28\substack{+0.24\\-0.23}$ & $1.58\substack{+0.12\\-0.12}$ &
        $0.07 \substack{+0.12\\-0.11}$ & $0.13 \substack{+0.10\\-0.10}$ &
        $-0.18\substack{+0.04\\-0.04}$ \\
        TNG     & $0.57\substack{+0.44\\-0.53}$ &
        $0.62\substack{+0.21\\-0.20}$ & $1.34\substack{+0.19\\-0.21}$ &
        $-0.18\substack{+0.20\\-0.19}$ & $-0.19\substack{+0.15\\-0.16}$ &
        $-0.07\substack{+0.08\\-0.08}$ \\
        EAGLE   & $0.59\substack{+0.33\\-0.33}$ &
        $0.18\substack{+0.20\\-0.17}$ & $0.81\substack{+0.14\\-0.15}$ &
        $-0.13\substack{+0.17\\-0.18}$ & $-0.22\substack{+0.14\\-0.14}$ &
        $-0.34\substack{+0.08\\-0.08}$\\
        \hline
    \end{tabular} \label{tab:posterior}
\end{table}

We take a closer look at the $M_*$ and $\ssfr$ dependence of the attenuation
curve in Figure~\ref{fig:avmsfr}, where we present dust attenuation at
$1500\AA$ ($A_{1500}$; top) and $5500\AA$
($A_V$; bottom) as a function of $\log M_*$ and $\log \ssfr$ predicted by the
\eda~for SIMBA (left), TNG (center) and EAGLE (right).
For each hexbin, the colormap represents the median attenuation for all
simulated galaxies in the bin.
We only include bins with more than 10 galaxies.
We include, for reference, the $M_* - \ssfr$ relation of all galaxies in the
simulations in black dashed contours, which mark the 68 and 95 percentiles.
We do not include a direct comparison with observations because there are large
variations between different measurements (Appendix~\ref{sec:slab}, see also
Figure~\ref{fig:av_obs}).

In each panel, we find that SIMBA, TNG, and EAGLE galaxies with higher
$M_*$ have higher dust attenuation at constant SSFR --- consistent with the literature.
\cite{burgarella2005}, for instance, found significant positive $M_*$
dependence in $FUV$ attenuation in NUV-selected and FIR-selected samples.
\cite{garn2010} and \cite{battisti2016} also found higher attenuation in
more massive SDSS star-forming galaxies.
Most recently, \cite{salim2018} found higher $V$ and $FUV$ attenuation for
more massive star-forming galaxies in GSWLC2.
For the $\ssfr$ dependence, we find that galaxies with higher $\ssfr$ have
higher $A_{1500}$ (top) and $A_V$ (bottom) as well as steeper slopes.
The $\ssfr$ dependence is not as prominent in EAGLE (see also
Table~\ref{tab:posterior}), which has a narrower $\ssfr$ distribution than
SIMBA and TNG with no starburst galaxies or quiescent galaxies with $\ssfr <
10^{-12}yr^{-1}$.
EAGLE has fewer intrinsically luminous star-forming galaxies
or UV-red galaxies (Figure~\ref{fig:obs}) and a narrower intrinsic $\gr$
and $\fnuv$ color distributions.
To reproduce observations, it requires an overall attenuation and reddenning
without a significant $\ssfr$ dependence.
Nevertheless, in all simulations, star-forming galaxies have slopes that
are consistent with observations (Section~\ref{sec:reproduce}) while
quiescent galaxies with the lowest $\ssfr$ have nearly flat attenuation
curves.
%Since observations have only focused on star-forming galaxies due to the difficulty in measuring dust attenuation in quiescent galaxies, the \eda~predictions provide new insight into the $\ssfr$ dependence of dust attenuation.
In summary, we find that \emph{SIMBA, TNG, and EAGLE galaxies with higher
$M_*$ require overall higher dust attenuation and galaxies with higher
$\ssfr$ require steeper attenuation curves}.


\section{From Whence the Specification}~\label{sec:req} Sound systems
engineering practices as well as regulatory guidelines typically
mandate safety-critical systems have detailed written requirements as
well as a thorough safety assessment. Safety is a system level
property, so if RV is to provide safety guarantees, then the monitor
specifications must flow down from the requirements and safety
analyses.  Indeed, Rushby's study~\cite{rvRushby,RushbyAIAA09}
demonstrated that a convincing safety case for an IA system protected
by RV demands evidence that the monitor specifications are derived
from validated requirements and that the monitor specifications
should include checks of the assumptions on which safe operation of
the system rests are indeed satisfied.

\paragraph{Challenge:} \emph{ Monitor specifications should derive from
  system level requirements and assumptions that have been validated
  by domain experts.}

\paragraph{}  As
machine intelligence replaces people, RV is likely to be called upon
to enforce safe behaviors that humans began learning in early
childhood.  For an autonomous automobile controlled by machine
learning, a safety property might be ``do not hit a pedestrian'' or
``do not behave erratically'', but what do these statements mean
precisely?  A more precise statement such as ``maintain five meters
distance from any object in the vehicle path'' may be more formal, but
is that what the expert wanted?  Phrases like ``erratic behavior''
may seem reasonable to a mature adult, but formalizing such statements
can be an open ended problem.

\paragraph{Challenge:} \emph{Precisely formalize safety properties  in
  a  logic.}

\paragraph{Copilot Approach:} We are conducting case studies informed
by collaborations with colleagues who are developing new concepts that
will enable aircraft to perform autonomous flight by self-optimizing
their four-dimensional trajectories while conforming to constraints
such as required times of arrival generated by air-traffic service
providers on the ground. Many of the proposed
algorithms~\cite{KarrVRC} do not behave with the predictability of
conventional systems.  Consequently, it is not possible to provide the
required level of assurance that the newly computed trajectories
preserve safe aircraft separation.  The separation requirement for two aircraft is specified by a minimum
horizontal separation $D$ (typically, $5$ nautical miles).
Fortuitously, colleagues at NASA have discovered analytical formula,
called \emph{criteria}~\cite{NM2011NASA}, that characterize resolution
maneuvers that both ensure safe separation when one aircraft maneuvers
and ensures separation when two conflicting aircraft both
maneuver. The criteria have been extensively validated by domain
experts who conducted sophisticated simulations as well as performing
formal mathematical proofs using the Prototype Verification System
(PVS) theorem prover. We have encoded these conditions as Copilot
specifications. The criteria for horizontal separation for two
aircraft is given as follows:
%$$ \texttt{horizontal\_criteria}(\bm{s}, \epsilon)(\bm{v'})
%\equiv  \bm{s} \cdot \bm{v'} \geq \epsilon  \frac{\sqrt{\bm{s} \cdot
%  \bm{s}  - D^2}} {D} \operatorname{det}(\bm{s},\bm{v'})$$

\begin{eqnarray*} \texttt{horiz\_criteria}(\bm{s}, \epsilon, \bm{v})
& \equiv &  \bm{s} \cdot \bm{v} \geq \epsilon  \frac{\sqrt{\bm{s} \cdot
  \bm{s}  - D^2}} {D} \operatorname{det}(\bm{s},\bm{v})  \\
& \wedge &  \epsilon \operatorname{det}(\bm{s},\bm{v})  \leq  0
\end{eqnarray*}

\noindent
where $\bm{s}$ is the relative position
vector for the two aircraft, $\epsilon$ is $1$ or $-1,$  and
$\bm{v}$ is the relative velocity vector after a planned maneuver.
The position is  assumed to be given in Earth Centered Earth Fixed (ECEF)
coordinates.  This is easily encoded in  Copilot's EDSL as:

%\begin{lstlisting}[frame=single]
%scalar2dim :: Stream Double -> Stream Double -> Stream Double -> Stream Double -> Stream Double
%scalar2dim x1 y1 x2 y2 = (x1 * x2) + (y1 * y2)
%
%normsq2dim :: Stream Double -> Stream Double -> Stream Double
%normsq2dim x y = scalar2dim x y x y
%
%norm2dim :: Stream Double -> Stream Double -> Stream Double
%norm2dim x y = sqrt (normsq2dim x y)
%
%det2dim :: Stream Double -> Stream Double -> Stream Double -> Stream Double -> Stream Double
%det2dim x1 y1 x2 y2 = (x1 * y2) - (x2 * y1)
%
%----------------------------------------
%-- Horizontal Criterion for Conflict Resolution
%----------------------------------------
\begin{lstlisting}[frame=single]
hor_rr :: Stream Double -> Stream Double -> Stream Double
hor_rr sx sy= (sqrt $ (normsq2dim sx sy) -
                      (minHorSep * minHorSep)) / (minHorSep)

horizontalCriterionForConflictResolution :: Stream Double ->
                Stream Double -> Stream Double -> Stream Double ->
                Stream Double -> Stream Bool
horizontalCriterionForConflictResolution sx sy e vx vy =
        ((scalar2dim sx sy vx vy) >=
               (e * (hor_rr sx sy) * (det2dim sx sy vx vy)))
        && (((det2dim sx sy vx vy) *e)  <= 0.0)
\end{lstlisting}

Formalizing the assumptions about the reliability of communicating
aircraft position data is ongoing work.  A case study formalizing what
it means for a UAS to behave erratically is planned as future
work.

%\begin{itemize}
%\item Safety-Critical that that are considered ultra-critical systems
% typically have detailed requirements
%
%\item established that only ``safety properties'' can be checked with
%  RV
% \item For adaptive systems where RV is the entrusted repository of
%  predictability and guarantor of safety we must make sure that we are
%  actually checking properties based on formal requirements.
% \item Need to make sure that the RV safety properties are derived
%  from requirements.
% \item Criteria based separation example an excellent example in that
%  has undergone extensive analysis both simulation and formal
%  analytical proof.  Assumptions are well spelled out.
% \item NASA is investigating sophisticated AI based  flight management
%  software. Such systems cannot be certified by traditional means, but
%  the criteria can an can serve as the basis for RV.
%\item EDSL we strive to build libraries allow specification in this
%  sort of notation.
%\end{itemize}
%
%
%The obvious specification is simply check to see if the aircraft have
%violated separation and take corrective action.  system level
%needs demanded a specification that the aircract will not come into
%conflict meaning separation is not lost in at some time $t>0.$

%These are expressed as a collection of formula
%involving the position and velocity vectors of the two
%aircraft.

%,NMH14ATIO