% !TEX root = ../zeth-protocol-specification.tex

\section{\zeth~Data Types}\label{zeth-protocol:zeth-data-types}

We begin by describing, and giving intuition about, the data types (see~\cref{preliminaries:structured-data}) used in \zeth.
We follow some design rationale from \zerocash~\cite{sasson2014zerocash}, and \zcash~\cite{zcashprotocol} in order to prevent the transaction malleability attack, and the Faerie Gold attack\cite[Section 8.4]{zcashprotocol}. We refer the reader to~\cref{appendix:trnm} for more details.

In what follows \Curve~represents a curve with scalar field $\FFx{\rCURVE}$, satisfying the requirements of \cref{instantiation:zksnark}. The specification is described in terms of this generic curve, with examples and notes relating to specific instances of interest (namely \BNCurve~and \BLSCurve, see \cref{chap:instantiation}).

\begin{description}
    \item[\zethNoteDType] Represents a note in \zeth. This data type consists of the note owner's public address \apk, identifier $\rrho$, randomness \noter{}~and value \notev.
        \begin{table}[H]
        \centering
        \begin{tabular}{ccc}
            Field & Description & Data type\\ \toprule
            $\apk$ & Note owner's paying key & $\BB^{\prfAddrOutLen}$\\ \midrule
            $\noter$ & Note randomness & $\BB^{\noterLen}$\\ \midrule
            $\notev$ & Note value & $\BB^{\zvalueLen}$ \\ \midrule
            $\rrho$ & Note identifier & $\BB^{\prfRhoOutLen}$\\ \bottomrule
        \end{tabular}
        \caption{\zethNoteDType~data type}\label{zeth-protocol:tab:zethnote}
        \end{table}
    \item[\jsInputDType] Denotes a \gls{joinsplit} input. It comprises the opening of a commitment $\cm{}$ which is in the set of leaves in the Merkle tree of $\mixer$ (i.e.~a $\zethnote$), its address $\mkaddr$ and authentication path $\mkpath$ on the contract's Merkle tree as well as the spending key $\ask$ of the note holder and the note nullifier $\nf{}$.
        \begin{table}[H]
        \centering
        \begin{tabular}{cp{20em}c}
            Field & Description & Data type\\ \toprule
            $\mkpath$ & Merkle authentication path to the commitment corresponding to the $\zethnote$ to spend & $(\FFx{\rCURVE})^{\mkTreeDepth}$ \\ \midrule
            $\mkaddr$ & Commitment address in the Merkle tree & $\BB^{\mkTreeDepth}$\\ \midrule
            $\znote$ & $\zeth{}$ note object & $\zethNoteDType$ \\ \midrule
            $\cm{}$ & $\zeth{}$ note commitment & $\FFx{\rCURVE}$ \\ \midrule
            $\ask$ & Note owner's spending key & $\BB^{\askLen}$\\ \midrule
            $\nf{}$ & Note nullifier & $\BB^{\prfNfOutLen}$\\ \bottomrule
        \end{tabular}
        \caption{\jsInputDType~data type}\label{zeth-protocol:tab:jsinput}
        \end{table}
    \item[\primInputDType]
    Represents the primary inputs used to generate the zk-SNARK proof $\zkp$. $\priminputs$ is a tuple defined as the current Merkle root \mkroot~of the Merkle tree maintained by $\mixer$, the input notes nullifiers $\nfs{} = \smalltuple{\nf{0}, \ldots, \nf{\jsin - 1}}$, the output notes commitments $\cms{} = \smalltuple{\cm{0}, \ldots, \cm{\jsout - 1}}$, the signature hash \hsig, the message authentication tags $\htags{} = \smalltuple{\htag{0}, \ldots, \htag{\jsin-1}}$ and the residual bits field $\resbits$, which aggregates the former's fields bits which could not be contained in a field element.
        \begin{table}[H]
        \centering
        \begin{tabular}{cp{20em}c}
            Field & Description & Data type\\ \toprule
            $\mkroot$ & Merkle root of the Merkle tree & $\FFx{\rCURVE}$ \\ \midrule
            $\nfs{}$ & Indexed set of nullifiers of the ``old'' notes to spend (see~\cref{instantiation:statement:pack} for definition of $\nfFLen$) & ${((\FFx{\rCURVE})^{\nfFLen})}^{\jsin}$\\ \midrule
            $\cms{}$ & Indexed set of commitments to the newly created notes & ${(\FFx{\rCURVE})}^{\jsout}$\\ \midrule
            $\hsig$ & Signature hash (non-malleability, see~\cref{appendix:trnm} and~\cref{instantiation:statement:pack} for definition of $\hsigFLen$)) & $(\FFx{\rCURVE})^{\hsigFLen}$\\ \midrule
            $\htags{}$ & Indexed set of message authentication tags (non-malleability, see~\cref{appendix:trnm} and~\cref{instantiation:statement:pack} for definition of $\htagFLen$)) & ${((\FFx{\rCURVE})^{\htagFLen})}^{\jsin}$\\ \midrule
            $\resbits$ & Residual bits corresponding to unpacked bits of former fields (see~\cref{instantiation:statement:pack} for definition of $\resBitsFLen$) & $(\FFx{\rCURVE})^{\resBitsFLen}$\\ \bottomrule
        \end{tabular}
        \caption{\primInputDType~data type}\label{zeth-protocol:tab:priminputs}
        \end{table}
    \item[\auxInputDType] Represents the auxiliary inputs used to generate the zk-SNARK proof \zkp. \auxinputs~is a tuple defined as \gls{joinsplit} inputs (i.e.~``old outputs to be spent''), the new \zethnotes{}, the joinsplit's randomness $\pphi$ as well the public values $\vin$ and $\vout$, the signature hash $\hsig$ and the message authentication tags $\htags{} = \smalltuple{\htag{0}, \ldots, \htag{\jsin-1}}$.
        \begin{table}[H]
        \centering
        \begin{tabular}{cp{20em}c}
            Field & Description & Data type\\ \toprule
            $\jsins{}$ & Indexed set of $\jsin$ \gls{joinsplit} inputs & $\jsInputDType^{\jsin}$\\ \midrule
            $\znotes{}$ & Indexed set of $\jsout$ newly created notes & $\zethNoteDType^{\jsout}$\\ \midrule
            $\pphi$ & The \gls{joinsplit} randomness (non-malleability, see~\cref{appendix:trnm}) & $\BB^{\phiLen}$\\ \midrule
            $\vin$ & Public input value to the \gls{joinsplit} & $\BB^{\zvalueLen}$\\ \midrule
            $\vout$ & Public output value to the \gls{joinsplit} & $\BB^{\zvalueLen}$\\ \midrule
            $\hsig$ & Signature hash (non-malleability, see~\cref{appendix:trnm}) & $\BB^{\crhhsigOutLen}$\\ \midrule
            $\htags{}$ & Indexed set of message authentication tags (non-malleability, see~\cref{appendix:trnm}) & ${(\BB^{\prfPkOutLen})}^{\jsin}$\\ \bottomrule
        \end{tabular}
        \caption{\auxInputDType~data type}\label{zeth-protocol:tab:auxinputs}
        \end{table}
    \item[\mixInputDType] Represents the set of inputs to the \mix~function of \mixer. The input of the \mix~function~is a tuple defined as the primary inputs \priminputs, the zk-proof \zkp, the ciphertexts of the newly created notes $\ciphers = \smalltuple{\ct_{0}, \ldots, \ct_{\jsout - 1}}$, a one-time signature $\sigma$ and the associated verification key \vk.
        \begin{table}[H]
        \centering
        \begin{tabular}{cp{20em}c}
            Field & Description & Data type\\ \toprule
            $\primInp$ & Primary input object associated with the zk-proof $\zkp$ & \primInputDType\\ \midrule
            $\zkproof$ & The zk-SNARK associated to the \zeth~statement (see~\cref{zeth-protocol:statement}) & $\zkpDType$ (see~\cref{instantiation:zksnark})\\ \midrule
            $\otssig$ & The one-time signature used to prevent transaction malleability (see~\cref{appendix:trnm}) & $\sigOtsDType$ (see~\cref{instantiation:otsig:data-types})\\ \midrule
            $\otsvk$ & The verification key associated with the signature $\otssig$ used to prevent transaction malleability (see~\cref{appendix:trnm}) & $\vkOtsDType$ (see~\cref{instantiation:otsig:data-types})\\ \midrule
            $\ciphers$ & Indexed set of ciphertexts of the newly generated notes & ${(\BB^{\encZethNoteLen})}^{\jsout}$ (see~\cref{instantiation:enc})\\ \bottomrule
        \end{tabular}
            \caption{\mixInputDType~data type}\label{zeth-protocol:tab:mixinputs}
        \end{table}
    \item[\mixEventDType] Represents the data emitted as an \ethereum{} event (\cref{sssec:ethereum-events}) during a successful execution of the \mix~function of \mixer{}. Clients are required to read this data and use it to update their representation of \mixer{}'s state.
      \begin{table}[H]
        \centering
        \begin{tabular}{cp{20em}c}
          Field & Description & Data type \\ \toprule
          $\mkroot$ & New root of Merkle tree of commitments & $\FFx{\rCURVE}$ \\ \midrule
          $\nfs{}$ & Nullifiers for input notes consumed & ${(\BB^{\prfNfOutLen})}^{\jsin}$ \\ \midrule
          $\cms{}$ & Commitments to the output notes & ${(\FFx{\rCURVE})}^{\jsout}$ \\ \midrule
          $\ciphers{}$ & Ciphertexts for the output notes & ${(\BB^{\encZethNoteLen})}^{\jsout}$ \\ \bottomrule
        \end{tabular}
        \caption{\mixEventDType~data type}\label{zeth-protocol:tab:mixoutputs}
      \end{table}
\end{description}


%! TEX program = pdflatex
\documentclass[journal]{IEEEtran}
\usepackage{siunitx}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{bm}
\usepackage[table,xcdraw]{xcolor}
\usepackage[pdftex]{graphicx}

\DeclareGraphicsRule{.ai}{pdf}{.ai}{}

\begin{document}

\title{QUIKS: an inexpensive motion tracking system}
\author{}
\maketitle
\begin{abstract}
Reducing cost for motion tracking system is essential for better virtual reality (VR) experience.
In this paper, we describe our low-cost system named QUIKS.
It is optimized for VR usage only.
Therefore, it can be constructed of inexpensive parts such as IMU and Cortex-M0+ MCU.
In addition, sensors not only sample motions but also calculate offset and apply transformation.
This lowers CPU usage of host PC.
We only spent about 10 dollars for tracking one joint.
In out tests, laptop PC without GPU can perform motion traccking and 3DCG rendering for 30 fps.
The quality and latency of motion is sufficient for VR experience.
\end{abstract}

\section{Introduction}
To capture the movement of a user is essential for the virtual reality (VR) experience.
With the increase of the number of VR user (including game player, streamer, etc.), perchasing motion tracking devices such as Vive tracker or Perception Neuron is thought to be a common case.
However, these products are still expensive and only a few user can perchase them.
Thus, a low-cost motion tracking system which is suitable for VR usage is needed.
Such a device should not track a motion much precisely or quickly.
That is, it is not suitable for the tasks like measurements and only produce 30 or 60 samples per second.
Also, it cannot be used with some particular motions which are difficult to track.

\section{Basis of motion tracking}
\subsection{Target software}
A motion data measured by a sensor is sent to some software to produce CG images for a user.
In this paper, we target Unity for that role because we can use it for free with few limitations.
However, most of our system can be reused for other software.

\subsection{Tracking method}
There are roughly two methods to track motions.
One is an optical method.
It can track almost every motions very precisely with low latency.
However, it needs multiple cameras and lots of calculation resources.
Thus, this method is not suitable for an inexpensive system.

The another is an inertial method.
It cannot track the absolute position of a body, but it is simple and costs less.
Therefore, we adopt this method.

The inertial method captures joint rotations and just assign them to a 3D model.
3DCG engine like Unity manages its internal model movements by rotations.
In general, inertial motion tracking can be achieved with following steps.
\begin{enumerate}
    \item Measure joint rotations by inertial measurement unit(IMU).
    \item Convert measured data for a 3DCG software.
    \item Assign joint rotations to a 3D model.
\end{enumerate}

\subsection{Quaternion}
A rotation in 3D space can be represented as the form of quaternion.
In this paper, we define a quaternion \(\bm{q}\) as (\ref{quat-def}).
\begin{equation}
    \bm{q} \overset{\mathrm{def}}{=} w + \mathbf{i}x + \mathbf{j}y + \mathbf{k}z = w + \begin{pmatrix}
        x \\
        y \\
        z
    \end{pmatrix} \label{quat-def}
\end{equation}
where \(w\) is the real part of a quaternion and \(x,\,y,\,z\) are the imaginary part.
\begin{table}[tb]
    \centering
    \caption{Quaternion multiplication} \label{quat-mult}
    \begin{tabular}{ccccc}
    \rowcolor[HTML]{9B9B9B}
                                           & 1              & \(\mathbf{i}\)  & \(\mathbf{j}\)  & \(\mathbf{k}\)  \\
    \cellcolor[HTML]{9B9B9B}1              & \(1\)          & \(\mathbf{i}\)  & \(\mathbf{j}\)  & \(\mathbf{k}\)  \\
    \rowcolor[HTML]{EFEFEF}
    \cellcolor[HTML]{9B9B9B}\(\mathbf{i}\) & \(\mathbf{i}\) & \(-1\)          & \(\mathbf{k}\)  & \(-\mathbf{j}\) \\
    \cellcolor[HTML]{9B9B9B}\(\mathbf{j}\) & \(\mathbf{j}\) & -\(\mathbf{k}\) & \(-1\)          & \(\mathbf{i}\)  \\
    \rowcolor[HTML]{EFEFEF}
    \cellcolor[HTML]{9B9B9B}\(\mathbf{k}\) & \(\mathbf{k}\) & \(\mathbf{j}\)  & \(-\mathbf{i}\) & \(-1\)
    \end{tabular}
\end{table}
Multiplication between each parts are defined as Table \ref{quat-mult}.
Also, (\ref{quat-norm}) always have to be valid.
When a quaternion holds this condition, it is called to be normalized.
\begin{equation}
    |\bm{q}| \overset{\mathrm{def}}{=} \sqrt{w^2 + x^2 + y^2 + z^2} = 1 \label{quat-norm}
\end{equation}
Sometimes we need the inverse (or complementary) quaternion \(\bm{q}^{-1}\) defined as (\ref{quat-inv})
\begin{equation}
    \bm{q}^{-1} \overset{\mathrm{def}}{=} \frac{w - \mathbf{i}x - \mathbf{j}y - \mathbf{k}z}{w^2 + x^2 + y^2 + z^2} = w - \mathbf{i}x - \mathbf{j}y - \mathbf{k}z \label{quat-inv}
\end{equation}

The rotation around the axis \(\bm{u} = (u_x,\,u_y,\,u_z)^\mathrm{T}\in\mathbb{R}^3\) by angle \(\theta\) can be expressed in (\ref{quat-rot}).
\begin{equation}
    \bm{q} = \cos\left(\frac{\theta}{2}\right) + \begin{pmatrix}
        u_x \\
        u_y \\
        u_z
    \end{pmatrix} \sin\left(\frac{\theta}{2}\right) \label{quat-rot}
\end{equation}
where \(|\bm{u}| = 1\).

The rotaion \(\bm{q}_2\bm{q}_1\) means the rotation applying \(\bm{q}_1\), then \(\bm{q}_2\).

Real world coordinate and 3DCG engine coordinate are often different.
Real world is represented as left-handed coordinate system and Unity uses right-handed one for example.
In this case, we should convert rotation axis \(\bm{u}\) to \(\bm{u}'\) with \(\bm{u}' = A\bm{u}\) where \(A\in\mathbb{R}^{3\times 3}\) is an transformation matrix.
Therefore, a quaternion \(\bm{q}\) can be converted to \(\bm{q}'\) by (\ref{quat-conv}).
\begin{equation}
    \bm{q}' = w + A\begin{pmatrix}
        x \\
        y \\
        z
    \end{pmatrix} \label{quat-conv}
\end{equation}

\section{Our system}
\subsection{Overview}
Figure \ref{block} shows the block diagram of our system, QUIKS.
\begin{figure}[tb]
    \centering
    \includegraphics[width=0.48\textwidth]{block.ai}
    \caption{Block diagram of QUIKS} \label{block}
\end{figure}
QUIKS is consists of multiple sensor nodes and a terminal node.
Each sensor node have an IMU and measures rotation of a joint.
The terminal node bridges PC and sensor nodes by converting USB and RS485 communication.
Basically, PC sends requests to each node and sensors return quaternions.
Every sensor node has a unique ID, which enables PC to communicate sensors separately.

\subsection{Electronic}
Sensor node is mainly consists of an IMU, a microcontroller unit(MCU), and an RS485 transceiver as shown in Figure \ref{circuit}.
\begin{figure}[tb]
    \centering
    \includegraphics{circuit.ai}
    \caption{Block diagram of sensor node} \label{circuit}
\end{figure}
We use ICM20948 from invensense for IMU.
This sensor provides Digital Motion Processing(DMP) function which enables a sensor to calculate imaginary parts of a quaternion from raw sensor values.
With this capability, we do not need to care about raw sensor value issues such as offset, drift, or random noise.
Therefore, our system can be built with a tiny software and we can reduce costs for MCU.
Measurement data quality is also guaranteed to be relatively good.

We use LPC802 from NXP for MCU.
This is the cheapest 32-bit MCU provided by NXP and one of the most cost-efficient ARM processor in the market.
The calculation resource of this MCU is limited, but it is enough to handle both the communicaton with PC and IMU.
Also, it offers great low-power capability which enables both the IMU and MCU be powered with small \SI{1.8}{\V} regulator.
The switch-matrix functionality of LPC802 makes wiring design easy and keep board size small.
And most importantly, it has ARM Cortex-M0+ core configured with some essential features like fast multiplication, software reset, and vector table offsetting.

Baud rate of RS-485 bus is \SI{460800}{bps} because the transciever IC supports up to \SI{500}{kbps}.
A UART packet is consists of 1 start bit, 8 bit data, and 1 stop bit.
Therefore, effective communication rate is \(\SI{460800}{bps} \times 8/10 = \SI{368640}{bps}\).

The quaternion request and return requires 80 bytes of payload, that is, 10 UART packets.
Thus, the number of quaternion transportable in every second is \(\SI{368640}{bps} / 80 = \num{4608}\).
If the frame rate is 30 fps, \(\num{4608} / 30 \sim 153\) quaternions can be handled in every frame.
This speed is enough to achieve full-body tracking.

\subsection{Software}
We developed software for sensor node and Unity.

Sensor node program is written in C and a few assembly.
The code uses lots of inline expansion to reduce subroutine call latancy since the MCU is not so powerful.
We do not use any RTOS because the software is small enough and the size of ROM/RAM cannot contain RTOS kernel/stack.

Unity program is written in C\#.
It abstructs sensor nodes and assign rotation from sensors to 3D model.
Unity will handle rotations and we do not need to be involved in 3DCG processing.

\section{Embedded software specification}
\subsection{Overview}
The software for sensor node boots with a bootloader, which provides a way to recover main program, then the bootloader branches to the main program.

\subsection{Bootloader}
The bootloader is the program which is launched firstly.
Figure \ref{boot-act} shows the activity diagram of it.
\begin{figure}[tb]
    \centering
    \includegraphics{bootloader.pdf}
    \caption{Activity diagram of the bootloader} \label{boot-act}
\end{figure}
It initializes some hardware and waits a magic packet for \SI{2}{seconds}.
If no packet is detected (normal case), it sets the interrupt vector and branches to the main program.

Firmware update sequence is performed as shown in Figure \ref{boot-upd}
\begin{figure}[tb]
    \centering
    \includegraphics{update.pdf}
    \caption{Firmware update sequence} \label{boot-upd}
\end{figure}
Update is performed for each \SI{64}{kB} flash pages.
PC sends a page of program and the sensor saves it in RAM.
Then, it sends the program back to the PC.
So PC side can validate whether the program is correctly sent or not.
If an error is detected, PC sends NG to the sensor and sends the same page again.
After all the program is flashed, the sensor will reboot and update finishes.

This update sequence still has a risk to flash a wrong program.
Clearly, it is impossible to \SI{100}{\percent} guarantee that the program is valid.
However, as long as the bootloader is valid, we can flash the main program again.

\subsection{Main program}
The main program is launched by the bootloader.
It obtains quaternions from the IMU, processes them suitable for Unity, then sends them to the PC.

The main program converts a quaternion \(\bm{q}_{\mathrm{IMU}}\) from the IMU to \(\bm{q}_{\mathrm{Unity}}\) for Unity as shown in (\ref{qconv}).
\begin{equation}
    \bm{q}_{\mathrm{Unity}} = T(\bm{q}_{\mathrm{IMUoffset}}\bm{q}_{\mathrm{IMU}})\bm{q}_{\mathrm{Unityoffset}} \label{qconv}
\end{equation}
Where \(T(\bm{q})\) represents the conversion between coordinates.

\(\bm{q}_{\mathrm{IMUoffset}}\) and \(\bm{q}_{\mathrm{Unityoffset}}\) are the offset rotations.
A user should first strike the same pose as 3D model, then initialize offsets as \(\bm{q}_{\mathrm{IMUoffset}} = {\bm{q}_{\mathrm{IMU}}}^{-1}\) and \(\bm{q}_{\mathrm{Unityoffset}} = \bm{q}_{\mathrm{Unity}}\).

\subsection{Real part of a quaternion}
ICM20948, the IMU we use, produces only imaginary part of a quaternion.
Therefore, we must calculate a real part in some way.
Since normalized quaternion holds (\ref{quat-norm}), \(w\) can be obtained through (\ref{quat-w}).
\begin{equation}
    w = \pm\sqrt{1 - x^2 - y^2 - z^2} \overset{\mathrm{def}}{=} \pm w_0 \label{quat-w}
\end{equation}
The problem is which sign should be used.
In short, both signs are fine.
From (\ref{quat-rot}), we can obtain (\ref{quat-wcos}).
\begin{equation}
    w = \cos\left(\frac{\theta}{2}\right) \label{quat-wcos}
\end{equation}
Therefore, \(\theta\) can be expressed as (\ref{quat-theta}).
\begin{align}
    \theta &= 2\cos^{-1}(w) = \begin{cases}
        2\cos^{-1}(w) & (w = w_0) \\
        2\pi - 2\cos^{-1}(-w) & (w = -w_0)
    \end{cases} \notag \\
    &= \pm 2\cos^{-1}(|w|) \label{quat-theta}
\end{align}
Thus, the final representation of \(\bm{q}\) becomes (\ref{quat-pmq}).
\begin{equation}
    \bm{q} = w + \bm{u}\sin\{\pm 2\cos^{-1}(|w|)\} = \pm w_0 \pm \bm{u}\sin(|\theta|) \label{quat-pmq}
\end{equation}
This means inverting \(w\) also inverts \(\theta\) and \(\bm{u}\).
Both rotations are clearly the same.
You can also validate this by checking the identity \(qq'^{-1} \equiv q^{-1}q' \equiv 1\) where the real part of \(q'\) is the negative of the one of \(q\).

\subsection{Fixed-point number}
ICM20948 represents real number as Q30, fixed-pont number.
In general, Q\(n\) means a fixed-point number point where fractional part has \(n\) bits.
Therefore, Q30 can represent the range \([-1:1)\).

In this paper, we express a Q30 number \(n\) as (\ref{q30-def}).
\begin{equation}
    n = 2^{-30}N \label{q30-def}
\end{equation}
Where \(N\) is the 32-bit integer representation of \(n\).
Since CPU can only handle \(N\), we must be careful about relationship between the two representations.

\subsubsection{Algebra}
In order to complete quaternion calculation (\ref{qconv}) and (\ref{quat-w}), we must define Q30 addition, subtraction, multiplication, square root.
Addition and subtraction are easily done with (\ref{q30-add}).
\begin{equation}
    n_1 \pm n_2 = 2^{-30}N_1 \pm 2^{-30}N_2 = 2^{-30}(N_1 \pm N_2) \label{q30-add}
\end{equation}
This means addition or subtraction in 32-bit integer representation can calculate the same operation in Q30 directly.

Q30 multiplication can be done as shown in (\ref{q30-mult}).
\begin{equation}
    n_1 n_2 = 2^{30}(2^{-30}N_12^{-30}N_2) = 2^{-30}N_1N_2 \label{q30-mult}
\end{equation}
This means multiplication can be done through shifting 32-bit integer product by 30 bits.
However, Cortex-M0+ core can only calculate lower 32 bits of a product efficiently.
Thus, we must transform the expression to achieve a good performance.

The easiest method can be expressed as (\ref{q30-fastmul}).
\begin{equation}
    n_1n_2 = \left(2^{-15}N_1\right)\left(2^{-15}N_2\right) \label{q30-fastmul}
\end{equation}
This implementation is very fast, but accuracy is only about the order of \(2^{14} \sim 10^{-4.2}\) in our test.

A more accurate method is based on the fact that multiplication of two 16-bit integers results a 32-bit product.
Thus, we can separate the whole multiplication into the four.
In order to do this, we define two 16-bit number \(N_{\mathrm{u}}\) and \(N_{\mathrm{l}}\) as shown in (\ref{q30-ul}).
\begin{equation}
    N \overset{\mathrm{def}}{=} 2^{16}N_{\mathrm{u}} + N_{\mathrm{l}} \label{q30-ul}
\end{equation}
Such that \(N_{\mathrm{u}}\) represents the upper 16-bit of \(N\) and \(N_{\mathrm{l}}\) represents lower.
Then, we define the Q30 multiplication as shown in (\ref{q30-mult2}).
\begin{align}
    n_1&n_2 = 2^{-30}\left(2^{16}N_{\mathrm{1u}} + N_{\mathrm{1l}}\right)\left(2^{16}N_{\mathrm{2u}} + N_{\mathrm{2l}}\right) \notag \\
    &= 2^2N_{\mathrm{1u}}N_{\mathrm{2u}} + 2^{-14}(N_{\mathrm{1u}}N_{\mathrm{2l}} + N_{\mathrm{1l}}N_{\mathrm{2u}}) + 2^{-30}N_{\mathrm{1l}}N_{\mathrm{2l}} \label{q30-mult2}
\end{align}
The accuracy of this method is about the order of \(2^{-27} \sim 10^{-8.1}\) in our test.
Clearly, the last part of the product shifted right by 30 bits is useless.
After emitthing this, we can obtain (\ref{q30-mult3}).
\begin{equation}
    n_1n_2 = 2^{2}N_{\mathrm{1u}}N_{\mathrm{2u}} + 2^{-14}(N_{\mathrm{1u}}N_{\mathrm{2l}} + N_{\mathrm{1l}}N_{\mathrm{2u}}) \label{q30-mult3}
\end{equation}

\subsubsection{Square root}
Square root can be generally obtained using Newton's method.
However, it needs Q30 division which cannot be executed fastly.
Therefore, we use taylor series for calculating Q30 square root.

According to the taylor's theorem, the \(k + 1\) differentiable function \(f(x)\) can be represented as (\ref{taylor}) around \(a\).
\begin{equation}
    f(x) = \sum_{n=0}^{k} \left.\frac{\mathrm{d}^n f}{{\mathrm{d}x}^n}\right|_{x=a}\frac{(x - a)^n}{n!} + R_{k+1}(x) \label{taylor}
\end{equation}
The remainder \(R_{n}(x)\) has a value of (\ref{remainder}).
\begin{equation}
     R_{n}(x) = \left.\frac{\mathrm{d}^n f}{{\mathrm{d}x}^n}\right|_{x=c}\frac{(x - a)^n}{n!} \label{remainder}
\end{equation}
where \(c\) is a real number between \(a\) and \(x\).
We use 4th ordered taylor series shown in (\ref{root-taylor}) and (\ref{r5}).
\begin{equation}
    \sqrt{x} = \sqrt{a} + \frac{x - a}{2\sqrt{a}} - \frac{(x - a)^2}{8a^{3/2}} + \frac{(x - a)^3}{16a^{5/2}} - \frac{5(x-a)^4}{128a^{7/2}} + R_{5}(x) \label{root-taylor}
\end{equation}
\begin{equation}
    R_5(x) = \frac{7(x-a)^5}{256c^{9/2}} \label{r5}
\end{equation}

(\ref{r5}) indicates that the distance between \(x\) and \(a\) induces the error.
Thus, it is essential to make \(x\) close to \(a\).
To achieve this, we can use an identity (\ref{root-shift}).
\begin{equation}
    \sqrt{x} \equiv 2^{-n/2}\sqrt{2^n x} \label{root-shift}
\end{equation}

Recalling that multiplying power of 2 is equivalent to bit shift, we can move a most significant set bit to a specific position.
If the most significant set bit of a Q30 number \(x\) is the bit 29, \(0.5 \leq x < 1.0\) holds.
A limitation of this method is \(n\) must be an even number bacause \(n/2\) is needed.

To shift the most significant set bit, we must know the number of clear bits from MSB.
This operation is called count-leading-zeros(CLZ).
In this paper, we define it as a function \(\operatorname{CLZ}(x)\).
\(\operatorname{CLZ}(\mathrm{0x0800000}) = 8\) for example.
\(n\) can be obrained using CLZ as shown in (\ref{nbycases}).
\begin{equation}
    n = \begin{cases}
        \operatorname{CLZ}(x) - 3 & (\operatorname{CLZ}(x)\operatorname{mod}2 = 1) \\
        \operatorname{CLZ}(x) - 2 & (\operatorname{CLZ}(x)\operatorname{mod}2 = 0)
    \end{cases} \label{nbycases}
\end{equation}
Then, the condition \(0.25 \leq 2^nx < 0.5\) or \(0.5 \leq 2^nx < 1.0\) holds.

\(a\) is chosen for different conditions to minimize the maximum error.
That is, the optimum value \(\hat{a}\) satisfies (\ref{ahat}) is chosen.
\begin{equation}
    \hat{a} = \underset{a}{\operatorname{argmin}}\;\underset{x}{\operatorname{max}}\;R_5(x) \label{ahat}
\end{equation}
We obtained 0.3685 and 0.737 for \(\hat{a}\) numerically.

\section{Testing results}
We connected 9 sensors to track both shoulders, elbows, hips, knees, and spine.
Host PC is MacBook Air 2019 model equipped with Dual-Core Intel Core i5 running at \SI{1.6}{\GHz} and RAM of \SI{16}{GB}.

\begin{figure}[tb]
    \centering
    \includegraphics[width=0.45\textwidth]{testing.jpg}
    \caption{Testing scene} \label{testing}
\end{figure}
Figure \ref{testing} shows the scene of testing.
It can be seen that 3D model in the display reflects the pose of the user.
CPU usage is about \SI{20}{\percent}.

\section{Conclusion}
We propose a low-cost motion tracking system optimized for VR experience.
It only spends about 10 dollars per joint and produces 30 samples per second.
The accuracy is enough to reflect real-world movements.

Our future work is to evaluate the system quantitatively.
Developing other nodes such as wireless terminal node or position tracking node is also our future work.

\appendix
Table \ref{list} lists components of sensor node and terminal node.
The price is in USD and if you order multiple units at once, prices can be lowered.

\begin{table*}[t]
    \centering
    \caption{Component list} \label{list}
    \begin{tabular}{cccc}
        \hline
        Part & Vendor & Model & Unit Price \\ \hline
        MCU & NXP & LPC802M001JDH16FP & \$1.29 \\
        IMU & TDK & ICM-20948 & \$8.04 \\
        Regulator & Microchip & MIC5504-1.8YM5-TR & \$0.11 \\
        RS-485 Transceiver & TI & THVD1500DR & \$1.51 \\
        USB to UART Interface & FTDI & FT234XD & \$2.04 \\
        \hline
    \end{tabular}
\end{table*}

\end{document}


\par
\section{Driver programs for the multithreaded functions}
\label{section:MT:drivers}
\par
%=======================================================================
\begin{enumerate}
%-----------------------------------------------------------------------
\item
\begin{verbatim}
allInOneMT msglvl msgFile type symmetryflag pivotingflag
           matrixFileName rhsFileName seed nthread
\end{verbatim}
This driver program reads in a matrix $A$ and right hand side $B$,
generates the graph for $A$ and orders the matrix,
factors $A$ and solves the linear system $AX = B$ for $X$
using multithreaded factors and solves.
Use the script file {\tt do\_gridMT} for testing.
\par
\begin{itemize}
\item
The {\tt msglvl} parameter determines the amount of output.
Use {\tt msglvl = 1} for just timing output.
\item
The {\tt msgFile} parameter determines the message file --- if {\tt
msgFile} is {\tt stdout}, then the message file is {\it stdout},
otherwise a file is opened with {\it append} status to receive any
output data.
\item
The {\tt type} parameter specifies a real or complex linear system.
\begin{itemize}
\item
{\tt type = 1 (SPOOLES\_REAL)} for real,
\item
{\tt type = 2 (SPOOLES\_COMPLEX)} for complex.
\end{itemize}
\item
The {\tt symmetryflag} parameter specifies the symmetry of the matrix.
\begin{itemize}
\item
{\tt type = 0 (SPOOLES\_SYMMETRIC)} for $A$ real or complex symmetric,
\item
{\tt type = 1 (SPOOLES\_HERMITIAN)} for $A$ complex Hermitian,
\item
{\tt type = 2 (SPOOLES\_NONSYMMETRIC)}
\end{itemize}
for $A$ real or complex nonsymmetric.
\item
The {\tt pivotingflag} parameter signals whether pivoting for
stability will be enabled or not.
\begin{itemize}
\item
If {\tt pivotingflag = 0 (SPOOLES\_NO\_PIVOTING)},
no pivoting will be done.
\item
If {\tt pivotingflag = 1 (SPOOLES\_PIVOTING)},
pivoting will be done to ensure that all
entries in $U$ and $L$ have magnitude less than {\tt tau}.
\end{itemize}
\item
The {\tt matrixFileName} parameter is the name of the files where
the matrix entries are read from.
The file has the following structure.
\begin{verbatim}
neqns neqns nent
irow jcol entry
...  ...  ...
\end{verbatim}
where {\tt neqns} is the global number of equations and {\tt nent}
is the number of entries in this file.
There follows {\tt nent} lines, each containing a row index, a
column index and one or two floating point numbers, one if real,
two if complex.
\item
The {\tt rhsFileName} parameter is the name of the files where
the right hand side entries are read from.
The file has the following structure.
\begin{verbatim}
nrow nrhs
irow entry ... entry
...  ...   ... ...
\end{verbatim}
where {\tt nrow} is the number of rows in this file
and {\tt nrhs} is the number of rigght and sides.
There follows {\tt nrow} lines, each containing a row index
and either {\tt nrhs} or {\tt 2*nrhs} floating point numbers,
the first if real, the second if complex.
\item
The {\tt seed} parameter is a random number seed.
\item
The {\tt nthread} parameter is the number of threads.
\end{itemize}
%-----------------------------------------------------------------------
\item
\begin{verbatim}
patchAndGoMT msglvl msgFile type symmetryflag patchAndGoFlag fudge toosmall
             storeids storevalues matrixFileName rhsFileName seed nthread
\end{verbatim}
This driver program is used to test the ``patch-and-go''
functionality for a factorization without pivoting.
When small diagonal pivot elements are found,
one of three actions are taken.
See the {\tt PatchAndGoInfo} object for more information.
\par
The program reads in a matrix $A$ and right hand side $B$,
generates the graph for $A$ and orders the matrix,
factors $A$ and solves the linear system $AX = B$ for $X$
using multithreaded factors and solves.
Use the script file {\tt do\_patchAndGo} for testing.
\par
\begin{itemize}
\item
The {\tt msglvl} parameter determines the amount of output.
Use {\tt msglvl = 1} for just timing output.
\item
The {\tt msgFile} parameter determines the message file --- if {\tt
msgFile} is {\tt stdout}, then the message file is {\it stdout},
otherwise a file is opened with {\it append} status to receive any
output data.
\item
The {\tt type} parameter specifies a real or complex linear system.
\begin{itemize}
\item
{\tt type = 1 (SPOOLES\_REAL)} for real,
\item
{\tt type = 2 (SPOOLES\_COMPLEX)} for complex.
\end{itemize}
\item
The {\tt symmetryflag} parameter specifies the symmetry of the matrix.
\begin{itemize}
\item
{\tt type = 0 (SPOOLES\_SYMMETRIC)} for $A$ real or complex symmetric,
\item
{\tt type = 1 (SPOOLES\_HERMITIAN)} for $A$ complex Hermitian,
\item
{\tt type = 2 (SPOOLES\_NONSYMMETRIC)}
\end{itemize}
for $A$ real or complex nonsymmetric.
\item
The {\tt patchAndGoFlag} specifies the ``patch-and-go'' strategy.
\begin{itemize}
\item
{\tt patchAndGoFlag = 0} --- if a zero pivot is detected, stop
computing the factorization, set the error flag and return.
\item
{\tt patchAndGoFlag = 1} --- if a small or zero pivot is detected,
set the diagonal entry to 1 and the offdiagonal entries to zero.
\item
{\tt patchAndGoFlag = 2} --- if a small or zero pivot is detected,
perturb the diagonal entry.
\end{itemize}
\item
The {\tt fudge} parameter is used to perturb a diagonal entry.
\item
The {\tt toosmall} parameter is judge when a diagonal entry is small.
\item
If {\tt storeids = 1}, then the locations where action was taken is
stored in an {\tt IV} object.
\item
If {\tt storevalues = 1}, then the perturbations are
stored in an {\tt DV} object.
\item
The {\tt matrixFileName} parameter is the name of the files where
the matrix entries are read from.
The file has the following structure.
\begin{verbatim}
neqns neqns nent
irow jcol entry
...  ...  ...
\end{verbatim}
where {\tt neqns} is the global number of equations and {\tt nent}
is the number of entries in this file.
There follows {\tt nent} lines, each containing a row index, a
column index and one or two floating point numbers, one if real,
two if complex.
\item
The {\tt rhsFileName} parameter is the name of the files where
the right hand side entries are read from.
The file has the following structure.
\begin{verbatim}
nrow nrhs
irow entry ... entry
...  ...   ... ...
\end{verbatim}
where {\tt nrow} is the number of rows in this file
and {\tt nrhs} is the number of rigght and sides.
There follows {\tt nrow} lines, each containing a row index
and either {\tt nrhs} or {\tt 2*nrhs} floating point numbers,
the first if real, the second if complex.
\item
The {\tt seed} parameter is a random number seed.
\item
The {\tt nthread} parameter is the number of threads.
\end{itemize}
%-----------------------------------------------------------------------
\item
\begin{verbatim}
testMMM msglvl msgFile dataType symflag storageMode transpose
        nrow ncol nitem nrhs seed alphaReal alphaImag nthread
\end{verbatim}
This driver program generates $A$, a
${\tt nrow} \times {\tt ncol}$
matrix using {\tt nitem} input entries, $X$ and $Y$,
${\tt nrow} \times {\tt nrhs}$ matrices,
is filled with random numbers.
It then computes $Y + \alpha*A*X$, $Y + \alpha*A^T*X$ or
$Y + \alpha*A^H*X$.
The program's output is a file which when sent into Matlab,
outputs the error in the computation.
\par
\begin{itemize}
\item
The {\tt msglvl} parameter determines the amount of output ---
taking {\tt msglvl >= 3} means the {\tt InpMtx} object is written
to the message file.
\item
The {\tt msgFile} parameter determines the message file --- if {\tt
msgFile} is {\tt stdout}, then the message file is {\it stdout},
otherwise a file is opened with {\it append} status to receive any
output data.
\item
{\tt dataType} is the type of entries,
{\tt 0} for real, {\tt 1} for complex.
\item
{\tt symflag} is the symmetry flag, {\tt 0} for symmetric,
{\tt 1} for Hermitian, {\tt 2} for nonsymmetric.
\item
{\tt storageMode} is the storage mode for the entries,
{\tt 1} for by rows, {\tt 2} for by columns,
{\tt 3} for by chevrons.
\item
{\tt transpose} determines the equation,
{\tt 0} for $Y + \alpha*A*X$,
{\tt 1} for $Y + \alpha*A^T*X$ or
{\tt 2} for $Y + \alpha*A^H*X$.
\item
{\tt nrowA} is the number of rows in $A$
\item
{\tt ncolA} is the number of columns in $A$
\item
{\tt nitem} is the number of matrix entries that are
assembled into the matrix.
\item
{\tt nrhs} is the number of columns in $X$ and $Y$.
\item
The {\tt seed} parameter is a random number seed used to fill the
matrix entries with random numbers.
\item
{\tt alphaReal} and {\tt alphaImag} form the scalar in the multiply.
\item
{\tt nthread} is the number of threads to use.
\end{itemize}
%-----------------------------------------------------------------------
\item
\begin{verbatim}
testGridMT msglvl msgFile n1 n2 n3 maxzeros maxsize seed type
         symmetryflag sparsityflag pivotingflag tau droptol
         nrhs nthread maptype cutoff lookahead
\end{verbatim}
This driver program tests the serial {\tt FrontMtx\_MT\_factor()}
and {\tt FrontMtx\_MT\_solve()} methods for the linear system $AX = B$.
The factorization and solve are done in parallel.
Use the script file {\tt do\_gridMT} for testing.
\par
\begin{itemize}
\item
The {\tt msglvl} parameter determines the amount of output.
Use {\tt msglvl = 1} for just timing output.
\item
The {\tt msgFile} parameter determines the message file --- if {\tt
msgFile} is {\tt stdout}, then the message file is {\it stdout},
otherwise a file is opened with {\it append} status to receive any
output data.
\item
{\tt n1} is the number of points in the first grid direction.
\item
{\tt n2} is the number of points in the second grid direction.
\item
{\tt n3} is the number of points in the third grid direction.
\item
{\tt maxzeros} is used to merge small fronts together into larger
fronts.
Look at the {\tt ETree} object for
the {\tt ETree\_mergeFronts\{One,All,Any\}()} methods.
\item
{\tt maxsize} is used to split large fronts into smaller
fronts.
See the {\tt ETree\_splitFronts()} method.
\item
The {\tt seed} parameter is a random number seed.
\item
The {\tt type} parameter specifies a real or complex linear system.
\begin{itemize}
\item
{\tt type = 1 (SPOOLES\_REAL)} for real,
\item
{\tt type = 2 (SPOOLES\_COMPLEX)} for complex.
\end{itemize}
\item
The {\tt symmetryflag} parameter specifies the symmetry of the matrix.
\begin{itemize}
\item
{\tt type = 0 (SPOOLES\_SYMMETRIC)} for $A$ real or complex symmetric,
\item
{\tt type = 1 (SPOOLES\_HERMITIAN)} for $A$ complex Hermitian,
\item
{\tt type = 2 (SPOOLES\_NONSYMMETRIC)}
\end{itemize}
for $A$ real or complex nonsymmetric.
\item
The {\tt sparsityflag} parameter signals a direct or approximate
factorization.
\begin{itemize}
\item
{\tt sparsityflag = 0 (FRONTMTX\_DENSE\_FRONTS)} implies a direct
factorization, the fronts will be stored as dense submatrices.
\item
{\tt sparsityflag = 1 (FRONTMTX\_SPARSE\_FRONTS)} implies an
approximate factorization.
The fronts will be stored as sparse submatrices, where
the entries in the triangular factors will be
subjected to a drop tolerance test --- if the magnitude of an entry
is {\tt droptol} or larger, it will be stored, otherwise it will be
dropped.
\end{itemize}
\item
The {\tt pivotingflag} parameter signals whether pivoting for
stability will be enabled or not.
\begin{itemize}
\item
If {\tt pivotingflag = 0 (SPOOLES\_NO\_PIVOTING)},
no pivoting will be done.
\item
If {\tt pivotingflag = 1 (SPOOLES\_PIVOTING)},
pivoting will be done to ensure that all
entries in $U$ and $L$ have magnitude less than {\tt tau}.
\end{itemize}
\item
The {\tt tau} parameter is an upper bound on the magnitude of the
entries in $L$ and $U$ when pivoting is enabled.
\item
The {\tt droptol} parameter is a lower bound on the magnitude of the
entries in $L$ and $U$ when the approximate factorization is enabled.
\item
The {\tt nrhs} parameter is the number of right hand sides to solve
as one block.
\item
The {\tt nthread} parameter is the number of threads.
\item
The {\tt maptype} parameter determines the type of map from fronts
to processes to be used during the factorization
\begin{itemize}
\item 1 -- wrap map
\item 2 -- balanced map
\item 3 -- subtree-subset map
\item 4 -- domain decomposition map
\item 5 -- improved domain decomposition map
\end{itemize}
See the {\tt ETree} methods for constructing maps.
\item
The {\tt cutoff} parameter is used for domain decomposition maps.
We try to construct domains (each domain is owned by a single
thread) that contain $0 \le {\tt cutoff} \le 1$ of the rows and
columns of the matrix.
Try to choose {\tt cutoff} to be {\tt 1/nthread}
or {\tt 1/(2*nthread)}.
\item
The {\tt lookahead} parameter controls the degree that a thread
will look past a stalled front in order to do some useful work.
{\t lookahead = 0} implies a thread will not look ahead, while
{\tt lookahead = k} implies a thread will look {\tt k} ancestors up
the front tree to find useful work.
Bewarned, while a thread is doing useful work further up the tree,
the stalled front may be ready, so large values of lookahead can be
detrimental to a fast computation.
In addition, a positive value of {\tt lookahead} means a larger
storage footprint taken by the factorization.
\end{itemize}
%-----------------------------------------------------------------------
\item
\begin{verbatim}
testQRgridMT msglvl msgFile n1 n2 n3 seed nrhs type
             nthread maptype cutoff
\end{verbatim}
This driver program tests the serial {\tt FrontMtx\_QR\_factor()}
and {\tt FrontMtx\_QR\_solve()} methods for the least squares problem
$\min_X \| F - A X \|_F$.
The factorization and solve are done in parallel.
\par
\begin{itemize}
\item
The {\tt msglvl} parameter determines the amount of output.
Use {\tt msglvl = 1} for just timing output.
\item
The {\tt msgFile} parameter determines the message file --- if {\tt
msgFile} is {\tt stdout}, then the message file is {\it stdout},
otherwise a file is opened with {\it append} status to receive any
output data.
\item
{\tt n1} is the number of points in the first grid direction.
\item
{\tt n2} is the number of points in the second grid direction.
\item
{\tt n3} is the number of points in the third grid direction.
\item
The {\tt seed} parameter is a random number seed.
\item
The {\tt nrhs} parameter is the number of right hand sides to solve
as one block.
\item
The {\tt type} parameter specifies a real or complex linear system.
\begin{itemize}
\item
{\tt type = 1 (SPOOLES\_REAL)} for real,
\item
{\tt type = 2 (SPOOLES\_COMPLEX)} for complex.
\end{itemize}
\item
The {\tt nthread} parameter is the number of threads.
\item
The {\tt maptype} parameter determines the type of map from fronts
to processes to be used during the factorization
\begin{itemize}
\item 1 -- wrap map
\item 2 -- balanced map
\item 3 -- subtree-subset map
\item 4 -- domain decomposition map
\item 5 -- improved domain decomposition map
\end{itemize}
See the {\tt ETree} methods for constructing maps.
\item
The {\tt cutoff} parameter is used for domain decomposition maps.
We try to construct domains (each domain is owned by a single
thread) that contain $0 \le {\tt cutoff} \le 1$ of the rows and
columns of the matrix.
Try to choose {\tt cutoff} to be {\tt 1/nthread}
or {\tt 1/(2*nthread)}.
\end{itemize}
%-----------------------------------------------------------------------
%-----------------------------------------------------------------------
\end{enumerate}


\chapter{Chi Squared Tests}

\section{Contingency tables}
\section{Fitting a theoretical distribution}
\section{Goodness of fit test}


\chapter{Gödel’s completeness theorem and the compactness theorem}


There is an ongoing tension between weak and strong logical
frameworks~\citep{DeBruijn91lf}, with which we can encode our
benchmarks. Weak frameworks are designed to accommodate advanced
infra-structural features for binders (HOAS/nominal syntax etc.) and
for judgments (hypothetical and parametric), but may struggle on other
issues, such as facilities for computation or higher-order
quantification/impredicativity. There are at least two coordinates in
which we can directly extend our benchmark, to further highlight this
dilemma:
\begin{itemize}
\item Logical relations for dependent types~\citep{Rabe:2013,AbelV14},
  up to the Calculus of Constructions. Here we need to go beyond
  first-order quantification, which is typically what is on offer in
  weak frameworks.
\item Proof by logical relation via
  \emph{step-indexing}~\citep{Appel:2001}. Here we have two issues:
  \begin{enumerate}
  \item the logical relation may even be harder to be accepted by the
    meta-language as an inductive definition
% in its original   formulation
than with simple types; in fact, the work around the
    negative occurrence of the defined relation cannot be based on
    structural induction on types, but it has to use some form of
    course-of-value induction.

\item   It involves a limited amount of arithmetic reasoning:
  \begin{quote}
    ``definitions and proofs have a tendency to become
  cluttered with extra indices and even arithmetic, which are really
  playing the role of construction line.'' (\cite{BentonH10}).
  \end{quote}
  This latter point may be problematic for frameworks such as Abella and Beluga,
  which do not (yet) have extensive libraries, nor computational mechanisms
  (rewriting, reflection) for those  tasks.
  \end{enumerate}
\end{itemize}

\section{Call for action}

We ask the community to submit solutions and
%  - Give a short (informal) talk at LFMTP to present their solution
we plan to invite everyone who does so to contribute
towards a joint paper discussing trade-offs between them. The authors
commit themselves to produce solutions in Agda,
Abella and Beluga. To resurrect the slogan from the PC, a small step
(excuse the pun) for us, a big step for bringing mechanized
meta-theory to the masses!

%%% Local Variables:
%%% mode: latex
%%% TeX-master: "poplr"
%%% End:

%  LocalWords:  DeBruijn lf Appel Coq Agda Abella HOAS Rabe
%  LocalWords:  impredicativity


\documentclass[12pt]{article}

\title{matchmaker-tm}
\author{Michael Tran}
\date{\today}

\begin{document}
\maketitle

\section{Outline}
This outlines the features that matchmaker-tm will have.

\subsection{Gathering Questions and Answers}
People will be matched based on their responses to high quality questions. These
questions will have at least two answers that can be distinctive of
compatibility.

Answers should be sorted on a spectrum. Those that choose the most left answer
should be very incompatible with those that choose the most right answer. This
could be subjective, so I will have this spectrum crowdsourced.

The answers should be sortable on the spectrum. \textbf{The containers should
not be sortable}; this will create conflicts between where the dragged object
should be placed.

\subsection{Representative Interface}
Representatives will be allowed to edit the Google Form containing the survey.
Representatives will authenticate survey partcipiants by allowing the user's
student ID to have access to the survey. All questions will already be final at
the time of the surveying. Users will have only one survey submission.

\subsection{Computing answers}
After the matchmaker has been available to the public for a good amount of time
and a solid amount of data is collected, each user will have their answers
compared against other user's answers. Answers have a difference from 0 to
1. A larger difference represents incompatibility, and a smaller difference
represents compatibility. This is done by comparing where the answers are
located on the spectrum. As stated, \textit{those that choose the most left answer
should be very incompatible with those that choose the most right answer.}

However, one question isn't indicative of everything, so the averages of the
differences are calculated and laid out onto a spreadsheet. A person that has a
smaller difference for a preference for another person is more compatible with
them. Additionally, question weights will be included for questions that are
mainly for fun and don't truly indicate incompatibility.

After averages are compiled, a user's preferences will ultimately be sorted from
the smallest difference to the largest difference.

\subsection{Data required}
In the survey form, each person will need to provide:
\begin{itemize}
    \item one's name,
    \item one's student ID,
    \item one's contact information for the matchmaker results to be reported to,
    \item one's contact information to be sent out to who one matched with,
    \item and the selected answers to survey questions.
\end{itemize}

\section{Spectrum Creation / Analysis}
Answers will be laid out on a spectrum, and this will allow for some easy math.
Questions with less choices imply that adjacent choices are more conflicting and
indicative of incompatibility. However, questions with many more choices will
have adjacent answer choices that have less of a difference between one
another.

\subsection{Spectrum Examples}
A question with two choices will have the biggest difference, as if Person A chooses the
first choice and Person B chooses the second choice, there is lower compatibility.
On the other hand, a question with one hundred choices will have less of an
impact, as differences between adjacent choices will only be 0.01. If Person A
chooses the first choice and Person B chooses the second choice (right next to
the first choice), then there is still a chance both can be very compatible.

\subsection{Question Weights}
Question weights can be included. These are multipliers. Multiplying a
difference by a number greater than 1 exaggerates the difference. Multiplying a
difference by a number less than 1 minimizes the difference. A difference that
is exaggerated truly indicates incompatibility, while a difference that is
minimized means that the answer to the question is mostly just for fun.

The following format will accept question weights:

\begin{verbatim}
    (Decimal) [question]
\end{verbatim}

An opening parenthesis is the first character provided, with the decimal
following it, and a closing parenthesis with a space before the question. This
is to be as unambiguous as possible.

\subsection{Output Format}
The output of questions and answers will be formatted like the following
example:

\begin{verbatim}
    What animal do you like as a pet?
    Cats | Birds | Dogs
\end{verbatim}

Vertical bars will act as delimiters. Answers can be loaded from this format,
and proposed answers will be submitted in this format. Ultimately, I will gather
data from all the spectrums and create the most appropriate one.

\subsection{Creating the spectrum interface}
Vanilla javascript and html should allow for dragging and dropping answers into
a row, and javascript automatically sorts the answers. This interface will
allow for ease of creating survey questions and submission of the answers.
Collecting answers will be done before the matchmaking is open to everyone.

Questions should be revised by others after they are made. Questions and answers
can populate the form with a previous answer file. This allows for adding of
answers and possible revision of the question.

\section{Generation and Usage of the Survey}
I will get diffs manually. Proposed spectrums will be compared and I will
ultimately decide upon the best one for people to answer to.

The survey form's metadata will be generated with nodejs on my own client. This
metadata will need to be moved into a directory for matchmaking.

\subsection{Output of answers}
From the Google Sheet integrated with the Google Form, the index for each
answer will need to be found from a Map. A list of indices will be made, and it
should be in order so that questions correspond.

Header information should also be specified. The CSV file may have headers in
different columns depending on who creates the survey. Have a file specifying
it so the extra data can be parsed appropriately.

\section{Data Computation}
I will be using Java for data computation. First, the survey file will be read.
The information needed is the number of answers for each question. Then, the
weights file will be read indicating each question's weight.

Each users' choice is an index of an answer for each question. The choice
will have a calculated value that is the index divided by the difference between
the number of answers and 1. Answer indices start at 0, so this will allow for
answers on opposite ends of a spectrum to have a difference of 1. Differences
will then be multiplied by the weight of the question.

\subsection{Data Output}
The final data spreadsheet will be a table with each person's compatibility
with one another. This spreadsheet will be kept private. CSVs are supported by
regular sheet readers like Excel and Calc.

\end{document}


\chapter{Introduction}
\label{c:intro}

Attention plays an important role in human vision. For example, when
we look at an image, our eye movements comprise a succession of {\em
fixations} (repetitive positioning of eyes to parts of the image)
and {\em saccades} (rapid eye jump). Those parts of the image that
cause eye fixations and capture primary attention are called {\em
regions of interest} (ROIs). Studies in visual attention and eye
movement have shown that humans generally only attend to a few ROIs.
Detecting these visually attentive regions in images is challenging
but useful in many multimedia applications, such as automatic
thumbnail cropping, object recognition, content-based image
retrieval, adaptive image compression and automatic browsing in
small-screen devices.

Many algorithms have been proposed for automatic ROI detection in
images. Unfortunately, these methods were often evaluated only on
specific and small data sets that are not publicly available. The
lack of published {\em benchmarks} makes experiments non-repeatable
and quantitative evaluation difficult. However, as recommended by
the latest ACM SIGMM retreat, repeatable experiments using published
benchmarks are important for advancing the multimedia research
field~\cite{Rowe:2005:ASR}.

\begin{figure}
\centering
\includegraphics[width=0.45\textwidth]{kl}
\caption{kl-distance}
\label{kl}
\end{figure}

\begin{table}[t]
\begin{center}
\begin{tabular}{lcc}

\hline
                    &  {\small Itti's method}     & {\small Fuzzy growing}    \\
\hline
{\small Precision}           &  0.4475    & 0.4506 \\
{\small Recall}              &  0.5515    & 0.5542 \\
\hline

\end{tabular}
\caption[Evaluation of FOA sets]{\small Evaluation of FOA sets. } \label{t:FOA}
\end{center}
\end{table}


\section{How To create a new Force Field}

In SOFA, the Force Field already existing are located in the namespace sofa::component::forcefield. They derive from the core class sofa::core::componentmodel::behavior::ForceField. It is templated by the type of elements you want to model. It can be a deformable object of 1,2,3 or 6 dimensions, or rigid bodies of 2 and 3 dimensions.\\
The simplest way to implement your own ForceField is :
\begin{enumerate}
 \item make it derive from sofa::core::componentmodel::behavior::ForceField
 \item implements the following virtual functions: addForce, getPotentialEnery. Others virtual functions exist like  addDForce, addDForceV (if you want to make dynamics), and you should read the doxygen documentation about ForceField.
 \item as with all the others component you might create, you have to add it to the project. \\Edit \$SOFA\_DIR/modules/sofa/component/component.pro, and add the path to new files in the section HEADERS and SOURCES.
 \item Edit \$SOFA\_DIR/modules/sofa/component/init.cpp and add your new forcefield to the list. This step is compulsory for Windows system, and does the linking of a new component to the factory. If you forget it, your component won't be created at initialization time.
\end{enumerate}

The method addForce computes and accumulates the forces given the positions and velocities of its associated mechanical state.\\
If the ForceField can be represented as a matrix, this method computes
    $$ f += B v + K x $$
     This method is usually called by the generic ForceField::addForce() method.


\documentclass{article}
\usepackage{leonine,amsmath,amssymb,amsthm,graphicx,algorithm,algorithmic}
\setkeys{Gin}{width=\linewidth,totalheight=\textheight,keepaspectratio}
\graphicspath{{graphics/}}
% Prints a trailing space in a smart way.
\usepackage{xspace}
% Inserts a blank page
\newcommand{\blankpage}{\newpage\hbox{}\thispagestyle{empty}\newpage}
% \usepackage{units}
% Typesets the font size, leading, and measure in the form of 10/12x26 pc.
\newcommand{\measure}[3]{#1/#2$\times$\unit[#3]{pc}}

\theoremstyle{definition}
\newtheorem{pred}[thm]{Prediction}

\def\ent{=\joinrel\mathrel|}

\title{Cerebellum} \author{Eric Purdy}

\begin{document}

\maketitle

\section{Overview}

We posit that the cerebellum as a whole tries to predict the output of
the cerebrum, and takes over the performance of activities that are
sufficiently predictable. Ito \cite{ito} posits a similar theory
(specifically, the inverse models from his paper).

\begin{figure}
\includegraphics[width=\linewidth]{computation.png}
\caption{Computations performed by the cerebellum}
\label{fig-computation}
\end{figure}

\begin{algorithm}
\caption{$cerebellum(\mathbf{command}^{(t-1)},\mathbf{context}^{(t-1)}, \mathbf{train}^{(t-1)})$}
\begin{algorithmic}
  \FNSTATE $\mathbf{state}^{(t-1)}$

  \FOR{$i = 0$ to $n$}
  \STATE{$\mathbf{pos\_state}_i^{(t)} \from positive\_state\left(\mathbf{state}^{(t-1)}, \mathbf{command}^{(t-1)}, \mathbf{context}^{(t-1)} \right)$}
  \ENDFOR
  \FOR{$j = 0$ to $m$}
  \STATE{$\mathbf{pos\_output}_j^{(t)} \from positive\_output\left(\mathbf{state}^{(t-1)}, \mathbf{command}^{(t-1)}, \mathbf{context}^{(t-1)} \right)$}
  \ENDFOR

  \COMMENT{Compute simple features}
  \FOR{$k = 0$ to $K$}
  \STATE{$\mathbf{feat}_k^{(t)} \from feature\_layer\left( \right)$}
  \ENDFOR

  \COMMENT{For each cluster of features, compute average activation}
  \FOR{$l = 0$ to $L$}
  \STATE{$\mathbf{cluster\_avg} \from cluster\_avg\left( \right)$}
  \ENDFOR

  \COMMENT{All cluster elements are lessened by the cluster average}
  \FOR{$k = 0$ to $K$}
  \STATE{$\mathbf{feat\_nms}_k^{(t)} \from subtract\_avg\left(
    \mathbf{feat}_k^{(t)},
    \mathbf{cluster\_avg}_l^{(t)}\right)$}
  \ENDFOR

  \COMMENT{Negative pathway is a collection of perceptrons on $\mathbf{feat\_nms}$}
  \FOR{$p = 0$ to $P$}
  \STATE{$\mathbf{neg\_output}_p^{(t)} \from negative\_output\left(
    \mathbf{feat\_nms}^{(t)}\right)$}
  \ENDFOR

  \COMMENT{Negative pathway output filters outputs of the positive pathway}
  \FOR{$j=0$ to $m$}
  \STATE{$\mathbf{output}_j^{(t)} = \begin{cases}1 & \mathbf{pos\_output}_j^{(t)} = 1, \forall p\in THING_j\ \mathbf{neg\_output}_p^{(t)}=0 \\ 0 & \mbox{otherwise}\end{cases}$}
  \ENDFOR
  \FOR{$i=0$ to $n$}
  \STATE{state filtering}
  \ENDFOR

  \STATE{---}
  \STATE{$\mathbf{pstate}_i^{(t)} \from \sigma\left(\theta_i +
    \sum_j w_{ij} \mathbf{state}_j^{(t-1)} + \sum_k x_{ik} \mathbf{context}_k^{(t-1)}\right)$}

  \RETURN $\mathbf{output}^{(t)}$
\end{algorithmic}
\label{alg-inside}
\end{algorithm}

\section{Positive Pathway}

Logistic regression.

\section{Feature Layer}

Learning in the feature layer occurs in an unsupervised manner, and
the feature layer together with non-maximal suppression is treated in
the chapter on The Granular Layer.

\end{document}


\subsubsection{Quality control of SNP array data}
\label{sec:QCchip}
%TODO: generate .miss and .multiple from Illumina files and flip according to reference sequence.
We will use SNP array data to generate a haplotype scaffold to assist in phasing of the refined sequence data derived genotypes. The SNP array data to be used for the haplotype scaffold has to undergo quality control. Some of the data has already undergone QC as part of the AGVP\cite{Gurdasani2015} and 1000G\cite{1000G2012}. Haplotype scaffolds are already publicly available for the 1000G populations. For the remaining new datasets we apply the same QC protocol that was used for the AGVP. We apply the QC individually to each population.

Low quality variants that map to multiple regions within the human genome or don't map to any region are removed. Duplicate variants genotyped on the chip are also excluded during filtering. We allow variants, which are not at the intersection between different platforms.

Samples with a call rate below 98\% and a heterozygosity greater than 3 SD from the mean are filtered sequentially. Sex check is carried out on the chromosome X SNPs with PLINK1.07 using default F values of \textless0.2 for males \textgreater0.8 for females. Samples not meeting the thresholds are removed. Following this SNP filtering is carried out across the remaning samples, and sites called in less than 98\% of samples are removed from each population. Sites not in Hardy Weinberg equilibrium ($p<10^{-7}$) will also be removed from each population. Identity by descent (IBD) is measured within each population and related individuals (IBD\textgreater0.05) are removed using an algorithm that retains the maximum number of indviduals by removing individuals related to the largest number in the dataset iteratively. For individuals related to equal numbers in the dataset, samples with lower call rates are preferentially removed.

Following the above QC process, principal component analysis i carried with EIGENSOFT5.0 for each population to inspect the data for outliers.


\section{Results} \label{sec:5}
\thispagestyle{plain} % surpress header on first page

The study is greatly governed by computer problems. On the one hand, DCDP models are computationally demanding, which is determined by their structure\citep{keane2011ChapterStructuralEstimation}. On the other hand, QBSM has no closed form expression to solve, thus they can only be numerically solved using MC simulation. For example, even DLR method is used, the number of function evaluations for computing QGSM for with $d$ inputs are $ N = N(dM + 1)$. To simplies the analyis, Figure \ref{fig:5} displays the estimate results of QBSM with 30 individuals and 3 chosen parameters. Even so, it is still computationally burdensome. However our aim with this quantile based sensitivity analysis of the structural econometric model is not to perform an in-depth analysis of the results, but rather to show how these statistical tools provide a better understanding of model complexity. \\

\noindent
 Table \ref{fig:6} displays QBSM of three carefully picked parameters, and through these poorly converged results we can still conclude that the parametric uncertainty in model prediction in previous literature has been overlooked. The red lines indicate the normalized QBSM of return to an additional year of schooling$\alpha_{11}$, while the blue lines indicate the normalized QBSM of return for collage education $\beta_1$. The red lines indicate the normalized QBSM of home rewards $\gamma_0$. $Q_{i}^{(1)}$ and  $Q_{i}^{(2)}$ corresponds to the measure listed in \eqref{eq:16} and \eqref{eq:17}, respectively. PDF of the output Y is symmetric(Figure \ref{fig:5}) lead to a symmetric behavior of $Q_i$ around $\alpha =0.5$. The general ranking of variables determined by QBSM is: $\alpha_{11}$, $\beta_1$, $\gamma_0$. Measures $Q_i$ approximately reaches the minimum for $\beta_1$ and $\gamma_0$ and and maximum for $\alpha_{11}$ at $\alpha=0.6$. When the quantities level $\alpha$ is very small or large, a larger sample size will required to achieve acceptable accuracy. However, the performance of these parameters around $\alpha = 0.5$ is quite stable, with $\alpha_{11}$ account for the greatest variability and $\gamma_0$ is nearly have no influence on the model uncertainty.


% !TEX root =../thesis-letomes.tex

\chapter{Software Engineering}

The project has been defined by a relatively wide scope of disciplines that we have worked on. Mathematic derivations, astrophysics intuition, machine learning, high-performance computing, and software engineering. The last of those particularly because of the simultaneous need for both rapid prototyping and strong performance characteristics. Essentially, we needed to be able to test a plethora of different ideas to answer all our questions, but since the computations are so relatively expensive, we wanted to build a few solid modules that could be treated as black boxes for our overarching purposes. Essentially, we rewrote the simulator from the precursor project \cite{Saxe2015} from the ground up, with a focus on modularity and extensibility. This was done on basis of the model inherent in the original, to give us a consistent "ground truth" with which to compare our new simulator.

\section{Software Architecture Overview}
Our system is organized as two general spheres with an interface between them. One sphere is the ES algorithm and its various auxiliary functions, and the other is the orbsim module, which takes care of the astrophysics aspect; turning launch parameters into trajectories. Between them is an interface that turns machine learning parameters into astrophysics parameters, and the reverse. Visualized in \cref{fig:software_architecture}, this architecture lets us keep variable names consistent with the literature for both domains. The multidisciplinary nature of this project and its authors made this a boon.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{fig/software_architecture}
    \caption{The architecture of the project, ES module inputs parameters to orbital simulator, which returns scores and trajectories. Not pictured are the many auxiliary analysis scripts which also use orbsim}
    \label{fig:software_architecture}
\end{figure}

\section{Software Architecture}
\subsection{Orbsim Module}
We have arranged our code into a python module that offers us some nice modularity and extensibility, at least within the bounds of our relatively specific problem. What follows is an explanation of the architecture of this module.

We packaged the new simulator into its own python module, installable with pip, to ensure proper reproducibility across machines and different python configurations. We then implemented the simulator into our ES framework, which we had previously developed using mock data, and tuned its launch parameters to let it function as a decent search strategy for the lunar case. As we pursued this goal, performance became a severe issue. The ES algorithm wants to compute trajectories thousands or millions of times, and that was not feasible with each trajectory taking \SIrange{60}{300}{\second} to compute, so we delved into the simulator again in order to optimize it with regards to runtime.

\subsubsection{Abstractions}

The simulator is a numerical solver for an analytical problem, implementing algorithms that have been defined through that analysis. Thus, we have taken care to keep nomenclature consistent between the simulator and its analytical foundations (i.e. consistency between ``code and paper''). This is complicated somewhat by the fact that we are applying Evolution Strategies as our search strategy, which carries with it its own set of nomenclature from the machine learning world. We have kept the two things largely separate, with the ES module interacting with the simulator through a relatively simple interface. This works fine, since ES is a black-box optimizer anyway.

\subsubsection{Simulator module}

Here, we have the interface between ES portion and simulator.

\noindent The function \texttt{launch\_sim} takes launch parameters that the ES algorithm uses for its optimization (in the form of a decision vector \(\psi\)), reformulates them, and starts a simulation based on them. It then returns a delta-v for that single run, along with the associated saved path. That delta-v is, as mentioned before, our fitness function, so based on that result, ES can continue to do its job.

\subsubsection{Integrators Module}

This contains the main loop of our simulator algorithm, along with subfunctions for the individual steps.

\subsubsection{Equations of Motion Module}

This module contains all coupled equations of motion and a ton of different convenience functions that compute some intermediate equation for use in the main algorithm. They are hidden away here to reduce clutter in \texttt{Integrators.py}.

\subsection{The ES module}
There are several different variations of the ES module, each executing the algorithm in a slightly different way. They are all conducting black-box optimization on a decision vector that describes the initial parameters for a trajectory. The basic, and first one works by calling the orbsim module every time the algorithm wants to find the fitness of a given point in our parameter space. It then waits a few seconds, depending on how long the given simulation takes, and uses the returned value as the function to minimize. The algorithm is parallelized to work on however many CPU cores are available, with different starting conditions.

We found out quickly that this approach was not reasonable for rapid iteration on the parameters of the ES algorithm, so we leveraged the fact that the angle of the burn vector is relatively unimportant, and is theoretically sub-optimal at any value but 0, to flatten the problem space to two dimensions and eventually compute the matrix seen in \cref{fig:golf_course_s1024}. We then made another implementation that simply made a lookup in that matrix, truncating its $\psi$ coordinates to integers. This let us actually make changes and see their effect quickly. This was the critical step necessary to make progress on the question of whether ES is a good method for the problem, and it did in fact not happen until relatively late into the project, with more of our effort in this regard having gone towards optimizing and developing the CUDA version.

The algorithm defines a list of starting points $\psi_{0..k}$, corresponding to k multi-start individuals. They are sampled from a set of disjoint bounds and applied to the evolution function. The evolution function  The fitness function takes the decision vector \(\psi\) as argument, calls on orbsim to run a simulation with the contained parameters, and returns the result of that simulation: The \(\Delta v\) value, or if the rocket did not hit the target, how close it came at the minimum, \(d_{min}\). The fitness is a single scalar, so the fact that that scalar can mean two different things (a change in velocity or a distance) gives some problems for estimating gradients in our space. We mitigate this by penalizing an unsuccessful mission heavily, by adding an upward bias and weight.

We then define the ES algorithm with an \texttt{evolve} function, which executes a single run of the algorithm. This function takes a population \(\psi\): A collection of decision vectors. For each vector at step t \((\psi_{i,t})\), \texttt{evolve} creates several jittered copies \(\epsilon\) of those vectors, akin to creating a gaussian point cloud in the 3-dimensional problem space that the vectors span. Each of the points in \(\epsilon\) is then run through the fitness function, and the weighted average over \(\epsilon\) -- according to fitness score -- is deemed as the direction for the next point \(\psi_{i,t+1}\). We then take a step in that direction, modified by our learning rate \(\alpha\). This process repeats, until the algorithm has converged upon a local minimum, or until some maximum number of steps have been taken.

For the points in \(\epsilon\), we took an adaptive approach, in the interest of performance. \(\epsilon_i\) should contain enough points to get a decent estimation of the gradient around \(\psi_i\). That number is of the order \num{1e3}. This implies a large number of fitness evaluations, and with our relatively expensive fitness function, we needed to compromise a bit. We let the fitness function accept a custom limit on its runtime, and let evaluations \(F(\theta_\epsilon)\) terminate after a number of steps that was two orders of magnitude lower than that for \(F(\theta_\psi)\). For such evaluations, we also increased the error tolerance commensurately, since it would otherwise mean that those paths would be only one a fraction of  the length, which would have been completely useless.

This adaptive precision approach was not a reasonable compromise, we found, since the paths diverged much too quickly from each other, and in particular because they diverged on their absolute length as well, puzzlingly. With the creation of the pre-computed fitness matrix, this method became obsolete.

This function is run in parallel on many cores, each with a different set of starting conditions \(\psi\). When all threads have finished, the best results are extracted and plotted, giving their \(\Delta v\).

Originally, we took care of parallelization with PyGMO's method of creating an archipelago, i.e a collection of islands. Each island would then run the algorithm on their own separate populations. Each island is assigned its own thread of execution. This convenient metaphor and easy implementation was the main draw of PyGMO for us, but we found that it introduced a significant performance overhead, and furthermore did not allow customization to the degree that we required. We therefore reimplemented the parallelization with less abstraction towards the end of the project.

\subsection{Early Adaptive Measures}
With our original, relatively unrestrained boundaries (two full circles, and a wide range of thrust power), we naturally saw a very large amount of failures. Many were cheap, computationally such as those that shot directly into the earth. Others were very expensive, like ones that were only barely above escape velocity, and just orbited the earth until the iteration counter maxed out. The model also wasted a lot of time looking at paths that were unmitigably bad: shooting at full power directly away from the moon, for example. With small perturbations of $\psi$, those paths see very little in the way of improvement, so for an inordinate number of evolution cycles, such paths would languish in that neighborhood, never getting better. Our human intuition tells us, of course, that it's more feasible to simply kill that candidate, and re-randomize. However, this has the effect of selecting for individuals that converge very quickly. That, as it turns out means selecting for Hohmann transfers, essentially, since they are much less sensitive to perturbations. We are not looking for the straight-forward solutions, but rather the unexpected ones. The paths that were successful in \cite{Saxe2015} were a hairs breadth in either direction from spinning off into oblivion, at least from the perspective of an algorithm working with a black-box simulator.

This means that it is at least principally important that we do not discard individuals for performing poorly initially, given the very 'all-or-nothing' nature of the problem space. We did, however, in the interest of computing time, try to nudge them on a little, by implementing a relatively naive system of adaptive learning rate and adaptive jitter spread. If an individual performs very poorly, we use our knowledge that its neighbors in a wide area will likely perform similarly, and increase its search radius $\sigma$ by some factor. This means that it will have a higher chance of selecting some jitter points that show some improvement, either by getting it closer to the target or by hitting it. Then, once the cloud of jittered points $\epsilon$ has been weighted, and our direction has been found, the step $\alpha$ that we take in that direction is commensurately increased as well. The idea is essentially a softer version of the notion of killing under-performing individuals, mentioned above, since if the scaling factor for $\alpha$ and $\sigma$ were very large, that would be essentially equivalent to re-initializing the point entirely. This method at least retains the idea of giving the mavericks a chance to find that one crazy solution that comes out of nowhere. This was a naive non-standard way of conduction variational optimization. The idea is good, but there are formal, empirically proven ways to do this properly.

We also restricted the search boundaries, though we were hesitant to do so overly. Shooting directly into the earth is not expensive, computationally (it terminates in one cycle), but it is pretty pointless to try, so we restricted the burn angle to only shoot outwards from earth. Additionally, we scaled back the maximum power of our burn so we wouldn't go rocketing into space with an impulse that would see us never coming back. Okay, not exactly rocket science so far (it is, actually, but not idiomatically at least). One other bad type of path that ended up taking a lot of our computing time was paths that fired roughly perpendicular to our starting angular velocity. If our burn vector was too short, we would be recaptured by earth, but instead of going back into an orbit, we would 'stall', and come straight back down, again hitting the earth. If the burn vector was high magnitude, we would either sail off into space, or in the lucky event we were captured by the moon, simply have conducted a really inefficient Hohmann-like transfer. This would become the flattened, pre-computed search space seen in \cref{fig:golf_course_s1024}. Restricting the search space with outside knowledge of the problem was and is always valuable when doing machine learning.

\section{High Performance Computing}
ES is an approach that requires a large number of evaluations of its fitness function with each iteration. This means, that for a problem like ours, where each fitness evaluation requires a significant numerical simulation to complete, performance optimization is critical. The update function is as such:

\begin{empheq}{align}
    \label{eq:Feval}
    \psi_{t+1} = \alpha\frac{1}{n\sigma}\sum_{i=1}^{n}F(\psi_t + \sigma\epsilon_i)\epsilon_i
\end{empheq}

With $F(x)$ corresponding to a fitness evaluation with parameters x. as such, each iteration requires $n$ fitness evaluations. Proper choice of $n$ varies with problem dimensionality; the higher, the better a gradient estimator we get. For our low-dimensional problem, 30 is decent, but 100-200 is noticeably smoother.
Performance optimization is two-dimensional in this case. Decrease the time taken by each fitness evaluation, and parallelize; execute several fitness evaluations at the same time. The good thing about ES is that the many fitness evaluations are independent, within the iteration of the algorithm at least. Since we are multi-starting, we evaluate multiple sets of iterations in parallel as well. The algorithm is thus incredibly well-suited for parallelism: We can utilize thousands of parallel execution threads with approximately linear efficiency gain.

\subsection{Numba}

Python is a notoriously slow language, due to its paradigm of not compiling code, but simply running on the script itself; with Numba however, we can mitigate this problem, without sacrificing too much of the very high productivity that Python offers. We attempted to have Numba optimize our code for us, but found that our "proper software engineering" philosophy was preventing this from working to any meaningful effect. Numba is only able to give significant performance increases if working with basic types: Integers, Floats, Strings, and lists of those, more or less. As soon as complicated data structures are introduced into the mix, Numba drops to `object mode', where it attempts to find subsections of the function that are purely defined with basic types. This mode was useless for us simply making the code run slower due to the overhead of the compilation.

In order to get the performance increases that the problem required, we had to make some sacrifices in our idealized architecture, ditching any complicated return types, any use of exceptions and the \texttt{Planet} class that we were using to contain our data. This was a painful sacrifice, since it prevented us from simply calling the algorithm with a named celestial body, and instantly have all the relevant parameters in place for computing a trajectory. Instead, we have been forced to split it into several functions with duplicated code, and a significant increase in visual and lexicographical complexity; a large reason for the redesign happening in the first place. Regrettable, but a valuable learning experience.

All this heart-wrenching work was not for nothing however, since with the pre-compilation of the symplectic Störmer-Verlet algorithm in place, we were seeing speedups of a factor 200-300.

Once we had this architecture in place, and giving the same results as the original simulator, we were able to make the first running versions of the ES algorithm on real data, though as it turned out it was still too slow. Fitness evaluations took on the order of 3-5 seconds each, prohibiting any true at-scale ES.

\subsection{CUDA}

Attempting to gain additional performance from our simulator, we tried to run it on GPUs. Given that the fitness evaluations are completely independent in ES, it is a great candidate for massive parallelization. This is also doable with Numba, but requires an even more granular refactoring of the simulator code, and conscious management of data transfer between CPU and GPU, since the communication overhead associated with GPU work can very easily eclipse the performance gained by the massive parallelization. This data transfer rate is very small for the ES algorithm, however, since we do not need to communicate the actual path, but rather just fitness and a boolean hit/no hit value. We began by looking at a performance optimized version for CPU, but its speed was not enough, even on the DTU HPC CPU clusters, that have two \SI{2.8}{\GHz} 10-core Intel Xeon processors. They are nothing to scoff at by themselves, but performance was still a limiting factor for much of the project, so it made sense to build a GPU version. In the end, most of the experimentation was conducted on a pre-computed fitness grid, to improve the iteration time, with experiments being tested on real-valued versions of the algorithm on GPU.

\section{Unit Testing} \label{sec:unit-testing}
For quite a few weeks of this MSc thesis project, our 3D simulator gave nonsense results when we tried running a simple circular orbit around the sun. Instead of staying in the circular orbit, it immediately plummeted into the sun in 2160 iterations of the program faster than the speed of light, as shown in \cref{fig:r4b-bug}. Something was clearly wrong with the program, and with so many lines of code containing lengthy equations of motion, coordinate transformation, initial conditions etc., it was very difficult to spot exactly where the problem was. After spending days on looking for the mistake, we decided to take a more systematic approach, and starting \emph{unit testing} the critical parts of the code, and steadily increased the \emph{test coverage} of the program.

\begin{figure}[H]
    \centering
    \includegraphics[width=0.50\linewidth]{fig/r4b-bug.png}
    \caption{3D plot of an early buggy 3D simulator. The yellow dot in the middle marks the Sun, Earth's orbit is traced in blue, Mars orbit in Red and the spacecraft trajectory traced in black. Instead of staying in orbit around the Earth, the spacecraft accelerated hard towards the sun in the very first time step iteration, crashing the program after 1473 iterations, just before reaching the sun faster than the speed of light.}
    \label{fig:r4b-bug}
\end{figure}

\subsection{What is unit testing?}
Complex computer software usually undergo various kinds of testing in its development cycle. For a computer program such as ours with lots of implemented mathematical formulae, the most relevant kind of systematic testing is \emph{unit testing}. In unit testing, various units of code are tested in isolation against some ``ground truth'', i.e. some test values that are obtained externally or manually written up. The units are often, as in our case, the \emph{functions} of the program.

\subsection{Our Mathematica-Python Unit Testing Workflow}
Our simulator program is written in the Python programming language, most of it inside functions. For example, we have a function that iterates a single coordinate or momentum a single time step forward, a function that converts positions or velocity vectors between different coordinate system, and a function that calculates circular orbital speed or period.

The whole point of doing this is that \textbf{Mathematica provides nice visual formatting that allows mathematical equations to be pretty much identical to the complicated mathematical formulae written in hand or LaTeX}. Thus we can easily visually compare the implemented equations in Mathematica and in handwriting / LaTeX, and see that they match up. To give an example, here in \cref{fig:unit-testing-sympletic-euler-step-python-1,fig:unit-testing-sympletic-euler-step-python-2} is the implementation of the \texttt{} function in Python:

\begin{figure}[H]
    \centering
    \includegraphics[width=0.90\linewidth]{fig/unit-testing-sympletic-euler-step-python-1.png}
    \caption{The main function of \texttt{symplectic\_euler\_step()} in Python.}
    \label{fig:unit-testing-sympletic-euler-step-python-1}
\end{figure}

\begin{figure}[H]
    \centering
    \includegraphics[width=0.90\linewidth]{fig/unit-testing-sympletic-euler-step-python-2.png}
    \caption{Function \texttt{get\_Bdot\_R(), which is just one of the sub-routines of \texttt{symplectic\_euler\_step()}}, in Python.}
    \label{fig:unit-testing-sympletic-euler-step-python-2}
\end{figure}

And here in \cref{fig:unit-testing-sympletic-euler-step-mathematica} is the same function in Mathematica:

\begin{figure}[H]
    \centering
    \includegraphics[width=1.0\linewidth]{fig/unit-testing-sympletic-euler-step-mathematica.pdf}
    \caption{The same function \texttt{symplecticEulerStep()} in Mathematica, which is obviously way more compact and readable than the Python counterpart.}
    \label{fig:unit-testing-sympletic-euler-step-mathematica}
\end{figure}

The point of comparison of \cref{fig:unit-testing-sympletic-euler-step-python-1,fig:unit-testing-sympletic-euler-step-python-2,fig:unit-testing-sympletic-euler-step-mathematica} above is that \emph{compactness} and \emph{readability} of the Mathematica version, thanks in large part to the 2D input syntax; Mathematica is simply a language built for handling mathematics really well. On the other hand, Mathematica doesn't scale well to larger and more complicated programs, performing on GPUs, using version control etc., where Python excels. But it does mean that Mathematica is a great application for testing your Python code – or rather providing test values that the Python code can be tested up against.

\subsection{The Python-Mathematica Testing Workflow}

The testing workflow can be described as such:

\begin{enumerate}
	\item \textbf{Write some piece of functionality as a function in Python.} It could be a mathematical formula, algorithm or just data-processing function.
	\item \textbf{Re-implement the same functionality Mathematica.} Ideally, one can use a built-in function if possible, such as when converting between spherical coordinates and cartesian coordinates. The benefit of Mathematica over Python is the aforementioned 2D display of input, which makes mathematical expressions much more compact and readable compared to Python code.
	\item \textbf{Export the test results (JSON) of various pairs of input/output.} Depending on the function a list of input can be a mix of random input, interesting edge-case input and invalid input. For functions with many input arguments, Mathematica can vary each input according to a list of numbers, and then generate all possible combinations of input arguments. \texttt{JSON} stands for JavaScript Object Notation, and is a standard file format for storing structured data. In this case, we used it for storing pairs of input and output for various functions.
	\item \textbf{Import test results (JSON) in python test script.} We used a Python test framework called pytest to set up automated tests that imported test data, and ran the test ``Output from Python-version-of-function(input) == Output from Mathematica-version-of-function(input)''.
	\item \textbf{Run tests using pytest and Coverage.py}. Coverage.py is a tool for measuring \emph{code coverage} of our Python program, meaning it can show what parts of the code is executed during running of all unit tests, both percentage-wise on an overall level, on file level and even line-by-line within code files, see \cref{fig:Coverage-py,fig:Coverage-py-file}. This was useful for systematically ensuring that all critical parts of the 3D simulator had been tested, and allowed everyone to directly see the progress in the debugging process.
\end{enumerate}

\begin{figure}[ht]
    \centering
    \includegraphics[width=0.90\linewidth]{fig/Coverage-py.png}
    \caption{Coverage.py generates a HTML file showing the \emph{code coverage} of the testing suite on a subset of the code. When we reached 70\% coverage, we were satisfied with the degree of certain of correctness of the critical parts of the code, and could verify that the 3D simulator now game the expected results in various scenarios, so we stopped testing then.}
    \label{fig:Coverage-py}
\end{figure}

\begin{figure}[ht]
    \centering
    \includegraphics[width=0.90\linewidth]{fig/Coverage-py-file.png}
    \caption{Clicking a file shown in previous \cref{fig:Coverage-py} shows details of which lines have test coverage in the given file. Here, no test covers the case of $v > 180$ in the function $\texttt{radians\_in\_domain().}$}
    \label{fig:Coverage-py-file}
\end{figure}

During the debugging process we found a $+$-sign and $*$-sign that was mixed up, and a parenthesis that was closed at the wrong place, but was syntactically correct. Following this, the 3D simulator started to work as expected during basic tests of closed circular orbits etc.