\section{Methods}
\label{sec:methods}

A self-organizing recurrent neural network (SORN) was used to learn specific input patterns, given by a defined Markov chain. It was checked how well the network was able to represent the given information. \acs{sorn} uses three plasticity rules to learn the input patterns:

\begin{itemize}
\item \ACF{stdp}
\item \ACF{sn}
\item \ACF{ip}
\end{itemize}

The time in the network is discrete and there is no refractory period modeled for the neurons. In most cases $\Nsim = 20$ networks were simulated and results were averaged.

% Zitiere grand averaging paper

The source code for all the simulation experiments is based on source code from \textcite{hartmann2015s}, which is written in \texttt{Python 2.7}. The team provided a small framework for SORN simulation experiments (\href{https://github.com/chrhartm/SORN}{https://github.com/chrhartm/SORN}). A lot of code was added or rewritten, especially regarding the evaluation of the network behavior, such that it fitted to the experiments, performed in this thesis.

\subsection{Network structure}

The reservoir consists of $N^E$ excitatory and $N^I$ inhibitory neurons. Additionally, some excitatory neurons where randomly connected to a specific number of input neurons $N^U = n$, which equals the number of Markov states. The weights $w_{ij} \in [0,1]$ between the neurons were initialized randomly. The network is build with three types of connections:

\begin{itemize}
\item Excitatory-excitatory weights: $W^{EE}$, where $\sum_{j} w^{EE}_{ij} = 1$
\item Inhibitory-excitatory weights: $W^{EI}$, where $\sum_{j} w^{EI}_{ij} = 1$
\item Excitatory-inhibitory weights: $W^{IE}$, where $\sum_{j} w^{IE}_{ij} = 1$
\item Input-excitatory weights: $W^{EU}$, where $w^{EU}_{ij} = 1$ for all $i,j$
\end{itemize}

As already explained in section \ref{sec:sorn}, there are no connections modeled between inhibitory neurons. On average, $10\%$ of the possible connections in $W^{EE}$ were realized. Hence, the matrix $W^{EE}$ is sparsely connected. The matrices $W^{IE}$ and $W^{EI}$ are fully connected. Note that the weights are normalized using equation \eqref{eq:sn}, such that the incoming weights to a neuron sum up to one.

In section \ref{sec:sorn}, figure \ref{fig:sorn} shows a sketch of the structure of the reservoir network. In this thesis $n = 4$ states were used. Hence, the state space consists of $S = \{x_1, x_2, x_3, x_4\} = \{A, B, C, D\}$. Three states are no enough to test a variety of combinations, while five or more states are already to complicated for a systematic approach. Therefore, four states seemed to be a good compromise. The four states are represented with neuron populations, called clusters. In most cases the network consists of $N^E = 200$ and $N^I = 40$. Furthermore $N_{x_i}^U = 10$ input connections are used per state or input neuron.

\subsection{Modification of the intrinsic plasticity}
\label{sec:ip-mod}

While the \acs{stdp} and the \acs{sn} where applied analog to \textcite{lazar2009sorn}, the \acs{ip} rule was modified. In the original \acs{sorn} model, the target rate $H_\IP$ was fixed. \textcite{hartmann2015s} found out that the model is more robust, if the average firing rate differs slightly between neurons. Therefore, the $H_\IP$ value from equation \eqref{eq:ip} was replaced by an individual $H_i^\IP$ target rate for every neuron $i$, resulting in

\begin{equation}
\label{eq:ip-ind}
T_i(t+1) = T_i(t) + \eta_\IP \left( x_i(t) - H_i^\IP \right).
\end{equation}

To get individual values, an average value $\bar H_\IP = 2\cdot N^U/N^E$ was used. It equals the original heuristic from \textcite{lazar2009sorn}. Additionally, a stochastic term $\varepsilon_i^\IP$ was added to the target rate of every neuron. It is uniformly \acs{iid} on an interval of $a = \sigma^\IP = 0.01$ and $b = -a = -\sigma^\IP = -0.01$. Therefore, $\varepsilon_i^\IP \sim \text{Uni}[a,b] = \text{Uni}[-\sigma^\IP,+\sigma^\IP] = \text{Uni}[-0.01,0.01]$. Together,

\begin{equation}
\label{eq:hip-ind}
T_i(t+1) = T_i(t) + \eta_\IP \left( x_i(t) - \bar H^\IP + \varepsilon_i^\IP \right)
\end{equation}

\nomenclature{$\bar H^\IP$}{Average target rate of IP mechanism}
\nomenclature{$\sigma^\IP$}{Target rate range of IP mechanism}

is the rule that was applied in the used network.

\subsection{Network training and testing}
\label{sec:train}

The network was trained in 3 phases:

\begin{itemize}
\item Self-organizing phase (plastic training phase)
\item Training phase (no-plastic training phase)
\item Testing phase
\end{itemize}

\paragraph{Self-organizing phase}

In the self-organizing phase (or plastic training phase) the input neurons are active, depending on the current sample $x_i$ from the Markov chain. In this phase all three plasticity rules are active. While STDP actively adjusts the weights, SN stabilizes the activity around every neuron without loosing the relative relationships between the neurons. IP also helps to ensure the stability of the network, since it adjusts thresholds of neurons depending on their past activity. The update of the network activity follows equation \eqref{eq:update-excitatory} for the excitatory neurons and equation \eqref{eq:update-inhibitory} for the inhibitory neurons.

During self-organizing, the information from the Markov chain is stored in the network and is coded in the weights between the neurons. Biologically, the encoding in the weights corresponds to the encoding in the strength of the synapses. In most cases the self-organizing phase was applied for $T_\plastic = 50,000$ steps. As shown below in the results, e.g. in figure \ref{fig:mc1-training}, latest after $50,000$ steps no further improvement can be noticed, in most cases much sooner.

\paragraph{Training phase}

In the training phase (or no-plastic training phase) the input neurons are still active and they still depend on the current state $x_i$ of the Markov chain. In this phase only \ac{ip} is active. Therefore, the stability of the network is ensured further on. \ac{stdp} is not active and thus also \ac{sn} is not necessary. Hence, the weights stay fixed in the training phase, meaning that $W^{EE}_{ij}(t) = W^{EE}_{ij}$ in equation \eqref{eq:update-excitatory}, where the rest of the network update equals the self-organizing phase.

The main role of the training phase is to find out how the states are represented in the activity patterns of the network after the weight structure was learned. After a burn in phase, \textcite{hartmann2015s} used the last $\tilde{T}_\compare = 2500$ steps of the no-plastic training phase for classification (see section \ref{sec:state-classification}). In the results section, it is shown that this fixed value causes a small bias, which was solved when choosing $\tilde{T}_{\text{compare}}$ more flexible, which is explained below in section \ref{sec:state-classification}.

The training phase needs enough steps to cover $\Tnoplastic > \tilde{T}_\compare$. However, the training was mostly chosen as $\Tnoplastic = 50,000$ no-plastic training steps. There are four reasons for this high value:

\begin{itemize}
\item The computational effort is very low in the training phase.
\item The number of no-plastic training steps do not influence the performance of the network (see appendix \ref{sec:appendix:noplastic}).
\item After switching off the self-organizing phase (namely the \ac{stdp} and \ac{sn}), there should be a short burn-in period. \textcite{hartmann2015s} used $\Tnoplastic = 20,000$, which seemed to be enough in all cases.
\item The updated classification algorithm, mentioned above, implies a flexible size of $\tilde{T}_\compare$, depending on the Markov chain. This change caused that a longer training phase is necessary in some cases. $\Tnoplastic = 50,000$ was enough for all applications.
\end{itemize}

\paragraph{Testing phase}

In the testing phase the input neurons are not active any more, there is no input at all. It means that $\bm v(t) = 0$ for all $t \in \{\Tevo+1, ..., \Tevo+\Ttest\}$, where $\Tevo = T_\plastic + \Tnoplastic$. Also \ac{stdp} and \ac{sn} is still switched off, as it was in the training phase before. \ac{ip} is still active. Beside the stabilizing character of the \acs{ip}, especially in the testing phase, it is responsible for spontaneous activity (see section \ref{sec:ip}). In the testing phase, the spontaneous activity patterns of the network are observed over time.

Since no input is given in this phase, it is necessary to classify the spontaneous activity patterns to a Markov state. For that purpose, the representations from the no-plastic training phase are used. A detailed explanation of the classification mechanism is given below.

The number of steps while testing has to take into account a short period of burn in. After the training phase, the input is switched off and therefore the network has to find equilibrium again, using just spontaneous activity from \acl{ip}. Mathematically, the burn in time depends on the learning rate of the \acs{ip} mechanism $\eta_\IP$. This relation is evaluated in appendix \ref{sec:appendix:eta}. In most cases the burn in phase is relatively short and $\Ttest > 20,000$ is sufficient.

\subsection{Classification of states}
\label{sec:state-classification}

In the last phase, the testing phase, it will be necessary to classify a Markov state $x_i$ to the activity pattern $\bm x(t)$ in every step of time. The activity patterns from the testing phase are compared to some representative activity patterns in the end of the no-plastic training phase.

\paragraph{Selection mechanism from Hartmann}

In \textcite{hartmann2015s} a fixed amount of $\tilde{T}_\compare = 2500$ steps in the training phase was used as representative activity patterns. From those steps the number of occurrences for every state $\tilde{T}_\compare^{x_i}$ was calculated. Let

\begin{equation}
\begin{split}
\tilde{R}_\compare &= \left\{ \bm x_u(T_\plastic+T_\noplastic-\tilde{T}_\compare), ..., \bm x_u(T_\plastic+T_\noplastic) \right\}\\
&= \left\{ \bm x_u(T_\evo-\tilde{T}_\compare), ..., \bm x_u(T_\evo) \right\}
\end{split}
\end{equation}

be a set of the last $\tilde{T}_\compare$ activity patterns in the no-plastic phase, where $u \in \{x_1, ..., x_n\}$ indicates the given input for the specific step. In order to avoid confusion, it is important to remind that the vector $\bm x_u(t)$ denotes the activity pattern of the excitatory neurons at time $t$ with input $u$ and the scalar $x_i$ denotes a state of the Markov chain. The obtained activity patterns $\tilde{R}_\compare$ are filtered by Markov state, denoted by $\tilde{R}_\compare^{x_i} = \{ \tilde{R}_\compare | u = x_i \}$. The size of this set is defined as $\tilde{T}_\compare^{x_i} = |\tilde{R}_\compare^{x_i}|$, which was mentioned above.

The minimum of those occurrences $T_\compare^\mini = \min\{\tilde{T}_\compare^{x_1}, ..., \tilde{T}_\compare^{x_n}\}$ was chosen as the number of steps for every state. If the number of representative activity patterns per state would differ, those with a higher amount of steps have a higher chance to be classified, since the amount of representative patterns cover more possible cases. Altogether, $n \cdot T_\compare^\mini$ steps are available for comparison, where $n$ is the number of Markov states or input neurons. The corresponding activity pattern for a specific state is denoted by $R_\compare^{x_i}$. To clarify how it works, a short example is provided:

Assume just two possible states $A$ and $B$, thus $n=2$. Further, assume that a Markov chain exists which provides a stationary distribution with $\pi_A = 2/5$ and $\pi_B = 3/5$. A likely sampling would result in $\tilde{T}_\compare^A = 997$ (where $\tilde{T}_\compare \cdot \pi_A = 2500 \cdot 2/5 = 1000$ would be expected) and $\tilde{T}_\compare^B = 1503$ (where $\tilde{T}_\compare \cdot \pi_B = 1500$ would be expected). Finally, in testing phase are more representative activity patterns of $B$ than $A$ available to compare with. Hence, $B$ has an advantage, compared to $A$, and activity patterns in the testing state will eventually more likely be classified as $B$. To avoid this possible bias, the minimum of both values is taken and $T_\compare^\mini = \min\{\tilde{T}_\compare^A, \tilde{T}_\compare^B\} = \min\{997,1503\} = 997$ is the number of steps which are used for \emph{every} state to compare with, meaning that $T_\compare^\mini = T_\compare^A = T_\compare^B$. Figure \ref{fig:classification-old} illustrates a similar example.

\begin{figure}
    \centering
    \begin{subfigure}{\textwidth}
    	\centering
        \includegraphics[width=\textwidth]{methods/classification_old}
        \vspace{5pt}
        \caption{A fixed number of potential activity patterns is chosen. In this example $\tilde{T}_\compare = 25$, meaning that the last $25$ steps in time are taken into account. In this period, state $A$ occurs $\tilde{T}_\compare^A = 8$ times and state $B$ $\tilde{T}_\compare^B = 17$ times. To ensure that the later classification of the spontaneous activity pattern have the same chance to be classified, $T_\compare^\mini = T_\compare^A = T_\compare^B = 8$ is chosen for both, state $A$ and $B$.}
        \vspace{20pt}
        \label{fig:classification-old}
    \end{subfigure}
    \begin{subfigure}{\textwidth}
    	\centering
        \includegraphics[width=\textwidth]{methods/classification_new}
        \vspace{5pt}
        \caption{Since the $T_\compare^\mini$ from the solution from \textcite{hartmann2015s} varies between input patterns with different probabilities, another bias needs to be taken into account when different learning patterns should be compared. Therefore, independent of the probability structure, $T_\compare = T_\compare^A = T_\compare^B = 10$ are chosen for all states. The search for those states begins at the end of the no-plastic training phase.}
        \vspace{20pt}
        \label{fig:classification-new}
    \end{subfigure}
    \caption[Selection mechanism for activity pattern representations]{Selection mechanism for activity pattern representations. In both illustrations, the first block is the plastic training phase, the second block represents the no-plastic training phase and the last block is the testing phase. Since input is given in the first two blocks, they perform with evoked activity. The last block has no input anymore and runs only with spontaneous activity from \acs{ip}. The example network is trained with two states: $A$, represented by the red lines, and $B$, drawn in green, where $\pi_A = 2/5$ and $\pi_B = 3/5$. Every line represents a point in time. The opaque lines are the selected steps, which are used as representative activity patterns for the specific state of the Markov chain. The activity of the chosen steps is collected in $R_\compare$ in general and in $R_\compare^A$ \& $R_\compare^B$ specifically. The first graphic shows the method provided by \textcite{hartmann2015s}, the second graphic illustrates the method which is applied in the network of this thesis.}
    \label{fig:classification}
\end{figure}

\paragraph{Updated selection mechanism}

While evaluating the network, another bias was found. The algorithm from \textcite{hartmann2015s} makes sure that the classification remains unbiased, if the same input pattern is chosen. But if different patterns are compared, the classification mechanism depends on the probabilities of the states. If one state in the input pattern is very rare, $T_\compare^\mini$ will be very small and therefore all activity patterns in the testing phase have just few representative patterns in the no-plastic training phase to compare with. On the other hand, if all states are equally probable, $T_\compare^\mini$ is quite high compared to the rare case. The performance between those two cases will eventually differ, just because of the number of available representative states in no-plastic training phase. In terms of Markov chains: If Markov chains with different stationary distributions are learned by the network and compared with each other, it is necessary to keep the number of $T_\compare^\mini$ constant for all chains. In section \ref{sec:input-dep-per} this effect is illustrated and discussed in detail.

An approach, which was chosen in this thesis, is to fix the number of representing steps for every state. A value of $T_\compare = 500$ was chosen. Independent of the probability of the state, starting from the end, $500$ representing activity patterns for every state were collected. If a state is very unlikely, for example with a probability of $\pi_{x_i} = 0.05$, for $500$ occurrences, $10,000$ steps are necessary in average to find enough representations for that state. Therefore, the $\Tnoplastic = 50,000$ was chosen, as indicated before. An illustration of this selection mechanism is shown in figure \ref{fig:classification-new}.

\paragraph{Classification mechanism}

After the representational activity patterns $R_\compare$ are collected with equal length $T_\compare^{x_1} = ... = T_\compare^{x_n}$, independent of the probabilities of the Markov chain, it remains to describe the classification mechanism for the activity patterns in the testing phase. First, a distance measure, the \emph{Hamming distance} is defined.

\begin{definition}[Hamming Distance]
Let $\Omega$ be an alphabet and $\bm x = (x_1, ..., x_N) \in \Omega^N$, $\bm y = (y_1, ..., y_N) \in \Omega^N$ two words with length $N \in \N$. Then the \emph{hamming distance} between those words $\bm x$ and $\bm y$ is defined as:

\begin{equation}
\label{eq:hamming}
d(\bm x, \bm y) := |\{i \in \{1, ..., N\} : x_i \neq y_i\}|
\end{equation}
\end{definition}

For the network holds that $\Omega = \{0,1\}$. The Hamming distances between every step in the testing phase $\bm x(t_\test) \in R_\test$, where $R_\test = \{ x(T_\evo + 1), ..., x(T_\evo + T_\test) \}$, and all steps in $\bm x(t_\compare) \in R_\compare$  are calculated. In that case the Hamming distance is defined as

\begin{equation}
\label{eq:hamming-applied}
d(\bm x(t_\test), \bm x(t_\compare)) = |\{ i \in \{ 1, ..., N^E \} : x_i(t_\test) \neq  x_i(t_\compare) \}|.
\end{equation}

\begin{table}[!b]
\centering
\footnotesize
\begin{tabular}{l|llllllllll|c}
$x_i$ & \multicolumn{10}{c|}{$\bm x(t_\compare)$} & $d\left(\bm x(t_\test), \bm x(t_\compare) \right)$ \\
\hline
A & \textbf{1} & \textbf{0} & 0 & \textbf{0} & \textbf{1} & 0 & \textbf{0} & 0 & \textbf{1} & 0 & 6 \\
A & 0 & \textbf{0} & 0 & \textbf{0} & 0 & \textbf{1} & \textbf{0} & \textbf{1} & 0 & \textbf{1} & 6 \\
A & 0 & 1 & 0 & \textbf{0} & 0 & \textbf{1} & \textbf{0} & \textbf{1} & \textbf{1} & 0 & 5 \\
A & 0 & \textbf{0} & \textbf{1} & \textbf{0} & 0 & \textbf{1} & 1 & 0 & \textbf{1} & \textbf{1} & 6 \\
A & 0 & \textbf{0} & 0 & 1 & 0 & 0 & \textbf{0} & \textbf{1} & \textbf{1} & \textbf{1} & 5 \\
B & 0 & 1 & \textbf{1} & \textbf{0} & 0 & 0 & 1 & 0 & 0 & 0 & 2 \\
A & 0 & 1 & 0 & 1 & 0 & \textbf{1} & \textbf{0} & \textbf{1} & \textbf{1} & \textbf{1} & 6 \\
B & 0 & 1 & \textbf{1} & 1 & 0 & 0 & \textbf{0} & 0 & 0 & 0 & 2 \\
A & \textbf{1} & \textbf{0} & 0 & \textbf{0} & 0 & \textbf{1} & 1 & \textbf{1} & \textbf{1} & \textbf{1} & 6 \\
B & \textbf{1} & 1 & \textbf{1} & 1 & 0 & 0 & 1 & 0 & 0 & \textbf{1} & 2 \\
B & 0 & \textbf{0} & \textbf{1} & 1 & 0 & 0 & \textbf{0} & 0 & 0 & 0 & 3 \\
A & \textbf{1} & 1 & 0 & \textbf{0} & 0 & 0 & 1 & \textbf{1} & \textbf{1} & \textbf{1} & 5 \\
A & 0 & \textbf{0} & 0 & \textbf{0} & 0 & \textbf{1} & \textbf{0} & \textbf{1} & \textbf{1} & \textbf{1} & 7 \\
B & 0 & 1 & \textbf{1} & 1 & 0 & 0 & 1 & 0 & 0 & 0 & \textbf{1} \\
B & 0 & 1 & \textbf{1} & \textbf{0} & 0 & 0 & \textbf{0} & \textbf{1} & \textbf{1} & 0 & 5 \\
A & 0 & \textbf{0} & 0 & \textbf{0} & 0 & 0 & \textbf{0} & 0 & 0 & \textbf{1} & 4 \\
B & 0 & \textbf{0} & \textbf{1} & 1 & 0 & 0 & \textbf{0} & 0 & 0 & 0 & 3 \\
B & \textbf{1} & 1 & \textbf{1} & 1 & 0 & 0 & \textbf{0} & \textbf{1} & 0 & 0 & 4 \\
B & 0 & 1 & \textbf{1} & 1 & 0 & 0 & \textbf{0} & \textbf{1} & 0 & 0 & 3 \\
B & 0 & \textbf{0} & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 & \textbf{1} \\
\hline
\hline
$\bm x(t_\test)$ & 0 & 1 & 0 & 1 & 0 & 0 & 1 & 0 & 0 & 0 &  \\
\hline
$\bm u_A$ & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 1 & 1 & 1 &  \\
$\bm u_B$ & 0 & 1 & 1 & 1 & 0 & 0 & 1 & 0 & 0 & 0 &
\end{tabular}
\vspace{5pt}
\caption[State classification using Hamming distance]{State classification using Hamming distance. Shown are $T_\compare^A = T_\compare^B = 10$ representative states from the no-plastic training phase, corresponding to figure \ref{fig:classification-new}. An activity vector $\bm x(t_\test)$ and examples of input vectors $\bm u_A$ and $\bm u_B$, corresponding to the Markov states $A$ and $B$ are shown in the bottom of the table. On the right side the Hamming distance between any $\bm x(t_\compare)$ and the $\bm x(t_\test)$ is calculated. The differing digits in the vectors are bold. The smallest distance is given by two vectors corresponding to an $B$ input, with a distance of just $1$, which is highlighted in bold on the right side. Therefore, the chosen step in testing phase would be classified as $B$. This procedure is repeated for every $\bm x(t_\test) \in R_\test$.}
\label{tb:hamming}
\end{table}

The activity pattern $\bm x(t_\compare)$ with the smallest distance $d(\bm x(t_\test), \bm x(t_\compare))$ for all $t_\compare$ is chosen and denoted by $\bm x^\mini_\compare(t_\test)$. Corresponding to $\bm x^\mini_\compare(t_\test)$ exists a specific input $u \in \{x_1, ..., x_n\}$ which is chosen as the Markov state for the corresponding step $t_\test$ in the testing phase. A simple example, which gives an intuition is shown in table \ref{tb:hamming}. The process is in principle a $k$-nearest neighbors algorithm with $k=1$, where the distance between the vectors is measured using a hamming distance.

The question arises what happens if two or more patterns from training phase having equal distances, assuming that the patterns in the training phase belong to different Markov states. \textcite{hartmann2015s} solved this problem by shuffling the training states, before calculating the distances and then taking the first element which has minimum distance. With this method a systematical bias is prevented.

However, the algorithm is relatively simple, which is its strength, at least regarding computational effort and simplicity in implementation. The classification method will be examined in the results and further discussed in the discussion.

\subsection{Measuring performance}

The reservoir network is trained with a specific Markov chain, where the states in the Markov chain represent input clusters in the neural network, modeled by input neurons. Ideally, the network finds a good representation of the transition probabilities and the stationary distribution, stored in the weights, to reproduce the Markov chain. Statistically, the network can be seen as an estimator of the true probability structure of the Markov chain. An estimator in that sense can be optimized by finding better parameter configurations of the network. The approach does not include an analytic study of the estimator characteristics, rather it is a simulation study.

\nomenclature{$\hat{\bm\pi}$}{Estimated stationary distribution of a Markov chain}
\nomenclature{$\hat M$}{Estimated transition matrix of a Markov chain}

Let $M$ denote the true transition matrix and $\hat M$ the estimator of the transition matrix. Analogously, $\bm\pi$ is the true stationary distribution of the Markov chain and $\bm{\hat\pi}$ the estimator of the stationary distribution. To quantify the performance of the estimators $\hat M$ and $\bm{\hat\pi}$, a common \acf{mse} can be implemented. For a general parameter $\bm\vartheta$ and associated estimator $\hat{\bm\vartheta}$, the \ac{mse} is defined as

\begin{equation}
\label{eq:mse}
\MSE(\bm{\hat\vartheta}) = \E_{\bm{\hat\vartheta}}[(\bm{\hat\vartheta} - \bm\vartheta)^T (\bm{\hat\vartheta} - \bm\vartheta)].
\end{equation}

In both cases, $\hat M$ and $\bm{\hat\pi}$, the analytic solution is probably impossible or at least very hard to obtain, since both estimators are highly non-linear random variables. But for both estimators an estimator for the \ac{mse} can be introduced. It is based on a Monte Carlo approximation. The simulation was repeated $\Nsim$ times. For the vector parameter $\bm\vartheta \in \R^d$

\begin{equation}
\label{eq:mse-est}
\widehat{\MSE}(\bm{\hat\vartheta}) = \frac{1}{d \cdot \Nsim} \sum_{i=1}^{\Nsim} (\bm{\hat\vartheta}_i - \bm\vartheta_i)^T (\bm{\hat\vartheta}_i - \bm\vartheta_i)
\end{equation}

will be the general formulation of the used performance measure.

In case of the transition matrix $M$, a stochastic matrix, the rows need to be concatenated. Therefore, $\bm m = (p_{11}, ..., p_{1n}, p_{21}, ..., p_{2n}, p_{31}, ..., p_{nn})^T$ (compare with equation \eqref{eq:trans-matrix}) defines a vector, containing all entries from the transition matrix. Finally, the estimator of the \ac{mse} for the transition matrix estimator $\hat M$ is defined as

\begin{equation}
\varepsilon_M = \widehat{\MSE}(\bm{\hat m}) = \frac{1}{n^2 \cdot \Nsim} \sum_{i=1}^{\Nsim} (\bm{\hat m}_i - \bm m_i)^T (\bm{\hat m}_i - \bm m_i).
\end{equation}

\nomenclature{$\varepsilon_M$}{Estimated mean squared error of transition matrix}

In case of the stationary distribution, the calculation is straight forward. The estimator of the \ac{mse} for the stationary distribution estimator $\bm{\hat\pi}$ is defined as

\begin{equation}
\varepsilon_\pi = \widehat{\MSE}(\bm{\hat\pi}) = \frac{1}{n \cdot \Nsim} \sum_{i=1}^{\Nsim} (\bm{\hat\pi}_i - \bm\pi_i)^T (\bm{\hat\pi}_i - \bm\pi_i).
\end{equation}

\nomenclature{$\varepsilon_\pi$}{Estimated mean squared error of stationary distribution}

It remains to show how $\hat M$ and $\bm{\hat\pi}$ are obtained. Since an analytic representation is highly complex, the estimators are empirically obtained from the output of the network. Assuming that the states of the Markov chain in the testing phase are already classified (using the method from section \ref{sec:state-classification}), the stationary distribution can directly be calculated by taking the frequencies of the states. If $n_{x_i}$ is the number of occurrences for state $x_i$ in the testing phase, the estimator is defined by

\begin{equation}
\bm{\hat\pi} = \left( \frac{n_{x_1}}{N_\test}, ..., \frac{n_{x_n}}{N_\test} \right)^T,
\end{equation}

where $N_\test = |\{ \bm x(t_\test) \,:\, \bm x(t_\test) \neq 0 \}|$ for all $t_\test \in \{ T_\evo + 1, ..., \Tevo+\Ttest \}$ is the number of non-silent steps in the training phase, where $\bm x(t_\test)$ is an activity vector of the network, as used before. Note that $\bm x(t_\test) \neq 0$ is equivalent to $x_i(t_\test) \neq 0$ for all $i \in \{1,...,N^E\}$. Equivalently $N_\test = n_{x_1} + ... + n_{x_n} = \sum_{j=1}^n x_{x_j}$ is just the sum of all non-zero state occurrences.

To track changes in the course of the testing phase, the testing phase is split into chunks of $\Delta T_\chunk = 5000$ steps. Hence, $\bm{\hat\pi}$ is calculated for every chunk. The number of chunks $K_{\text{chunks}} = T_\test/\Delta T_\chunk$ is simply the number of overall steps in the testing phase divided by the chunk size. Again, $N_\test^k = |\{ \bm x(t_\test^k) \:|\, \bm x(t_\test^k) \neq 0 \}|$ for all $k \in \{ 1, ..., K_{\text{chunks}} \}$ describes the number of non-silent steps in every chunk $k$. The calculation of the estimator $\bm{\hat\pi}^k$ for a specific chunk $k$, changes to

\begin{equation}
\bm{\hat\pi}^k = \left( \frac{n_{x_1}^k}{N_\test^k}, ..., \frac{n_{x_n}^k}{N_\test^k} \right)^T.
\end{equation}

For the calculation of the estimator for the transition matrix, the silent steps are ignored, too. If $n_{x_i, x_j}^k$ denotes the number of transitions from state $x_i$ to state $x_j$ inside of chunk $k$, $\hat M^k$ is calculated as

\begin{equation}
\label{eq:transition-estimation}
\hat M^k = \left(\frac{n_{x_i, x_j}^k}{N_\test^k}\right)_{x_i,x_j \in S} = \begin{bmatrix}
\frac{n_{x_1,x_1}^k}{N_\test^k} & \frac{n_{x_1,x_2}^k}{N_\test^k} & \hdots  & \frac{n_{x_1,x_n}^k}{N_\test^k}\\
\frac{n_{x_2,x_1}^k}{N_\test^k} & \ddots &  & \vdots \\
\vdots &  & \ddots & \vdots \\
\frac{n_{x_n,x_1}^k}{N_\test^k} & \hdots & \hdots & \frac{n_{x_n,x_n}^k}{N_\test^k}
\end{bmatrix}.
\end{equation}

Finally, the \ac{mse} estimator can be applied. As the estimator of the stationary distribution $\bm{\hat\pi}$ just contains information about how often a state occurs, the estimated transition matrix also contains information about the transition properties. Therefore, the $\widehat{\MSE}(\bm{\hat m})$ is preferred in the following.

As a remark, it should be stated that also the hamming distance can be used as a performance measure. If the mean of the hamming distance of all steps in testing phase is calculated, it should be smaller for networks which perform better, since the learned states should be closer to the initial states. Since the differences in the hamming distance means do not differ much between different settings (see appendix \ref{sec:appendix:hamming}), this kind of performance measure was not further investigated.

A last problem was the amount of silent passes. The more silent passes exist, the worse will be the estimator. This problem was solved by just implementing a threshold. If the number of silent steps exceeded a threshold of $25\%$ of $\Delta T_\chunk$, which means that if $1 - N_\test^k/\Delta T_\chunk > 0.25$ for any $k$, an error is thrown and the network simulation is canceled.


\documentclass{scrartcl}

\usepackage{listings}
\usepackage{amsfonts}
\usepackage{amsmath}
\usepackage{mathtools}
\usepackage{tabularx}
\usepackage{ragged2e}
\usepackage{hyperref}

\usepackage{syntax}

\DeclareMathOperator{\ifop}{if}
\DeclareMathOperator{\thenop}{then}
\DeclareMathOperator{\elseop}{else}
\DeclareMathOperator{\lenop}{len}
\DeclareMathOperator{\nil}{nil}
\DeclareMathOperator{\fby}{fby}
\DeclareMathOperator{\where}{where}
\DeclareMathOperator{\nextop}{next}

\DeclareMathOperator{\numtype}{\mathbb{R}}

\DeclareMathOperator{\ceval}{\overset{C}{\rightarrow}}

\DeclareMathOperator{\initrel}{\overset{init}{\Rightarrow}}
\DeclareMathOperator{\comprel}{\overset{compensate}{\Rightarrow}}
\DeclareMathOperator{\flattenrel}{\overset{flatten}{\Rightarrow}}

\DeclareMathOperator{\buffer}{\mathbf{buffer}}
\DeclareMathOperator{\compress}{\mathbf{compress}}

\makeindex

\begin{document}
    \title{Pixy progress report}
    \author{A. Finn Hackett, Reed Mullanix}

    \maketitle

    \tableofcontents

    \section{Introduction}

    Our initial goal was to adapt the Lucid programming language to have first-class support for real-time programming\cite{bencina}, while retaining its nature as a general purpose language.

    This means a few things: every step of computation must be able to meet a deadline, unrelated parts of a program must not be able to interfere with each others' timing and ideally even related parts of a program must never stop each other.

    This requires a heavily asynchronous mindset for which Lucid is a good starting point, since almost everything in Lucid is indeed asynchronous. It also requires avoiding the original Lucid implementation's reliance on indexed streams, since they worked based on a heuristic \cite[pages 67-68]{wadgeashcroft85} that would induce arbitrary recomputation in some cases. \footnote{The original implementation would store the last 200 or so steps of the computation, recomputing anything older than that on demand. The heuristic was that this was unnecessary most of the time. This is hardly useful if we want precise bounds on the amount of memory a Pixy program might use or how long it will take.}

    Our plan was to derive our own set of evaluation semantics for the language that was to become Pixy that were able to compute all the values for each timestep using bounded memory and time, and then to work on enforcing more and more correctness properties such as co-inductive deadlock-freedom and custom invariants via a temporal type system.

    Our progress so far is that we have designed Pixy' evaluation semantics from the ground up to operate using a fixed amount of state and compute every timestep within an arbitrary but constant deadline.

    \section{Pixy's programming model}

    We have defined Pixy's operational semantics in terms of nested state machines - any Pixy expression will evaluate to a state machine that will operate asynchronously, yielding a value at each timestep $t \in \mathbb{N}$.
    This approach allows for a much finer-grained control of the asynchronous relationships between dataflow nodes than Lucid's original lazy computation model, at the cost of some flexibility.

    Specifically, Pixy does not support nested iteration in that same way that Lucid does, nor do custom operators enjoy the same degree of freedom as they did in Lucid.

    To introduce Pixy's supported subset of Lucid, here is Pixy's grammar:

    \begin{grammar}
        <expr> ::= <number>
        \alt <var>
        \alt <bool>
        \alt "nil"
        \alt "?" <expr>
        \alt "if" <expr> "then" <expr> "else" <expr>
        \alt <expr> "fby" <expr>
        \alt <expr> "where" "{" <wheredecls> "}"
        \alt "next" <expr>
        \alt <var> "(" <exprlist> ")"
        \alt <expr> "+" <expr>
        \alt <expr> "-" <expr>
        \alt <expr> "*" <expr>
        \alt <expr> "/" <expr>
        \alt <exprtuple>

        <exprtuple> ::= "(" <exprlist> ( "|" <expr> )? ")"

        <bool> ::= "true" | "false"

        <exprlist> ::= <expr> "," <exprlist> | <expr>

        <varlist> ::= <var> "," <varlist> | <var>

        <wheredecl> ::= <var> "=" <expr>
        \alt <var> "(" <varlist> ")" "=" <expr>
        \alt "(" <varlist> ( "|" <var> )? ")" "=" <expr>

        <wheredecls> ::= <wheredecl> ";" <wheredecls> | <wheredecl>
    \end{grammar}

    In principle, any Pixy expression with no free variables can be viewed as a self-contained program. Our current implementation reads a series of wheredecls and executes a main operator for the sake of ease of use.

    You will notice that the grammar above contains $\left<exprtuple \right>$, which can be used to construct tuples. This is a work in progress and will not be described in any depth in this paper, but it is expected to become part of the language in future work. Broadly speaking, tuples would be able to be constructed and manipulated as an extra data type. The only feature of note is that we expect that tuple literals will function as synchronisation points between streams during pre-delay calculations.

    \subsection{From an end-user point of view}

    When programming in Pixy, one can consider expressions as representing streams of values. This does not match our state-machine based formal model, but can aid in explaining its idiosyncrasies.

    In this section we will discuss programs written in Pixy and how one can reason about them.

    \subsubsection{Literals}

    A single literal is the most trivial Pixy program. Any literal in Pixy will repeatedly yield itself. For example, $1$ will yield the sequence $1, 1, 1, ...$ forever.

    The same goes for $true$ and $false$. The exception is $nil$, which describes an empty stream. It will never yield a value.

    One can also bind Pixy expressions to names like so $x = 1$, in the context of which the expression $x$ will also yield $1, 1, 1 ...$.

    \subsubsection{Dataflow operators}

    Any Pixy expression is composed of sometimes-stateful dataflow operators. Many operators mirror common operations in mainstream programming languages like $+$, $-$, etc...

    Pixy also features some more unique operators to control the flow of time.

    \paragraph{fby}

    We will first introduce fby: fby expresses sequencing. Combining fby with literals, one can write $1 \fby 2$. This program will yield the infinite sequence $1, 2, 2 ...$.

    Strictly speaking, fby will take one value from its left-hand side then yield all values from its right-hand side.

    Mixing assignment with fby allows us to elegantly specify different recurrences:

    \begin{lstlisting}
x = 1 fby x+1
    \end{lstlisting}

    A quintessential example due to the ability to trivially identify which timestep any value of $x$ came from, the expression $x$ will yield the infinite increasing sequence $1,2,3,4 ...$.

    To be precise, fby will yield one value from the expression $1$, then all the values from the expression $x+1$ which expresses the sequence $2,3,4 ...$.

    Likewise, it is quite easy to define alternating sequences using only fby:

    \begin{lstlisting}
x = 1 fby 2 fby 3 fby x
    \end{lstlisting}

    In this case the expression $x$ will yield $1, 2, 3, 1, 2, 3 ...$.

    \paragraph{next}

    The other important temporal operator in Pixy is the next operator. It allows us to express a lookahead into a stream.

    For example, taking one of our previous examples $x = 1 \fby x+1$, the expression $\nextop x$ will yield the infinite sequence $2,3,4 ...$.

    We can also use it in direct conjunction with fby to express more sophisticated sequences:

    \begin{lstlisting}
x = 1 fby 1 fby (x+ next x)
    \end{lstlisting}

    In this case, the expression $x$ yields the sequence $1, 1, 2, 3, 5 ...$, or the Fibonacci sequence.

    The intuition to how this works is to consider $x$ to yield the "first element" of the stream and $\nextop x$ to yield the "second element". So, initially both are $1$, then as we move forward in time we have $1, 2$, and so forth. $x + \nextop x$ simply calculates the sum of these two values going forward in time.

    \subsubsection{Where and user-defined operators}

    To help keep Pixy code organised and allow code reuse, we provide some primitives to control the scoping of values.

    \paragraph{where}

    Where clauses are similar to the construct of the same name and similar syntax in Haskell. They allow you to create a local scope with variables private to a specific expression.

    Here is an example of a Pixy program using a where expression:

    \begin{lstlisting}
x = a + b
    where {
        a = 1 fby b + 1
        b = 2 fby a + 1
    }
    \end{lstlisting}

    For more involved programs like this one, we encourage readers to check their behaviour using tables like these:

    \begin{align*}
    a_1 = 1 && b_1 = 2 && x_1 = 3 \\
    a_2 = 3 && b_2 = 2 && x_2 = 5 \\
    a_3 = 3 && b_3 = 4 && x_3 = 7 \\
    a_4 = 5 && b_4 = 4 && x_4 = 9 \\
    ...
    \end{align*}

    This table for example shows that we have defined a Rube Goldberg-esque odd number generator.

    This also exercises the key features of where expressions: aside from allowing assigning expressions to variables and then using them in the $a+b$, where expressions allow defining complex mutually-referential streams.

    It is also worth noting that definition order has no impact on program semantics - the following program is semantically identical to the first:

    \begin{lstlisting}
x = a + b
    where {
        b = 2 fby a + 1
        a = 1 fby b + 1
    }
    \end{lstlisting}

    \paragraph{User-defined operators}

    As well as giving names to streams, Pixy also allows defining custom operators. Syntactically they are similar to function definition and application, but they have an important difference: they operate on streams, so we call them operators.

    Here is an example program equivalent to the example above that demonstrates custom operator definition:

    \begin{lstlisting}
f1(p, q) = p fby q

x = a + b
    where {
        f2(a, b) = a fby b + 1
        a = f1(1, b+1)
        b = f2(2, a)
    }
    \end{lstlisting}

    Custom operators can be visualised as operating by substitution. They can be defined at any location and do not perform any kind of variable capture - their inner scope is composed entirely of their arguments. $f2$ demonstrates that you can safely use the same name to refer to a parameter and a variable defined outside an operator and not run into conflicts.

    Custom operators do not support recursion at the moment - that is a topic for future work.

    \subsubsection{Conditionals}

    Pixy also features an if expression syntactically similar to those found in Haskell or ML. Comparison past superficial syntax may be misleading however.

    Often, Pixy's conditionals act like one might naively expect:

    \begin{lstlisting}
y = 1 fby y + 1
x = if y % 2 == 0 then y else 0
    \end{lstlisting}

    This will unsurprisingly cause the expression $x$ to yield the sequence $0, 2, 0, 4, 0, 6 ...$.

    The intuition that the condition is evaluated, then based on the result one of the two branches is returned is largely correct.

    Now consider this code:

    \begin{lstlisting}
y = 1 fby y + 1
x = if y % 2 == 0
    then
        z where { z = 1 fby z + 1 }
    else
        0
    \end{lstlisting}

    This yields the same sequence, but highlights an important detail: time progresses as the same rate in each branch of the if statement, regardless of which branch is chosen. In effect, an if expression acts as a 2-way multiplexer.

    \paragraph{Choking and nil}

    Now let us introduce $nil$ more fully and demonstrate a confusing but important extra detail of if expressions.

    Consider the following program:

    \begin{lstlisting}
y = 1 fby y + 1
x = if y % 2 == 0 then y else nil

z = if ?x then x + 1 else 0
    \end{lstlisting}

    In this program, $z$ will yield the sequence $3, 5, 7 ...$. This may raise a few questions.

    First, we need to explain that $?x$ is a new operator. It is called a nil-check and maps from a stream of values to a stream of booleans. If a given value in the original stream was $nil$ then that element of the new stream will be $false$, otherwise the stream will be comprised of repeated $true$.

    The nil-check is useful for reacting to a stream not yielding a value at the current timestep. Formally, $x$ does not yield a value when $y$ yields a value that is odd. When a stream does not yield a value its value is called $nil$, and the only thing you can do to that value is check for it and forward it. In the case of $z$, we chose to default $z$'s value to $0$ when $x$ does not have a value.

    This then raises the second question: if an if expression evaluates both of its arguments, why wouldn't the program crash on evaluating $nil + 1$, even if the $0$ is chosen at that time?

    The answer is the concept of choke-evaluation. If if expressions really did evaluate both their arguments then the problem would have no solution - you would not be able to shield a given subexpression from encountering $nil$ values.

    Choke evaluation is applied to a non-taken branch of an if expression in order to make sure any state it has progresses, while "pretending" to not evaluate it. That is, a choke-evaluated expression must always yield $nil$ but may perform upkeep internally if need be. We achieve this by defining a second mode of evaluation where all literals and variable references return $nil$, as well as defining semantics for all the operators to tolerate $nil$ under most circumstances.

    For example, in the above program what is really evaluated when $x$ is $nil$ is $nil + nil$, which evaluates to $nil$ without any issue.

    The closest thing to an "exception" to choke-evaluation is when you try to choke-evaluate a where expression. As before, it is required that the expression yields $nil$ in the end, but all the internal definitions are still evaluated.

    This is why the following code from the previous section also does what one would expect - $z$ is still updated, since the expression that updates it exists inside a where expression:

    \begin{lstlisting}
y = 1 fby y + 1
x = if y % 2 == 0
    then
        z where { z = 1 fby z + 1 }
    else
        0
    \end{lstlisting}

    We find that choke-evaluation strikes a balance between the ability to skip evaluation of a non-taken branch and the ability to keep time synchronised between both branches of an if expression, despite its sometimes strange-looking behaviour.

    \subsection{Operational semantics}

    Pixy's operational semantics have several stages in order to keep memory allocation and analysis distinct from the actual evaluation steps. This helps keep evaluation simple and allows us to prove quite trivially that for every timestep a Pixy program will perform a bounded amount of computation. We chose to avoid time-indexed computation in order to stay in keeping with the idea that a Pixy program only needs its immediate state in order to keep functioning - that is to say, a Pixy program could in theory be coinductively verified to remain correct and on time by symbolically evaluating reachable states, and will run using a well-defined finite amount of memory.

    The first stage is used to flatten out any operator calls. Since recursive operator calls could lead to unbounded memory usage during evaluation, Pixy requires all operator calls including recursive ones to be flattened out before evaluation. In order to predict the maximum possible recursion depth and allocate memory for it ahead of time, we use a technique based on sized types.

    The second stage is the allocation of initial state for all builtin operators. This stage has two subsections, namely state allocation and delay compensation.

    The last stage is the evaluation derivations themselves, which operate on the modified AST and initial state defined in the prior stages. This stage models the life cycle of the program using what can be considered both small- and big-step semantics depending on your point of view. If you consider the program's entire lifespan, then we express that in a series of small steps. If you consider one step, we provide big-step semantics for analysing one single step. To avoid confusion, we will notate evaluation using $\rightarrow$.

    \subsubsection{Operator calls}

    To begin with, we flatten out all operator calls via substitution. This is both to ensure that there is a clear bound on the time a Pixy expression can take to perform one time step and to simplify the formulation of any further stages.

    We have an idea of how to deal with recursive operator application, but it is incomplete. Similar to \cite{abel}, we expect to reason about recursion depth using sized types. Specifically, we hope to generate upper and lower bounds for all operator arguments and enforce that the range of possible inputs to a recursive operator strictly decreases as recursion depth increases. Thus, we should be able to either generate a series of substitutions equivalent to the deepest recursion depth possible (terminating in an infinitely choked if branch) or reject an expression as ill-formed.

    Our problem at the moment is that we do not have a precise enough representation of numbers in Pixy. While it is simple enough to calculate upper and lower bounds for an expression given its type, performing operations on these bounds requires a precise definition of numerical edge cases that we lack. We leave to future work exactly how to do this, but expect that once a more precise set of numerical semantics are derived the issue will be simpler.

    For the simple case where we assume no recursion occurs, operator call flattening can be expressed using the syntax $\Gamma \vdash E \flattenrel E'$ where $\Gamma$ represents the current scope, $E$ represents the initial expression and $E'$ represents the modified result with no operator calls.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma \vdash B \flattenrel_{W1} B'; \Gamma' \\
        \Gamma, \Gamma' \vdash B' \flattenrel_{W2} B'' \\
        \Gamma, \Gamma' \vdash E \flattenrel E'
        \end{matrix}
    }{
        \Gamma \vdash E \where \{ B \} \flattenrel E' \where B''
    }[\text{Flatten-where}]
    \end{align*}

    Since order inside a where clause is arbitrary, we have to be careful how we treat scoping. First we pass through and collect all the operator definitions, then we perform a second pass while applying the operators wherever necessary.

    \begin{align*}
    \frac{
        \Gamma \vdash R \flattenrel_{W1} R'; \Gamma'
    }{
        \Gamma \vdash F(A...) = E; R \flattenrel_{W1} R'; F(A...) = E, \Gamma'
    }[\text{Flatten-where-1-operator}]
    \end{align*}

    \begin{align*}
    \frac{
        \Gamma \vdash R \flattenrel_{W1} R'; \Gamma'
    }{
        \Gamma \vdash V = E; R \flattenrel_{W1} (V = E; R'); \Gamma'
    }[\text{Flatten-where-1-variable}]
    \end{align*}

    \begin{align*}
    \frac{}{
        \Gamma \vdash \flattenrel_{W1} ; \emptyset
    }[\text{Flatten-where-1-empty}]
    \end{align*}

    Once we've collected all the operator definitions, we traverse all the subexpressions and substitute them in when necessary:

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma \vdash E \flattenrel E' \\
        \Gamma \vdash R \flattenrel_{W2} R'
        \end{matrix}
    }{
        \Gamma \vdash V = E; R \flattenrel_{W2} V = E'; R'
    }[\text{Flatten-where-2-variable}]
    \end{align*}

    \begin{align*}
    \frac{}{
        \Gamma \vdash \flattenrel_{W2}
    }[\text{Flatten-where-2-empty}]
    \end{align*}

    When we encounter an operator application, we replace it the operator's body and a scope modifier to avoid name conflicts:

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma(F(A...) = E) \\
        \Gamma \vdash E \flattenrel E'
        \end{matrix}
    }{
        \Gamma \vdash F(V...) \flattenrel [A=V...]E'
    }[\text{Flatten-apply}]
    \end{align*}

    Note: the scope modifier pictured here is not a substitution - it will be interpreted specially by later stages.

    \subsubsection{Delay compensation}

    Often, delay compensation must be applied to the evaluation of a Pixy expression in order for it to operate correctly. \footnote{As an interesting analogy, this is quite similar to accounting for propagation delay in electronic circuit design. \cite{propwiki}}

    To give a brief rationale, consider the following Pixy snippet:

    \begin{lstlisting}
x = 1
y = x + next x
    \end{lstlisting}

    Following a naive reading of the evaluation semantics, at $t_1$ $x$ will be 1, and when computing $y$ we will take 1 and try to add it to $\nextop x$, which dropped the value $1$ and is equal to $nil$ in this case. This is counter-intuitive since $x + \nextop x$ looks like it means "wait for $\nextop x$" then add $x$ to it. When considering this issue we found that it was possible to reimplement the intuitive interpretation using implicit buffering, or delay compensation.

    Initially we attempted to express this in terms of ad-hoc buffering at the offending operator, + in this case. This however encountered problems when under choke-evaluation. Take the following expression:

    \begin{lstlisting}
if ?x then (x+x) + next x else 1
    \end{lstlisting}

    Assuming $x$ may be $nil$, the problem here is that the if statement will try to choke-evaluate $(x + x)$ on the wrong timestep. While the if expression should be preventing $(x+x)$ from being computed when $x$ is nil, the value of $x$ would reach the condition clause one timestep after $(nil+nil)$ had been evaluated and the Pixy program had crashed. This is because making + buffer its inputs assumes that the operands will be valid at every timestep, which may not be true.

    To fix this issue, we ensure that only variable references are buffered. No computation occurs when referencing a variable, so there can be no errors due to incorrectly timed $nil$ values.

    To achieve this, we specify a delay compensation relation that may be used by the init stage in order to add buffers to nested variable references.

    The syntax for the relation looks like this: $\mathcal{V}; d; \delta; S \vdash E \comprel S'; \delta_e'; \Delta_e'$.

    Unspecified letters have the same function as those used in the init stage.

    $S$ refers to the existing initial state computed for expression $E$.

    $\delta$ refers to the amount by which any identifiers are to be delayed on top of their existing delay.

    $S'$ refers to the new initial state after delay compensation has been computed.

    $\delta_e'$ refers to the new delay of the expression $E$ after compensation.

    $\Delta_e'$ refers to the constraints applicable to the delays in $E$ after compensation.

    \paragraph{Id}

    The only derivation we will give for compensation is that for identifiers - all others are identical to the init stage except for passing along the extra information present during delay compensation.

    What happens for identifiers is that the extra delay $\delta$ is applied to the identifier's buffer. As you will see from the init stage, this starts as buffer of length 0 and becomes longer as needed to buffer enough of a variable's history.

    In principle one could hold a single common buffer for every identifier and grow/index into it as needed, but this would significantly complicate the topics so we will leave that as an exercise for the implementation.

    \begin{align*}
    \frac{
        I \in \mathcal{V}
    }{
        \mathcal{V}; d; \delta; \mathcal{B}[N] \vdash I \comprel \mathcal{B}[N+\delta]; \delta(I)+1; \emptyset
    }[\text{Compensate-id-where}]
    \end{align*}

    \begin{align*}
    \frac{
        I \notin \mathcal{V}
    }{
        \mathcal{V}; d; \delta; \mathcal{B}[N] \vdash I \comprel \mathcal{B}[N+\delta]; \delta(I); \emptyset
    }[\text{Compensate-id-default}]
    \end{align*}

    \subsubsection{init stage}

    Pixy has an initialisation stage during which all the state needed for any later computation will be initialised and buffers are put in place to implement delay compensation - this is the only stage that controls the allocation of memory.

    The syntax for derivations of this stage looks like this:

    \begin{align*}
    \mathcal{V}; d \vdash E \initrel S; \delta; \Delta
    \end{align*}

    $\mathcal{V}$ is the set of names that are currently being defined by a where expression.

    $d$ is the number of nested fby expressions containing $E$ (initially 0)

    $S$ is the starting state for expression $E$

    $\delta$ is the pre-delay of expression $E$, that is, the number of timesteps before $E$ will yield values.

    $\Delta$ is the set of constraints on pre-delay time. These are to be solved by a constraint solver in order to derive the appropriate pre-delays of any expression.

    \paragraph{Binop}

    Binary operators are one of the main places where delay compensation occurs. Their state allocation is simple - we recursively allocate enough space for the operands. The important detail however is that we must ensure the operands are synchronised. If their $\delta$ values are different, we must apply delay compensation.

    \begin{align*}
    \frac{
        \begin{matrix}
        \mathcal{V}; d \vdash A \initrel S_a; \delta_a; \Delta_a \\
        \mathcal{V}; d \vdash B \initrel S_b; \delta_b; \Delta_b \\
        \mathcal{V}; d; \max(\delta_a, \delta_a) - \delta_a; S_a \vdash A \comprel S_a'; \delta_a'; \Delta_a' \\
        \mathcal{V}; d; \max(\delta_a, \delta_a) - \delta_b; S_b \vdash B \comprel S_b'; \delta_b'; \Delta_b'
        \end{matrix}
    }{
        \mathcal{V}; d \vdash A \; op \; B \initrel (S_a', S_b'); \max(\delta_a', \delta_b'); \Delta_a', \Delta_b'
    }[\text{Init-binop}]
    \end{align*}

    In each case we compensate the faster of the two operands to be slower - this ensures that only values from the same time will meet each other.

    \paragraph{Conditionals}

    If statements also require delay compensation, this time across 3 different operands. Otherwise, we just recursively allocate space for the conditional and two branches.

    \begin{align*}
    \frac{
        \begin{matrix}
        \mathcal{V}; d \vdash C \initrel S_C; \delta_C; \Delta_C \\
        \mathcal{V}; d \vdash T \initrel S_T; \delta_T; \Delta_T \\
        \mathcal{V}; d \vdash F \initrel S_F; \delta_F; \Delta_F \\
        \mathcal{V}; d; m - \delta_C; S_C \vdash C \comprel S_C'; \delta_C'; \Delta_C' \\
        \mathcal{V}; d; m - \delta_T; S_T \vdash T \comprel S_T'; \delta_T'; \Delta_T' \\
        \mathcal{V}; d; m - \delta_F; S_F \vdash F \comprel S_F'; \delta_F'; \Delta_F'
        \end{matrix}
    }{
        \mathcal{V}; d \vdash \ifop C \thenop T \elseop F \initrel (S_C', S_T', S_F'); m'; \Delta_C', \Delta_T', \Delta_F'
    }[\text{Init-if}]
    \end{align*}

    where $m = \max(\delta_C, \delta_T, \delta_F)$ and $m' = \max(\delta_C', \delta_T', \delta_F')$

    \paragraph{Next}

    Since next needs to remember how many values it needs to skip in order to reach the second value yielded by $E$, we store that number. The number in question happens to be $\delta+1$, since it is one more than the time needed to wait for the first value yielded by $E$. We also store the initial state of its only operand $E$.

    \begin{align*}
    \frac{
        \begin{matrix}
        \mathcal{V}; d \vdash E \initrel S; \delta; \Delta
        \end{matrix}
    }{
        \mathcal{V}; d \vdash \nextop E \initrel (\delta+1, S); \delta+1; \Delta
    }[\text{Init-next}]
    \end{align*}

    \paragraph{Fby}

    Fby is the only binary operator in Pixy that does not perform typical delay compensation. Since it is used for sequencing, instead we ensure that exactly one value from $A$ can be sequenced before all the values from $B$.

    We do this with a delay-based buffer but use a different strategy. We allocate a buffer internal to the fby expression's state that holds $\max(0, \delta_a - \delta_b +1)$ elements. This means that if the delays for $A$ and $B$ are equal, that is, they are expected to arrive at the same time, then $B$'s stream should be delayed by one timestep. If $A$ is at all earlier than $B$, then $B$ does not have to wait at all and the buffer size becomes $0$ (pass-through). If $A$ is later than $B$, $B$ should wait one more than the number of timesteps $A$ is late by, since it needs to wait for both $A$ to arrive and be yielded (which takes one extra timestep).

    We also make the first element of the state $false$ in order to initialise the latch between reading from $A$ and reding from $B$. See the evaluation rules for how this is used.

    \begin{align*}
    \frac{
        \begin{matrix}
        \mathcal{V}; d \vdash A \initrel S_a; \delta_a; \Delta_a \\
        \mathcal{V}; d+1 \vdash B \initrel S_b; \delta_b; \Delta_b \\
        \end{matrix}
    }{
        \mathcal{V}; d \vdash A \fby B \initrel (false, \mathcal{B}[\max(0, \delta_a - \delta_b + 1)], S_a, S_b); \delta_a; \Delta_a, \Delta_b, \delta_a \geq \delta_b - d
    }[\text{Init-fby}]
    \end{align*}

    To explain the constraint $\delta_a \geq \delta_b - d$ that we add to the delay constraint set $\Delta_a, \Delta_b$, this is to prevent $x = 1 \fby \nextop x$ from being valid. In effect, that Pixy definition defines a stream whose first element is $1$ and whose second element is ... its own first element, which has no computable value. When viewed from angle of delay constraints, such a situation would lead to $\delta_a = 0, \delta_b = 1$ and the constraint $0 \geq 1$ which fails. The other case that would fail this constraint is when $B$ is late - this would insert a $nil$ value into the output of the fby between the first value of $A$ and the first value of $B$. This would be unintuitive implicit behaviour, so we forbid it in the current version of the language.

    To explain the $-d$ term, it is to deal with constraints on nested fby expressions. Given the definition $x = 1 \fby 2 \fby \nextop x$, solving the constraint without $-d$ would yield $\delta_a = 0, \delta_b = 1$ and the unsatisfied constraint $0 \geq 1$. This is undesirable, since this definition does work and defines the sequence $1, 2, 2, 2 ...$. This is fixed by changing the constraint to $0 \geq 1 - 1$. The intuition behind subtracting $d$, the number of nested fby expressions we are in, is that we know there is an enclosing fby that will hide at most $d$ nils from the user, so $B$ being late by at most $d$ timesteps is permissible in this case.

    \paragraph{Id}

    When an identifier is encountered, we need to initialise the variable's buffer in case delay compensation needs to be done. Since it is possible that no compensation at all is required, we initialise the buffer to 0. In the delay compensation derivation for identifiers, this is why initially $N=0$.

    \begin{align*}
    \frac{
        \begin{matrix}
        I \in \mathcal{V}
        \end{matrix}
    }{
        \mathcal{V}; d \vdash I \initrel \mathcal{B}[0]; \delta(I) + 1; \emptyset
    }[\text{Init-id-whereclause}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        I \notin \mathcal{V}
        \end{matrix}
    }{
        \mathcal{V}; d \vdash I \initrel \mathcal{B}[0]; \delta(I); \emptyset
    }[\text{Init-id-nodelay}]
    \end{align*}

    The reason there are two versions of this derivation is due to pre-delay: if we are still among the where clauses where the variable was defined, we will access it with a delay of 1 since all where clauses run synchronously and all variables defined by a where clause will be $nil$ at time 0. Otherwise, such as when a variable is passed as an argument to an operator or when we are in the body of a where expression, the value will be available immediately so we do not introduce any additional delays.

    \paragraph{Operator scope}

    Initialising operator scope is deceptively simple. The main issue we need to watch out for is that operator application acts as a synchronisation point - in order to ensure we pass in sets of values from the same point in time we must perform delay compensation on the operands.

    \begin{align*}
    \frac{
        \begin{matrix}
        \mathcal{V} \setminus \{A...\}; d \vdash E \initrel S; \delta; \Delta \\
        \overline{\mathcal{V}; d \vdash V \initrel S_v; \delta_v; \Delta_v}^{V...} \\
        \overline{\mathcal{V}; d; \max(\delta_v...) - \delta_v; S_v \vdash V \comprel S_v'; \delta_v'; \Delta_v}^{V...}
        \end{matrix}
    }{
        \mathcal{V}; d \vdash [A=V...]E \initrel (S, (S_v'...)); \delta; \Delta, \Delta_v'..., \delta(A)=\delta_v'...
    }[\text{Init-operator-scope}]
    \end{align*}

    \paragraph{Where}

    During init, where expressions must enforce scoping rules and allocate the state of all their subexpressions.

    We only add the names $\{N...\}$ to $\mathcal{V}$ when recursing over the assignments, since where expressions imply an ordering between their definitions and their body. The only effect of changing $\mathcal{V}$ is that identifiers in the set will gain $+1$ to their delay as references between different names being defined in a where expression are effectively buffered by the initial $nil$ value they are set to during init. This helps keep order-independent evaluation of where expressions simple, since all identifiers refer to the previous value of that variable instead of the currently computed one.

    \begin{align*}
    \frac{
        \begin{matrix}
        \overline{\mathcal{V} \cup \{N...\}; d \vdash V \initrel S_v; \delta_v; \Delta_v}^{V...} \\
        \mathcal{V}; d \vdash E \initrel S; \delta; \Delta
        \end{matrix}
    }{
        \mathcal{V}; d \vdash E \where \{ N=V... \} \initrel (S, (nil...), (S_v...)); \delta; \Delta, \Delta_v..., \delta(N)=\delta_v...
    }[\text{Init-where}]
    \end{align*}

    To give an example of the effect of $\mathcal{V}$ in practice, consider the following set of mutually referential definitions:

    \begin{lstlisting}
x = 1 fby x + 1
y = x
z = y fby x
    \end{lstlisting}

    Due the way mutual references (even non-recursive ones) work, seeing an execution trace of each variable value at each timestep may be initially confusing:

    \begin{align*}
    x_0 = nil && y_0 = nil && z_0 = nil \\
    x_1 = 1 && y_1 = nil && z_1 = nil \\
    x_2 = 2 && y_2 = 1 && z_2 = nil \\
    x_3 = 3 && y_3 = 2 && z_3 = 1 \\
    x_4 = 4 && y_4 = 3 && z_4 = 1 \\
    x_5 = 5 && y_5 = 4 && z_5 = 2 \\
    ...
    \end{align*}

    All the values seem to be out of step with each other.

    Once you take into account delay compensation however, we realize that within this scope $\delta_x = 1, \delta_y = 2, \delta_z = 3$. This explains the apparent asynchrony: each expression is being evaluated one timestep later than the other, and buffering is used in various places to keep each expression from seeing any values from the wrong timestep.

    A further exercise is to then consider the body of this hypothetical where expression, which is initialised without $\{x, y, z\}$ added to $\mathcal{V}$:

    \begin{lstlisting}
z where
    x = 1 fby x + 1
    y = x
    z = y fby x
    \end{lstlisting}

    Now, if we name the output of this expression $p$ we can see how timings in the body of a where expression relate to those in the definitions:

    \begin{align*}
    x_0 = nil && y_0 = nil && z_0 = nil && p_0 = nil \\
    x_1 = 1 && y_1 = nil && z_1 = nil && p_1 = nil \\
    x_2 = 2 && y_2 = 1 && z_2 = nil && p_2 = 1 \\
    x_3 = 3 && y_3 = 2 && z_3 = 1 && p_3 = 1 \\
    x_4 = 4 && y_4 = 3 && z_4 = 1 && p_4 = 2\\
    x_5 = 5 && y_5 = 4 && z_5 = 2 && p_5 = 3\\
    ...
    \end{align*}

    How come $p$ appears to be predicting the future? A partial hint is in the delay for $p$: $\delta_p = 2$. Since the body of a where expression is executed strictly after all the definitions, the body of the where expression will see all the values of the definitions in the same timestep that they are computed, not one timestep later as with mutual references between where definitions. In fact, in the body the apparent delay of $x$, $y$ and $z$ is one shorter: $\delta_x = 0, \delta_y = 1, \delta_z = 2$.

    \subsubsection{Runtime evaluation semantics}

    Now that we have described how to prepare a Pixy program to be run, here are the semantics of a running Pixy program.

    Since a Pixy program's evaluation takes place over an infinite series of timesteps, the evaluation semantics describe a single timestep in detail.

    The syntax of one evaluation step looks like this: $\Gamma; S \vdash E \rightarrow V; S'$. Choke-evaluation has the same syntax, except that the relation is changed from $\rightarrow$ to $\ceval$.

    $\Gamma$ represents a mapping of variable names to value, that is, the current scope. When resolving conflicts we assume the right-most element will be chosen.

    $S$ represents the accumulated state associated with the current expression. It is important to note that the structure of state is strongly enforced: $(1, 2, 3)$ and $(1,(2, 3))$ are semantically different states, one being a tuple of three integers and the other being a 2-tuple of $1$ and $(2,3)$. Order also matters - it would be an error to initialise a state to $(1,2)$ and expect the pattern $(A, B)$ to match with $A=2, B=1$. When we pattern-match on state we rely on this property in order to avoid ambiguities and keep the states of different subexpression strictly separate.

    $E$ represents the current expression.

    $V$ is the value that $E$ evaluates to in the context of $\Gamma$ and $S$ at the current timestep.

    $S'$ is the state that will be used to evaluate $E$ in the next timestep.

    \paragraph{Conditionals}

    These evaluations are a key part of the formal description of choke-evaluation. If expressions are one of the few expressions whose non-choke evaluation rules initiate choke-evaluation.

    Specifically, if expressions choke the non-taken branch. Otherwise, they act just like a ternary operator, advancing the state of their subexpressions and correctly multiplexing values from $T$ and $F$.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_c \vdash C \rightarrow true; S_c' \\
        \Gamma; S_t \vdash T \rightarrow V; S_t' \\
        \Gamma; S_f \vdash F \ceval nil; S_f' \\
        \end{matrix}
    }{
        \Gamma; (S_c, S_t, S_f) \vdash \ifop C \thenop T \elseop F \rightarrow V; (S_c', S_t', S_f')
    }[\text{Eval-if-true}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_c \vdash C \rightarrow false; S_c' \\
        \Gamma; S_t \vdash T \ceval nil; S_t' \\
        \Gamma; S_f \vdash F \rightarrow V; S_f' \\
        \end{matrix}
    }{
        \Gamma; (S_c, S_t, S_f) \vdash \ifop C \thenop T \elseop F \rightarrow V; (S_c', S_t', S_f')
    }[\text{Eval-if-false}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_c \vdash C \rightarrow nil; S_c' \\
        \Gamma; S_t \vdash T \ceval nil; S_t' \\
        \Gamma; S_f \vdash F \ceval nil; S_f' \\
        \end{matrix}
    }{
        \Gamma; (S_c, S_t, S_f) \vdash \ifop C \thenop T \elseop F \rightarrow nil; (S_c', S_t', S_f')
    }[\text{Eval-if-nil}]
    \end{align*}

    To cover all cases, we also specify that if an if statement's conditional receives $nil$ then it chokes both its branches and yields $nil$. This is to ensure that expressions are $nil$-safe within reason. In other words, to ensure that expressions "wait" when not provided with input values by yielding $nil$ themselves.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_c \vdash C \ceval nil; S_c' \\
        \Gamma; S_t \vdash T \ceval nil; S_t' \\
        \Gamma; S_f \vdash F \ceval nil; S_f' \\
        \end{matrix}
    }{
        \Gamma; (S_c, S_t, S_f) \vdash \ifop C \thenop T \elseop F \ceval nil; (S_c', S_t', S_f')
    }[\text{Eval-if-C}]
    \end{align*}

    \paragraph{Binop}

    This is a catch-all derivation for any binary operator in Pixy. Binary operators recurse over their arguments and perform their respective operation. They pass through choke evaluation, and have no effect when passed 2 $nil$ values, returning $nil$ themselves.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_l \vdash L \rightarrow V_l; S_l' \\
        \Gamma; S_r \vdash R \rightarrow V_r; S_r' \\
        \end{matrix}
    }{
        \Gamma; (S_l, S_r) \vdash L \; op \; R \rightarrow V_l \; op \; V_r; (S_l', S_r')
    }[\text{Eval-binop}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_l \vdash L \ceval nil; S_l' \\
        \Gamma; S_r \vdash R \ceval nil; S_r' \\
        \end{matrix}
    }{
        \Gamma; (S_l, S_r) \vdash L \; op \; R \ceval nil; (S_l', S_r')
    }[\text{Eval-binop-C}]
    \end{align*}

    \paragraph{Where}

    Where expressions perform 3 functions:

    \begin{enumerate}
        \item Reading their scope from their saved state as part of our system to implement mutual reference between definitions in where expressions
        \item Recursively evaluating all their definitions, then their body (in that order)
        \item When choked, they only choke their body. The definitions are still evaluated as normal in order to avoid the state inside the where lagging behind the rest of the program.
    \end{enumerate}

    We also ensure that we always append names to the right of $\Gamma$. This is so that when a variable is defined more than once the right-first lookup semantics will pick the most local definition.

    \begin{align*}
    \frac{
        \begin{matrix}
        \overline{\Gamma, \overline{N=S_v...}^{S_v...}; S_s \vdash E_v \rightarrow V_v; S_s'}^{E_v...} \\
        \Gamma, N=V_v...; S \vdash E \rightarrow V; S' \\
        \end{matrix}
    }{
        \Gamma; (S, (S_v...), (S_s...)) \vdash E \where \{ N=E_v... \} \rightarrow V; (S', (V_v...), (S_s'...))
    }[\text{Eval-where}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        \overline{\Gamma, \overline{N=S_v...}^{S_v...}; S_s \vdash E_v \rightarrow V_v; S_s'}^{E_v...} \\
        \Gamma, N=V_v...; S \vdash E \ceval nil; S' \\
        \end{matrix}
    }{
        \Gamma; (S, (S_v...), (S_s...)) \vdash E \where \{ N=E_v... \} \ceval nil; (S', (V_v...), (S_s'...))
    }[\text{Eval-where-C}]
    \end{align*}

    \paragraph{Operator scope}

    Operator scope has similar properties to a where expression, only they do not store any state. They simply ensure that all the arguments to the operator have been evaluated then add the results to the body's scope, evaluating it in turn.

    \begin{align*}
    \frac{
        \begin{matrix}
        \overline{\Gamma; S_v \vdash E_v \rightarrow V_v; S_v'}^{E_v...} \\
        A = D_v...; S \vdash E \rightarrow V; S'
        \end{matrix}
    }{
        \Gamma; (S, (S_v...)) \vdash [A=E_v...]E \rightarrow V; (S', (S_v'...))
    }[\text{Eval-apply}]
    \end{align*}

    Similar to a where expression, operator scopes negate choking. Opposite to where expressions however, it is the arguments that are choked whereas the body is evaluated normally. It is up to the implementer of the operator to ensure that the operator does not naively use its arguments when they might be nil.

    \begin{align*}
    \frac{
        \begin{matrix}
        \overline{\Gamma; S_v \vdash E_v \ceval nil; S_v'}^{E_v...} \\
        A = nil...; S \vdash E \rightarrow V; S'
        \end{matrix}
    }{
        \Gamma; (S, (S_v...)) \vdash [A=E_v...]E \ceval nil; (S', (S_v'...))
    }[\text{Eval-apply-C}]
    \end{align*}

    \paragraph{fby}

    Evaluating fby has some extra requirements compared to other operators. One one hand, it operates as a latch waiting for one value from $A$ then choking $A$ and exclusively forwarding values from $B$. On the other hand, it buffers values from $B$ while waiting for the first value from $A$ as well as when dealing with its backlog of values from $B$ after it is done waiting for $A$.

    Here we use $\buffer(b, V_b) = b', V$ to represent pushing $V_b$ onto $b$ then popping $V$ from $b$, resulting in the upated queue $b'$. The semantics of this operation are the same as when buffering identifier values - the buffer is assumed to start full of $nil$ values, and the buffer acts like a queue. As you may notice in fby-A, the only correct value that can be popped from $b$ at that stage is $nil$. The expectation is that the buffer is large enough to not run out of $nil$ values before switching to fby-B. This property must be guaranteed by a correct implementation of the init stage.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_a \vdash A \rightarrow V_a; S_a' \\
        \Gamma; S_b \vdash B \rightarrow V_b; S_b' \\
        \buffer(b, V_b) = b', nil
        \end{matrix}
    }{
        \Gamma; (false, b, S_a, S_b) \vdash A \fby B \rightarrow V_a; (V_a \neq nil, b', S_a', S_b')
    }[\text{Eval-fby-A}]
    \end{align*}

    In fby-A you will notice that the first element of that state is pattern-matched to be $false$. This means the fby expression is still waiting for the first value from $A$. Correspondingly, the new value for that part of the state is whether a value was received from $A$, or $V_a \neq nil$. When that becomes true, that part of the state will be latched to $true$ and future evaluations will use fby-B instead.

    In fby-B, we continue cycling the buffer with new values from $B$. The difference here is that the buffer functions act purely as a delay with constant size - the deadline for receiving a value from $A$ was the only thing requiring it to be a certain size so now we just forward whatever comes out of the buffer.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_a \vdash A \ceval nil; S_a' \\
        \Gamma; S_b \vdash B \rightarrow V_b; S_b' \\
        \buffer(b, V_b) = b', V
        \end{matrix}
    }{
        \Gamma; (true, b, S_a, S_b) \vdash A \fby B \rightarrow V; (true, b', S_a', S_b')
    }[\text{Eval-fby-B}]
    \end{align*}

    When choked, fby chokes its arguments and does not update anything. This can happen irrespective of its latch state or buffer contents, since we forced $A$ and $B$ to yield $nil$ and all of fby's transitions depend on non-nil values from one of the operands.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S_a \vdash A \ceval nil; S_a' \\
        \Gamma; S_b \vdash B \ceval nil; S_b'
        \end{matrix}
    }{
        \Gamma; (C, b, S_a, S_b) \vdash A \fby B \ceval nil; (C, b, S_a', S_b')
    }[\text{Eval-fby-C}]
    \end{align*}

    \paragraph{next}

    Next expressions need to ensure that they drop the first non-nil value they receive. To do this, they choke $E$ while counting down its pre-delay ($N$, stored in the state by the init stage) plus one, thus beginning to yield value from $E$ on the timestep after $E$ has yielded its first value.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S \vdash E \ceval nil; S' \\
        N > 0
        \end{matrix}
    }{
        \Gamma; (N, S) \vdash \nextop E \rightarrow nil; (N-1, S')
    }[\text{Eval-next-countdown}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S \vdash E \rightarrow V; S'
        \end{matrix}
    }{
        \Gamma; (0, S) \vdash \nextop E \rightarrow V; (0, S')
    }[\text{Eval-next-forward}]
    \end{align*}

    When choked, next still counts down. This is to ensure the timing matches the computed pre-delay.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S \vdash E \ceval nil; S' \\
        N > 0
        \end{matrix}
    }{
        \Gamma; (N, S) \vdash \nextop E \ceval nil; (N-1, S')
    }[\text{Eval-next-countdown-C}]
    \end{align*}

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma; S \vdash E \ceval nil; S'
        \end{matrix}
    }{
        \Gamma; (0, S) \vdash \nextop E \ceval nil; (0, S')
    }[\text{Eval-next-C}]
    \end{align*}

    \paragraph{Id}

    Identifiers are both in charge of looking up the value pointed to by $I$ and buffering it by the appropriate amount. This is so that any downstream operators receive correctly synchronised values - ensuring the buffers are the correct size is one of the main responsibilities of the init phase.

    As elsewhere, $\buffer(b, V) = b', V_b$ is a shorthand for pushing $V$ onto $b$ and popping $V_b$ from the from of $b$, producing a new queue $b'$. $b$ acts as a queue that is initially full of $nil$ values, causing a delay as long as the buffer in the values yielded by this expression compared to the actual timing of the values at $\Gamma(I)$.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma(I) = V \\
        \buffer(b, V) = b', V_b
        \end{matrix}
    }{
        \Gamma, b \vdash I \rightarrow V_b; b'
    }[\text{Eval-id}]
    \end{align*}

    Choke evaluation still cycles the buffer but drops the value the comes out of the buffer. This is in order to maintain the illusion that $I$ is yielding values at a delay equal to buffer size - if instead we pushed a $nil$ value onto the back of the queue, we would effectively be replacing "future" values with $nil$s instead of the one that would be yielded at the current timestep.

    \begin{align*}
    \frac{
        \begin{matrix}
        \Gamma(I) = V \\
        \buffer(b, V) = b', V_b
        \end{matrix}
    }{
        \Gamma, b \vdash I \ceval nil; b'
    }[\text{Eval-id-C}]
    \end{align*}

    \section{Implementation}

    The implementation of Pixy presents a few specific challenges. For starters,
    to be able to determine the delays of variables, you need to be able to
    solve what amounts to a constraint solving problem. On top of that, you
    have to be able to properly line up the buffers, and ensure that all access happen seamlessly.

    The implementation can be found at (\url{https://github.com/fhackett/pixy-lang}).

    \subsection{Constraint Solving}
    The constraint problem at hand is in the domain of Linear Integeger Arithmetic. This consists
    of linear equations like $x * 2 = y$ and $max(x,y) > 4$. To solve this, we implemented a
    simple constraint solver in Haskell, CLPHS (\url{https://github.com/TOTBWF/clphs}).

    \subsection{Buffering}
    Implementing buffering poses a few challenges. For efficiency reasons, we want to minimize data duplication.
    This means that having individual buffers for each variable reference is out of the question. Instead, we compute
    offsets for each variable reference, and use those to index into our buffers. For example, consider the code:
    \begin{lstlisting}
        fib = 0 fby 1 fby (fib + next fib)
    \end{lstlisting}
    \lstinline{fib} must have a 2-cell buffer,as references access it at 2 distinct points in time.

    The basic sketch of the algorithm for computing the offsets is as follows:
    \begin{enumerate}
        \item Traverse the tree, and recursively build up a map of variables to delays.
        \item Whenever you need to union together those maps, (For example, on a binop), take the
            larger delay.
        \item Whenever you hit a where clause, union together all of the maps generated by the variable
            expressions, and then create buffers for each variable in the where clause.
    \end{enumerate}
- can the system afford to wait? What other components might be affected?
    It is worth noting that while the algorithm is simple in concept a lot of care needs to go into correctly calculating and combining variable delays - this algorithm is heavily context-based. That said, we have achieved decent results on our test cases with this setup.

    \subsection{Test cases}

    We have a suite of runnable test cases available at (\url{https://github.com/fhackett/pixy-lang/tree/master/samples}).

    \subsubsection{Prime sieve}

    Our most complex example is a naive constant-memory implementation of Erastothenes' prime sieve:

    \begin{lstlisting}
div(c,o) = c % o == 0 || o * o > c

main() = prime where {
    candidate =
        5 fby (if div(candidate, odds) then
            candidate + 2 else candidate)
    odds =
        3 fby (if div(candidate, odds) then 3 else odds + 2)
    isPrime =
        if div(candidate, odds) then
            (candidate % odds != 0) else nil
    prime =
        2 fby 3 fby (if ?isPrime && isPrime then
            candidate else nil)
}
    \end{lstlisting}

    Notably, while it will not always provide an answer at every timestep the algorithm will report either a result or $nil$ in constant time every step. This means that if this algorithm were used as part of a larger program it would not slow down the rest of the program's execution over time, unlike a naive imperative or functional implementation of the same program.

    \subsubsection{Running average}

    We also provide a running average calculation that exercises Pixy's stateful custom operators. For any stream consisting of numbers, $avg(n)$ will yield a correct running average at every timestep. Here we show how it can be applied to a sequence of increasing even numbers.

    \begin{lstlisting}
avg(x) = sum/count where {
    sum = x fby (sum + next x)
    count = 1 fby (count + 1)
}

main() = avg(n) where {
    n = 2 fby (n + 2)
}
    \end{lstlisting}

    \section{Comments and future work}

    There is clearly a lot more to be done to the language. At the evaluation level we need to properly refine recursive operator application, and most of our ideas for a type system are discussed but not fully explored. While we have described some of these ideas inline, here we will present a set of possible future directions and our thoughts on them.

    \subsection{Circuits and lowest common denominator of computation}

    As we worked more on the language, we discovered that Pixy is not very powerful compared to most languages. In its current form it is unable to perform basic tasks we might expect from a general-purpose language like performing tasks that require unbounded memory or interacting with the world.

    While on its own this can be seen as a weakness in the language, we noticed at the same time that due to its fully static control flow Pixy can be used to describe algorithms that would work on more unusual platforms such as field programmable gate arrays - since it derives from Lucid, which had both a text and a diagram representation, and removes most of Lucid's dynamism, Pixy should be able to express algorithms targeted at the lowest common denominator of computation: digital circuits. Since we also envisage interesting implementation strategies for commodity hardware, once we build a better base for the language it may be interesting to investigate what happens if you apply Pixy to circuit design.

    \subsection{State of the type system(s)}

    Current discussion places Pixy type system as 2-dimensional along the data and control axes. On the one hand we envisage a traditional type system similar to that used in typical functional programming languages that would be used to make sure basic data is coherent across all program timesteps, and on the other we hope to experiment with a temporal logic-based \cite{ltlwiki} system that tries to reason about tricky corner cases and synchronisation problems that span multiple timesteps.

    Part of our work on synchronising pre-delays might be generalisable to become a part of the temporal type system, and can already catch some basic programming errors such as leaving "gaps" in between the left and right operand of a fby.

    One strategy we have in mind for implementing the temporal logic system is coinductive symbolic evaluation across different possible program states, taking advantage of the finite state space we describe in the init phase in order to argue termination of the checking process.

    \subsection{Expressing iteration}

    One notable missing feature from Lucid is support for nested iteration. Here we explore why this feature disappeared and how we expect to replace it.

    In Lucid one would have been able to write something like this:

    \begin{lstlisting}
c until c eq N where
    N is current n;
    c = 1 fby c+1;
end
    \end{lstlisting}

    Given some external variable $n$, for each value of $n$ this would yield the sequence $1 \dots n-1$. While this is a reasonable program to write, within the context of Pixy's state machine-based semantics one might ask the question: if used within another program, when will these values be yielded? For this program, the question does not have an answer. Since for every timestep a new value of $n$ may be presented, there can exist no state machine that will yield more than one value in a single timestep.

    While this can be seen as a limitation of Pixy relative to Lucid's expressiveness, this also demonstrates that Pixy's state machine model enforces that each timestep of a Pixy program will take a known amount of time - it is impossible to write a program whose timesteps take an unbounded amount of time to execute.

    The closest alternative we imagine is Turing-incomplete synchronous recursion, also referred to as recursive operator application earlier in this document. This would allow processing batches of data in one timestep, but using a mechanism more similar to C++'s templates. All calls would be expanded at compile time and the maximum recursion depth would be checked in order to ensure that a deadline can be given and met for the completion of each program timestep.

    For truly infinite computation, the programmer would have to express it over multiple timesteps. This is reasonable, since Pixy specifically prioritises liveness and concurrency over expressiveness. It also encourages developers to think about the liveness of their systems.

    \subsection{Pixy's representation and treatment of numbers}

    In order to reason precisely about the symbolic evaluation of Pixy terms we may need to revisit and refine Pixy's model of numbers. Since we assume any value in Pixy will have a bounded size, it becomes important to define precisely how different operations interact with this size. What happens if you add 1 to the largest number a variable could represent? For sized types as described in \cite{abel} to work we also need to reason about lower bounds - what happens if you inductively subtract 2 from an unknown value? How do we deal with the possibility of reaching $1-2$ while unfolding a recursive operator application?

    We suspect some of these issues will be clarified by implementing our planned explicit type system, but we can also see that the subtleties of some of these numerical edge cases may in turn inform what our type system should look like.

    \section{Conclusion}

    While working on the formal description of the language and the implementation, we found that generating a formal description of a new language required a large amount of effort to go into understanding that language. We initially expected to be able to quickly describe Pixy's evaluation process then move on to devising a type system, but repeatedly discovered that we did not truly understand all of the ramifications of Pixy's semantics.

    While a naive set of dynamic semantics for dataflow languages is not that hard to produce, reasoning about exactly what values need to be buffered due to (sometimes indirect) lookaheads and what can be safely discarded proved one of the biggest challenges. Our major breakthrough in this area turned out to be separating the very hard problem of the general-purpose synchronisation of heterogenously timed streams from the more tractable problem of ensuring that each state machine was receiving values that came from the same timestep by calculating correct pre-delays.

    The other major discovery we made was choke-evaluation. We had not properly considered the issue until we ran into examples where fully evaluating a non-taken branch of an if statement would lead to an error at the same time as finding examples where skipping non-taken if branches was also incorrect. Trying to reconcile the two lead to some interesting discussions on what would be the semantically correct option, and we went through several alternatives before arriving at choke-evaluation.

    Overall, the back and forth of designing and implementing a new programming language while constantly discovering new issues with both the implementation and theory has been educational. We've gained a lot of experience in writing out a consistent formal description of a language, and we've gained a much better appreciation for what it takes to get from a basic concept to a working theory.

    Going forward we hope to follow up on the future work we identified and continue learning more about what it takes to make the general idea of Pixy work in reality.

    \begin{thebibliography}{9}
        \bibitem{abel}
        Abel A.
        \textit{MiniAgda: Integrating Sized and Dependent Types}
        Department of Computer Science,
        Ludwig-Maximilians-University Munich, Germany
        \bibitem{wadgeashcroft85}
        Wadge W. W.,
        Ashcroft E. A.
        \textit{Lucid, the Dataflow Programming Langugage}
        Academic Press, 1985
        \bibitem{bencina}
        Bencina, R.
        \textit{Real-time audio programming 101: time waits for nothing}
        \url{http://www.rossbencina.com/code/real-time-audio-programming-101-time-waits-for-nothing}, accessed 2018/04/25
        \bibitem{ltlwiki}
        Wikipedia
        \textit{Linear temporal logic}
        \url{https://en.wikipedia.org/wiki/Linear_temporal_logic}, accessed 2018/04/25
        \bibitem{propwiki}
        Wikipedia
        Propagation delay
        \url{https://en.wikipedia.org/wiki/Propagation_delay}, accessed 2018/04/25
    \end{thebibliography}

\end{document}


To arrive at a computable analysis we must abstract the infinite concrete state
space with a finite abstract state space.
%
As we sketch the abstract state space in the AAM methodology we will hint at
corrosponding changes that need to take place in the semantics.
%
The full abstract semantics for the final abstract state space will be given in
section~\ref{section:AAMByExample:AbstractSemantics}.

%--%

\subsubsection{Cutting Recursion}
\input{sections/03AAMByExample/02AbstractStateSpace/00Cut.tex}

\subsubsection{Address and Time Abstraction}
\input{sections/03AAMByExample/02AbstractStateSpace/01AAM.tex}

\subsubsection{Introducing Nondeterminism}
\input{sections/03AAMByExample/02AbstractStateSpace/02NonDet.tex}

\subsubsection{Delta Abstraction}
\input{sections/03AAMByExample/02AbstractStateSpace/03Delta.tex}


 \doublespace

 \section{Abstract} We hypothesize that treatment of individual measures of music\footnote{i.e. four beats in 4/4 time}, as the basic units of analysis across a large corpora of compositions, will allow generation of uniquely identifying probabilistic grammars. These grammars are composed of Markov-chains, which are generated by incrementing through each measure of a selected piece while tallying both that measure’s features, and those of the measures which immediately precede and succeed it. Initially, the radius, or neighborhood, is set at $r = 1$ so that one prior and following measure is evaluated for each increment. The neighborhood is then increased, until the hypothesis can be confirmed or rejected. Confirmation or rejection of the hypothesis occurs when a given radius either proves sufficient for intuitively satisfactory classification of analyzed pieces, or the radius is increased to such an extent that distinctions between pieces cease to be fine-grained enough to allow for comparison of pieces in our chosen corpora. Successful research in this area would entail quantitative characterization and comparison of musics both within, and across cultures.