%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Lachaise Assignment
% LaTeX Template
% Version 1.0 (26/6/2018)
%
% This template originates from:
% http://www.LaTeXTemplates.com
%
% Authors:
% Marion Lachaise & François Févotte
% Vel (vel@LaTeXTemplates.com)
%
% License:
% CC BY-NC-SA 3.0 (http://creativecommons.org/licenses/by-nc-sa/3.0/)
%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

%----------------------------------------------------------------------------------------
%	PACKAGES AND OTHER DOCUMENT CONFIGURATIONS
%----------------------------------------------------------------------------------------

\documentclass{article}
\usepackage{hyperref}
\usepackage{bm}
\usepackage{booktabs}
\usepackage{titlesec}

\input{structure.tex} % Include the file specifying the document structure and custom commands
\setlength{\parskip}{0.2em}

\titlespacing*{\section}
{0pt}{1em}{1em}
\titlespacing*{\subsection}
{0pt}{0.5em}{0.5em}

\def\x{{\mathbf x}}
\def\X{{\mathbf X}}

\def\L{{\cal L}}

%----------------------------------------------------------------------------------------
%	ASSIGNMENT INFORMATION
%----------------------------------------------------------------------------------------

\title{ \large ECS7012P Music and Audio Programming • Spring 2021 \\ \huge Final Project: Zero-latency convolution on Bela} % Title of the assignment

\author{Christian Steinmetz\\ \texttt{c.j.steinmetz@qmul.ac.uk}} % Author name and email address

\date{Queen Mary University of London --- \today} % University, school and/or department name(s) and a date

%----------------------------------------------------------------------------------------

\begin{document}

\maketitle % Print the title

\section*{Abstract}

Convolution has many applications in audio, such as equalization and artificial reverberation.
Unfortunately, performing convolution with large filters, as is common in artificial reverberation,
can be computationally complex when utilizing the direct-form implementation.
Block-based convolution that takes advantage of the FFT, performing filtering in the frequency domain,
can significantly accelerate this operation, but comes at the cost of higher latency.
To balance these forces, so-called `zero-latency' convolution has been proposed,
which breaks up the filter so that the early part is implemented with the direct form and later blocks are implemented with the FFT.
In this project, the goal is to implement a `zero-latency' convolution on the Bela platform to enable real-time effects utilizing large convolutions.
Furthermore, the ability to utilize this implementation for the creation of a temporal convolutional neural network (TCN) was also investigated.
While this implementation was able to implement a single large convolution for artificial reverberation,
this alone utilizes a significant portion of the CPU resources on Bela,
A simple real-time convoluational reverb effect is demonstrated,
along with a UI that enables users to adjust the quality of the reverb.
Due to the high computational cost of a single convolution, cascading more convolutions in series as is common to implement a TCN was not feasible.
Future work will incorporate additional optimizations, as well as investigate the applicability of sparsity,
in order to enable large kernel convolutions for the implementation of TCNs in real-time neural audio effects.

\section{Introduction}\label{sec:intro}

The discrete convolution between two signals $x[n]$ and $h[n]$ is given by
\begin{equation*}
    y[n] = \sum_{k=0}^{N-1} h[n] \cdot x[n-k],
\end{equation*}
where $N$ is the number of samples in $h[n]$, often referred to as a filter.
It is common to use $*$ to denote the convolution,
which can then be written more concisely without the summation as
\begin{equation*}
    y[n] = x[n] * h[n].
\end{equation*}

Convolution is a foundational operation in signal processing, which has an array of applications.
From a signal processing perspective, convolution enables the application of finite impulse response (FIR) filters.
These filters are composed of a series of coefficients, which produce an output as a linear combination of the current and past outputs.
In audio, filtering is one of the predominant use cases, which enables equalization~\cite{valimaki2016all} and artificial reverberation~\cite{valimaki2016more}.
Interestingly, room reverberation can be modeled as an FIR filter given a sufficient number of filter coefficients.
This entails modeling the room as a linear-time invariant (LTI) system, which is often a reasonable assumption.
The impulse response of the room must be first measured in order to create a filter.
This process requires both a transducer to generate a signal in the room as well as a receiver to record the resultant sound pressure.

While early approaches considered impulsive excitation signals,
like popping a ballon, firing a gun, or simply playing an impulse over a loudspeaker,
these approaches generally are considered less accurate in capturing the room response.
This is due to two main reasons.
First, these excitations signals are often not sufficiently wideband, and therefore exhibit some non-flat frequency response.
This likely colors the resulting room impulse response (RIR) measurement.
Furthermore, playing an impulse over a loudspeaker is also not ideal due to the nonlinearities of the speaker and amplifier.
Second, these approaches are also very prone to corruption by environmental noise.
Therefore, common methods for measuring RIRs today generally involve
either exponential sine sweeps (ESS)~\cite{farina2000simultaneous}, which spread the energy of the impulse out across time,
as well as the maximum length sequences (MLS), which utilizes multiple noise signals repeated over time~\cite{stan2002comparison}.

\subsection{Implementation}

Assuming an impulse response of a room has been recorded, the next step in the process is to perform the convolution with an incoming audio signal.
The common way to implement such an FIR filter is by using what is known as the direct form convolution
\begin{equation*}
    y[n] = h_0 x[n] + h_1 x[n-1] + h_2 x[n-2] + h_3 x[n-3] + \dots + h_{N-1} x[n-N-1]
\end{equation*}
where each output of the filtering process $y[n]$ is computed as given in the previous summation,
considering a linear combination of past values, with the values of the filter $h_n$ multiplied by the appropriate values.
This is shown graphically in the block diagram in Figure~\ref{fig:direct}.
While direct form convolution is quite efficient when the number of coefficients remains small,
this operation becomes costly as the number of coefficients increases.
This results in linear scaling, with $O(N)$ multiplies per output point, where $N$ is the number of filter coefficients.
This is problematic for the application of RIRs where the filter may contain upwards of
80,000 coefficients for a 2 second long impulse response at a sample rate of 44.1\,kHz.
This entails over 80,000 multiplies that must occur within a single sampling period, which is 22.6\,$\mu$s.

\begin{figure}
    \centering
    \includegraphics[width=0.7\linewidth]{figures/direct.pdf}
    \caption{Block diagram for the finite impulse response filter implemented with a direct form convolution.}
    \label{fig:direct}
\end{figure}

\subsection{Frequency domain convolution}

This complexity can be reduced by taking advantage of the convolution theorem,
which states that multiplication in the frequency domain is equivalent to convolution in the time domain.
Therefore, the convolution can be computed by first transforming the input signal and the filter to the frequency domain.
The complex multiplication (recall that the Fourier transform of a real valued signal is a complex signal)
of $X[m]$ and $H[m]$ will produce the convolution of the original signals in the frequency domain
\begin{equation*}
    \mathcal{F}\big[ x[n] * h[n] \big] = X[m] \cdot H[m].
\end{equation*}
The time domain output of the convolution can be recovered by taking the inverse fourier transform,
\begin{equation*}
    y[n] =  x[n] * h[n] = \mathcal{F}^{-1}\big[ X[m] \cdot H[m] \big].
\end{equation*}
Applying the filter directly in the frequency domain enables us to take advantage of the fast-Fourier transform (FFT).
This results in log scaling, with $O(\log N)$ multiplies per output point.
As the number of coefficients in the filter $N$ increases, the frequency domain convolution becomes significantly more efficient than the direct form convolution.

\begin{figure}
    \centering
    \includegraphics[width=0.7\linewidth]{figures/block-conv.pdf}
    \caption{Block-based FFT convolution between $x_n$ and $h$.}
    \label{fig:block-conv}
\end{figure}

In practice, frequency domain convolution is implemented with a block convolver, as shown in Figure~\ref{fig:block-conv}.
In order to avoid time aliasaing, i.e. to capture the resultant ringing of the convolution between the filter and the input signal,
an FFT size twice as large of the input blocks is utilized, padding the second half of the time domain input with zeros.
The frequency domain representation of the filter block $H$ is computed using the FFT, along with incoming blocks of $x[n]$ of equal size.
After complex multiplication in the frequency domain,
the IFFT is performed on the result to produce the time domain output of the filter, $y[n]$.
In practice, the impulse response is often very large and therefore it is commonly split into smaller
blocks, often a power of 2 in length ($N \in$ 32, 64, ... 512, ...).
An overlap-add or overlap-save method can then be used to accumulate the outputs from the convolution with each block,
adding the filtered signal at the proper offset within the output signal.

\subsection{Low-latency convolution}

While frequency domain convolution achieves superior efficiency as compared to the direct form convolution with larger filters (often $N > 128$),
it imparts a minimum latency of $N$ samples, given by the FFT block size.
This is due to the fact that $N$ samples must first be accumulated from $x[n]$ before performing the computation.
This contrasts with direct form convolution which requires only a single input sample to produce the next output,
but requires significantly more computation when the filter is large.

Approaches that balance these two forces to achieve computational efficiency and achieve low latency have been proposed~\cite{gardner1994efficient, muller1999low, hurchalla2010time, wefers2015partitioned}.
The implementation in this work focuses on the one proposed by Gardner~\cite{gardner1994efficient},
which is also referred to as the `zero-latency' convolution.
While signal processing is never completely free of latency since some time is required in the hardware,
in this case a 'zero-latency' convolution is an operation wherein the only delay imparted by the system is that of carrying out the computation.
This is another way of saying that no time is needed to `wait around', while samples accumulate in a buffer, imparting latency as they do in the block-based convolution.

In order to reduce latency while reducing the computation complexity of the convolution,
this approach utilizes a combination of the direct form and frequency domain convolution implementations.
As in the block-based frequency domain convolution, the impulse response is broken into separate blocks.
To achieve the optimal latency and computation tradeoff these impulse response is split into blocks given the pattern in Figure~\ref{fig:blocks}.
The size of the first block $h_0$ is set to $2N$ samples, where N is selected as the block size where the FFT convolution becomes more efficient than the direct form convolution.
Then $h_1$ and $h_2$ are of size $N$, followed by two blocks of size $2N$, and so on.
The first block is implemented with a direct form convolution immediately in the main processing loop.
Since the size of this block is selected to be sufficiently small there will be no problem in computing the output within time.
Then, separate tasks are scheduled to compute the following larger blocks when enough input samples have been collected.
These tasks are scheduled with decreasing priority, such that the smallest blocks will always be computed with a higher priority
since they are needed more rapidly than larger blocks.

\begin{figure}[h]
    \centering
    \includegraphics[width=\linewidth]{figures/blocks.pdf}
    \caption{Splitting the impulse response in blocks.}
    \label{fig:blocks}
\end{figure}

\section{Convolutional neural networks}\label{sec:conv}

Changing gears, convolution has found utility in modern deep learning.
The convolutional neural network (CNN)~\cite{lecun1998gradient} has been of significant interest since
the success of AlexNet~\cite{krizhevsky2012imagenet} in 2012,
which outperformed previous image classification models by a significant margin.
These neural network architectures are conceptually quite simple,
but a multitude of architectural variations have been investigated.
Nevertheless, the general design consists of a convolution operation,
followed by downsampling, and then a nonlinear activation function.

Unlike RIRs, most CNNs in computer vision utilize very small filters.
This is often $3 \times 3$, $5 \times 5$, or $7 \times 7$ filters.
While these filters have very limited receptive field,
they gain larger context by cascading many convolution operations in series,
with interleaved downsampling operations.
This achieves and effectively large filter by composing many small filters,
which are efficient to implement using the direct form convolution.
While there has been some interest in the applicability of FFT-based convolution
to accelerate these networks~\cite{vasilyev2015cnn, lavin2016fast, abtahi2017accelerating, abtahi2018accelerating},
this technique has not seen widespread adoption
due to the somewhat limited gains when using such small filters.

\subsection{Temporal convolutional networks}

While there has been significant interest in 2D CNNs both in computer vision as well as in audio, operating on spectral representations,
the temporal convolutional network (TCN)~\cite{bai2018tcn} has seen growing interest in end-to-end audio processing architectures~\cite{rethage2018wavenet}.
First popularized for audio by WaveNet~\cite{oord2016wavenet}, these architectures operate on waveform using a series of dilated 1-D convolutions.
Since achieving sufficient receptive field with small filters at audio samples rates is not feasible by adding simply additional layers,
dilated convolutions have become a popular technique for achieving large receptive fields efficiently.
Dilated convolutions take a small convolutional filter and increase its effective size by inserting zeros within the filter.
A larger dilation factor inserts more zeros between values of this filter.
Since the multiplication between the input and any of the zeroed values in the filter will also be zero, these computations can be ignored.
This provides a high computational cost savings via sparsity as compared to a dense filter of comparable size.
Interestingly, this approach of very sparse filters have brought acceptable performance.

\subsection{Efficient temporal convolutional networks}

Independent of dilations, one of the easiest ways to increase the efficiency of TCNs is to reduce the number of layers,
since these computations must occur in series.
However, reducing the depth of the network significantly reduces the receptive field, which is critical for performance.
Even when utilizing dilated convolutions as is standard in TCNs, where the dilation factor grows as a function of the depth,
given by $d_n = 2^n$, where $n \in 1, 2, ..., N$ for an $N$-layer network, 20 to 30 layers are often required.
This present a computational load that restricts real-time performance on CPU.

In contrast, utilizing fewer layers with larger kernels is one route to reducing the computational load~\cite{steinmetz2021efficient}.
This comes at the cost of increased sparsity in the convolutions, yet can enable real-time operation on CPU.
Another potential route to address this could be the use of very few layers, potentially as little as two,
which utilize very large filters, and are implemented using a block-based FFT convolution.
This enables significant receptive field, and could potentially see significant benefit from
FFT convolution, whereas previous FFT based approaches for small filters have been less impressive.
While block-based convolutions significantly reduce the computational cost,
as introduced previously, the block size imparts a lowerbound on the latency of the system.
While this is not often of significant concern in computer vision applications,
the ability for not only efficient, but also low-latency operation, is critical in real-time audio effects.

For that reason, there is the potential to first train a TCN that uses very large filters, but few layers,
using a standard large block size FFT convolution.
This will provide the most computationally efficient operation when latency is of no concern.
However, it will also be possible to use a different convolution implementation during inference as compared to training.
Treating the weights of the convolutional layers in the pre-trained network as large filters means that the
previously proposed low-latency convolution could be used to implement the neural network.
This is partly the motivation for the investigations presented in this project.

\section{Design}

As introduced in Section~\ref{sec:intro}, the design of the zero-latency convolution implementation is based heavily on the system proposed by Gardner~\cite{gardner1994efficient}.
In this configuration, the impulse response $h$ is split into a number of blocks of increasing size, as shown in Figure~\ref{fig:blocks}.
To implement this in code, three classes were designed.
There are two lower level classes, \texttt{DirectConvolver} and \texttt{FFTConvolver},
which implement an isolated direct form convolution and block-based FFT convolution respectively.
Each of these classes access an external input and output circular buffer in order to read current and past values,
and then write filtered values to the appropriate place in memory.
Instances of these classes are instantiated in the higher level \texttt{ZLConvolver} class,
which implements a complete zero-latency convolution.

The \texttt{DirectConvolver} class is the simplest.
During initialization, this class requires a vector containing the coefficients of the filter,
along with the index of this filter block within the full filter.
For all practical cases this index is 0, since the direct convolver is generally used to process the first block of the filter.
At initialization pointers to vectors containing the input and output circular buffer are also required.
These pointers will be used during the processing phase in order read and write the appropriate data,
and will be shared among all of the convolvers in the system.

In order to operate the convolver the \texttt{process()} method is called,
passing a index to the current position to read from within the circular buffer.
Then a single output value will be computed by iterating over the filter coefficients, stored in memory,
accumulating the value as a function of the current and past output values.
Finally, this single value is written to the correct position within the output circular buffer,
and the output write pointer is incremented by a single step.

The \texttt{FFTConvolver} is somewhat more complex than the direct convolver, but shares many of the same design elements.
During initialization this class also requires a vector containing the filter coefficients, the offset in the complete filter,
and pointers to the input and output circular buffer.
In addition, the FFT size must be provided and it must always be twice as large as the number of samples in the supplied filter.
In \texttt{setup()} the Bela \texttt{Fft} objects are instantiated for both the current input block \texttt{fftX},
the associated filter block \texttt{fftH}, along with a FFT buffer \texttt{fftBuffer} for storing the results of the complex multiplication.
After setting the FFT size for these objects, the values from the provided vector of filter coefficients is loaded into the
time domain attribute of the \texttt{fftH} object, and the FFT is precomputed.

The process of launching the FFT convolver is somewhat more complicated since it is designed such that it can operate on its own thread.
The convolver must first be queued by passing an index of where to read from the input circular buffer.
This index corresponds to the most recent value read into the buffer, and the convolver will read backwards
up to one half the FFT size.
After being queued, the \texttt{process()} method must be called to begin the process.
This is designed so that the main thread in \texttt{render()} can queue the convolver with the correct read pointer,
but the process will not be launched until there is time, which will be explained in more detail in the following paragraph.
Within \texttt{process()}, values from the input buffer will be read for the computation, up to one half the FFT size.
The remaining values for the input block will be zeroes. Then the FFT of the input block will be computed.
To apply the filter, complex multiplication must be carried out between the frequency domain representations.
This can be achieved very simply.
Consider two complex numbers $a + bi$ and $c + di$.
Their produce can be computed in a similar manner as if these were binomials.
\begin{align*}
    (a + bi) \cdot (c + di) \\
    ac + (ad)i + (bc)i + (bd)i^2 \\
    ac + (ad + bc)i - (bd) \\
    (ac - bd) + (ad + bc)i
\end{align*}
After the complex multiplication, the IFFT is computed,
which produces the same number of output samples as the FFT size.
These values are then written to the appropriate place in the output buffer.
Since the initial output write index was initialized as the offset of this block within complete filter,
the output samples should be written to the appropriate location .
As is the usual, modulo arithmetic is utilized to ensure that indices in the circular buffers are wrapped around.
The final step involves incrementing the write index.
Even though the convolution will produce FFT size number of samples,
the write index is only advanced by one half of the FFT size, since this was the number of samples from the input block.

The \texttt{ZLConvolver} class encapsulates the convolver objects to create an entire convolution system.
This class is significantly more complex than the lower level classes, and requires managing multiple threads
to run individual processes for each of the \texttt{FFTConvolver} objects.
During initialization the most important parameter is a path to the appropriate wav file from which to load the impulse response.
The provided \texttt{MonoFilePlayer} class is then utilized to read this data from disk.
The most important step during this process is that of splitting the complete impulse response into a set of blocks, each with their own convolver.
The original pattern from Figure~\ref{fig:blocks} is followed, where the FFT size is always twice the block size.
After reading the appropriate number of samples from file player, a new convolver is created for that block.
Only one \texttt{DirectConvolver} is created for the first block, while \texttt{FFTConvolver} objects are stored in a vector for the remaining blocks.
The final step during initialization is the creation of a unique thread for each of the \texttt{FFTConvolver} objects.
This is achieved with \texttt{Bela\_createAuxiliaryTask}, storing the resulting task in a vector.
In order to access the correct convolver, each task passes a pointer to the convolver instance as an argument.

Along with the convolvers, the \texttt{ZLConvolver} class also maintains the input and output circular buffers.
When calling the \texttt{process()} method, the next input sample is written into the input buffer.
Next the \texttt{DirectConvolver} is called to produce the result of this filter block.
After this, all of the \texttt{FFTConvolvers} are iterated over in order to determine if enough samples
have accumulated for them to begin processing the next block.
To facilitate this, the \texttt{convolverBufferSamples\_} vector contains the number of samples
that have been read since the last time each convolver was run.
If the value in a position of this vector is equal to one half the FFT size of the respective convolver,
then this convolver task is launched from the \texttt{convolverThreads\_} array.
Each convolver task is created with a priority based on the FFT size.
Smaller blocks have a higher priority since they occur more rapidly, while larger blocks are needed less frequently.
It was found that due to the tight timing constraints, without using these priority values it was not possible to achieve smooth performance.

Following the launching of the convolvers,
the next value from the output buffer is read, so that it can be written to the audio output on Bela for the current frame.
This value is then set to 0 in the output circular buffer for the next time it is utilized.
As a final step the output read pointer is incremented for the next sample.
Before passing the output sample to the audio output, a wet/dry mix is applied, along with an optional tanh nonlinearity.

\emph{Note: As a starting point, the design of the code was based upon the code in Lecture 20: Phase vocoder.}

\subsection{Optimizations}

Gardner~\cite{gardner1994efficient} lists a number of optimizations including:

\begin{enumerate}
    \item Precomputing the spectra for all blocks of the filter $h$.
    \item Utilize real valued FFT to exploit symmetry.
    \item Expoloit symmetry in the complex multiplication.
    \item Reuse input spectra from $x$ whenever possible.
    \item Reuse input spectra from small blocks for large block computations.
\end{enumerate}

In this implementation the first three optimizations are considered.
The final two optimizations are likely to further improve performance,
but will require more significant rework of the \texttt{FFTConvolver} class.

\subsection{User interface}

\begin{figure}
    \centering
    \includegraphics[width=0.6\linewidth]{figures/ui.png}
    \caption{User interface}
    \label{fig:ui}
\end{figure}

A number of sliders are provided in a simplistic UI as shown in Figure~\ref{fig:ui}.
This enables user interaction with the convolution algorithm in real-time.
The first slider, named \emph{Room} enables the user to switch between six different pre-loaded RIRs.
These RIRs are sourced from various locations and include a range of rooms, from small, medium, to a large church, as well as plate reverb.
Setting the \emph{Room} slider to 6 will result in a bypass of the reverb effect.

The \emph{Max blocks} control adjust the maximum number of blocks of the filter applied.
As the value is reduced, and blocks at the end of the filter beyond the maximum allowed will be excluded.
This can be utilized as a way to shorten the reverb time, and also reduce the computational load.

In addition, the \emph{Sparsity} control enables another way to ignore blocks in the convolutional filter.
Instead of only removing blocks from the end of the filter, this control will progressively remove
blocks from the across the entire filter.
While a very low \emph{Sparsity} provides the best sonic results, higher values produce an interesting delay like effect,
due to the periods of sparsity in the resultant effective filter.

A simple toggle is provided to activate a tanh nonlinearity.
This is followed by \emph{Wet} and \emph{Dry} mix controls, which are common in most reverb effects.
Finally \emph{In gain} and \emph{Out gain} controls are provided in order to either push the nonlinearity,
or reduce the output gain in the case of loud impulse responses.

\section{Evaluation}\label{sec:eval}

As the first part of the evaluation, the runtime of both the \texttt{DirectConvolver} and \texttt{FFTConvolver} are compared.
To do so, the time required to compute $N$ output samples with a filter of length $N$ was averaged over 100 blocks.
To eliminate the effect of the block size in the these measurements, a large block size of 4096 samples was used.
The results are reported in Figure~\ref{fig:runtime}.
This agrees with our intuition, where as the filter size grows the compute time for the direct form implementation increases linearly.
For the the FFT implementation, the computation time grows significantly slower as the filter size and input size increase.
Looking closely at the results, after the filter size increases beyond 32 samples, the FFT computation becomes more efficient.
This was then used as the base value for $N$ when setting up the \texttt{ZLConvolver}.

\begin{figure}[h]
    \centering
    \includegraphics[width=0.65\linewidth]{figures/runtime.pdf}
    \caption{Runtime for filters of size $N$ and input blocks of size $N$.}
    \label{fig:runtime}
\end{figure}

As a more informal evaluation, the ability to operate at different block sizes was investigated.
Performance is manageable down to a frame size of 8 samples, but underuns occur at a frame size of 4.
Things function well at frame sizes of both 16 and 32, but utilizing a frame size larger than 32 causes an issue in the current implementation.
This is due to the choice of $N=32$, since when the frame size is 64, all 64 samples must be computed by the end of the period.
Due to the auxiliary tasks that will be computing the following samples, these lower priority tasks will not be completed by the time that
the main thread reads values from the output buffer and sends them to the audio output.
Adjusting $N$ to be a larger value does not help this significantly.
Instead, this would require a way to ensure that FFT convolvers
on other threads complete their computation for output samples in the
current frame before the values are read out and sent to the audio output.

Since one of the other purposes of this work was to investigate the feasibility of implementing a real-time neural audio effects
with the low-latency convolution the ability to run multiple convolutions in series was tested.
Unfortunately, the computational cost of a single convolution on Bela is already quite high, and therefore it was not possible
to go beyond two medium sized convolutions. This is simply not enough to implement an effective TCN,
which likely requires at least two layers, with the first layer featuring a multiple channel convolution.
For that reason, the final design of the program is focused on the simple convolutional reverb, which is fully functional.

\subsection{Demonstration}

A short demonstration of the convolutional reverb is available online\footnote{\url{https://youtu.be/PuP8an5F2rk}}.
This video demonstrates the basic functionality of the zero-latency convolver on Bela.
First a simple looping guitar riff is used as a source signal,
which is processed in real-time using a small block size of 16 samples.
Note that the audio quality is not ideal since sound from Bela is recorded being played over a loudspeaker.
The function of the different controls are presented along with a demonstration of the six different room impulse responses.
In addition to the guitar signal, a flute recording is also used to demonstrate some of the functionality.

\subsection{Limitations}
While this implementation was capable of achieving a low-latency convolution on Bela with long convolutional filters up to 3 seconds,
there are certainly some performance instabilities.
The CPU usage required by the current implementation is often quite high and on the longest impulses responses
this can sometimes result in the Bela losing connection to the host, while audio remains playing.
Additionally, this implementation was not efficient on Bela to enable a multiple channel convolution for emulating a TCN.

\section{Conclusion}\label{sec:conclusion}

In this project a `zero-latency' convolutional reverb was implemented on the Bela platform.
While it was demonstrated that this implementation was successful in achieving real-time low-latency convolution
with a large impulse response of over 3 seconds, this required all of the compute resources on Bela.
Initial experiments to implement a simple temporal convolutional network utilizing this implementation
were therefore not successful, since it was not feasible to run more than two short convolutions in series.
Nevertheless, the resultant low-latency convolution works well on Bela.
To demonstrate the potential of this implementation a simple UI was constructed that enables user
to switch through six different room impulse responses and adjust various parameters.
Max blocks and sparsity controls were introduced to both adjust the sound of the the prerecorded impulse responses,
as well as potentially reduce the computational load of very long impulse responses.
Future work will consider other potential optimizations mentioned in the literature,
as well as the investigation of other methods for inducing sparsity in the block-based FFT convolution,
which may be applicable for implementing low-latency convolution for real-time neural audio effects.

\cleardoublepage
\bibliographystyle{unsrt}
\bibliography{references}

\end{document}


% !TEX root = ../main.tex
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\section{Example}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Seminal paper}\vspace{0.75cm}
\fullcite{Keane.1997}.\vspace{0.5cm}

\begin{itemize}
	\item The study follows individuals over their working life from young adulthood at age 16 to retirement at age 65 where the decision period $t = 16, \dots, 65$  is a school year.
	\item Individuals decide $a\in\mathcal{A}$ whether to work in a blue-collar or white-collar occupation ($a = 1, 2$), to serve in the military $(a = 3)$, to attend school $(a = 4)$, or to stay at home $(a = 5)$.
\end{itemize}

\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Decision tree}

\begin{figure}
  \scalebox{0.60}{\input{../material/fig-decision-tree}}
\end{figure}
\end{frame}

%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}\frametitle{Immediate utility}\vspace{0.3cm}

  \begin{multicols}{2}

  \begin{align*}
  u_a(s_t) =
  \begin{cases}
      \,\zeta_a(s_t)  + w_a(s_t)   & \text{if}\, a \in \{1, 2, 3\}  \\[0.2cm]
      \,\zeta_a(s_t)                 &  \text{if}\, a \in \{4, 5\}
  \end{cases}
  \end{align*}

  \columnbreak

  \heading{Informed by reduced-form evidence}\vspace{0.3cm}
  \begin{itemize}\setlength\itemsep{1em}
     \item Mincer equation
     \item Sheepskin effects
     \item Skill depreciation
     \item Mobility and search costs
     \item Monetary and psychic cost of schooling
  \end{itemize}

\end{multicols}

\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Transitions}\vspace{0.25cm}

\begin{itemize}\setlength\itemsep{1em}
  \item Work experience $\bm{k}_t$  and years of completed schooling $h_t$ evolve deterministically.
  \begin{align*}
  k_{a,t+1} = k_{a,t} + \ind[a_t = a]  &\qquad \text{if}\, a \in \{1, 2, 3\} \\
  h_{t + 1\phantom{,a}} = h_{t\phantom{,a}} +   \ind[a_t = 4]  &\qquad
  \end{align*}

  \item Productivity shocks $\bm{\epsilon}_t$ are uncorrelated across time and follow a multivariate normal distribution with mean $\bm{0}$ and covariance matrix $\bm{\Sigma}$.

  \item Given the structure of the utility functions and the distribution of the shocks, the state at time $t$ is $s_t = \{\bm{k}_t, h_t, t, a_{t -1}, \bm{e},\bm{\epsilon}_t\}$.
\end{itemize}

\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Utility of blue-collar occupation}\vspace{0.25cm}

	\begin{itemize}\setlength\itemsep{1em}
	\item Non-pecuniary
  \begin{align*}
  \zeta_{1}(\cdot)  = \alpha_1 & + c_{1,1} \cdot \ind[a_{t-1} \neq 1] + c_{1,2} \cdot \ind[k_{1,t} = 0] \\
                              & + \vartheta_1 \cdot \ind[h_t \geq 12] + \vartheta_2 \cdot \ind[h_t \geq 16] + \vartheta_3 \cdot \ind[k_{3,t} = 1]
  \end{align*}

	 \item Wage component
	 \begin{align*}
	 w_{1}(\cdot) & = r_{1} \, x_{1}(\cdot),
	 \end{align*}
where  $x_{1}(\cdot)$ is the occupation-specific skill level.
 \end{itemize}
\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Skill production for blue-collar occupation}

\begin{align*}
	x_{1}(\cdot) & = \exp \big( \Gamma_{1}(\bm{k}_t,  h_t, t, a_{t-1}, e_{j,1}) \cdot \epsilon_{1,t} \big)
\end{align*}
\begin{itemize}\setlength\itemsep{1em}
\item Parameterization of the deterministic component of the skill production function:\vspace{-0.5cm}

\begin{align*}
     \Gamma_1(\cdot) = e_{j,1} & + \beta_{1,1} \cdot h_t + \beta_{1, 2} \cdot \ind[h_t \geq 12] + \beta_{1,3} \cdot \ind[h_t\geq 16]\\
                                   & + \gamma_{1, 1} \cdot  k_{1,t} + \gamma_{1,2} \cdot  (k_{1,t})^2 + \gamma_{1,3} \cdot  \ind[k_{1,t} > 0] \\
                                 & + \gamma_{1,4} \cdot  t + \gamma_{1,5} \cdot \ind[t < 18]\\
                                   & + \gamma_{1,6} \cdot \ind[a_{t-1} = 1] + \gamma_{1,7} \cdot  k_{2,t} + \gamma_{1,8} \cdot  k_{3,t}
\end{align*}
\end{itemize}

\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{National Longitudinal Survey of Youth 1979}\vspace{0.25cm}

	\begin{itemize}\setlength\itemsep{1em}
	\item 1,373 individuals starting at age 16
	\item Life cycle histories\medskip
	\begin{itemize}\setlength\itemsep{1em}
		\item School attendance
		\item Occupation-specific work status
		\item Wages
	\end{itemize}
\end{itemize}
\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Sample size}
  \begin{figure}
  \scalebox{0.30}{\includegraphics{fig-data-sample-size}}
  \end{figure}
\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Data descriptives}
  \begin{figure}[h!]\centering
  \setcounter{subfigure}{0}
  \subfloat[Choices]{\scalebox{0.225}{\includegraphics{fig-data-choice-all}}}\hspace{0.5cm}
  \subfloat[Wages]{\scalebox{0.225}{\includegraphics{fig-data-wage-occupations}}}
  \end{figure}
\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Calibration results}
  \begin{figure}[h!]\centering
  \setcounter{subfigure}{0}
  \subfloat[Average wage]{\scalebox{0.225}{\includegraphics{fig-model-fit-wage-all}}}\hspace{0.5cm}
  \subfloat[Blue-collar]{\scalebox{0.225}{\includegraphics{fig-model-fit-choice-blue}}}
  \end{figure}
\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------
\begin{frame}{Economic insights}
  \begin{figure}[h!]
  \centering
  \setcounter{subfigure}{0}
  \subfloat[Time preference]{\scalebox{0.225}{\includegraphics{fig-economic-mechanism}}}\hspace{0.5cm}
  \subfloat[Tuition subsidy]{\scalebox{0.225}{\includegraphics{fig-policy-forecast}}}
  \end{figure}
\end{frame}
%---------------------------------------------------------------------------------------------------
%---------------------------------------------------------------------------------------------------


\section{Brutal Attack}\label{maneuver:brutalAttack}
\textbf{Cost:} 100 CP\\
\textbf{Requirements:} 3 Strength\\
\textbf{Maneuver, Active, Repeatable, Source(100 Gold)}\\
When making a melee weapon attack based on Strength, you can pay 5 Stamina and reduce your attack roll by up to 3 and add that value to the damage.
For example, if your weapon would deal 2D4 damage and you had 8 Strength, you could pay 5 Stamina and reduce your attack roll by up to 2, changing the weapon's damage to 2d4+2.\\
When using this weapon with a two-handed weapon attack, the damage increase is doubled.\\
\\
Level Progression:\\
\\
\rowcolors{2}{lightgray}{white}
\begin{tabular}{l | l | l | l}
	Level & Cost & Requires & Effect\\
	II & 500 & 6 Strength & 10 Stamina for up to 6 points.\\
	III & 2,500 & 9 Strength & 15 Stamina for up to 9 points.\\
	IV & 7,500 & 12 Strength & 20 Stamina for up to 12 points.\\
\end{tabular}


% This file contains the content for the Procedure
\regularsectionformat
\chapter{Recommended procedure for IDT creation}
%% Modify below this line %%

\label{chap:procedure}

The core of a digital camera system IDT is the 3x3 matrix by which radiometrically linear camera RGB code values are transformed to produce ACES RGB relative exposure values. This procedure describes the derivation of such a matrix.

To have fully followed the recommendations given in this document, IDT authors should provide an IDT for daylight and an IDT for tungsten illumination sources. The daylight illumination source should have the spectral power distribution of CIE Standard Illuminant D55; the tungsten illumination source, ISO 7589 Studio Tungsten.

\note{All references to CIE XYZ tristimulus values in this recommendation refer to colorimetric coordinates defined with respect to the CIE 1931 XYZ color space and the CIE 1931 2$^\circ$ Standard Colorimetric Observer.}

\section{Selection and Weighting of Training Data}
The quality of an IDT constructed as per this recommendation will depend heavily on the diversity and distribution of the set of spectral radiances used in IDT matrix optimization. Training spectra selection should attempt to sample the entire space of real-world scene spectra. Particularly important scene spectra, such as memory color spectra, may be weighted more heavily than colors rarely found in everyday life.

Training spectra should represent in situ stimuli if possible: the training spectrum representing a blade of grass illuminated by sunlight should be representative of that blade of grass in the context of surrounding leaves of grass, rather than in isolation on a laboratory slide, since very few motion pictures feature grass examined under a microscope. Only a small part of the reflected light from a single blade of grass is a first-bounce reflection of sunlight; the larger part of the illumination of a blade of grass in production photography will bounce off of or pass through neighboring blades of grass.

\section{Assumptions and prerequisites}
This procedure is appropriate when the IDT manufacturer desires to implement some form of neutral chromaticity difference compensation. It presumes the following information is available to the IDT manufacturer before applying the procedure:

\begin{itemize}
	\item	A set of training spectra against which the IDT matrix will be optimized, expressed either as spectral radiances, or as spectral reflectances which can be multiplied with the spectral power distribution of the scene adopted white to form spectral radiances.
	\item	The spectral power distribution of the scene adopted white.
	\item	A neutral chromaticity difference compensation strategy when the scene adopted white chromaticity differs from that of ACES neutrals.
	\item	A transform from camera system image data code values to three-channel radiometrically linear image data.
	\item	The spectral responsivities of the digital camera system for which the IDT is being created.
	\item	A transform from CIE XYZ colorimetry into an error minimization color space.
	\item	The error metric (cost function) that will be minimized, and any weights that will be applied differentially to the training spectra.
\end{itemize}

The sampling bounds and sampling interval of the spectra for the scene adopted white, for the RICD spectral responsivities, for the camera system spectral responsivities and for the training spectra are assumed to be the same so that mathematical operations can be performed without ambiguity. In addition, the procedure presumes the IDT author has access to software capable of mathematical operations including vector and matrix algebra, linear regression and minimization with user-defined cost functions.

\section{Symbols}
\label{sec:symbols}
In the table and equations below, lower case italic letters ($m$) represent scalars, lower case bold face letters ($\mathbf{h}$) represent vectors, and upper case bold face letters ($\mathbf{M}$) represent matrices.

{\renewcommand{\arraystretch}{1.25}
\begin{tabularx}{\textwidth}{|l|l|X|}
	\hline
	\bfseries{Symbol} & \bfseries{Domain} & \bfseries{Description} \\ \hline
	$m$ & $\Re$ & Number of samples in spectral data. \\ \hline
	$n$ & $\Re$ & Number of training spectra. \\ \hline
	$\mathbf{C}$ & $\Re^{m\times3}$ & Spectral responsivities of camera system (with channel gain differences maintained). \\ \hline
	$\big(\mathbf{r}_{i}\big)^{n}_{i=1}$ & $\Re^m$ & Spectral reflectances of training spectra. \\ \hline
	$\big(\mathbf{t}_{i}\big)^{n}_{i=1}$ & $\Re^m$ & Spectral radiances of training spectra. \\ \hline
	$\big(\mathbf{x}_{i}\big)^{n}_{i=1}$ & $\Re^3$ & CIE XYZ tristimulus values of training spectra. \\ \hline
	$\big(\mathbf{d}_{i}\big)^{n}_{i=1}$ & $\Re^3$ & Native color encoding of the camera system response values of training spectra. \\ \hline
	$\big(\mathbf{v}_{i}\big)^{n}_{i=1}$ & $\Re^3$ & Scaled white-balanced radiometrically linear camera system response values for training spectra. \\ \hline
	$\mathbf{h}_s$ & $\Re^m$ & Spectral power distribution of scene illumination source. \\ \hline
	$\mathbf{w}_s$ & $\Re^3$ & CIE XYZ tristimulus values of the scene illumination source. \\ \hline
	$\mathbf{h}_w$ & $\Re^m$ & Spectral power distribution of scene adopted white. \\ \hline
	$\mathbf{w}_w$ & $\Re^3$ & CIE XYZ tristimulus values of the scene adopted white. \\ \hline
	$\mathbf{M}$ & $\Re^{3\times3}$ & Matrix converting ACES RGB relative exposure values to CIE XYZ tristimulus values. \\ \hline
	$\mathbf{w}$ & $\Re^3$ & CIE XYZ tristimulus values of ACES unity neutral. \\ \hline
	$\mathbf{X}$ & $\Re^{m\times3}$ & CIE 1931 2$^\circ$ Standard Observer color matching functions. $\mathbf{X}_\mathrm{Y}$ is the spectral luminous efficiency function. \\ \hline
	$\mathbf{b}$ & $\Re^3$ & Camera system white balancing and scaling factors. \\ \hline
	$\mathbf{B}$ & $\Re^{3\times3}$ & Matrix converting white-balanced scaled camera system RGB response values to ACES RGB relative exposure values. \\ \hline
	$\mathbf{A}_{Bradford}$ & $\Re^{3\times3}$ & Bradford matrix for chromatic adaptation, with value \\
	& &
		$\begin{bmatrix*}[r]
			0.8950 & 0.2664 & -0.1614 \\
			-0.7502 & 1.7135 & 0.0367 \\
			0.0389 & -0.0685 & 1.0296
		\end{bmatrix*}$ \\ \hline
	$\mathbf{A}_{CAT02}$ & $\Re^{3\times3}$ & CAT02 matrix for chromatic adaptation, with value \\
	& &
		$\begin{bmatrix*}[r]
			0.7328 & 0.4296 & -0.1624 \\
			-0.7036 & 1.6975 & 0.0061 \\
			0.0030 & 0.0136 & 0.9834
		\end{bmatrix*}$ \\ \hline
	$\mathbf{A}_{re-illum}$ & $\Re^{3\times3}$ & Example matrix for re-illumination\footnote{Both the contents of this matrix and the method by which it is calculated may be protected by patent(s) including but not limited to U.S. Patent No. 7,298,892 (Inventors: Spaulding, Kevin E.; Woolfe, Geoffrey J.; and Giorgianni, Edward J.).}, with value \\
	& &
		$\begin{bmatrix*}[r]
			1.6160 & -0.3591 & -0.2569 \\
			-0.9542 & 1.8731 & 0.0811 \\
			0.0170 & -0.0333 & 1.0163
		\end{bmatrix*}$ \\ \hline
\end{tabularx}
}

\section{Functions}
\label{sec:functions}
In the table and equations below, functions are indicated by a (possibly subscripted) lower case letter or word, followed by comma-separated arguments enclosed in parentheses. Arguments represented by italic lower case letters are scalars, those represented by lower case bold face letters are vectors, and those represented by upper case bold face letters are matrices.

\begin{tabularx}{\textwidth}{|l|l|X|}
	\hline
	\bfseries{Function} & \bfseries{Domain} & \bfseries{Description} \\ \hline
	$\mathrm{f_{CAM}}(\mathbf{x}, \mathbf{w})$ & $\Re^3\rightarrow\Re^3$ & Return color appearance correlates for tristimulus values $\mathbf{x}$ under adopted white $\mathbf{w}$ suited for measurement of color difference. \\ \hline
	$\min(\mathbf{v})$ & $\Re^m\rightarrow\Re$ & Return the minimum element of vector $\mathbf{v}$. \\ \hline
	$\gamma(\mathbf{c})$ & $\Re^3\rightarrow\Re^3$ & Decode camera image data code values $\mathbf{c}$ to normalized radiometrically linear camera system exposures. \\ \hline
	$\psi(\mathbf{A}, \mathbf{w})$ & $\Re^3\rightarrow\Re^3$ & Create a re-illumination matrix which transforms tristimulus values to an alternative color space using matrix $\mathbf{A}$, multiplies transformed values by the reciprocal of transformed white $\mathbf{w}$, and inverse transforms the scaled transformed values to their original color space using the inverse of matrix $\mathbf{A}$. \\
	& & $\psi(\mathbf{A}, \mathbf{w})=\mathbf{A}^{-1} \begin{bmatrix*}
			\rho & 0 & 0 \\
			0 & \gamma & 0 \\
			0 & 0 & \beta
		\end{bmatrix*}\mathbf{A}$ \\
	& & with \\
	& & $\begin{bmatrix*}
			\rho \\
			\gamma \\
			\beta
		\end{bmatrix*} = \dfrac{1}{\mathbf{Aw}}$ \\ \hline
	$\phi(\mathbf{A}, \mathbf{w_1}, \mathbf{w_2})$ & $\Re^3\rightarrow\Re^3$ & Create a chromatic adaptation matrix which transforms tristimulus values to an alternative color space using matrix $\mathbf{A}$, scales transformed values by ratio of transformed white $\mathbf{w_2}$ to transformed white $\mathbf{w_1}$, and inverse transforms the scaled transformed values to their original color space using the inverse of matrix $\mathbf{A}$. \\
	& & $\phi(\mathbf{A}, \mathbf{w_1}, \mathbf{w_2})=\mathbf{A}^{-1} \begin{bmatrix*}
			\rho & 0 & 0 \\
			0 & \gamma & 0 \\
			0 & 0 & \beta
		\end{bmatrix*}\mathbf{A}$ \\
	& & with \\
	& & $\begin{bmatrix*}
			\rho \\
			\gamma \\
			\beta
		\end{bmatrix*} = \dfrac{\mathbf{Aw}_2}{\mathbf{Aw}_1}$ \\
	& & For equal whites, $\phi$ shall be the identity function: \\
	& & $\mathbf{w}_1=\mathbf{w}_2\Rightarrow\phi(\mathbf{A}, \mathbf{w_1}, \mathbf{w_2})=\mathbf{I}$ \\ \hline
	$\min(\mathbf{v})$ & $\Re^m\rightarrow\Re$ & Return the minimum element of vector $\mathbf{v}$. \\ \hline
	$\mathrm{clip}(\mathbf{v})$ & $\Re^m\rightarrow\Re^m$ & Clip channel values in vector $\mathbf{v}$ to [1, 1, 1], or extrapolate the channels clipped on capture from the unclipped channels. \\ \hline
\end{tabularx}

\section{Operations}
\label{sec:operations}
In the table and equations below, lower case italic letters ($m$) represent scalars, lower case bold face letters ($\mathbf{h}$) represent vectors, and upper case bold face letters ($\mathbf{M}$) represent matrices.

\begin{tabularx}{\textwidth}{|l|l|X|}
	\hline
	\bfseries{Operation} & \bfseries{Domain} & \bfseries{Description} \\ \hline
	$\mathbf{v} + \mathbf{w}$ & $\Re^m\rightarrow\Re^m$ & Element-wise addition of vectors in $\Re^m$. \\ \hline
	$\mathbf{v} * \mathbf{w}$ & $\Re^m\rightarrow\Re^m$ & Element-wise multiplication of vectors in $\Re^m$. \\ \hline
	$\mathbf{v} / \mathbf{w}$ & $\Re^m\rightarrow\Re^m$ & Element-wise division of vectors in $\Re^m$, $w_i>0$. \\ \hline
	$\mathbf{v}\mathbf{w}$ & $\Re^m\rightarrow\Re$ & Dot product of vectors in $\Re^m$, $\mathbf{vw}=\displaystyle\sum_{i}^{m} v_iw_i$. \\ \hline
	$\mathbf{M}\mathbf{v}$ & $\Re^{nxm},\Re^m\rightarrow\Re^m$ & Matrix-vector multiplication $\mathbf{v}\ni\Re^m$, $\mathbf{M}\ni\Re^{nxm}$. \\ \hline
	$\mathbf{M}_{\mathrm{T}}$ & $\Re^{nxm}\rightarrow\Re^{mxn}$ & Transposition of matrix $\mathbf{M}$. \\ \hline
\end{tabularx}

\section{Constant symbol values}
\subsection{Compute CIE XYZ tristimulus values of the ACES unity neutral}
The CIE XYZ tristimulus values $\mathbf{w}$ of the ACES unity neutral will be the same for all IDTs, as the elements involved in its computation do not include scene- or camera-system-specific data; the values are computed by post-multiplication of $\mathbf{M}$ (the ACES RGB to CIE XYZ conversion matrix defined in Section 4.1.1 of the ACES document) by the ACES unity neutral:

$\mathbf{M} =
    \begin{bmatrix*}[r]
		0.9525523959 & 0 & 0.0000936786 \\
		0.3439664498 & 0.7281660966 & -0.0721325464 \\
		0 & 0 & 1.0088251844
    \end{bmatrix*}$

$\mathbf{w} =\mathbf{M}\begin{bmatrix} 1 \\ 1 \\ 1 \\ \end{bmatrix}$

\section{Computation}
\subsection{Compute (if required) spectral radiances of illuminated training spectral reflectances}
\label{sec:compstart}
When the training colors $\mathbf{t}_i$ are expressed as spectral radiances, they may be used directly in the equations below. When they are expressed as spectral reflectances $\mathbf{r}_i$ under a scene adopted white with spectral power distribution $\mathbf{h}_w$ the resulting spectral radiances are calculated as

$\mathbf{t}_i = \mathbf{h}_w*\mathbf{r}_i$

\subsection{Compute CIE XYZ tristimulus values of training colors spectral radiances}
The training color CIE XYZ tristimulus values are calculated as

$\mathbf{x}_i=\mathbf{X}^{\mathrm{T}}\mathbf{t}_i$

\subsection{Compute CIE XYZ tristimulus values of scene adopted white}
The scene adopted white CIE XYZ tristimulus values are calculated as

$\mathbf{w}_w=\mathbf{X}^{\mathrm{T}}\mathbf{h}_w$

\subsection{Adjust training color CIE XYZ tristimulus values to compensate for difference between scene adopted white chromaticity and ACES neutral chromaticity}
\label{sec:chromaticadaptation}
Unless the scene adopted white happens to have the chromaticity of ACES neutrals (which have the chromaticity of CIE Standard Illuminant D60), some transformation is required to map the scaled white-balanced camera system response values that would result from the capture of the scene adopted white to the ACES unity neutral.

The training color CIE XYZ values are transformed to a balancing color space, scaled, then transformed back to CIE XYZ. When this balancing color space is structured around the cone responses of the human visual system (HVS), the scaling transform models chromatic adaptation; when the balancing color space is one of a class of color spaces particularly optimized for color adjustment, the scaling models re-illumination.

Three possible matrices representing a white balance transform are given in this section. The first and second model chromatic adaptation. The third models re-illumination.

\subsubsection{Alternatives 1 and 2: chromatic adaptation}
The training color CIE XYZ values are chromatically adapted such that neutral reflectors illuminated by the scene adopted white are encoded with the chromaticities of ACES neutrals. Two matrices are provided in this document, one defining the Bradford chromatic adaptation transform, and one defining the CAT02 transform.

$\mathbf{x}'_i = \phi(\mathbf{A}_{Bradford},\mathbf{w}_w,\mathbf{w})\mathbf{x}_i$ or $\mathbf{x}'_i = \phi(\mathbf{A}_{\mathit{CAT02}},\mathbf{w}_w,\mathbf{w})\mathbf{x}_i$

\subsubsection{Alternative 3: re-illuminate white-balanced ACES RGB relative exposure values for ACES neutral chromaticity}
The training color CIE XYZ values are scaled such that the scene adopted white CIE XYZ values are mapped to the ACES unity neutral CIE XYZ values and the white-balanced camera system exposure values are left unchanged.

$\mathbf{x}'_i = \phi(\mathbf{A}_{re-illum},\mathbf{w}_w)\mathbf{x}_i$

\subsection{Compute camera system white balance factors}
The camera system white balance factors map and normalize the linearized camera system responses corresponding to the capture of a perfect reflecting diffuser illuminated by the scene adopted white to camera system RGB values [1, 1, 1].

$\mathbf{b}=\dfrac{1}{\mathbf{C}^{\mathrm{T}}\mathbf{h}_w}$

\subsection{Compute white-balanced linearized camera system response values of training \\*colors}
The camera system spectral responsivity functions are post-multiplied by the training spectral radiances and scaled by the camera system white balance factors to produce the white-balanced, linearized, and normalized camera system response values for the set of training colors.

$\mathbf{v}_i=\mathbf{b}*\mathbf{C}^{\mathrm{T}}\mathbf{t}_i$

\subsection{Create initial values for the unoptimized IDT matrix entries}
\label{sec:compmat}
The matrix $\mathbf{B}$ is made up of 6 free parameters $\big(\beta_i\big)^{6}_{i=1}$.

$   \mathbf{B} =
    \begin{bmatrix}
		\beta_1 & \beta_2 & 1-\beta_1-\beta_2 \\
		\beta_3 & \beta_4 & 1-\beta_3-\beta_4 \\
		\beta_5 & \beta_6 & 1-\beta_5-\beta_6
    \end{bmatrix}$

Although other methods are possible, this document recommends the use of linear regression between the ACES RGB relative exposure values $\mathbf{x}'_i$ calculated in Section 4.7.3 and the white balanced camera system response values for the training colors $\mathbf{v}_i$ determined in Section 4.7.6.

The constraint that each row of matrix entries sums to unity can be formulated into the regression\footnote{Finlayson and Drew: White-point preserving color correction, Proceedings of the Fifth Color Imaging Conference, 1997.} or can be enforced by scaling the parameters obtained from unconstrained regression (although the latter gives only an approximation of the true solution).

\subsection{Select a cost function for error minimization}
\label{sec:compcostfn}
The cost function sums the distances between the target values and the transformed camera system response values after both sets of values have been transformed to a suitable color appearance space. The merit function is

$\chi^2=\displaystyle\sum_{i}^{n}\begin{Vmatrix}\mathbf{f}_{\mathrm{CAM}}(\mathbf{x}'_i,\mathbf{w})-\mathbf{f}_{\mathrm{CAM}}(\mathbf{MBv}_i,\mathbf{w})\end{Vmatrix}$

Choices for a suitable color appearance space for $\mathbf{f}_{\mathrm{CAM}}$ include

\begin{itemize}
	\item Nonlinear, neutral preserving RGB
	\item CIE L*u*v*
	\item CIE L*a*b*
	\item CIECAM

\end{itemize}

\note{Other color spaces may also be used.}

\note{More complex cost functions may also be used to weight the importance of particular training spectra, or to weight particular aspects of the approximation of the RICD's capture of training spectra (hue angle, luminance, colorfulness) more heavily than others, or in other ways chosen by the IDT author.}

\subsection{Find final matrix values minimizing cost function}
\label{sec:compend}
An iterative algorithm is used to minimize the cost function with respect to the parameters $\beta$. Existing implementations of this IDT calculation use the nonlinear least squares regression from MATLAB's Optimization Toolbox, or the \textbf{\texttt{optim}} function in R, or the \textbf{\texttt{FindMinimum}} function in Mathematica.

\subsection{Application}
\label{sec:application}
The IDT converts image data to ACES RGB relative exposure values $\mathbf{a}$ as follows:

$\mathbf{a} = k \mathbf{B} \mathrm{clip}\left(\dfrac{\mathbf{b}*\gamma(\mathbf{d})}{\min(\mathbf{b})}\right)$

where $\gamma$, $\mathbf{b}$, $\min()$, clip$()$ and $\mathbf{B}$ are as defined in Sections \ref{sec:symbols} and \ref{sec:functions}; $\mathbf{d}$ is an image data code value vector; and $k$ is the factor that results in a nominally ``18\% gray'' object in the scene producing ACES values [0.18, 0.18, 0.18].

\subsubsection{Application using CTL}
A CTL function taking a normalized linear camera system RGB code value and returning an ACES RGB relative exposure value could be implemented as follows:

\cleardoublepage
\begin{lstlisting}
// D200 IDT
// D55 illuminant
// camSPECS-measured D200 spec sens
// Woolfe/Spaulding/Giorgianni space for reconciliation of
// scene adopted white chromaticity and ACES neutral chromaticity
// (cf. U.S. Patent No. 7,298,892).

float
min(float a, float b) {
  if (a < b)
    return a;
  else
    return b;
}

float
clip(float v) {
  return min(v, 1.0);
}

void
main
(   input varying float rIn,
    input varying float gIn,
    input varying float bIn,
    input varying float aIn,
    output varying float rOut,
    output varying float gOut,
    output varying float bOut,
    output varying float aOut)
{

const float b[] = { 0.00102373, 0.000570585, 0.000797976 };

const float B[][] = { {  0.674849,    0.231105,  0.0940463 },
                      {  0.0431725,   1.06981,  -0.112982  },
                      {  0.0312748,  -0.151054,  1.11978   } };

const float min_b = min(b[0], min(b[1], b[2]));
const float e_max = 1.0;
const float k     = 1.0;

float clippedRGB[3];
clippedRGB [0] = clip((b[0] * rIn) / (min_b * e_max));
clippedRGB [1] = clip((b[1] * gIn) / (min_b * e_max));
clippedRGB [2] = clip((b[2] * bIn) / (min_b * e_max));

rOut = k * (B[0][0] * clippedRGB[0] + B[0][1] * clippedRGB[1] + B[0][2] * clippedRGB[2]);
gOut = k * (B[1][0] * clippedRGB[0] + B[1][1] * clippedRGB[1] + B[1][2] * clippedRGB[2]);
bOut = k * (B[2][0] * clippedRGB[0] + B[2][1] * clippedRGB[1] + B[2][2] * clippedRGB[2]);

}
\end{lstlisting}


\documentclass[a4paper]{article}

\title{Trajectory Model Code}
\author{Jamie Kettleborough}

\begin{document}
\maketitle

\section{Introduction}

This document outlines the numerical method and code used by the BADC
trajectory model.  It does not outline the details of the CGI
interface.
These are given in separate document.

\section{Numerical methods}

\subsection{Forward Time Stepping}
The time stepping of the trajectory positions is performed by a fourth
order Runge Kutta method.  Details of the Runge Kutta method can be
found in standard texts such as `Numerical Recipes'.  Stepping the
trajectories forward in time is simply solving the equation
\begin{displaymath}
 \frac{d\bf{x}}{dt}=\bf{u}(\bf{x},t)
\end{displaymath}
where $\bf{x}$ are the parcel positions, $\bf{u}$ are the parcel
velocities, and $t$ is time.

The method requires interpolation of the forcing wind
fields onto the parcel positions.  The Runge Kutta requires the
velocities at three time levels during each time step: the current
time ($t$), halfway through the time step ($t+\frac{\delta t}{2}$), and
at the end of the time step ($t+\delta t$).  Depending on the relative
number of parcels being advected and the number of grid points on the
wind forcing fields it can be more efficient to interpolate in time
first or in space first.  When interpolating in time first
interpolation has to be done at all the forcing field grid points.
This can be more efficient, however, since the wind from the end of
one time step ($t+\delta t$) can be carried over to become the gridded
wind at the beginning of the next time step ($t$).  The code, as
written, supports both interpolation in time first or interpolation in
space first.  (In retrospect the way that this is supported in the
code is not very well implemented and complicates the code
unnecessarily.)

The main time stepping routine is called {\tt runge\_kutta}.

Some important relevant variables are

\begin{tabular}{|l|l|}
\hline
Variable Name & Meaning \\
\hline
{\tt velt} & Gridded wind at current time step \\
{\tt velthdt} & Gridded wind at centre of current time step \\
{\tt veltdt} & Gridded wind at end of current time step \\
{\tt ltfirst} & Switch time interpolation first on or off \\
\hline
\end{tabular}

\subsection{Spherical Geometry}
The advection is done on a sphere (so the geometry of the surfaces of
constant height variable are approximated by spheres).  This is
achieved by using Cartesian coordinates $(x,y,z)$ on a sphere.  The
local vertical coordinate (pressure, or potential temperature) is
supported by a fourth dimension ($zhm$).  The gridded winds are given
on a latitude-longitude grid and with eastward and northward
components.  The Cartesian parcel positions have to be transformed
into latitude-longitude coordinates, and the winds then transformed
into Cartesian components. During the time stepping the Cartesian
coordinate of the parcels are renormalised to keep them on the surface
of a unit sphere.

Relevant routines include

\begin{tabular}{|l|p{15pc}|}
\hline
Routine & Purpose \\
\hline
{\tt latlon\_from\_cartesians} & convert Cartesian parcel positions in
lat-lon coordinates \\
{\tt find\_parcel\_cartesians} & convert lat-lon coordinates of parcels
to Cartesian \\
{\tt uvw} & part of this routine converts Eastward/Northward winds to
Cartesian coordinates\\
{\tt adjust} & remornamlise the Cartesian parcel positions to lie on a
unit sphere \\
\hline
\end{tabular}

\subsection{Time interpolation}
Simple linear interpolation is used to get the velocities onto the
current time.  The routine {\tt time\_interp} performs the time
interpolation.

\subsection{Space interpolation}
The space interpolation is also performed by linear interpolation.
The routine {\tt uvw} is responsible for the interpolation (as well as
transforming the interpolated winds into Cartesian components).  {\tt
  uvw} calls helper routines:

\begin{tabular}{|l|p{15pc}|}
\hline
Routine & Purpose \\
\hline
{\tt get\_horizontal\_weights } & Calculate the interpolation weights
for the horizontal direction \\
{\tt find\_vert} & find the location of the parcels on the vertical
grid and calculate vertical interpolation weight\\
{\tt interpolate} & built in IDL routine for linear interpolation\\
\hline
\end{tabular}

\subsection{Vertical Coordinates}
The trajectory model supports parcel vertical coordinates of hybrid
pressure or potential temperature.  When the vertical
coordinate of the parcel positions is the same as that of the wind
field then the wind grid information simply carries the one
dimensional level information.  If the vertical coordinate of the
parcel positions is not the same as those of the gridded wind field
then there has be be a way of calculating the the gridded values of
the parcel vertical coordinate from the gridded fields, this gridded
vertical coordinate is then carried around as the three dimensional
level information.  This has to be done, for instance, when the parcel
vertical coordinate is potential temperate, but the gridded wind
vertical coordinate is pressure, or hybrid sigma-pressure.

\subsection{Undefined points}
Parcels that are initialised close to the boundaries of the wind
domain may be advected out of the domain.  These parcels are set as
undefined.  Within the code a list of undefined parcels is carried
around as the variable {\tt in\_range}.  Only parcels that are defined
are advected.  Carrying the {\tt in\_range} array around probably is
not the best way to account of for undefined parcels.  Simply setting
the coordinates as $-999$ may be better.

\section{Outline}
An outline of the steps made by the code is as follows:

\begin{enumerate}
\item initialise parcel positions (external to main model)
\item set inputs
\item initialise model grid (defines grid wind files are present on)
\item read first wind file
\item loop around time
\begin{enumerate}
\item read and swap winds if needed
\item new release if needed (include interpolation of temperature etc)
\item perform Runge-Kutta step
\item increment time step
\item output on this time step if needed.
\item delete an old release if needed.
\end{enumerate}
\item close any unclosed output files
\end{enumerate}

\section{Multiple Releases}
Many users want to run the same trajectories for a number of different
times, say to build up a climatology of air origins at their observing
station.  To help with this kind of study trajectories can be run as a
set of multiple release trajectories.  In this case trajectories are
run with identical spatial starting positions for a set of times.

\section{Time step and date handling}
The trajectory model uses a set of routines, such as {\tt dt\_add}
that  were available at IDL 5.2, but which have since been
removed.  They have to be made available (in Traj:Job.pm) in order to
run the trajectory model.

\end{document}


% declare document class and geometry
\documentclass[12pt]{article} % use larger type; default would be 10pt
\usepackage[english]{babel} % for hyphenation dictionary
%\setdefaultlanguage{english} % polyglossia command for use with XeTeX / LuaTeX
\usepackage[margin=1in]{geometry} % handle page geometry

% import packages and commands
\input{../header2.tex}

% title information
\title{Phys 221A -- Quantum Mechanics -- Lec13}
\author{UCLA, Fall 2014}
\date{\formatdate{17}{11}{2014}} % Activate to display a given date or no date (if empty),
         % otherwise the current date is printed

\begin{document}
\maketitle

\section{Landau levels and quantum Hall effect}

We're going to talk a bit more about the stuff from last time because it's important and subtle. Recall that classically if we have an electron subjected to a uniform magnetic field $\v B = B \uv z$. From the Lorentz force we have
\begin{eqn}
m \vdd r = \frac{e}{c} \v v \times \v B.
\end{eqn}
If the particle is stopped, it will remain stopped. However if it is moving it will undergo cyclotron motion (in the $xy$-plane). The radius can be found from balancing the centripetal and centrifugal forces,
\begin{eqn}
\frac{mv^2}{R} = \frac{\abs{e}}{c} v \abs{B} \qquad
\implies \qquad
R_c = \frac{mc}{\abs{eB}} v, \quad
\omega_c = v / R_c = \frac{\abs{eB}}{mc}.
\end{eqn}
This turns out to give us quantized energy levels all separated by $\hbar \omega_c$, called Landau levels. Furthermore, we will not go into much detail but this turns out to imply quantized Hall conductance in a free electron gas,
\begin{eqn}
g_H = n \frac{e^2}{2\pi \hbar}, \quad n = 1, 2, 3, \dots.
\end{eqn}
This is the smallest measure of conductivity we have---in fact it is the standard set by NIST. Except maybe not, because actually there is the fractional quantum Hall effect, where we can have certain fractional quantizations of this value.

\subsection{things and stuff}

Now suppose we have a particle on a ring of radius $R$, subjected to a uniform time-independent magnetic field $\v B = B \uv z$. The Hamiltonian for such a particle not on the ring is
\begin{eqn}
H_k = \frac{\Pi^2}{2m} = \frac{1}{2m} (\v p - \frac{e \v A}{c})^2
\end{eqn}
where $\v \Pi = \v p - (e/c) \v A$ is the kinematical momentum (as opposed to the canonical momentum $\v p$). When we constrain the particle to the ring we get a new Hamiltonian $H_p$. Since our wavefunctions are now just functions $\psi(\theta)$ of the azimuthal angle $\theta$, we have a new canonical momentum
\begin{eqn}
\v p \rightarrow - \uv \theta \frac{i\hbar \partial_\theta}{R}.
\end{eqn}
Of course we can write a vector potential
\begin{eqn}
\v A = \frac{BR}{2} \uv \theta
\end{eqn}
for the magnetic field $\v B$ on the ring, so the old Hamiltonian becomes
\begin{eqn}
H_k = \frac{1}{2m} (- \frac{i\hbar \partial_\theta}{R} - \frac{eB}{2c})^2.
\end{eqn}
What is this vector potential? It's a vortex or some shit. From Stokes' theorem, the geometrical action is just
\begin{eqn}
\oint \dif{\v r} \cdot \v A = \int \dif{\v S} \cdot \v B = B \pi R^2,
\end{eqn}
which is constant so for some reason we find that the energy is really just $H = H_k$. So we haven't really done anything or something?

So we find that
\begin{eqn}
p_\theta = -\frac{i\hbar \partial_\theta}{R}
\end{eqn}
is the generator of translations along the ring (which is a symmetry). Then
\begin{eqn}
[p_\theta, H] = 0
\end{eqn}
since
\begin{eqn}
H = \frac{1}{2m} (p_\theta - \frac{eBR}{2c})^2.
\end{eqn}
The quantum numbers are given by the eigenvalues of $p_\theta$, where eigenstates are
\begin{eqn}
\psi_n \frac{1}{2\pi} e^{in \theta}, \qquad
n = 0, \pm 1, \pm 2, \dots
\end{eqn}
so the energy spectrum is
\begin{eqn}
E_n = \frac{1}{2m} (\frac{\hbar n}{R} - \frac{eBR}{2c})^2
\end{eqn}
where the momentum is quantized $p_n = \hbar n / R$. Suppose we plot the spectrum on a parabolic plot along the $n$ axis, so there is a point at each $n = 0, \pm 1, \pm 2$ etc. Then if we change the field $B$ it will shift the points along the parabola by
\begin{eqn}
\Delta n = \frac{e B R^2}{2c \hbar}.
\end{eqn}
So if we change the $B$ field in increments of
\begin{eqn}
\Delta B = \frac{2c\hbar}{eR^2}
\end{eqn}
we just get back the original spectrum. So the spectrum is invariant under incrementing $B$ by $\Delta B$. This gives us a corresponding magnetic flux
\begin{eqn}
\phi_0 = \frac{2c\hbar}{eR^2} \pi R^2 = \frac{2\pi \hbar c}{e}.
\end{eqn}

This is beginning to lead us to a proper understanding of the Aharonov-Bohm effect. Suppose we have a particle on a ring but this time there is a solenoid passing through the ring, with sufficient field containment that $\v B = 0$ outside the solenoid near the ring. Classically, the particle should not be affected by the solenoid because there is no field to interact with it. However, quantum mechanically we do obtain dynamics. We can model the vector potential as
\begin{eqn}
\v A = f(r) \frac{B \uv \theta}{2}
\end{eqn}
where
\begin{eqn}
f(r) =
\begin{cases}
r & r < R_S \\
\alpha / r & r > R_S
\end{cases}
\end{eqn}
where $\alpha$ is some constant to make it continuous. Then all the arguments from before carry over and we have an energy spectrum
\begin{eqn}
E_n = \frac{\hbar^2}{2mR^2} (n - \phi / \phi_0)^2.
\end{eqn}

Historically, the Aharonov-Bohm effect is something that many people missed.

\end{document}


\section{ARMAX}


    \filetitle{DAILY}{Calendar view of a daily tseries object}{tseries/daily}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
daily(X)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{X} {[} tseries {]} - Tseries object with indeterminate
  frequency whose date ticks will be interpreted as Matlab serial date
  numbers.
\end{itemize}

\paragraph{Description}\label{description}

\paragraph{Example}\label{example}


\documentclass[a4paper,oneside,11pt]{memoir}

\newif\ifhex
\hextrue

% ============================================================================ %
% MEMOIR SETUP                                                                 %
% ============================================================================ %

% -------------------------------------------------------- %
% Choose a page layout                                     %
% -------------------------------------------------------- %
%                                                          %
% - The spine margin is 30 mm. This is so that binding the %
% document with  a ringbinder does  not come too  close to %
% the text.                                                %
% - The edge  margin is  1.5 times the spine  margin, i.e. %
% 45 mm.                                                   %
% - The top margin is 1/9  of the paper height, same as in %
% \isopage                                                 %
% - The bottom margin is decreased from isopage's value to %
% 1.5 times the top margin, i.e. about 50 mm.              %
% -------------------------------------------------------- %
%    +----------^--------------------+
%    |          | 33mm               |
%    |     +----v------------+       |
%    |30mm |                 | 45mm  |
%    |<--->|                 |<----->|
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     |                 |       |
%    |     +----^------------+       |
%    |          | 49.5mm             |
%    |          |                    |
%    +----------v--------------------+
\isopage%
\setlrmarginsandblock{0.142857111\paperwidth}{0.190476190\paperwidth}{*}
\setulmarginsandblock{0.111111111\paperheight}{*}{1.5}%
\checkandfixthelayout%

% -------------------------------------------------------- %
% Pagestyle (Headers, Footers)                             %
% -------------------------------------------------------- %
\pagestyle{headings}

% -------------------------------------------------------- %
% Choose a chapter style                                   %
% We're  going  with  veelo, but  removing  the  "Chapter" %
% designation in front of the chapter number.              %
% -------------------------------------------------------- %

\newcommand\setupfadings{%
    \pgfmathsetmacro{\numberOfHorizSides}{12}%
    \pgfmathtruncatemacro\numberOfHorizElements{round(\numberOfHorizSides * 5 / 4)}
    \pgfmathsetmacro{\maxElementNumber}{\numberOfHorizSides - 1}%
    \dimendef\referenceWidth=0
    \pgfmathsetlength{\referenceWidth}{\paperwidth * 25.4 / 72}%
    \pgfmathsetmacro{\side}{0.5\referenceWidth/\maxElementNumber}%
    \pgfmathcos{30}%
    \pgfmathsetmacro{\height}{\pgfmathresult * \side}%
    \foreach \j in {0,...,\maxElementNumber}{%
        \pgfmathsetmacro{\leftTransp} {100 *  \j    / \numberOfHorizSides}%
        \pgfmathsetmacro{\rightTransp}{100 * (\j+1) / \numberOfHorizSides}%
        \tikzfading[name=hexfade \j,right color=transparent!\rightTransp,left color=transparent!\leftTransp]%
    }%
}

\newcommand\hexagons{%
    \setupfadings
    \tikzset{external/export next=false}%
    \begin{tikzpicture}[remember picture,overlay,x=1mm,y=1mm]
        \pgfmathtruncatemacro\iterstop{floor(\numberOfHorizElements / 2)}
        \pgfmathtruncatemacro\iterstopvert{floor(\numberOfHorizElements / 3 )}
        \path[clip] (current page.north west) rectangle (current page.east);
        \foreach \n in {0,...,\iterstopvert} {%
            \foreach \m in {0,...,\iterstop} {%
                \pgfmathtruncatemacro\k{round(\m + 1)}
                \fill[br0,draw=br1,path fading=hexfade \k]
                    ($(current page.north west) + (1.5 * \side + 3 * \m * \side,+0.866025 * \side - \n * 1.732051 * \side)$)
                    -- ++(   0:\side)
                    -- ++( -60:\side)
                    -- ++(-120:\side)
                    -- ++(-180:\side)
                    -- ++(-240:\side) -- cycle;
                \fill[br0,draw=br1,path fading=hexfade \m]
                    ($(current page.north west) + (0 * \side + 3 * \m * \side,-\n * 1.732051 * \side)$)
                    -- ++(   0:\side)
                    -- ++( -60:\side)
                    -- ++(-120:\side)
                    -- ++(-180:\side)
                    -- ++(-240:\side) -- cycle;
            }%
        }%
    \end{tikzpicture}%
}

\newcommand\chapnumhexagons{%
    \setupfadings
    \tikzset{external/export next=false}
    \begin{tikzpicture}[remember picture,overlay,x=1mm,y=1mm]
        \path[clip] (current page.north west) rectangle (current page.east);
        \foreach \n in {1,...,\thechapter} {%
            \pgfkeysgetvalue{/chapter \thechapter/hexagon \n/x}{\randNumHoriz}
            \pgfkeysgetvalue{/chapter \thechapter/hexagon \n/y}{\randNumVert}
            \pgfmathmod{\randNumHoriz}{2}
            \pgfmathtruncatemacro\fadingIndex{floor(\randNumHoriz / 2)}
            \pgfmathsetmacro\offsetVert{\pgfmathresult}
            \fill[dv-7,draw=br1,path fading=hexfade \fadingIndex]
                ($(current page.north west) + (1.5 * \randNumHoriz * \side,- 0.866025 * \offsetVert * \side - \randNumVert * 1.732051 * \side)$)
                -- ++(   0:\side)
                -- ++( -60:\side)
                -- ++(-120:\side)
                -- ++(-180:\side)
                -- ++(-240:\side) -- cycle;
        }%
        % Fade from below:
    \end{tikzpicture}%
}

\newcommand\appendixnumhexagons{%
    \setupfadings
    \tikzset{external/export next=false}
    \begin{tikzpicture}[remember picture,overlay,x=1mm,y=1mm]
        \path[clip] (current page.north west) rectangle (current page.east);
        \foreach \n in {A,...,\thechapter} {%
            \pgfkeysgetvalue{/appendix \thechapter/hexagon \n/x}{\randNumHoriz}
            \pgfkeysgetvalue{/appendix \thechapter/hexagon \n/y}{\randNumVert}
            \pgfmathmod{\randNumHoriz}{2}
            \pgfmathtruncatemacro\fadingIndex{floor(\randNumHoriz / 2)}
            \pgfmathsetmacro\offsetVert{\pgfmathresult}
            \fill[ct4,draw=br1,path fading=hexfade \fadingIndex]
                ($(current page.north west) + (1.5 * \randNumHoriz * \side,- 0.866025 * \offsetVert * \side - \randNumVert * 1.732051 * \side)$)
                -- ++(   0:\side)
                -- ++( -60:\side)
                -- ++(-120:\side)
                -- ++(-180:\side)
                -- ++(-240:\side) -- cycle;
        }%
    \end{tikzpicture}%
}

\newcommand\bottomscreen{%
    \setupfadings
    \tikzset{external/export next=false}%
    \begin{tikzpicture}[remember picture,overlay,x=1mm,y=1mm]
        \path[clip] (current page.north west) rectangle (current page.east);
        \fill[white,path fading=north]%
            ($(current page.north west) - (0pt, 10.5 * \side)$) rectangle ($(current page.north east) - (0pt,2 * \side)$);%
    \end{tikzpicture}%
}

\newlength{\edgewidth}
\makeatletter
\makechapterstyle{alpenthesis}{%
    % Lengths
    \setlength{\afterchapskip}{40pt}%
    \renewcommand*{\chapterheadstart}{\ifhex\hexagons\fi\vspace*{40pt}}%
    \renewcommand*{\afterchapternum}{\par\nobreak\vskip 25pt}%
    \setlength{\beforechapskip}{18mm}%
    \setlength{\midchapskip}{\paperwidth}%
    \addtolength{\midchapskip}{-\textwidth}%
    \addtolength{\midchapskip}{-\spinemargin}%
    % Fonts
    \renewcommand*{\chapnamefont}{\sffamily\LARGE\flushright}%
    \renewcommand*{\chapnumfont}{\sffamily\HUGE}%
    \renewcommand*{\chaptitlefont}{\sffamily\HUGE\flushright}%
    % Output:
    \renewcommand*\printchaptertitle{\chaptitlefont}%
    \renewcommand*{\printchaptername}{%
        \ifhex\chapnumhexagons\fi%
        \bottomscreen%
        \chapnamefont\MakeTextUppercase{\@chapapp}%
    }%
    \renewcommand*\printchapternonum{\bottomscreen}%
    % What comes between chaptername and the number:
    \renewcommand*{\chapternamenum}{}%
    % Print chapter number etc.
    \renewcommand*{\printchapternum}{%
        \setlength{\edgewidth}{\marginparwidth}%
        \addtolength{\edgewidth}{\marginparsep}%
        \makebox[0pt][l]{%
            \hspace{.8em}%
            \resizebox{!}{\beforechapskip}{\chapnumfont \thechapter}%
            \hspace{.8em}%
            %\rule{\midchapskip}{\beforechapskip}%
        }%
   }%
   \makeoddfoot{plain}{}{}{\thepage}}%
\makechapterstyle{alpenappendix}{%
    % Lengths
    \setlength{\afterchapskip}{40pt}%
    \renewcommand*{\chapterheadstart}{\ifhex\hexagons\fi\vspace*{40pt}}%
    \renewcommand*{\afterchapternum}{\par\nobreak\vskip 25pt}%
    \setlength{\beforechapskip}{18mm}%
    \setlength{\midchapskip}{\paperwidth}%
    \addtolength{\midchapskip}{-\textwidth}%
    \addtolength{\midchapskip}{-\spinemargin}%
    % Fonts
    \renewcommand*{\chapnamefont}{\sffamily\LARGE\flushright}%
    \renewcommand*{\chapnumfont}{\sffamily\HUGE}%
    \renewcommand*{\chaptitlefont}{\sffamily\HUGE\flushright}%
    % Output:
    \renewcommand*\printchaptertitle{\chaptitlefont}%
    \renewcommand*{\printchaptername}{%
        \ifhex\appendixnumhexagons\fi%
        \bottomscreen%
        \chapnamefont\MakeTextUppercase{\@chapapp}%
    }
    \renewcommand*\printchapternonum{\bottomscreen}%
    % What comes between chaptername and the number:
    \renewcommand*{\chapternamenum}{}%
    % Print chapter number etc.
    \renewcommand*{\printchapternum}{%
        \setlength{\edgewidth}{\marginparwidth}%
        \addtolength{\edgewidth}{\marginparsep}%
        \makebox[0pt][l]{%
            \hspace{.8em}%
            \resizebox{!}{\beforechapskip}{\chapnumfont \thechapter}%
            \hspace{.8em}%
            %\rule{\midchapskip}{\beforechapskip}%
        }%
   }%
   \makeoddfoot{plain}{}{}{\thepage}}%
\makeatother
\chapterstyle{alpenthesis} % requires graphicx package

%\newcommand{\chapterheadstart}{\vspace*{\beforechapskip}}
%\newcommand{\printchaptername}{\chapnamefont \@chapapp}
%\newcommand{\chapternamenum}{\space}
%\newcommand{\printchapternum}{\chapnumfont \thechapter}
%\newcommand{\afterchapternum}{\par\nobreak\vskip \midchapskip}
%\newcommand{\printchapternonum}{}
%\newcommand{\printchaptertitle}[1]{\chaptitlefont #1}
%\newcommand{\afterchaptertitle}{\par\nobreak\vskip \afterchapskip}
%\newcommand{\chapnamefont}{\normalfont\huge\bfseries}
%\newcommand{\chapnumfont}{\normalfont\huge\bfseries}
%\newcommand{\chaptitlefont}{\normalfont\Huge\bfseries}
%\setlength{\beforechapskip}{50pt}
%\setlength{\midchapskip}{20pt}
%\setlength{\afterchapskip}{40pt}

% ============================================================================ %
% PACKAGES                                                                     %
% ============================================================================ %
\usepackage{xcolor-solarized}
\usepackage{tikz}
\usetikzlibrary{calc}
\usetikzlibrary{fadings}
\usepackage{pgfplots}
\pgfplotsset{compat=1.14}
\tikzfading[name=topbot,top color=transparent!0,bottom color=transparent!100]
\usepackage{lipsum}
\usepackage[nott,light]{kpfonts}
\usepackage{lettrine}
\usepackage{graphicx}
\usepgfplotslibrary{external}
\tikzexternalize[prefix=cache/,optimize command away=\includepdf]
\usepackage{ifthen}

% ============================================================================ %
% RANDOM INTEGERS                                                              %
% ============================================================================ %
\input{randInts.tex}

% ============================================================================ %
% COLORS                                                                       %
% ============================================================================ %
\definecolor{da0}{HTML}{002B36}
\definecolor{da1}{HTML}{073642}
\definecolor{da2}{HTML}{008080}
\definecolor{da3}{HTML}{800080}
\definecolor{da4}{HTML}{404040}
\definecolor{ct0}{HTML}{CB4B16}
\definecolor{ct1}{HTML}{DC322F}
\definecolor{ct2}{HTML}{DC3682}
\definecolor{ct3}{HTML}{268BD2}
\definecolor{ct4}{HTML}{2AA198}
\definecolor{ct5}{HTML}{B58900}
\definecolor{ct6}{HTML}{859900}
\definecolor{br0}{HTML}{BEB9AA}
\definecolor{br1}{HTML}{EEE8D5}
\definecolor{br2}{HTML}{FDF6E3}
\definecolor{sq0}{HTML}{F7FBFF}
\definecolor{sq1}{HTML}{EFF3FF}
\definecolor{sq2}{HTML}{DECDF7}
\definecolor{sq3}{HTML}{C6DBEF}
\definecolor{sq4}{HTML}{BDD7E7}
\definecolor{sq5}{HTML}{9ECAE1}
\definecolor{sq6}{HTML}{6BAED6}
\definecolor{sq7}{HTML}{4292C6}
\definecolor{sq8}{HTML}{3182BD}
\definecolor{sq9}{HTML}{2171B5}
\definecolor{sqA}{HTML}{08519C}
\definecolor{sqB}{HTML}{084594}
\definecolor{sqC}{HTML}{08306B}
\definecolor{dv-7}{HTML}{8E0152}
\definecolor{dv-6}{HTML}{C51B7D}
\definecolor{dv-5}{HTML}{D01C8B}
\definecolor{dv-4}{HTML}{DE77AE}
\definecolor{dv-3}{HTML}{E9A3C9}
\definecolor{dv-2}{HTML}{F1B6DA}
\definecolor{dv-1}{HTML}{FDE0EF}
\definecolor{dv+0}{HTML}{F7F7F7}
\definecolor{dv+1}{HTML}{D1E5F0}
\definecolor{dv+2}{HTML}{92C5DE}
\definecolor{dv+3}{HTML}{67A9CF}
\definecolor{dv+4}{HTML}{4393C3}
\definecolor{dv+5}{HTML}{0571B0}
\definecolor{dv+6}{HTML}{2166AC}
\definecolor{dv+7}{HTML}{053061}
\definecolor{q0}{named}{ct3}
\definecolor{q1}{named}{sqB}
\definecolor{q2}{named}{ct2}
\definecolor{q3}{named}{da3}
\definecolor{q4}{HTML}{FEC44F}
\definecolor{q5}{named}{ct0}
\definecolor{q6}{HTML}{80EA82}
\definecolor{q7}{HTML}{2E7D32}

\title{Hexagon-Inspired Chapter Style}
\author{Raphael Frey}

% **************************************************************************** %
\begin{document}
% **************************************************************************** %

\begin{titlingpage}
    \maketitle
\end{titlingpage}

\frontmatter
\begin{abstract}
    \lipsum[1-2]
\end{abstract}
\newpage
\tableofcontents*

\mainmatter
\chapter{This Is the First Chapter}
\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]

%\chapter{This Is the Second Chapter With a Longer Title}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Third Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Fourth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Fifth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Sixth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Seventh Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Eigth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Ninth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Tenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Eleventh Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Twelfth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Thirteenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Fourteenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Fifteenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Sixteenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Seventeenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Eighteenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Nineteenth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]
%
%\chapter{Twentieth Chapter}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-3]

\appendixpage
\appendix
\chapterstyle{alpenappendix}
\chapter{First Appendix}
\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]

\chapter{Second Appendix}
\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]

%\chapter{Third Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Fourth Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Fifth Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Sixth Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Seventh Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Eigth Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Ninth Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{Tenth Appendix}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\backmatter
%\chapter{Backmatter Stuff}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
%
%\chapter{More Backmatter Stuff}
%\lettrine{\textcolor[gray]{0.5}{L}}{orem} \lipsum[1-2]
\end{document}