\documentclass[12pt]{amsart}
%\documentclass[12pt,oneside,letter]{article}
%\documentclass[12pt]{article}

\usepackage{geometry}                % See geometry.pdf to learn the layout options. There are lots.
\geometry{letterpaper}                   % ... or a4paper or a5paper or ...
%\geometry{landscape}                % Activate for for rotated page geometry
%\usepackage[parfill]{parskip}    % Activate to begin paragraphs with an empty line rather than an indent
\usepackage{graphicx}
\usepackage{amssymb}
\usepackage{epstopdf}
\DeclareGraphicsRule{.tif}{png}{.png}{`convert #1 `dirname #1`/`basename #1 .tif`.png}

\title{Manual for est\_noise7.3x}
\author{John Langbein \\
US Geological Survey \\
Menlo Park, CA \\
langbein@usgs.gov}
%\date{July 14, 2016}
\date{May 15, 2021}                                          % Activate to display a given date or no date

\begin{document}
\maketitle
\section{Introduction}

The program \textit{est\_noise7.3x} does time-series analysis to extract trends (velocities), offsets, and other
parameters that characterize GNSS observations (or any other time series) and simultaneously estimate the parameters the characterize the background noise
of the data. Basically, it does least-squares regression, but importantly, it quantifies the temporal correlations of the data to
provide unbiased estimates of the parameters that model the time-dependence of the data and their statistical errors. Standard least-squares regression typically assumes that each observation is independent, and drawn from a Gaussian distribution. However, for many time series, the data are temporally correlated.  \textit{est\_noise7.3x} simultaneously "measures" the temporal correlations and estimates the usual parameters that one often seeks from a time series.
The optimization is measured using the maximum likelihood estimator.  Although this code was written primarily for
GNSS observations of positions, it can be applied to other time series, too.

\textit{est\_noise7.3x} is an update of my older program, \textit{est\_noise6.50}.  There are several improvements:
\begin{enumerate}
\item {It provides a mechanism to analyze large data sets rapidly. For data with no gaps in the time series, the speed-up
can be up to a factor of 50. Typically, for many GPS time series, which have a few gaps, the speed-up is about a factor of 8.}
\item {It can take a variety of time formats including modified julian day and GMT style formats}
\item {The data and ancillary matrices are now double precision which minimizes apparent singularities in the inversion of data
covariance matrices having large temporal correlations.}
\item {Since the input or configuration to this program is a series of answers to questions, the program provides a
\textit{journal} file of the responses.  One can edit and then reuse the \textit{journal} file.}
\item {All subroutines are public domain. The legacy \textit{est\_noise6.50} used some subroutines from \textit{Numerical Recipes, Press et al.}. Those included a Fourier Transform, and the  \textit{Nelder and Mead} (1965), Downhill simplex optimization codes.
I have written my own version of Nelder and Mead and used the public domain FFT from netlib.org}
\item {The same set of inputs for \textit{est\_noise6.50} can be used with \textit{est\_noise7.3x} which yields the same
results; that is  \textit{est\_noise7.3x} is backwards compatible with  \textit{est\_noise6.50}.}
\end{enumerate}

This current version, 7.30, is an incremental update to version 7.22.  The primary change is related to the formation of the
covariance matrix that characterized Gauss-Markov noise. The new version better accounts for the time period prior to the start
of the time series. Neglecting this caused the old version to underestimate the rate uncertainty and biased the estimate of rate.
For other noise models, this has not been a problem.

Included in the top directory for \textit{est\_noise} is a file called \texttt{BugFix.txt} which provides the changes in version 7 of  \textit{est\_noise} in
chronological order.  There have been numerous enhancements, mostly to the scripts that can be used to "drive" \textit{est\_noise}.

The development and discussion of \textit{est\_noise} can be found in the references.

\subsection{Time dependent models}

The following types of time dependence are included with \textit{est\_noise};  rate, changes in rate, offsets, sinusoids,
exponential, $1-e^{-t/{\tau}}$, and Omori decay, $log(1 + t/{\tau})$. In addition, more functions can be added.  For instance,
some observations of interest, such a borehole strainmeter data, one might include the dependence of another set of observations such as atmospheric pressure. The pressure data can be used as another time-dependent function and the
program finds the admittance (or gain) of the pressure upon the strain data. Specifying these functions is done by answering
questions that the program asks. You will need to know the times of the offsets, the time of the start of the exponential trend,
and the period of time spanning the rate change.  More information will be provided later in the manual.

\subsection{Noise models for temporal correlations}

The temporal correlations can be modeled in terms of a power spectra, $P(f)$, where $f$ is frequency. \textit{est\_noise}
has a library of a few, simple noise models which can replicate a variety of temporal correlations. These include,
white noise, where $P(f)={\sigma^2}/f_{ny}$ where $f_{ny}$ is the Nyquist frequency and $\sigma$ is the amplitude
of white noise.  This is represents, uncorrelated observations, and that the data are Normally distributed with a standard
deviation of  $\sigma$.

However, most data sets have temporal correlations that can be represented by a power-law power-spectrum, $P(f)=P_o/f^{\beta}$. If $\beta =1$, then this is termed flicker noise, and if $\beta =2$, then this is termed random-walk noise.

In addition, \textit{Langbein} (2004) provided two more types of noise; generalized Gauss-Markov and bandpass filtered noise.
Generalized Gauss Markov noise can be represented by
\begin{equation}
P(f)~= ~{{P_o} \over [f^2 + ({\alpha}/{2{\pi}})^2]^{{\beta}/2}}
\end{equation}
where the power spectra is frequency independent at the lowest frequencies, but becomes a power law when $f > {\alpha}/{2{\pi}}$. (Note: this corrects equation~15 in \textit{Langbein} (2004)). When $\beta=2$, this is known as first-order Gauss-Markov (FOGM) noise.

For band pass filtered noise, it is represented by:
\begin{equation}
P(f)~= ~{{\sigma_{bp}}^2  [ {{(f/f_h)^2} \over {(1 + (f/f_l)^2)(1 + (f/f_h)^2)} }] ^{2p}}
\end{equation}
where $f_l$ and $f_h$ are the high and low frequency limits of the filter and $p$ is the number of poles.

All of these noise sources can be added together to provide a more complex and realistic representation of the temporal correlations.  As \textit{Langbein} (2017) demonstrates, there are two different ways to add these noise sources and
that can impact the computational requirements.  It customary to assume that each noise source is independent
and add them in quadrature, which has been done with the legacy program \textit{est\_noise6.50} and is offered
as one option in \textit{est\_noise7.3x}.  Alternatively, each of these noise models is represented by
a time-domain equivalent, or filter functions.  These filters can be added together to form the covariance
matrix. Computationally, this offers some advantages that is incorporated in \textit{est\_noise7.3x}.

\subsection{Related programs}

The tar-ball contains several other programs that augment \textit{est\_noise} which can help test the code and/or help evaluate the results.
\begin{description}
\item[bust\_5]  This removes outliers from time series based upon a running median
\item[compare\_wander7]  Computes a spectrum of \textit{wander} or drift based upon \textit{Agnew} [1992]. This is
useful to assess the quality of the estimated noise model obtained from \textit{est\_noise}.
\item[psd\_calc7] Computes the equivalent power spectral density using the noise model consistent with \textit{est\_noise}.
This involves re-scaling the coefficients.
\item[gen\_noise7] This program generates a time series of simulated, colored noise using the noise models discussed
above.
\item[adjust\_1]  removes sinusoids, offsets, rate, changes in rate, and exponential/Omori decay when given their
specified amplitudes. This is used in companion with \textit{bust\_5}.
\end{description}

In the \texttt{example0} directory, I provide 3 scripts that combine the use of all of these programs to remove outliers
and to perform the time series analysis to extract trends and the background noise of the data.

Finally, \textit{Bos et al.} (2012) provides an alternative to \textit{est\_noise} and that code can be found
at
\\
\texttt{http://segal.ubi.pt/hector/}
\\
\textit{est\_noise} primary advantage is that it has no approximations.  For power-law noise with indices greater than $1$,
\textit{hector} makes an approximation to force the covariance matrix to be Toeplitz and uses a fast Toeplitz solver
to invert the covariance matrix. \textit{est\_noise}, when using the simple addition of filter functions provides similar computational
speed to \textit{hector} without the approximation. In reality, for \textit{hector}, approximation has relatively minor influence on
its results based upon tests between the two programs.

\section{Compiling the program}

The program is written in Fortran, primarily the legacy Fortran 77. Unlike the legacy \textit{est\_noise6.50}, I'm not
able to provide an executable file. Instead, you will need to compile the program and its subroutines. I've successfully
compiled the program on Mac OS and Linux OS, using both gfortran and the Intel Fortran compilers.
I have not tried compiling and running the program on Windows OS. The best performance
comes with the Intel compiler and its MKL libraries.  By using gfortran and the associated openblas library, the computation speed is reduced by approximately 30\%.  For Linux and gfortran, you may need to download the openblas library.

I have provided a \textit{compile} script; to invoke,
\\
\texttt{./compile.sh}
\\
For which:
\\
\texttt{Script used to compile est\_noise \\
Usage:  compile.sh mac/linux i/g  \\
  where argument one designates the OS, mac or linux  \\
    and argument two designates the compiler \\
    i   intel ifort  \\
    g   gfortran
}

For compiling on a Mac, you will need Apple's Xcode installed which is obtained via Apple's online "App" store.
With the Intel option, \textit{i},
this requires Intel Fortran and their Math Kernal Library.  Otherwise, you'll need \textit{gfortran} which can
be obtained at \texttt{http://hpc.sourceforge.net/} or via \texttt{brew install gfortran}, assuming that the \texttt{brew} package
manager has been installed.

For compiling on Linux either Intel Fortran or \textit{gfortran} will work.  If the Intel compiler with MKL
is not available, then gfortran is easily obtained (if not already installed).  Depending on the Linux
distribution, a package installer, such as \texttt{yum}, will install gfortran.

In addition, if your are using gfortran on linux, the openblas library needs to be installed either in  \texttt{/usr/lib} or \texttt{/usr/lib64} and the listing will look like:
\\
\tiny{
\texttt{
ls -lt /usr/lib/*openblas*
\\
lrwxrwxrwx 1 root root       22 Jan 28  2015 /usr/lib/libopenblas.so.0 -> libopenblas-r0.2.11.so
\\
-rwxr-xr-x 1 root root 15134496 Aug 24  2014 /usr/lib/libopenblas-r0.2.11.so
}
}
\normalsize{ }
\\
If not present, you can simply download the openblas libraries using a package installer such as \texttt{yum}, that
is \texttt{sudo yum install openblas}, which is used with the CentOS dialect of Linux. IN ADDITION, you will need to
\textit{comment-out} one of two lines in compile.sh script near line 40.

Note for ubuntu OS:  For this OS, I downloaded openblas from
\\
\texttt{http://www.openblas.net/} (look for download tab near top
of page) and follow the install instructions
\\
\texttt{https://github.com/xianyi/OpenBLAS/wiki/Installation-Guide}
\\
using \texttt{make}.
You will need to run \texttt{make} a few times with different arguments but those are provided by the output of the previous \texttt{make} command.  The \texttt{apt-get install} may work, too.

\textit{A word of apology:}  Unfortunately, compiling \textit{est\_noise} is some-what a "crapshoot".  Depending on the
versions of compilers and the versions of your OS, you may need to change the various compiler options to be able
to not only successfully compile \textit{est\_noise}, but to get it to run, too. In the \texttt{compile.sh} script, you will
see a number of "commented-out" compile statements; these might provide you some hints about what you might
need to change to get the program(s) to work. Recently, for gfortran, I had to add an additional option of \texttt{-std=legacy}
to the compilation since the FFT subroutines are written either for Fortran 77 or Fortran 66.

\section{Initial test}

An example is included with the distribution. Critically, it contains the seed file, \texttt{seed.dat}, the data \texttt{canp.e}, and the
input driver file, \texttt{est\_canp.in}. These are found in the \texttt{example} directory. To run,
\\
\texttt{../bin/est\_noise7.30 < est\_canp.in} \\
On a single core, 64 bit Linux computer running 2.2GHz, the program take ~48 seconds when compiled with the Intel Fortran and ~68 seconds with gfortran.

Although the program can run over multiple cores, I find that it is most efficient to use a single core.  That is, if you are
analyzing several time series, one should assign a single process of \textit{est\_noise} to run on a single core processing
a single time series. That way, if you have 8 core available, one can be simultaneously analyzing 8 different time series. However, one needs to
create a directory structure for each time series. The Linux utility, \texttt{taskset}, assigns a process to specific cores. For example,
\\
\texttt{taskset -c 0 est\_noise7.30 < est\_canp.in}\\
assigns the process to the first core.  Unfortunately, this utility may not available on Macs. (Note,  \texttt{brew install util-linux} may
include \texttt{taskse}t)

The input file, \texttt{est\_canp.in}, contains some annotations about the meaning of each line.  You can compare
the standard out (aka screen dump) with \texttt{est\_canp.out}.  The most critical information is found at the tail
end of the output which includes the estimate of rates, offset, log-trends, sinusoids, and the noise model.  More information
is contained in the following sections.

\section{Input files}

At the minimum, \textit{est\_noise} requires two input files, one \texttt{seed.dat}, which is a single integer number
to act as a seed to a random number generator, and a file of data.  When the program is run, it will start
asking questions on the standard output and responses are required to be typed into the
standard input. Your answers are recorded in a journal file which you
can re-use after changing its name. This section is broken into two parts, the first part discusses formats, in particular
for the data and specifying time, and the second part documents the questions and answers required by
\textit{est\_noise}

\subsection{Data formats}

\textit{est\_noise} handles five different formats for time.  Within the program, the basic unit is day.  The time formats
are specified by:
\begin{itemize}
\item \texttt{otr} Time is specified as year, and day of year ranging from 1 to 365 (or 366 for leap year). Finer resolution
is specified as fraction of a day.  For instance, 1515 and 10 seconds on 1, December 2015 is \texttt{2015 335.635532407}. These are read and stored as double precision values.
\item \texttt{otd} Time is specified as year, month, and day of month.  Month is a number from 1 to 12. For
1515 and 10 seconds on 1, December 2015, it is specified as \texttt{20151201.635532407}
\item \texttt{otx} Time is specified as year, month, and day of month.  Month is a number from 1 to 12. For
1515 and 10 seconds on 1, December 2015, it is specified as \texttt{2015 12 1.635532407}
\item \texttt{gmt} This uses the time format consistent with the GMT plotting package, \textit{www.soest.hawaii.edu/gmt/}.
Therefore, 1515 and 10 seconds on 1, December 2015, it is specified as \texttt{2015-12-01T15:15:10.0}
\item \texttt{mjd} Time is specified as modified Julian day; Therefore, 1515 and 10 seconds on 1, December 2015, it is specified as \texttt{57357.635532407}. Day zero is 17, November, 1858.
\end{itemize}

The time series of data requires a time stamp, using one of 5 formats, the observation, followed by its "error". Actually,
that last column, although required by the \texttt{read} statement, is not used. Some thought is needed with
regards to the data-value.  Although the program is written for double precision, the Fortran format statements
provide limited resolution. For instance, if the actual scale of the variations in the observations are a few
millimeters, I suggest that the observations be scaled to millimeters and not meters.

\subsection{\textit{est\_noise} questions and responses}

The question/answers are generally grouped into two parts, one that specifies the components of the time-dependent functions and the second part that specifies the components of the noise model.  The time dependent model is the summation of a rate, changes in rate, offsets, sinusoids, exponentials, $1 - e^{-t/{\tau}}$, logarithmic trends,
$log(1 + {t/{\tau}})$, and auxiliary functions or data, $a(t)$.

\begin{description}
\item[Format of input]  Provide the format of dates used to specify the limits of the data and the time dependent model; use one of the five abbreviations listed in \textit{Data formats}.

\item[Number of time series] This will usually be \texttt{1}, but in special cases, such as legacy electronic distance meter
data where near simultaneous measurements were made to neighboring monuments, this value could be greater than \texttt{1}. This option adds additional columns in the design matrix having either $0$ or $1$ depending upon whether
the measurement was made to monument 1 or 2...

\item[Time interval] Specify the time interval to analyze the data. Initial date followed by ending date. The dates must be
consistent with the date format descriptor specified in the first answer. It is possible that your data set spans a longer
time interval than you wish to analyze. The input provides a means to limit the interval for analysis. In addition, the
\textbf{start time has a special meaning} as it provides the reference time.  For instance, when fitting a secular trend to data, $d_i = B + R (t_i -t_{0})$, the nominal value $B$ represents the estimate of $d$ at the \textbf{start time}, $t_{0}$.

\item[Secular rate, y/n]  Estimate the secular rate, \textit{yes or no}.  That is, estimate $R$ in $d_i = B + R (t_i -t_{0})$.
If \texttt{n}, then $R=0$.

\item[Number of rate changes]  Normally \texttt{0}, but if a rate change is specified, then the number of rate changes is specified.  Following this entry, if not equal to \texttt{0}, are \texttt{n} lines that specify each interval of the rate change, both the start and end date of the change using the same time format entered above.  Each rate change interval must be
within the specified interval of the data given in \textbf{Time interval}. The $k^{th}$ rate change, $r_k$ over the interval
$t_{k1}$ to $t_{k2}$ is:
\begin{equation}
\begin{aligned}
d_i & = B + R (t_i - t_0)  & \text{ for } t_i < t_{k1} \\
      & = B + R (t_i - t_0) + r_k (t_i - t_{k1}) & \text{ for } t_{k1} \leq t_i \leq t_{k2} \\
      & = B + R (t_i - t_0) + r_k (t_{k2} - t_{k1}) & \text{ for } t_i >  t_{k2}
\end{aligned}
\end{equation}
None of the time intervals for the rate changes should fall outside the limits specified in \textit{Time interval} discussed above.

\item[Number of sinusoids] Input the number of sinusoids. If the number is more than \texttt{0}, then list the
period of each sinusoid in \textbf{days}. The $k^{th}$ sinusoid is the sum of the sine and cosine;
\begin{equation}
d_i  = B + R (t_i - t_0) + C_k \cos (2 {\pi}(t_i - t_0)/{p_k})+ S_k \sin (2 {\pi}(t_i - t_0)/{p_k})
\end{equation}
where $p_k$ is the period of the $k^{th}$ sinusoid with amplitudes $C_k$ and $S_k$.  Likewise, $t_0$ is the \textbf{start time}

\item[Number of offsets] The number of offsets is specified. If not \texttt{0}, then the time of each offset
is listed. The offset is modeled as a \textit{Heaviside} or unit step with the first value of the step corresponding
to the time specified of the offset.

\item[Number of exponential and/or log functions]  Many time series have exponential or logarithmic trends. This
option allows one to estimate the amplitude, and if desired, the time constant for each of these trends. If the number
is not \texttt{0}, then for each of the $k_{th}$ trends require:
\begin{description}
\item[Time of start of trend] Input the initial date of the trend, $t_T$, using format entered previously
\item[Time constant and either fix/float] This is two entries, the value of the time constant in \textbf{years} and
whether you want \textit{est\_noise} to estimate the time constant, \texttt{float} or hold the time constant \texttt{fix}'ed.
If \texttt{float} then the entered time constant is a guess and the program uses the Nelder and Mead (1965) algorithm
to find the optimal time constant assuming the current guess at the values of the data covariance.
\item[ Type of trend, exponential or logarithmic] Enter \texttt{e} for \textit{exponential}, $A e ^{-(t_i - t_T)/{\tau}}$, or
\texttt{m} for \textit{logarithmic} or \textit{Omori's law}, $A log (1 + {(t_i - t_T)/{\tau}})$
\end{description}

\item[Specify the date format of the data] The date format of the data may be different than the format of the dates
the specify the time dependence. See above for acceptable formats.

\item[Name of Data file] Provide the name of the data file. See above for acceptable formats.

\item[Specify then number of auxiliary data files] Usually, this will be \texttt{0}.  However, for some data sets,
such as borehole strainmeter data, this will be required as atmospheric pressure data
is used to adjust the raw strain data. One can enter other functions which might impact the observations.  The time
dependent function is set-up as:
\begin{equation}
d_i  = B + R (t_i - t_0) + ...  A_k (a_{k,i-j}) + ...
\end{equation}
where $A_k$ is the \textit{gain} or \textit{sensitivity} of the $k^{th}$ function (or data-channel) made up of $a_{k,i-j}$
$i$ \textit{observations} with a lag (in time) of $j$.  If the number of auxiliary data is not \texttt{0}, then the following entries
are required;
\begin{description}
\item[Format of data] Use one of the abbreviations listed under data formats.
\item[Name of Data file] Provide the name of the data file; Again, each record of data must have time,
the data value, and value for error, which is not used.
\item[Lead/Lag time] Usually \texttt{0} but can take on other values. It must be integer multiples
of the data sample interval given units of days.
\end{description}
The program will match the $a_{k,i-j}$ with the corresponding data, $d_i$. If one of the pair is missing, then
that line in the design matrix relating to the observation/data is deleted.

\item[Sampling interval] The nominal sampling interval, in days of the data.

\item[Type of error model] Acceptable values are \texttt{n}, \texttt{a}, \texttt{f}, or \texttt{c}.  See discussion in
the introduction, but usually this will either be \texttt{n}, or \texttt{a}. With \texttt{n}, the components of the noise
model are added in quadrature consistent with the legacy \textit{est\_noise6.50}. With the other three, filter functions of the noise model are simply added.  For data sets with few gaps or missing data, the program runs most efficiently
taking advantage of the algorithm discussed by \textit{Langbein} (2017) option \texttt{f} should be selected.
For gappy data, Cholesky decomposition is employed and option \texttt{c} should be selected.  By using
option \texttt{a}, the program automatically selects between the \textit{fast} or \textit{Cholesky} algorithms;
if there are more than 25\% missing data, the program defaults to Cholesky decomposition to invert the data
covariance matrix.

\item[Substitute synthetic noise for data]  This will be typically \texttt{n} or \textit{no}.  If \texttt{y}, the program will
ask a set of questions that specify the parameters of a noise model. Synthetic data are created with the same
sampling interval as the data, then at time when data are \textit{missing}, the synthetic data are deleted. \textbf{Note that
this option has not been tested with the current version of \textit{est\_noise7.sx} but had been used extensively in
the past. Use with caution!}

\item[Decimation type]  Typically, this will be \texttt{0}, but one can put anything between \texttt{0} and \texttt{3}.
With \texttt{0}, all measurements are used.  Otherwise, the data will be decimated using the following, ad-hoc schedule:
\begin{description}
\item[1]  Keep two observations, delete the third, keep two, delete the third; uses $2/3$ of the available data.
\item[2] Keep two observations, delete the next two, keep two, delete two; uses $1/2$ of the available data.
\item[3] Keep two observations, delete the next three, keep two, delete three; uses $2/5$ of the available data.
\end{description}
If \textit{Type of error model} is anything but \texttt{n}, the \textit{decimation type} defaults to \texttt{0}, or no decimation.

\item[White noise amplitude] Set-up the parameters to model the white noise component. Two entries are required
with the first being a \textit{guess} at the value of the white noise and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value of white noise that maximizes the log-likelihood. If \texttt{fix},
the program holds the value of white noise \textit{fixed}.  As an option, one can let \textit{est\_noise} make the guess
at the initial white noise value.  To do this, enter \texttt{-99999.0 float}.

\item[Amplitude of first power law noise model] Set-up the amplitude for the first power law/Gauss Markov noise model. Two entries are required
with the first being a \textit{guess} at the value of the amplitude of power law and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value of the amplitude of power-law noise that maximizes the log-likelihood. If \texttt{fix},
the program holds the amplitude of power-law noise \textit{fixed}.  Note that the unit of the amplitude is $unit/{yr^{{\beta}/4}}$.
As an option, one can let \textit{est\_noise} make the guess
at the initial amplitude of power-law noise.  To do this, enter \texttt{-99999.0 float}.  The program will complete its guess
after the next parameter is entered as the power-law index is required.

\item[Index of first power law noise model] Continuation of the set-up of the first power-law/Gauss Markov noise model.
\textbf{Three} entries are required
with the first being a \textit{guess} at the of power law index and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value of the index of power-law noise that maximizes the log-likelihood. If \texttt{fix},
the program holds the index of power-law noise \textit{fixed}.  If the index, $\beta=1$, then this is considered
as flicker noise and if $\beta=2$, then this is considered
as random-walk noise.  The last entry is places a limit on the size of the index during the optimization. The value
$3$ is a reasonable choice. See equation~1.

\item[Gauss Markov frequency] Continuation of the set-up of the first power-law/Gauss Markov noise model.
Two entries are required
with the first being a \textit{guess} at the ${\alpha}/{2{\pi}}$ frequency and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value G-M frequency that maximizes the log-likelihood. If \texttt{fix},
the program holds the G-M frequency \textit{fixed}.   See equation~1. In most cases, your entry will be \texttt{0 fix}.
Note that the unit of ${\alpha}$ is radians/year.

\item[Bandpass filter limits] Pass-band for band-pass filtered noise model in units of cycles/year. Two numbers are
required. For typical seasonal noise, use \texttt{0.5 2.0}.  See equation~2.

\item[Number of poles for bandpass filtered noise] A integer value between \texttt{1} and \texttt{4} needs to be entered. See equation~2.

\item[Amplitude of bandpass filtered noise] Two entries are required
with the first being a \textit{guess} at the value of the amplitude of bandpass filtered noise and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value of the amplitude  that maximizes the log-likelihood. If \texttt{fix},
the program holds the amplitude  \textit{fixed}.  If bandpass filtered noise is not considered, enter \texttt{0 fix}.

\item[Index of second power law noise model] Set-up of the second power-law noise model.
Two entries are required
with the first being a \textit{guess} at the of power law index and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value of the index of power-law noise that maximizes the log-likelihood. If \texttt{fix},
the program holds the index of power-law noise \textit{fixed}.  If the index, $\beta=1$, then this is considered
as flicker noise and if $\beta=2$, then this is considered
as random-walk noise.

\item[Amplitude of second power law noise model] Continuation of the set-up the second power law noise model. Two entries are required
with the first being a \textit{guess} at the value of the amplitude of power law and the second being \texttt{float} or \texttt{fix}.
If \texttt{float}, the program will find the optimal value of the amplitude of power-law noise that maximizes the log-likelihood. If \texttt{fix},
the program holds the amplitude of power-law noise \textit{fixed}. Note that the unit of the amplitude is $unit/{yr^{{\beta}/4}}$.
Here, for the second power law entry, \textit{est\_noise} does \textbf{not} provide an option for the program to make
its guess at the power law amplitude.

\item[Add white noise to data] Usually, this will be \texttt{0}.  In some rare cases where the data are characterized
by extreme power-law noise with no detectable white noise, one may need to add white noise to the observations
to keep the inversion of the data covariance becoming numerically singular.

\end{description}

Although this is a long list of inputs, the program supplies descriptive queries.  The answers supplied to the queries
are preserved into a \textit{journal file} call \texttt{estin.jrn}. Note; one may need to delete extra lines following the
line specifying the name of the data file(s).

As suggested above, \textit{est\_noise} can provide a guess for both the white noise and the power-law amplitude. This
is useful if one has no idea of the scale of the error in the data.
To do this, the program computes two values of "root mean squared"; one that represents the high frequency component
and a second one for the complete data set.  The high frequency RMS is a measure of the white noise component while
the RMS for the entire data set is a measure of the combination of white noise plus colored noise components. The
white noise estimate, $\sigma_w$, is simply RMS $[d(i+1)-d(i)]/ \sqrt{2}$.  To compute the colored noise, one first computes
RMS of the data residuals after fitting a time-dependent model to the data using least-squares with a diagonal covariance matrix
and this is designated as $\sigma_t$.  The colored noise portion is the difference between the total and the white noise;
$\sigma_c = \sqrt { ( {\sigma_t}^2 - {\sigma_w}^2 )}$.  From a power spectrum of colored noise, one integrates over
frequency to obtain  ${\sigma_t}^2 = (1-n) P_c f ^ {1-n}$ evaluated at the nyquist frequency and the low-frequency limit.
Then, using equation~11 of \textit{Langbein} (2004), the power-law amplitude for the specified index of $n$ is estimated:
${\sigma_{pl}}^2 = (2*f_{ny})^{(1 - n/2)} P_c (2 \pi)^n / 2$, where $f_{ny}$ is the Nyquist frequency.  Finally, for the case
where the covariance matrix is \textit{additive} rather than \textit{quadrature}, the white noise estimate needs to be reduced by
$\sqrt {{\sigma_w}^2 - \sigma_w  \sigma_{pl} {\delta t}^{n/4} }$ where $ {\delta t}$ is the sampling interval.

\section{Output}

The program provides several outputs, with the standard out being the most informative. Often, especially
when running the program in batch mode, one should direct the standard output to a file. An example
is shown in the \texttt{./example} directory where \textit{est\_noise} was run as \\
\texttt{../bin/est\_noise7.30 < est\_canp.in > est\_canp.out} \\
In addition to the standard output are \texttt{resid.out} and \texttt{covar.out} which augments the information
presented by the standard output. Specifically, each line in \texttt{resid.out} is the time, the difference
between the observed and predicted observation, the predicted value from the model, and the observation.
\texttt{covar.out} provides the covariance between the parameters of the time-dependent model and, secondly,
their cross correlations.  For \textit{trouble-shooting}, a file, \texttt{tauexp.out} is provided showing the estimates
of time-constants if the program is required to estimate these terms for either an exponential or logarithmic trend.

The standard output is divided roughly into four sections: Some statistics of the sampling of the data; The estimates
of the parameters of the time-dependent model \textit{assuming that the data are temporally independent, Gaussian
distributed}; A sequence of trials of noise parameters as the program iterates to an optimal value of log-likelihood;
And finally a section listing the estimates of the parameters of the time-dependent model and the accompanying
model for noise.

The various statistics compiled from the raw data include the number of points, the statistics of the data sampling,
the number of missing data, and an estimate of the white noise component made by taking the differences
between adjacent observations.

The program then assumes that the data covariance is a diagonal matrix and carries out the least-squares analysis
to estimate the size of each parameter and its standard error.  The standard error is rescaled based upon the
standard deviation of the fit of the model to the data, assuming a diagonal covariance matrix. (Presumably,
these would be the values you would get by plugging the data and the design matrix into spreadsheet).

The bulk of the standard output shows the values of each trial noise model and its corresponding likelihood. If
the time-constant of either an exponential or log-function is being estimated, it is shown, too.

Once the program maximizes the likelihood, the estimates of the parameters of time-dependent model
and their standard errors are shown (they are recomputed at each iteration of noise modeling).
The program attempts to estimate the uncertainty of the parameters of the noise model by
examining the curvature of the log-likelihood function during the optimization.  I don't believe these
results; they look too small.

Along with the value of the logarithm of the likelihood, MLE, both AIC and BIC, Akaike/Bayesian Information Criterion
which both factor in the total degrees of freedom from both the time-dependent and the noise modeling.  However,
I am skeptical of using  AIC/BIC as a method of choosing one noise model over another; More comments
can be found in the companion report called \textit{Threshold\_dMLE} found in this \textit{tar-ball}.

\section{Known problems}

The source program \texttt{time.f} is written by me a long time ago prior to the internet. I keep
updating the code to include the current, and future dates, but I think the current version is
limited to time between 1960 and 2040.

As noted above, the estimates of the standard errors for the parameters of the noise model are
probably too small.

The program has been extensively tested and run using the \texttt{otr} format. Recently, I did some
testing with the \texttt{gmt} format. I wouldn't be
surprised to find bugs with the other time-formats.

When using the \textit{f} or \textit{fast} option to create and invert the data covariance matrix and using
bandpass filtered noise as one component of the data-covariance, \textit{est\_noise7.3x} might "crash"
or provide non-sensible results. This is usually identified by seeing that there are a lot of NaN for some or
all of the trials of covariance parameters in the standard output. On the flip side, \textit{est\_noise7.3x} works
fine using the \textit{c} or \textit{cholesky} option, but for data sets with few gaps, the calculation is slower.
(The legacy option/quadrature \textit{n} works fine, too).  I believe that the problem of deconvolving a filter function
having a bandpass component is that the filter function is oscillatory with some points that are near zero. The inverse
filter function is one that, when convolved with the original filter, yields the \textit{delta} function. The algebra
required to solve this problem has division, and given that the original filter functions has terms that are close
to zero, their is a potential of obtaining large, oscillatory terms in the inverse function. Additional work, using
DFT rather than deconvolution, did not provide satisfactory results. My advise when bandpass filter noised is
desired, run  \textit{est\_noise7.3x} as usual, then, if the \textit{f} option is used and the results of the analysis either
doesn't make sense and/or there are NaN and Infinities in screen dump, then re-run with the \textit{c} option.

\section{Companion Programs}

As mentioned briefly in section 1.3, the \textit{est\_noise} package includes five other programs to
either generate temporally correlated noise, \textit{gen\_noise}, remove outliers, \textit{bust} in conjunction
with \textit{adjust}, and evaluate the results, \textit{compare\_wander} and \textit{psd\_calc}. All of these are
fortran programs that the user provides answers from a list of questions from each program. Since \textit{gen\_noise}
has a number of questions, a \textit{journal} file is created. With the exception of \textit{compare\_wander}, the documentation provided above for \textit{est\_noise}
should provide sufficient information to run each of these programs.

The input to  \textit{compare\_wander} requires two output files from \textit{est\_noise}, \texttt{resid.out} and \texttt{max.dat}.
The \texttt{resid.out} lists the misfits to the time-dependent model prescribed for \textit{est\_noise}, while \texttt{max.dat}
provides the estimated parameters of the noise model.  The output of \textit{compare\_wander} is found in \texttt{wander.out}
which provides a measure of the drift; this is characterized by computing the RMS of $(x(t+{\tau})-x(t))$ as a function of various
intervals, ${\tau}$, for  time series (\textit{Agnew} 1992). The quality of the noise model provided by \textit{est\_noise} can be assessed through
simulating data using the noise model from \textit{est\_noise} and computing the drift from each set of simulated data.
I recommend that at least 200 simulations.  The drift from each simulation is stored, then, for each ${\tau}$ interval,
the RMS drift is sorted and then tabulated to establish the median drift along with the 68\% and 95\% confidence levels.
These statistics, along with the drift of the original data should be plotted for a visual assessment of the quality of the
modeled noise and time dependence.

\section{Sample scripts}

In the directory \texttt{example0}, I provide three scripts to analyze two different time series. Each script should be
sufficiently documented such that they could be reused for other applications.  However, to see the results,
they do need the GMT plotting library; \textit{www.soest.hawaii.edu/gmt/}. These scripts however, use an older version
of GMT, which is version 4. (\textit{ftp://ftp.soest.hawaii.edu/gmt/legacy})

The three scripts are:
\begin{description}
\item[cleanEst.sh] This should be used first as it 1) makes a preliminary estimate of the parameters of the time dependence of the
data using a default, \textit{canned} noise model that is roughly appropriate for continuous GNSS data; 2) detrends those data, 3) removes the outliers, and 4) reinserts the trends. The residuals to the fit are plotted so that one can
add additional time-dependent models as required or remove outliers not caught by \textit{bust\_5}.   In addition, the input to successive calls to \textit{est\_noise} is set-up, which are required in the next two scripts.
\item[EstNoise.sh]  Uses the cleaned data, \texttt{data.cl} and the file \texttt{est0.in} from \textit{cleanEst.sh} along with a user selected noise
model and runs \textit{est\_noise} to find the optimal parameters of the noise model.  The residuals to this fit are plotted.
In addition, the output of \textit{est\_noise} is feed into \textit{compare\_wander} and the RMS drift of the data, along with
the statistics from a set from simulations are plotted.  Finally, the equivalent power spectral density is plotted.
\item[EstNoiseAll.sh]  Instead of evaluating a single type of noise model, this script evaluates a sequence of noise models
as a batch process such that the best noise model can be identified after evaluating the changes in Log(likelihood) and
the corresponding drift.  Initially, both the random-walk plus white noise and the flicker plus white noise models are
computed.  The script picks the one with the \textit{maximum likelihood} and identifies it as the so called \textit{null} model.
Next, the power law model is evaluated and its likelihood is compared with that from the \textit{null} model. Similarly,
the flicker plus random-walk noise is evaluated. Of these four, it is likely that one of these is a clear "winner". However,
the script goes on to evaluate both First order Gauss-Markov and Generalized G-M noise. A final set of evaluations uses
bandpass filtered noise superimposed on either flicker or random-walk noise, and flicker plus random walk noise (the default) or power-law noise with the \texttt{-P} option.  Using the \texttt{-S} option,
one can limit the number of noise models tested from the original eight. For all of these, a drift curve is evaluated.
\end{description}
These scripts currently set for the \texttt{otr} format and mostly set-up for
the \texttt{gmt} format.

Prior to running each of these scripts, you will need to edit each script to identify where the executable programs reside.
This line should be located near the top of the file containing the script with an environment variable \texttt{progs}.  The actual
work by these scripts is carried out in \texttt{/tmp/SCRATCH}.

By typing the command, one should get documentation with regards to the inputs to these files. The \texttt{example0} directory
contains the scripts, two data sets, files that are required for the inputs, and the output plot files.

\subsection{Example 1}

The data are collected from a site with observations spanning both the San Simeon and Parkfield  Earthquakes (2003 and 2004
respectively). The time dependent model includes sinusoids, a secular rate, offsets due to both earthquakes, \texttt{canp.off},
two Omori law, decay functions and a rate change, \texttt{canp.trd}, and a few observations deleted just after the Parkfield
earthquake, \texttt{canp.edit}.  The initial "cleaning" is obtained by:
\\
\texttt{cleanEst.sh -d canp.e -f otr -O canp.off -T canp.trd -E canp.edit}
\\

Because the time constant for the Omori function for the Parkfield earthquake is being estimated, \textit{est\_noise} runs
slower than normal.  The results can be found in \texttt{canp.e.ps} and \texttt{est\_canp.e.out}.  To speed-up the
next set of evaluations, I have modified the \texttt{.trd} file by \textbf{fixing} the Omori time constant for the Parkfield
Earthquake to $0.0061746$ years which is shown in \texttt{canp1.trd}. Consequently, I re-run \textit{cleanEst.sh}
\\
\texttt{cleanEst.sh -d canp.e -f otr -O canp.off -T canp1.trd -E canp.edit}
\\

To evaluate a specific noise model, I run \textit{EstNoise.sh} as:
\\
\texttt{EstNoise.sh -d canp.e  -M PL}
\\
The outputs of this analysis are shown in  \texttt{canp.e\_PL.ps} and \texttt{est\_canp.e\_PL.out}.  The plot shows the
residuals to a fit to a time dependent model, the drift of the data compared to simulated data having the same
model of noise, and the equivalent power spectra of the noise model.  The drift suggests that observed data has slightly
more noise over the 300 to 1000 day periods that the noise provided by a power-law noise model.  Consequently, I could try a combination of flicker and random walk.
\\
\texttt{EstNoise.sh -d canp.e  -M FLRW}
\\
Indeed, examination of the drift, \texttt{canp.e\_FLRW.ps} suggests an improvement in the fit. In addition, the increase
in Log likelihood, from -7667.7 to -7660.9 from the PL to the FLRW model suggests that the flicker and random-walk
model of noise is more appropriate for this data set.
Likewise, other noise models can be evaluated.

However, a more comprehensive search for the optimal noise model is carried out with \textit{EstNoiseAll.sh}. For this example,
it is invoked with:
\\
\texttt{EstNoiseAll.sh -d canp.e }
\\
Examination of the results of the drift spectra along with the values of Log(likelihood) (MLE), AIC, and BIC shown in
\texttt{canp.e\_all.ps} suggests that the FLRW noise model is best since it has the largest MLE and the smallest AIC and BIC.

\subsection{Example 2}

This is an example of simulated noise having first order Gauss Markov noise. In addition, there is an offset, and the
data use the \textit{gmt} format.  Initially, the data are "cleaned";
\\
\texttt{cleanEst.sh -d noise2.gmt -f gmt -E noise.edit -O noise.off}
\\
The results are presented in \texttt{noise2.gmt.ps} and \texttt{est\_noise2.gmt.out}.
\\

Noise modeling is accomplished by:
\\
\texttt{EstNoise.sh  -d noise2.gmt  -M RW}
\\
Although other noise models can be evaluated, PL, FOGM, and GM, none seem to be much better than the random walk
model.

Likewise, a search of optimal noise models is provided by invoking:
\\
\texttt{EstNoiseAll.sh  -d noise2.gmt}
\\
Examination of the results, \texttt{noise2.gmt\_all.ps} suggests that RW is probably the best noise model since
its closest competitor, FOGM, has marginally larger MLE (1.39) and lower AIC (-0.8) relative to the null model of RW.
On the other hand, BIC for FOGM is worse than for RW.  The most conservative choice would be to chose the model
having random-walk represent the data at its longest periods which provides larger standard error in rate relative to
the FOGM model.

\section{Updates}

Updates are included in a file call \texttt{BugFix.txt}.

\section{References}

Agnew, D. (1992), The time domain behavior of power law noises, \textit{Geophys. Res. Lett.},  doi:{10.1029/91GL02832}.

Bos, M.~S., R.~M.~S.~ Fernandes, S.~P.~D.~Williams, and L.~Bastos (2012), Fast error analysis of
continuous GPS observations,
\textit{J. Geod.},  doi:{10.1007/s00190-007-0165-x}.

%Hackl, M., R.~Malservisi, U.~Hugentobler, and R.~Wonnacott (2011), Estimation of velocity uncertainties
%from GPS time series: examples from analysis of the South African TrigNet network, \textit{J.  Geophy. Res.},
 %\textit{116B15:11404}

Langbein, J., and H. Johnson (1997), Correlated error in geodetic time series: Implications for
time-dependent deformation, \textit{J.  Geophy. Res.},  \textit{102}, 591--604.

Langbein, J. (2004), Noise in two-color electronic distance meter measurements revisited,
 \textit{J.  Geophy. Res.}  doi:{10.1029/ 2003JB002819}.

Langbein, J. [2008], Noise in GPS displacement measurements from Southern California and Southern Nevada,
 \textit{J.  Geophy. Res.}  doi{10.1029/2007JB005247}.

Langbein, J. [2009], Computer algorithm for analyzing and processing bore- hole strainmeter data, \textit{Comput. Geosci.}, 36(5), 611-619, doi:{10.1016/j.cageo.2009.08.011}

Langbein, J. [2012], Estimating rate uncertainty with maximum likelihood: differences between power-law and flicker/random-walk
models, \textit{J. Geod.} doi:{10.1007/s00190-012-0556-5}

Langbein, J. [2017], Improved method for maximum likelihood analysis of time series with temporally correlated errors,
\textit{J. Geod.} doi{0.1007/s00190-017-1002-5}.  note that a copy of this paper is included in the documentation

Nelder, J. A., and R. Mead (1965), A simplex method for function minimization, \textit{Computer Journal 7:} 308?313. doi:10.1093/comjnl/7.4.308

Press,, W.H, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, (1992) \textit{Numerical Recipes in Fortran 77}, second edition,
Cambridge University Press.

Williams, S. D. P., Y. Bock, P. Fang, P. Jamason, R. M. Nikolaidis,
L. Prawirodirdjo, M. Miller, and D. J. Johnson (2004), Error analysis of continuous GPS position time series,
 \textit{J.  Geophy. Res.}  doi:{10.1029/2003JB002741}.

\end{document}


%!TEX root = ../tikz-figures.tex

\section*{Gaps in a poset}

\index{brace}
\begin{equation*}
	\begin{tikzpicture}
		\path
			(0,0) node[point] {}
			-- ++(0,0.6) node {$\vdots$}
			-- ++(0,0.4) node[point] [label=left:$v_1$] {}
			edge[draw,densely dotted] +(-0.3,0.3)
			edge[draw,densely dotted] +(0.3,0.3)
			-- +(-0.25,0.75) node[point] [label=right:$w_1$] {}
			-- ++(0,1.5) node[point] [label=left:$u_1$] {}
			edge[draw,densely dotted] +(-0.3,-0.3)
			edge[draw,densely dotted] +(0.3,-0.3)
			-- ++(0,0.6) node {$\vdots$}
			-- ++(0,0.4) node[point] [label=left:$v_2$] {}
			edge[draw,densely dotted] +(-0.3,0.3)
			edge[draw,densely dotted] +(0.3,0.3)
			-- +(0.25,0.75) node[point] [label=left:$w_2$] {}
			-- ++(0,1.5) node[point] [label=left:$u_2$] {}
			edge[draw,densely dotted] +(-0.3,-0.3)
			edge[draw,densely dotted] +(0.3,-0.3)
			-- ++(0,0.6) node {$\vdots$}
			-- ++(0,0.4) node[point] {};
			\draw[decoration={calligraphic brace,amplitude=0.2cm}, decorate, line width=1.25pt] (0.5,1) -- (0.5,0);
			\draw[decoration={calligraphic brace,amplitude=0.2cm}, decorate, line width=1.25pt] (0.5,3.5) -- (0.5,2.5);
			\draw[decoration={calligraphic brace,amplitude=0.2cm}, decorate, line width=1.25pt] (0.5,6) -- (0.5,5);
			\node at (3,3) (X) {X};
			\draw (X)
				edge (0.7,0.5)
				edge (0.7,3)
				edge (0.7,5.5);
	\end{tikzpicture}
\end{equation*}


\documentclass{article}

\input{../preamble}

\title{A Graphical Representation of the Solos Calculus \\
	\large CM30082 Project Proposal}

\author{Adam Lassiter}

\date{October 2017}

\begin{document}

\maketitle

\vfill

\tableofcontents

\pagebreak

\section{Problem Description}
\subsection{Current Problem}
As computer processor clock speeds have begun to stagnate in performance increase per year, manufacturers have begun to shift focus to multiple cores in search of greater performance. Supercomputers for decades have operated around the idea of clusters and massive parallelism. For this reason there is now more than ever a need to study the mathematical effects of concurrency in computation.

\subsection{Existing Solutions}
\subsubsection{$\lambda$-calculus}\label{ssubsec:solns_lambda_calc}
Developed by Alonzo Church in the 1930s, the $\lambda$-calculus is a more mathematical representation of a computable function. It forms a universal model of computation and can contain an encoding of any single-taped Turing machine. The $\lambda$-calculus became a popular formal system within which to study properties of computation.
The $\lambda$-calculus consists of an expression $M$ built from the following terms:
\begin{equation*}
	M  \quad::=\quad  a  \quad|\quad  (\lambda x . M)  \quad|\quad  (M N)
\end{equation*}
where the expressions describe a variable, an abstraction and an application of a function on an argument. From this, any computable function can be constructed and computation is achieved through a series of reductions.
\\
While the $\lambda$-calculus has been successful and been studied by various areas of academia outside of Computer Science, it is limited in application by its fundamentally `single-process' model and struggles to describe multiple systems working and communicating together.

\subsubsection{$\pi$-calculus}
Similar to the $\lambda$-calculus as described in~\ref{ssubsec:solns_lambda_calc}, there exists the $\pi$-calculus for the study of concurrent computation.
\\
The $\pi$-calculus is constructed from the recursive definition of  an expression $P$:
\begin{equation*}
	P  \quad::=\quad  P|P  \quad|\quad  c(x).P  \quad|\quad  \bar{c}(x).P  \quad|\quad  vx.P  \quad|\quad  !P  \quad|\quad  0
\end{equation*}
where the expressions describe each of concurrency, waiting for input, providing output, name binding (similar to $\lambda x . P$), replication (which itself is $!P \coloneq P|!P$) and process termination respectively.
While it provides the expressiveness required for Turing-completeness, it does not lend itself to understandability nor clarity of the problem encoding when presented as a standalone expression.

\subsubsection{Solos Calculus}
As defined by~\cite{solo-concert}, the Solos calculus is constructed from \textit{solos} ranged over by $\alpha,\beta,\ldots$ and \textit{agents} ranged over by $P,Q,\ldots$ as such:
\begin{center}
	\begin{tabular}{ l l l }
		$\alpha ::=$	& $u\tilde{x}$		& (input) \\
				& $\bar{u}\tilde{x}$	& (output) \\ \\
		$P ::=$		& $0$			& (inaction) \\
				& $\alpha$		& (solo) \\
				& $Q | R$		& (composition) \\
				& $(x)Q$		& (scope) \\
				& $[x=y]Q$		& (match) \\
	\end{tabular}
\end{center}
This was further developed by~\cite{solo-diagrams} to provide a one-to-one correspondence between these  expressions and `diagram-like' objects. This provides a strong analog to real-world systems and an applicability to be used as a modelling tool for groups of communicating systems.

\subsubsection{Others}
There also exist many further variants, such as the Fusion calculus --- itself a superset of $\pi$-calculus --- and CSP (or communicating sequential processes). These aim to improve on the $\pi$-calculus in making the problem encoding more understandable, but at the cost of a more complicated language, making them harder to implement and prove reductions in.

\subsection{Project Aims}
The project seeks to provide an implementation of both the Solos calculus, for which there exists an encoding of the $\pi$-calculus within itself, and also of Solos diagrams, an intuitively graphical representation of the calculus. These diagrams can then aid in understanding of reductions of process calculi, while maintaining a clean and simple language with strong provable properties.
Furthermore, the design of the Solos calculus is such that it is fully asynchronous. That is, unlike in the $\pi$-calculus where processes `block' while waiting for inputs/outputs, it is shown through the construction of the Solos calculus that no such system is required --- however there still exists a way of building such a system to give the effects of the $\pi$-calculus should it be desired.
For these reasons, the Solos calculus is found to be an interesting alternative to the more common $\pi$-calculus and is therefore chosen as the project focus.

\subsection{Objectives}
Throughout the project, the following objectives and landmarks are hoped to be achieved:
\begin{itemize}
	\item Research current models of concurrency calculus and understand common use-cases and environments.
	\item Build a system capable of computing Solos calculus expressions.
	\item Be able to convert expressions to diagrams.
	\item Visualise and interact with/modify diagrams and watch reductions step-by-step.
	\item Evaluate qualities of the system as a learning tool.
\end{itemize}

\section{Requirements Specification}
The following seeks to provide an estimate for the progress path the project will take, judging by initial research and planning. These are likely to evolve over time, as are priorities of each requirement (\textit{must $>$ should $>$ may $>$ might}).

\subsection{Solos Expressions}\label{subsec:reqs_solo_expr}
The first functional part of the system will be focused on text-based Solos Expressions only. This will form a basis and back-end to build upon to implement other features.
This system:
\begin{itemize}
	\item \textit{Must} support input and output of text representing Solos Expressions.
	\item \textit{Must} be able to perform reductions on inputted expressions and return them.
	\item \textit{Should} provide this functionality through an accessible user interface.
	\item \textit{May} be implemented to evaluate expressions concurrently.
\end{itemize}

\subsection{Solos Diagrams}
This will be the first extension of the system to allow interoperability with Solos Diagrams, an equivalent representation of the calculus, which has intuitive graphical representations.
This system:
\begin{itemize}
	\item \textit{Must} support input and output of said Solos Diagrams.
	\item \textit{Must} provide a conversion between Diagrams and Expressions.
	\item \textit{Should} be able to provide the same reductions as in the Expressions.
	\item \textit{May} be able to produce step-by-step reductions.
	\item \textit{Might} be implemented to evaluate diagrams concurrently.
\end{itemize}

\subsection{Diagram Visualisation and Interaction}
The second extension will build on the previous to provide a graphical interface for input/output/manipulation of Solos Diagrams giving a clear visual representation of how the calculus works.
This system:
\begin{itemize}
	\item \textit{Must} support visualisation of Solos Diagrams through a GUI\@.
	\item \textit{Must} support input/output of a (potentially manipulated) Diagram from/to an Expression.
	\item \textit{Should} support manipulation of a Diagram through use of the GUI only.
	\item \textit{Should} be both intuitive and feature-complete to provide an obvious improvement over text-based input as described in Section~\ref{subsec:reqs_solo_expr}.
	\item \textit{May} show step-by-step reductions of Diagrams.
\end{itemize}

\section{Project Plan}
\subsection{Phases}
The project will be broken down into the following stages, with each building on the previous. This allows flexibility if there is not enough time as an incomplete extension will still leave a working system.
\begin{enumerate}
	\item Literature review
	\item Core system (Solos Expressions) implementation and unit testing
	\item Extension 1 (Solos Diagrams) implementation and integration testing
	\item Extension 2 (Graphical Visualisation) implementation and integration testing
	\item Analysis, write-up and conclusion
\end{enumerate}

\subsection{Milestones and Deliverables}
\begin{center}
	\begin{tabular}{ l l }
		\toprule
		\textit{Milestone} & \textit{Target Date} \\
		\midrule
		\textbf{Project proposal} & {27th October, 2017} \\
		\textbf{Literature review} & {24th November, 2017} \\
		{Minimum viable product --- Solos Expressions} & {20th December, 2017} \\
		{Extension 1 --- Solos Diagrams} & {20th January, 2018} \\
		{Extension 2 --- Graphical representation} & {19th February, 2018} \\
		\textbf{Progress demonstration} & {19th February, 2018} \\
		{User testing and critical analysis} & {10th March, 2018} \\
		\textbf{Dissertation} & {4th May, 2018} \\
		\bottomrule
	\end{tabular}
\end{center}

\subsection{Time Breakdown}
\begin{center}
	\begin{ganttchart}[
		hgrid,
		vgrid={*{6}{draw=none}, dotted},
		time slot format=isodate,
		y unit title=0.6cm,
		y unit chart=0.6cm,
		x unit=0.065cm,
		milestone/.append style={xscale=5},
	]{2017-11-01}{2018-04-29}
		\ganttset{bar height=0.6}
		\gantttitlecalendar{year, month=name, week} \\
		\ganttbar{Literature Study}{2017-11-01}{2017-11-24} \\
		\ganttlinkedmilestone{Literature Review}{2017-11-24} \\
		\ganttlinkedbar{Solos Expressions}{2017-11-24}{2017-12-20} \\
		\ganttlinkedbar{Solos Diagrams}{2018-01-07}{2018-01-20} \\
		\ganttlinkedbar{Graphical Interface}{2018-01-20}{2018-02-19} \\
		\ganttlinkedmilestone{Progress Demo}{2018-02-19} \\
		\ganttlinkedbar{Finishing, User Testing}{2018-02-19}{2018-03-10} \\
		\ganttlinkedbar{Critique, Analysis}{2018-03-10}{2018-04-01} \\
		\ganttlinkedbar{Write-Up}{2018-04-01}{2018-04-27} \\
		\ganttlinkedmilestone{Dissertation}{2018-04-29}
	\end{ganttchart}
\end{center}

\section{Resources}
\subsection{Technology}
The system will be developed as a Python application, as this allows for quick, modular development as performance is not critical, but still allows for multiple processes without hassle.
The GUI is likely to be written using Tcl/Tkinter, as it is the common choice for Python graphical interfaces and is suited to 2D visualisation well-enough to be useful, while allowing easy implementation of interactive features.

\subsection{Literature}
Most literature for the project has already been obtained through both digital copies of papers and journals as well as physical books on the subject. While the topic of Solos calculus forms a niche area of research, the study of $\pi$-calculus is well-studied and has a large amount of relevant information that may be carried over.

\subsection{Testing and Evaluation}
In the scope of a learning and teaching aid on the subject, the system will need to be assessed for ease-of-use and for how well it explains the topic. For this, a small group of volunteers with moderate Computer Science understanding must be sourced, likely from a group of peers.

\section{References}
\bibliography{proposal}

%\section{Agreement}
%\subsection{Student Signature}
%\vspace{1in}

%\subsection{Supervisor Signature}
%\vspace{1in}

\end{document}


\chapter{Introduction}
\label{chap_intro}
Representation Learning is one of the key area in machine learning, where intense research works are being done spanning over decades. The choice of the representation has a considerable influence in the performance of learning algorithms (\cite{bengioRL} et al.). Representation Learning is also considered as an effective mechanism for transferring the prior knowledge of humans to the machine learning system. \cite{bengioAI} et al. broadly classified the architecture of the representation learning system  into deep and shallow architectures. Shallow architectures often assumes strong task specific priors, whereas deep architectures can support broad priors. Algorithms like SVM, single layer neural network etc. come under shallow architectures, and algorithms like deep belif networks, convolutional neural networks etc. belong to deep architectures.

Deep Learning is a rapidly developing area among representation learning algorithms, which makes use of depth as a key criteria for producing good representations (hence they belong to deep architectures). \textit{In general, Deep Learning is a set of algorithms in machine learning that attempts to model high-level abstractions in data by using model architectures composed of multiple non-linear transformations}(\cite{bengioLDA} et al., \cite{bengioRL} et al.). Deep Learning methods are preferred over shallow ones in many complex learning tasks such as computer vision, speech recognition etc. due to: \textit{the wide range of functions that can be parameterized by composing weakly nonlinear transformations, the broad range of prior knowledge they can transfer to the learning system, the invariance being modelled by such systems against local changes in the input and their ability to learn more abstract concepts in an hierarchical fashion}(\cite{saul} et al., \cite{bengioAI} et al.).

Though the idea of using such deep architectures for learning representations was known to machine learning researchers, two main factors deterred the wide use of deep learning algorithms: (i) the gradient-based methods often gets trapped in local minima which results in poor generalization capacity, (ii) training the networks with more than 2 or 3 layers was a big computational challenge. The \textit{greedy layerwise unsupervised pre-training} proposed by \cite{hintonDBN} et al. was a breakthrough in machine learning research as it tackled the difficulty in training such deep models. Using this method  a hierarchy of features can be learned one level at a time, using  unsupervised  techniques(\cite{bengioRL} et al.). This greedy layerwise unsupervised pre-training is used to initialize the weights of the network (unsupervised pre-training stage) and then the weights are fine-tuned with the given label information(supervised fine-tuning stage).

The recent rise in computing power alleviated the second problem upto a considerable level. Efficient training methods are devised for learning networks with millions of parameters both on multi-core CPUs and GPUs(\cite{gpu1} et al., \cite{gpu2} et al.). Scalable deep learning models like \cite{graphlab} etc. are also developed in distributed cloud based environments.

Armed with these tools deep learning witnessed a sudden boost since 2006, producing state-of-the-art results in several complex machine learning tasks like speech recognition(\cite{speechBest} et al.), visual object recognition(\cite{imagenet} et al., \cite{cifarBest} et al., \cite{mnistBest} et al.), natural language processing(\cite{nlpBest} et al.) etc. However, most of the deep learning algorithms are developed on top of neural network based models(\cite{lecunnCNN} et al., \cite{hintonDBN} et al.).

Many researchers had extensively studied the limitations of kernel machines in such problems. The analysis shown in \cite{bengioAI} et al. claims that the number of training examples required for a kernel machine with Gaussian kernel may grow linearly with the number of variations (change in the directions) of the target function. \cite{saul} et.al explored the concept of \textit{kernel methods based Deep Learning} by proposing Multi-layer Kernel Machines(MKMs). The architecture of MKMs consists multiple layers with each layer performing unsupervised feature extraction using kernel PCA\nomenclature{PCA}{Principal Component Analysis} proposed by \cite{kpca} et al. followed by supervised feature selection by ranking features based on their mutual information with class labels. They also proposed a multi-layer kernel function, which can mimic the computations of neural network based architecture.

Inspired by the work of \cite{saul} et al., multi-layer kernel machines were explored  by many researchers (\cite{2l_mkl} et al., \cite{deep_mkl} et al.). Most of these researchers used multiple kernel learning (MKL) concepts in the layered architecture. Some of these works are using deep layerwise kernel models for automating the kernel learning task in SVMs. But there is less work done combining feature learning with MKMs and MKL. MKL in single layer for unsupervised feature extraction was studied by \cite{zhuang} et al. Much of the work done in this thesis is also focussed on MKL and MKMs for unsupervised feature learning to build a deep learning architecture for kernel machines.

\section{Thesis Outline}
\label{chap1_outline}
The contents of this thesis are organized as six chapters.

The \autoref{chap_mkm} discusses the MKMs in detail, with empirical study on popular deep learning datasets. MKMs with mixed kernels are also tested on these datasets. Finally, some visualization (using tSNE algorithm) of the features learned by the model are also provided.

The \autoref{chap_mlmkl}  describes the study of using MKL in MKMs. The formulation of the unsupervised MKL is given first, followed by empirical study and exploratory analysis on MKL in each layer of MKMs.

The \autoref{chap_kfda} is devoted to discriminant analysis with multi-layer kernels. It includes a brief summary of KFDA algorithm and the results from empirical study.

The \autoref{chap_struct} contains a study of multi-layer kernels on structured output spaces. Struct SVM, a large margin generalization of SVM to structured output spaces is chosen as the candidate algorithm for study, and analysis is done on multi-label and multi-class classification problems.

The \autoref{chap_conc} gives the conclusion of this thesis work, along with some potential future directions to explore.

\section{Contributions}
\label{chap1_contri}
The main contributions of this thesis can be summarized as follows:
\begin{itemize}
\item An in depth study is made on multi-layer kernel machines with single and mixed kernels.
\item MKL concepts are introduced in MKMs by taking a combination of kernels in each layer, following an unsupervised learning paradigm. The contribution of each kernel in the MKL at every layer is analyzed. A comparative study is done with existing deep learning algiritms.
\item A study on the discriminating power of multi-layer kernels is done with KFDA algorithm. Empirical results are compared with the performance of existing deep architectures.
\item A comparative study between multi-layer and single layer kernels are done on structured output spaces. The problem instances chosen for the study is multi-label and multi-class classification problems.
\end{itemize}


\documentclass[main.tex]{subfiles}
\begin{document}

\section{Boltzmann equation for CDM}

\marginpar{Thursday\\ 2020-3-26, \\ compiled \\ \today}

[From yesterday: the distribution function for Compton scattering is general, as long as electrons and photons are in thermal equilibrium.]

Last time we derived the first order Boltzmann equation for the thermal anisotropy.

Now, however, we need to discuss CDM, and then we will discuss baryons..

For Dark Matter the collision term is negligible (since by the observations of galactic dynamics we know that DM is collisionless):
so in the Boltzmann equation we have
%
\begin{align} \label{eq:Boltzmann-DM}
\bigd{f _{\text{dm}}}{\lambda } = 0
\,.
\end{align}

The key fact is that DM is nonrelativistic. The calculation, except for this, is quite analogous.
The momentum \(P^{\mu }\) has to be normalized: \(P^{\mu }P_{\mu } = g_{\mu \nu } P^{\mu }P^{\nu } = -m^2\), as opposed to \(0\), which we had with the photon. We define the norm of the three-momentum
%
\begin{align}
p^2 = g_{ij} P^{i} P^{j}
\,,
\end{align}
%
and with it the energy: \(E = \sqrt{p^2+m^2}\). Then, using our perturbed metric we get
%
\begin{align}
P^{0} = E (1-\Phi )
\qquad \text{and} \qquad
P^{i} = p \hat{p}^{i} \frac{1 + \Psi }{a}
\,,
\end{align}
%
where \(\hat{p}^{i}\) is a unit vector in the direction of \(p^{i}\).
This makes sense: the velocity, without perturbation, decays like \(a^{-1}\).

Notice that we can make the derivative with respect to the affine parameter \(\lambda \) to one with respect to \(t\), the factor \(\dv*{t}{\lambda }\) can be moved to the RHS and does not affect our discussion.
The LHS of the Boltzmann equation \eqref{eq:Boltzmann-DM} then reads
%
\begin{align}
\bigd{f_{\text{dm}}}{t} = \pdv{f_{\text{dm}}}{t}
+ \pdv{f _{\text{dm}}}{x^{i}} \dv{x^{i}}{t}
+ \pdv{f _{\text{dm}}}{E} \dv{E}{t}
+ \underbrace{\pdv{f _{\text{dm}}}{\hat{p}^{i}} \dv{\hat{p}^{i}}{t}}_{\text{higher order}}
\,,
\end{align}
%
where the last term is second order since it is a product of factors which are both zero to zeroth order.
With steps analogous to the massless case we find
%
\begin{align}
0= \pdv{f _{\text{dm}}}{t} + \frac{\hat{p}^{i}}{a} \frac{p}{E} \pdv{f _{\text{dm}}}{x^{i}}
- \pdv{f _{\text{dm}}}{E} \qty[
\frac{1}{a} \dv{a}{t} \frac{p^2}{E} - \frac{p^2}{E} \pdv{\Psi }{t}
+ \frac{\hat{p}^{i} p}{a} \pdv{\Phi }{x^{i}}
]
\,.
\end{align}

The factor \(\dv*{a}{t} / a\) is just \(H\), we write it this way in order not to overload the dot (which represents derivatives with respect to conformal time only in our notation).

In order to work with this, we shall consider only terms up to first order in \(p / E\), which is justified since DM is non-relativistic.
Also, we will take moments: in general, a moment is an integral of a function (with angular dependence) multiplied by some power of the cosine of the angle between the direction considered and some fixed other direction.
In principle, the hierarchy goes to any order: in practice, we can usually truncate the hierarchy to some order according to some physical principle.
Fortunately, DM behaves like a fluid, which allows us to work to first order only.

\todo[inline]{Are the NS equations not exact?}

This approach is fully relativistic, but linear.
In star formation, we use Newtonian approximation, but we account for  nonlinearity.
Having both is impossible.

The lowest (0th) moment is just given by the integral in \(\dd[3]{p} / (2 \pi )^3\) of the distribution, that is, over all angles: we find
%
\begin{subequations}
\begin{align}
\begin{split}
0&= \pdv{}{t} \int \frac{\dd[3]{p}}{(2 \pi )^3} f _{\text{dm}}
+ \frac{1}{a} \pdv{}{x^{i}} \int \frac{\dd[3]{p}}{(2 \pi )^3} f _{\text{dm}} \frac{p \hat{p}^{i}}{E} \\
&\phantom{=}\
- \qty[\frac{1}{a} \dv{a}{t} - \pdv{\Psi }{t}]
\int \frac{\dd[3]{p}}{(2 \pi )^3} \pdv{f _{\text{dm}}}{E} \frac{p^2}{E}
- \underbrace{\frac{1}{a} \pdv{\Phi }{x^{i}} \int \frac{\dd[3]{p}}{(2 \pi )^3} \pdv{f _{\text{dm}}}{E} \hat{p}^{i} p}_{\text{higher order}}
\marginnote{We take out of the integral any term not depending on the angles in momentum space.}
\,.
\end{split}
\end{align}
\end{subequations}
%
why is the last term higher order? There is no way to have a nonzero result to zeroth order in the integral by isotropy, and it is multiplied by a first-order perturbation.

Now, we introduce the dark matter density \(n _{\text{dm}}\) --- for more details, see the notes for the course in Fundamentals of Astrophysics and Cosmology \cite[section 3.2]{tissinoFundamentalsAstrophysicsCosmology2020}, but do note that the number density defined there is global, while the one we are considering here is a local number density, which can be different for different points in spacetime.
Also, we define its velocity \(v^{i}\):
%
\begin{subequations}
\begin{align}
n _{\text{dm}} &= \int \frac{ \dd[3]{p}}{(2\pi )^3} f _{\text{dm}}
\marginnote{Note that \(f _{\text{dm}}\) accounts for the number of degrees of freedom \(g _{\text{dm}}\).}\\
v^{i } &= \expval{ \frac{P^{i}}{E}}_{\text{particle}}
= \frac{1}{n _{\text{dm}}} \int \frac{ \dd[3]{p}}{(2 \pi )^3} f _{\text{dm}} \frac{p \hat{p}^{i}}{E}
\,.
\end{align}
\end{subequations}

\todo[inline]{These are not covariant, right? Since \(n\) is the 0th component of a 4-vector\dots}

In the first and second terms we can insert \(n _{\text{dm}}\) and \(v^{i}\) directly (to first order --- we always do manipulations neglecting second order terms), while for the third term we need to integrate by parts, using \(\dv*{E}{p} = p/E\):\footnote{This comes from \(E = \sqrt{p^2+m^2}\):
%
\begin{align}
\pdv{E}{p} = \frac{ 2 p }{2 \sqrt{p^2+m^2}} = \frac{p}{E}
\,.
\end{align}
} we obtain:
%
\begin{subequations}
\begin{align}
\int \frac{\dd[3]{p}}{(2 \pi )^3} \pdv{f _{\text{dm}}}{E} \frac{p^2}{E}
&= \int \frac{ \dd[3]{p}}{(2\pi )^3} p \pdv{f _{\text{dm}}}{p}
\marginnote{Changed the derivative of the distribution} \\
&= \frac{4 \pi }{(2 \pi )^3} \int_{0}^{ \infty } \dd{p} p^3 \expval{\pdv{f _{\text{dm}}}{p}}  \\
&= -3 \frac{4 \pi }{(2 \pi )^3} \int_{0}^{ \infty } \dd{p} p^2 \expval{f _{\text{dm}}}= -3 n _{\text{dm}}
\,,
\end{align}
\end{subequations}
%
which is a calculation discussed in the FAC course \cite[]{tissinoFundamentalsAstrophysicsCosmology2020} [I'll insert the section when I get to it].

This yields:
%
\begin{align}
\pdv{n _{\text{dm}}}{t} + \frac{1}{a} \pdv{}{x^{i}} \qty(n _{\text{dm}} v^{i}) + 3 \qty[\frac{1}{a} \dv{a}{t}  - \pdv{\Psi }{t}] n _{\text{dm}} = 0
\,,
\end{align}
%
which to 0th order yields
%
\begin{align}
\pdv{n _{\text{dm}}^{(0)}}{t} + 3 \frac{1}{a} \dv{a}{t} n _{\text{dm}}^{(0)} \iff a^3 n _{\text{dm}}^{(0)} = \const
\,.
\end{align}
%
Now, we can write the full number density as
%
\begin{align}
n _{\text{dm}} = n _{\text{dm}}^{(0)} \qty[1 + \delta (\vec{x}, t)]
\,,
\end{align}
%
where \(\delta \) is the standard notation in cosmology for these kinds of dimensionless fractional perturbations.

Then, after some simplifications the evolution of this perturbation looks like
%
\begin{align}
\pdv{ \delta  }{t} + \frac{1}{a} \pdv{v^{i}}{x^{i}} - 3 \pdv{\Psi }{t}=0
\,,
\end{align}
%
which is our first order continuity equation.

This was for the first moment, for the next one we integrate after multiplying by \(\hat{p}^{j} / E\).
We neglect a term because it is too relativistic, that is, second order in \(p / E\).

Integrating by parts, for a different term we get
%
\begin{align}
\int  \frac{ \dd[3]{p}}{(2\pi )^3} \pdv{f _{\text{dm}}}{p} \frac{p^2 \hat{p}^{i}}{E} &= \int \frac{ \dd{\Omega  } \hat{p}^{j}}{(2 \pi )^3}
\int \dots lni
&= - 4 n _{\text{dm}} v^{i}
\,.
\end{align}

We finally obtain
%
\begin{align}
\pdv{(n _{\text{dm}} v^{i})}{t} + 4 \frac{1}{a} \dv{a}{t} n _{\text{dm}}v^{j} + \frac{n _{\text{dm}}}{a} \pdv{\Phi  }{x^{i}} =0
\,,
\end{align}
%
so we factor out the background and get, to first order,
%
\begin{align}
\pdv{v^{j}}{t} + H v^{j} + \frac{1}{a} \pdv{\Phi }{x^{j}}=0
\,.
\end{align}

This is basically a linear Euler equation. In Fourier space, we get
%
\begin{subequations}
\begin{align}
\widetilde{\dot{\delta}} + i k \widetilde{v} - 3 \widetilde{\dot{\psi}} &= 0  \\
\widetilde{\dot{v}} + \frac{\dot{a}}{a} \widetilde{v} + i k \widetilde{\Phi} &= 0
\,.
\end{align}
\end{subequations}

\section{Boltzmann equation for baryons}

Which interaction are relevant?
At recombination, electrons are tightly coupled with photons, while the interaction between protons and photons is suppressed by a factor \((m_{e} / m_p)^2\) with respect to to electron-photon scattering.
We need to deal with Coulomb scattering between electrons and proton.
So, we can say that the quantity
%
\begin{align}
\frac{ \rho_{e} - \rho_{p}^{(0)}}{\rho_{e}^{(0)}}  =
% \frac{ \rho_{e} - \rho_{p}^{(0)}}{\rho_{e}^{(0)}}  =
\rho_{b}
\,,
\end{align}
%
is a perturbation. So we write two Boltzmann equations, for electrons and protons, assuming that they move with the same velocity. The collision term looks like
%
\begin{align}
C_{e \gamma }= (2 \pi)^{4} \delta^{(4)} (p + q - p' - q')
\frac{ \abs{\mathcal{M}}^2 \qty[f_{e}(q') f_{\gamma }(p') - f_e (q) f_\gamma (p)] }{8 E(p) E(p') E_e(q) E_e(q')}
\,,
\end{align}
%
So the zeroth moment equation for electrons is the same as the one for Dark Matter: both of the terms in the RHS vanish in
%
\begin{align}
\pdv{n_e}{t} + \frac{1}{a} \pdv{}{x^{i}} (n_e v^{i}_{b}) + 3 \qty[H - \pdv{\Psi }{t}]n_e = \expval{C_{ep}} + \expval{C_{e \gamma }}
\,,
\end{align}
%
since they multiply antisymmetric terms in exchange of momenta [clear up].

For the first moment we sum the proton and electron equations multiplied by the respective charges.
The symmetry of the momenta of electrons and protons cancels the term of Coulomb scattering in the first moment equation. So, we get
%
\begin{align}
\pdv{v^{j}_{b}}{t} + H v_b^{j} + \frac{1}{a} \pdv{\Phi }{x^{j}}
=\frac{1}{\rho_{b}} \expval{c_{e \gamma } q^{j}}_{pp'q'q}
\,,
\end{align}
%
but by total momentum conservation we can switch the momentum \(q^{j} \) in the equation to a \(p^{j}\).
We Fourier transform, and multiply by \(\hat{k}^{j}\): then we get \(\vec{p} \cdot \hat{k} = p \mu \), where \( \mu = \cos \alpha \).

Two pages from Dodelson \cite[]{dodelsonModernCosmology2003}.

% In the Planckian, the only dependence is the time.

\end{document}


\documentclass[fleqn]{article}
\usepackage{amsmath}
\usepackage{setspace}
\usepackage{algorithmic}
\usepackage{vmargin}
%\setpapersize{USletter}
\setmarginsrb{1.0in}{1.0in}{1.0in}{1.0in}{0pt}{0mm}{0pt}{0mm}
\doublespacing
%..............................
% Alias commands
%..............................
\newcommand{\SUM}[1]    {\ensuremath{\sum \limits_{{#1}=1}^N }}
\newcommand{\bigS}[1]   {\ensuremath{S^{r \leftrightarrow m}_{#1}}  }
\newcommand{\B}[1]      {\biggr{#1}}
\newcommand{\U}         {{\vec{U}}}
\newcommand{\uo}        {\ensuremath{\vec{U}_o}}
\newcommand{\rhoM}      {\ensuremath{\rho^{o}_m}}
\newcommand{\rhoMFC}    {\ensuremath{\langle \rhoM \rangle}}
\newcommand{\delt}      {\ensuremath{\Delta{t}} }
\newcommand{\delx}      {\ensuremath{\Delta{x}} }
\newcommand{\dely}      {\ensuremath{\Delta{y}} }
\newcommand{\delz}      {\ensuremath{\Delta{z}} }
\newcommand{\f}         {\ensuremath{f^{\theta}_m} }
\newcommand{\sv}[1]     {\ensuremath{v^o_{m_{#1}} }}
\newcommand{\dpdrho}    {\ensuremath{ \frac{dP}{d\rhoM}} }
\newcommand{\velFC}     {\ensuremath{\vec{U_m}^{*^{f}} } }
\newcommand{\thetaUW}   {\ensuremath{\widehat{\theta_m}} }
\newcommand{\iter}      {\ensuremath{\text{ iter}}}
\newcommand{\massSrc}   {\ensuremath{ \SUM{m}  \frac{\dot{m}} {V \rhoM} } }
\newcommand{\termA}     {\ensuremath { \B{\langle} \frac{ \rho_m \U_m }{\rho_m} \B{\rangle} } }
\newcommand{\termB}     {\ensuremath{ \frac{ \delt }{\rhoMFC} \nabla^{f} } }
 %.....................................................
% This command is necessary so that the subscripts, some of which are
% in capital letters come out
%\DeclareMathSizes{10}{10}{6}{5}
%______________________________________________________________________
\begin{document}
\setlength{\abovedisplayskip}{0.1in}
\setlength{\mathindent}{0.0in}

\section{\underline{Overview of implicit ICE\hspace{ 0.5in}12/14/05}  }

\begin{algorithmic}
\STATE --Compute $ \velFC$ (add Exchange contribution, set boundary
             conditions on $\velFC$ )

\STATE --Form RHS $B = -$\text{Advection} $(\theta_m \velFC)$

\WHILE{ $| \text{outer residual} | <$ convergence criteria}

    \STATE --Compute $A$ matrix \\ \quad
      $A_r = \frac{\dely\delz\delt^2}{\delx} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{r}$ \quad
      $A_l = \frac{\dely\delz\delt^2}{\delx} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{l}$ \\ \quad
      $A_t = \frac{\delx\delz\delt^2}{\dely} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{t}$ \quad
      $A_b = \frac{\delx\delz\delt^2}{\dely} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{b}$ \\ \quad
      $A_f = \frac{\delx\dely\delt^2}{\delz} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{f}$ \quad
      $A_{bk} = \frac{\delx\dely\delt^2}{\delz} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{bk}$ \\ \quad
      $A_{p} =V\SUM{m} \theta_m \kappa_m +( A_r + A_l + A_t + A_b + A_f + A_ {bk})$

    \STATE --Solve for $\delta{P}$
    \STATE --Update pressure $\Delta{P} = \sum \limits_{n=1}^{iter} \delta{P}, \quad P = P^{Old} + \Delta{P}$, set boundary condition on $P$ and $\delta{P}$
    \STATE --Recompute $ \velFC$ (using $\delta {p}$ ), add Exchange contribution, set boundary
             conditions on $\velFC$, compute $ \rhoMFC $)

    \STATE --Form RHS $B = -V\SUM{m} \theta_m \kappa_m \Delta{P} - \SUM{m}$ V\text{Advection} $(\theta_m \velFC)$,\\ \quad
             where outer residual = $Max(RHS), \kappa_m = \frac{v^o_m}{c_m^2}$

\ENDWHILE

\end{algorithmic}
%______________________________________________________________________
\newpage
\section{\underline{Derivation of the matrix stencil \hspace{3in}}}
Step 1: \hspace{0.25in} Derive the implicit form of the pressure, starting with
change equation
\begin{equation}
     \label{GE}\frac{dP}{dt} =
     \frac{\massSrc -  \SUM{m} \nabla \cdot \thetaUW \velFC}
                      {\SUM{m} \frac{\theta_m}{\rhoM c_m^2} }
\end{equation}
%
and the face-centered velocity
\[
    \vec{U_m}^{*^{f}}
    = \termA - \termB{P^{\iter}} + \text{Exchange term}   + \delt\vec{g}
\]
%
Write eq.{\ref{GE}} in terms of $P^{\iter} = P^{\iter -1 } + \delta{P}$, $\Delta{P} = P^{\iter} - P^{old}$
\begin{equation}
\label{eq2}
\SUM{m} \theta_m \kappa_m (P^{\iter} -P^{old}) =
     \delt \massSrc
  -  \delt \nabla \cdot \thetaUW \B{(} \termA -\termB{P^{\iter}} +\text{Exch} + \delt\vec{g} \B{)}
\end{equation}
Substitite in for$P^{\iter} = P^{\iter -1 } + \delta{P}$
\[
  \SUM{m} \theta_m \kappa_m ((P^{\iter -1 } + \delta{P}) -P^{old}) =
    \delt \massSrc
  - \delt \nabla \cdot \thetaUW \B{(} \termA -\termB{(P^{\iter -1 } + \delta{P})}
  +\text{Exch} + \delt\vec{g} \B{)}
\]
Now factor out $\delta{P}$
\begin{align}
  \SUM{m} \theta_m \kappa_m \delta{P} -
  \delt^2 \nabla \cdot \SUM{m} \frac{\thetaUW}{\rhoMFC} \nabla \delta{P} \notag
  & =
     \SUM{m} \theta_m \kappa_m(P^{\iter} - P^{old} ) + \delt \massSrc \notag \\
  &- \delt \nabla \cdot \SUM{m} \thetaUW  \notag
  \overbrace {\B{(} \termA -\termB{P^{\iter}} +\text{Exch} + \delt\vec{g} \B{)} }^{\velFC} \notag
\end{align}
%
%
The ODE we have to solve is
\begin{equation}
\label{eq3}
  \SUM{m} \theta_m \kappa_m \delta{P} -
  \delt^2 \nabla \cdot \SUM{m} \frac{\thetaUW}{\rhoMFC} \nabla \delta{P}
  =
  -\SUM{m} \theta_m \kappa_m(P^{\iter} - P^{old} )
  + \delt \massSrc
  -  \delt \nabla \cdot \B{(} \SUM{m} \thetaUW \velFC \B{)} ^{\iter}
\end{equation}
%
%-------------------------------------------------------------------
\newpage
Step 2: \hspace{0.25in} Integrate eq. {\ref{eq3}} over a cell volume
\begin{align}
  \int_{V} \SUM{m} \theta_m \kappa_m \delta{P} dV
  - \delt^2 \int_{V} \nabla \cdot \SUM{m} \frac{\thetaUW}{\rhoMFC} \nabla \delta{P} dV
  =
  &- \int_{V} \SUM{m} \theta_m \kappa_m(P^{\iter} - P^{old} ) dV \notag\\
  &+ \int_{V} \delt \massSrc \notag \\
  &- \delt \int_{V} \nabla \cdot \B{(} \SUM{m} \thetaUW \velFC \B{)} ^{\iter} dV \notag
\end{align}
%
%
Invoke the divergence theorem
    $\int_V \nabla \cdot \vec{f} dV = \int_S \vec{f} \cdot d\vec{s}$
to convert the volume to surface integrals.
%
\begin{align}
  V \SUM{m} \theta_m \kappa_m \delta{P}
  - \delt^2 \int_{S}  \SUM{m} \frac{\thetaUW}{\rhoMFC} \nabla \delta{P} \cdot d\vec{s}
  =
  &-V \SUM{m} \theta_m \kappa_m(P^{\iter} - P^{old} )
  + \delt  \SUM{m}  \frac{\dot{m}} {\rhoM} \notag \\
  - \delt \int_{S}  \B{(} \SUM{m} \thetaUW \velFC \B{)} ^{\iter} \cdot d\vec{S} \notag
\end{align}
%
%
Step 3: \hspace{0.25in} Write out the contributions for each face of the computational cell
\begin{align}
\label{eq4}
\delt^2 \int_{S}  \SUM{m} \frac{\thetaUW}{\rhoMFC} \nabla \delta{P} \cdot d\vec{s}
 = \delt^2 \B{[}
  &\SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_R \dely \delz \frac {( \delta{P_R}  - \delta{P} )}{\delx} \quad
-  \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_L \dely \delz \frac {( \delta{P}    - \delta{P_L} )}{\delx}   \notag \\
+ &\SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_T \delx \delz \frac {( \delta{P_T}  - \delta{P} )}{\dely} \quad
-  \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_B \delx \delz \frac {( \delta{P}    - \delta{P_B} )}{\dely} \notag\\
+ &\SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_F \delx \dely \frac {( \delta{P_F}  - \delta{P} )}{\delz} \quad
-  \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{BK} \delx \dely \frac {( \delta{P} - \delta{P_{BK}} )}{\delz}
\B{]}
\end{align}
%
%
Form the stencil of Matrix A from eq. {\ref{eq4}}\\
  $A_{R/L}  = \frac{\dely \delz \delt^2}{\delx} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{R/L}  \quad
   A_{T/B}  = \frac{\delx \delz \delt^2}{\dely} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{T/B}  \quad
   A_{F/BK} = \frac{\dely \delx \delt^2}{\delz} \SUM{m} \frac{\thetaUW}{\rhoMFC} \B{|}_{F/BK}$ \\
%
%
\begin{align}
V \SUM{m} \theta_m \kappa_m \delta{P}
- \B{[}
   &A_R(\delta{P_R} - \delta{P} ) - A_L(\delta{P} - \delta{P_L} )  \\
+  &A_T(\delta{P_T} - \delta{P})  - A_B(\delta{P} - \delta{P_B} ) \notag \\
   &A_F(\delta{P_F} - \delta{P} ) - A_{BK}(\delta{P} - \delta{P_{BK}} )
\B{]}
=
-V \SUM{m} \theta_m \kappa_m(P^{\iter} - P^{old} )
+ \delt  \SUM{m}  \frac{\dot{m}} {\rhoM}
- \delt \text{Advection}  \B{(} \SUM{m} \thetaUW \velFC \B{)} ^{\iter} \notag
\end{align}
%
Factor out $\delta{P}$ the equation (In impICE.cc we don't normalize
by $V$ to make it non-dimensional.  We need to keep it dimensional for implicit AMRICE)\\
%
$\B{(} V\SUM{m} \theta_m \kappa_m + \B{[} A_R +A_L+ A_T + A_B + A_F + A_{BK} \B{]}  \B{)}\delta{p}
-
\B{[} A_R \delta{P_R} + A_L \delta{P_L} + A_T \delta{P_T} + A_B \delta{P_B}
+ A_F \delta{P_F} + A_{BK} \delta{P_{BK}} \B{]}
= \\
- V \SUM{m} \theta_m \kappa_m(P^{\iter} - P^{old} )
+ \delt  \SUM{m}  \frac{\dot{m}} {\rhoM}
- \delt V \text{Advection}  \B{(} \SUM{m} \thetaUW \velFC \B{)} ^{\iter}
$
%-------------------------------------------------------------
\newpage
\section{Nomenclature}
\begin{tabular}{lll}
\\
\underline{Variable} & \underline{Dimensions} & \underline{Description }\\
$\Delta{t}$   &  $[t]$      &    Timestep\\
$\rho$        &  $[M/L^3]$  &    Density of material\\
$\rhoMFC$     &  $[M/L^3]$  &    Microscopic density at the face center\\
$\vec{U}$     &  $[L/t]$    &    Velocity vector\\
$V$           &  $[L^3]$    &    Cell Volume\\
$P$           &  $[M/L^2]$ &     Pressure\\
$\theta_m$    &             &    volume fraction of material $m$\\
$\thetaUW$    &             &    upwinded volume fraction\\
\\
\textsf{\underline{Superscript}}\\
$old$         &             &    Previous timestep\\
$n$           &             &    Current timestep\\
$iter$        &             &    outer iteration number \\

\textsf{\underline{Subscript}}\\
$m$           &             &    Material index\\
$o$           &             &    Pure material\\
$l,r,t,b,f,bk $       &     &    Left, right, top and bottom, front and back cell faces\\
\textsf{\underline{Misc}}\\
$\langle\rangle$ &          &    value averaged to the face\\
\end{tabular}
\end{document}