\chapter{Relational E-matching}

\begin{figure}
    \centering
    % \caption{The main algorithm}\label{alg:main}
    \begin{algorithmic}[1]
    \Procedure{RelationalEMatching}{$E,\textit{ps}$}
    \State $\textit{I}\gets \textsc{EgraphToDatabase}(E)$
    \State \Return $\{\textsc{EvalCQ}(q, I)\ |\textbf{ for } p\in \textit{ps}, q\gets \textsc{PatternToCQs}(p)\}$
    \EndProcedure
    \end{algorithmic}
    \caption{The relational \ematching algorithm}
    \label{fig:main}
\end{figure}

\begin{figure}
    \centering
    \begin{align*}
        \textsc{EgraphToDatabase}(E) &= \{\textsc{EnodeToAtom}(n)\ |\text{ for \enode\ } n\in E\}\\
        \textsc{EnodeToAtom}(f(c_1,\ldots,c_n)) &= R_f(\textit{lookup}(f(c_1,\ldots,c_n)), c_1,\ldots, c_n)
    \end{align*}
    \caption{Transforming \egraphs to relational database}
    \label{fig:egraph_to_db}
\end{figure}

\def\aux{\textsc{Aux}}
\def\compile{\textsc{Compile}}
\begin{figure}
\begin{align*}
  \compile(p) &= Q(\textit{root}, v_1,\ldots,v_n) \leftarrow A\\
  &\quad \begin{aligned}
  \text{ {where} $v_1\dots v_n$ are variables in $p$}\\
  \text{{ and} $\aux(p)=\textit{root}\sim A$}\\
  \end{aligned}\\
  \aux(f(p_1,\ldots, p_n)) &= v \sim R_f(v, v_1, \ldots, v_n), A_1, \ldots, A_n \\
  & \text{\quad { where} $v$ is fresh and }\aux(p_i) = v_i \sim A_i \\
  \aux(x) &= x \sim \emptyset \text{\qquad { where} $x$ is a pattern variable}\\
\end{align*}
\caption{Compiling a pattern to a conjunctive query.}\label{alg:compile}
\end{figure}
In this chapter, we introduce our relational \ematching algorithm, shown in \autoref{fig:main}.
The algorithm takes an e-graph $E$ and a set of patterns $\textit{ps}$.
It first transforms the \egraph to its relational representation, denoted as $I$.
Next, for each pattern, it compiles the pattern to a conjunctive query $q$, and executes this conjunctive query over $I$.
This is particularly suited for scenarios like equality saturation.
In such scenarios, the program optimizer switches between e-matching and batched rewrite applications in each iteration to amortize the run-time required for maintaining the congruence invariant, a technique known as ``rebuilding'' \citep{egg}.
In such scenarios, the run time for materializing the \egraph can be neglected compared to the \ematching workload.
In other scenarios, where the \egraph is frequently updated and \ematching is performed interactively, it is possible to use a totally relational representation to avoid the cost of frequent materialization of the relational database and use standard update operations from a SQL-like language\footnote{We actually implemented a prototype \egraph purely on top of an in-memory relational database that does not use graph data structure nor union-finds.}.
However, we suspect this would incur a larger constant factor to the run time compared to the graph representation of \egraph.
In general, we left the problem of interactive \ematching, where \ematching is performed on a frequently updated \egraph, as future work (See Chapter~\ref{sec:future}).

\section{Reducing e-graphs to relations}

The first step of relational e-matching is to transform the \egraph to a relational database.
An \egraph $E$ representing a congruence relation over $T(\Sigma,\emptyset)$ will be mapped to a relational database over \id, the set of \eclass ids.
For every $n$-ary symbol $f$ in $\Sigma$, We map it to an $n+1$-ary symbol $R_f$ in schema $S_\id$.
We refer to fields of $R_f$ as $\eclassid,\child_1,\ldots,\child_n$ respectively, and they denote the \eclass this \enode belongs to and the children \eclasses of this \enode respectively.
For every \enode in the \egraph, we map it to an atom in the relational database.
The algorithm is presented in \autoref{fig:egraph_to_db}.

Take the \egraph in \autoref{fig:egraph} for example. There are 8 \eclasses in total. The $g$-application \enode in \eclass $c_1$ translates to atom $R_g(1, 2)$, where $1$ is the id of the \eclass that contains it and $2$ is the id of its child \eclass. Similarly, the $f$-application \enode in \eclass $c_2$ translates to the atom $R_f(2, 4, 4)$, where $2$ is the id of the \eclass that contains it and $4,4$ are the ids of its child \eclasses. \autoref{fig:egraph_to_db} only shows the relation representing $f$ and $g$. The relations representing constants $a,b,c$, which are all relations with $\leq 1$ atom, are omitted for brevity.

\section{Reducing \ematching patterns to conjunctive queries}
\label{sec:pattern_to_cq}

We reduce every \ematching to a conjunctive query so that it can be run on the relational representation of \egraphs.
We use the algorithm in \autoref{alg:compile} to ``unnest'' a pattern to a conjunctive query. The \aux{} function returns a variable and a conjunctive query atom list.
Particularly, for non-variable pattern $f(p_1,\ldots p_n)$, \aux{} produces a fresh variable $v$ and a concatenation of $R_f(v, v_1,\ldots, v_n)$ and atoms from $A_i$, where $v_i\sim A_i$ is the result of calling $\aux(p_i)$. For variable pattern $x$, \aux{} simply returns $x$ and an empty list.
Given a pattern $p$, the \compile{} function returns a conjunctive query with body atoms from $\aux(p)$ and the head atom consisting of the root variable and variables in $p$. The compiled conjunctive query and the original \ematching query are equivalent because there is an one-to-one correspondence between the output of them. Specifically, each output atom of the form $Q(c_\textit{root}, c_{x_1},\ldots c_{x_k})$ denotes an \ematching output $\left(c_\textit{root}, \{x_1\mapsto c_{x_1},\ldots,x_k\mapsto c_{x_k} \}\right)$.

With this algorithm, the example pattern $f(\alpha, g(\alpha))$
 is compiled to the following conjunctive query $$Q(\textit{root},\alpha)\,\textit{:-}\,R_f(\textit{root},\alpha,x),R_g(x,\alpha).$$
Compared to the original \ematching pattern, this flattened representation enables the pattern search to utilize both the structural and the equality constraints. For example, most of the database optimizers will synthesize query plans that build and lookup indices on both join variables (i.e., $x$ and $\alpha$), which runs in linear time. In contrast,  backtracking-based \ematching takes quadratic time. Backtracking-based \ematching can be seen as a hash join that only builds and look-ups a single variable (i.e., $x$), and filter the outputs using the equality predicates on $\alpha$. In other words, existing \ematching algorithms will consider all $f(\alpha, g(\beta))$ regardless whether $\alpha$ is congruent to $\beta$, while the generated conjunctive query gives the query optimizer the freedom to synthesize query plans that will consider only atoms where $\alpha\cong\beta$.

\section{Answering conjunctive queries with generic join}

Finally, we consider the problem of efficiently solving the compiled conjunctive
queries. We propose to use the generic join algorithm to solve the generated
conjunctive queries. Although traditional query plans, which are based on two
way joins such as hash joins and merge-sort joins, are
extensively used in industrial relational database engine, they may suffer from
certain conjunctive queries \ematching~generates. For example, consider the
pattern $f(g(\alpha),g(\alpha))$. The synthesized conjunctive query
is $$Q(\alpha)\gets R_f(\textit{root}, x, y), R_g(x, \alpha), R_g(y, \alpha),$$
which is a cyclic conjunctive query. Two-way join plans suffer on these cyclic
queries because they are unable to leverage all the constraints in the
conjunctive query to prune the space during enumeration. Consequently, two-way
join plans will enumerate terms that do not satisfy the constraint and prune
them away \textit{a posteriori}. In contrast, the generic join algorithm will
only consider a concrete term when the term satisfies all the constraints.
Moreover, generic joins has comparable
performance on acyclic queries with two-way join plans. These properties make
generic join our ideal solver for conjunctive queries generated from \ematching
patterns.


\section{Personalized Medicine}

A medium-term goal of medicine is ``personalized medicine\index{personalized medicine}'',
whose goal is to provide custom-tailored health care on an individual
basis. For example, a standard treatment for breast cancer is chemotherapy,
but not all patients profit from this treatment. The event that a
patient has no sign of breast cancer after reductive surgery followed
by chemotherapy is called \emph{pathologic complete response}\index{pathologic complete response}\index{pCR},
and the opposite event that the patient still has cancerous tissue
after this procedure is called \emph{residual disease}\index{residual disease}\index{RD}.

Suppose there were a predictor that could tell the physicist how likely
a patient is to benefit from chemotherapy. If the prediction for a
certain patient was such that complete response to chemotherapy was
unlikely, chemotherapy could be replaced by another therapy.

The goal of this work is to contribute to such a predictor. The input
to the predictor is the molecular expression data, i.e. measures of
the number of RNA copies of specific genes present in the cancer tissue.
These gene expression measurements are ususally measured using microarrays
or next generation sequencing. An artificial neural network then processes
this data. The prediction is output by the network in the form of
a number between 0 and 1. Here, 0 means the patient is predicted with
absolute certainty to have residual disease, 1 means the patient is
predicted with absolute certainty to have pathologic complete response,
and a number in-between is interpreted as the probability for pathologic
complete response.

The study of neural networks in biology prompted the development of
artificial neural networks as models of biological neural networks.
After an introduction to biological and artificial neural networks
we will give an overview of the relevant topics of machine learning
and then introduce the own work done in this manuscript.

\section{Biological and Artificial Neural Networks}

Artificial neural networks\index{artificial neural networks} are
mathematical constructs, designed to imitate the signal processing
capabilities of real neurons, found in nearly all animals. Neurons
can be connected to form complex neural networks. Like their biological
counterparts, artificial neural networks consist of simpler building
blocks, the neurons.

\subsection{Neurons As Basic Signal Processing Units}

The biological neurons are defined (according to the neuron doctrine
\cite{BullockDouglas2005}) as the smallest units whose state change
may be called signal processing, so they are the basic signal processing
units. They have multiple inputs at dendrites, and multiple outputs
at axon terminals \cite{ByrneDafny1997}. Figure \ref{fig:Schematic-image-of-biological-neurons}
gives a schematic overview of these elements.

\begin{figure}
\begin{centering}
\includegraphics[width=0.8\columnwidth]{images/neurons}
\par\end{centering}
\caption[Schematic image of three biological neurons.]{\label{fig:Schematic-image-of-biological-neurons}Schematic image
of three biological neurons. A: neuron body B: nucleus C: dendrite
D: synapse E: axon projecting from a distant neuron.}
\end{figure}

In most real neurons, the signal transmission and processing is facilitated
by alternating small electric (action) potentials (along the axons)
and chemical transmissions (at chemical synapses between axon and
dendrite). The electric potential is transmitted along the dendrites
of a neuron, and flows to the axon of the neuron, where it can lead
to the release of neurotransmitters stored in the axon terminals into
the synaptic cleft. The released neurotransmitters are detected by
receptors and cause ion channels in the adjacent dendrites of other
neurons to open, which changes their membrane potential. See figure
\ref{fig:Schema-of-chemical-synapse} for a depiction of axon, synaptic
cleft, and dendrite.

\begin{figure}[t]
\begin{centering}
\includegraphics[width=0.8\columnwidth]{images/synapse}
\par\end{centering}
\caption[Schema of a chemical synapse.]{\label{fig:Schema-of-chemical-synapse}Schema of a chemical synapse.
The signal is transmitted from the axon terminal (left) to the dendrite
(right). Grey: membranes of neurons. Green and blue: ion transporters
maintain intracellular ion concentrations. Red: neurotransmitter is
stored inside the cell in vesicles and emitted into the synaptic cleft
upon an electric potential arriving at the axon terminal. Purple:
receptors signal to the inside of the cell the absence or presence
of neurotransmitter on the outside of the cell.}
\end{figure}

\subsubsection{Action Potentials, Their Propagation, and Chemical Synapses }

The action potentials are realized by cells in the form of different
ion concentrations inside and outside the cell. These ion gradients
are maintained in the resting state by the $Na^{+}/K^{+}$-ATPases
that pump 3 $Na^{+}$ ions out of and 2 $K^{+}$ ions into the cell
for every ATP molecule \cite{LodishZipursky2000}. Because ions are
charged, there is an electric potential between the outside and inside
of the cell. The resting potential is between $-80$mV and $-40$mV,
depending on the type of neuron. The electric potential becoming more
positive is called depolarization, and the opposite hyperpolarization.

The propagation of the action potentials along dendrites is realized
by the opening and closing of ion channels. Once depolarization of
an adjacent region of a neuron causes the electric potential between
the inside and outside of a $Na^{+}$ ion channel to reach a critical
value, the ion channel opens, causing further depolarization in adjacent
regions of the neuron. This positive feedback loop continues until
all $Na^{+}$ channels are open. At the peak of depolarization, $K^{+}$
ion channels open, causing hyperpolarization, and the potential returns
to the resting  potential. This makes the action potential travel
along the neuron. Once it has reached an axon terminal, it causes
neurotransmitter release.

Neurotransmitters binding to receptors present on the outside of
the neuron's membrane cause ion channels to open, and the ions flow
into or out of the cell to achieve equilibrium of ion concentration.
The type of ion channel being opened upon binding of a neurotransmitter
can cause either depolarization or hyperpolarization of the dendrite,
depending on the charge of the ion, and whether the resting concentration
of the ion is higher intracellular or extracellular. If a critical
threshold of depolarization is reached, the $Na^{+}$ ion channels
will open, and an action potential ``spike'' is generated as described
above.

\subsubsection{Encoding of Information in Action Potentials}

The presence of a critical threshold suggests that it is not the ``analog''
electric potential, but the ``digital'' spike that carries the information
from one neuron to the next. For example, the strength muscles are
innervated with, is encoded in the number of action potentials per
time delivered by the muscle neuron to the muscle fiber. However,
some neurons involved in perception directly transmit information
in the fluctuations of neurotransmitter released. This analog mode
of transmission allows more information to be transmitted per time.
Sub\-threshold emission of neurotransmitter also seems to modulate
subsequent action potentials, allowing for a mixture of analog and
digital information transmission \cite{DebanneRama2013}.

Examples for neural networks that have been partly decoded are the
eye (visual system) and the nose (olfactory system).

\subsection{Examples of Biological Neural Networks}

\subsubsection{The Eye, a Visual System}

In the eye, specialized cells called rods and cones detect light\cite{Biochemistry2002,Kolb2003}.
Rods are more sensitive to dim light, while the three types of cones
react to bright light only but can differentiate between colors. Both
rods and cones release the neurotransmitter glutamate continuously
into the synaptic cleft, but when hit by light, suspend this emission
for the duration of the light. This is implemented by the cell by
a long pathway.

Specifically, light elicits a transformation of cis-rhodopsin to trans-rhodopsin,
which presents on its surface a G protein binding site. The G protein
transducin binds to the activated rhodopsin, and in this process GDP
acquires a phosphate group to form GTP. The $\alpha$-subunit of transducin
activates a cGMP phosphodiesterase, which in turn hydrolyzes cGMP
to GMP. The reduction in the concentration of cGMP causes cGMP-gated
ion channels to close. This in turn hyperpolarizes the photosensitive
cell, causing glutamate to be released into the synaptic cleft at
a slower rate. This long pathway between cis-rhodopsin and glutamate
release inhibition facilitates an amplification of the signal at every
step, which allows rod cells to signal a spike in response to it being
hit by a single photon.

The area that elicits a response in the cell upon being illuminated
is called the \emph{receptive field}\index{receptive field}, and
is just as large as the top of the photoreceptor for rods and cones.
The released glutamate binds to receptors present on the outside of
bipolar cells, and, depending on the type of bipolar cell, cause either
an action potential to be generated when the photoreceptor is lit
and the surrounding area is dark (\emph{ON }bipolar cell), or when
the photoreceptor is dark against a bright background (\emph{OFF }bipolar
cell). Another type of cell, the horizontal cell integrates signals
from surrounding cone cells, and feed their signal back to the cones,
or directly to bipolar cells. This enhances contrast. The signal from
several bipolar cells is fed into a ganglion cell, which therefore
has a larger receptive field than its connected bipolar cells. ON
bipolar cells only excite ON ganglion cells, and OFF bipolar cells
excite only OFF ganglion cells. Finally, in primates, there are more
than a million nerve fibers from ganglion cells to the visual cortex
of the brain. Altogether, the basic cell types are, depending on the
species, 1 to 4 types of horizontal cells, 11 types of bipolar cells,
22 to 30 types of amacrine cells, and 20 types of ganglion cells.
Among those cell types' known functions are integration of a large
number of rods to provide sight in little light, brightness-dependent
size regulation of the receptive field of amacrine cells, and an additional
photoreceptor distinct from rods and cones\cite{Kolb2003}.

\subsubsection{Odor Sensing in the Olfactory System}

The olfactory system of mammals and insects contains neurons that
detect odor molecules, called glomeruli\cite{ZhangSharpee2016}. In
humans, there are about $500$ different types of glomeruli {[}1{]},
but it is hypothesized that a human can perceive around $10,000$
different odors. Each ``atomic'' odor consisting of a few ($<100$)
molecular species excites one or more glomeruli, and the compression
requires each glomerulus to signal the presence of one or more than
one odor. The excitation pattern of multiple glomeruli must be resolved
in the olfactory neuronal system so that a low-dimensional vector
of ($\approx500$) glomeruli activations is decompressed to a high-dimensional
representation of ($\approx10,000$) odors in the brain.

Each glomerulus is connected to one or more Kenyon Cells in insects.
It is assumed that the activation of a Kenyon cell signals to the
insect nervous system the presence of one specific odor. (In the mammalian
brain, a single odor is represented by neurons in the olfactory cortex.)
Experiments show that the circuit connecting glomeruli to Kenyon cells
is feed-forward\index{feed-forward neural network} only, i.e. without
recurrent connections (loops). The structure of a feed-forward compressed
sensing circuitry is of interest, because standard compressed sensing
circuits are recurrent dynamic systems that converge to one of their
attractor states. In addition to quick decoding of odors, experimental
evidence shows that the biological compressed sensing circuitry is
robust to noise, i.e. to spurious neuronal spikes in glomeruli, or
noise due to experimental inhibition\footnote{Also experimental modifications of glomeruli have been made so that
half of all glomeruli always express only one type of receptor.} of glomeruli.

The theoretical work of \cite{ZhangSharpee2016} proposed that a feed-forward
architecture could facilitate odor decoding simply by implementing
a logical AND. They suggest that in the neuronal AND-circuit a specific
odor's Kenyon cell is activated when at least (for example) $80\%$
of the glomeruli with receptors to this odor are active. On a cellular
level, this could be realized by connecting the glomeruli associated
with an odor with the odor's Kenyon cell, and a threshold at the Kenyon
cell's input.

A prediction of \cite{ZhangSharpee2016} is that the number of glomeruli
activated by a single odor should be close to the number of glomeruli
that are connected to a Kenyon cell. They postulated further that
the validity of their feed-forward model can be tested by measuring
the odor sparseness\footnote{Odor sparseness is the average number of different molecular species
in an odor.} in the environment of an animal species and comparing it to its average
number of connections from glomeruli to Kenyon cells.

For example, in \emph{Drosophila}, about $9\%$ of the glomeruli are
excited by an odorant, and the connectivity rate between glomeruli
and Kenyon Cells is between $6.5\%$ and $12.5\%$. In the locust,
a projection neuron (the equivalent to a glomerulus) is activated
by half of the odorants and the connectivity rate is about $50\%$.
This is in agreement with the proposed model.

The model also predicts that species with sparse connectivity have
better odor perception of complex odor mixtures. On the other hand,
species with dense connectivity should have better olfactory performance
in detecting simple odor mixtures.

\subsection{Artificial Neurons as Simple Models of Biological Neurons\label{subsec:Artificial-Neurons-as-Simple-Models-of-Biological-Neurons}}

The machinery facilitating propagation and transmission of information
in and between biological neurons is highly simplified in artificial
neurons\index{artificial neuron signal processing}. Signal processing
of a real neuron is modelled in an artificial neuron as a mathematical
function that has multiple input variables, computes a value according
to the function formula and its parameters and outputs its computed
value to multiple neurons, which use it as an input variable. Herein,
the processes of neurotransmitter release, de- and hyperpolarization,
and propagation of the action potential are abstracted away into discrete
time steps.

Each artificial neuron's function is evaluated once per time step.
Often, the \emph{sigmoid}\index{sigmoid function}\emph{ }function
is used to describe the output of an artificial neuron, the so-called
\emph{activation}\index{activation of an artificial neuron}

\begin{equation}
o_{i}=\sigma(v_{i})=\frac{1}{1+\exp(-v_{i})},\label{eq:sigmoid-function}
\end{equation}
where $v_{i}\in\mathbb{R}$ is the accumulated input to neuron $i$,
and $o_{i}\in[0;1]$ is the activation of neuron $i$. See the left
panel of figure \ref{fig:sigmoid-function} for a plot of the sigmoid
function.
\begin{figure}
\begin{centering}
\includegraphics[width=0.45\columnwidth]{images/plot-sigmoid}\hfill{}\includegraphics[width=0.45\columnwidth]{images/neuronal-network-example}
\par\end{centering}
\caption{\label{fig:sigmoid-function}Left: The sigmoid function $\sigma$.
Right: Schema of the computational steps in an artificial neural feed-forward
network\index{feed-forward neural network} from input layer to output
layer. The ``+'' nodes accumulate their input values, and the ``$\sigma$''
nodes compute the output of a neuron, to be used as input for the
next layer.}
\end{figure}

The effect of an incoming axon onto a neuron, that is, the different
types of receptors that can be present on the outside of a real dendrite,
and the effected de- or hyperpolarization of the dendrite are abstracted
away by using real-numbered weights. Weights are parameters to the
mathematical function describing the conversion of outputs of neurons
to the single input of the next connected neuron. Usually the input
$v_{i}$ of neuron $i$ is computed from the outputs of its connected
neurons $\mathbf{c_{i}}$ as in
\begin{equation}
v_{i}=b_{i}+\sum_{j\in\mathbf{c_{i}}}o_{j}w_{ij},\label{eq:input-to-a-neuron}
\end{equation}
where $b_{i}\in\mbox{\ensuremath{\mathbb{R}}}$ is the so-called \emph{bias}\index{bias of an artificial neuron}\emph{
}of neuron $i$, $\mathbf{c_{i}}$ is the vector of indices of its
in-going connected neurons, $o_{j}\in\mathbb{R}$ is the activation
of the connected neuron $j$, and $w_{ij}\in\mathbb{R}$ is the weight
of the connection going out of neuron $j$ and into neuron $i$.

In an neural network the neurons are often arranged in layers. See
the right panel of figure \ref{fig:sigmoid-function} for an example
of the structure of an artificial neural network.

\subsection{Learning in Biological Neural Networks}

Nervous systems do not only process signals, but they also learn,
that means that they adapt their signal processing over time. One
reason for this is an organism's need for a change in behavior, as
response to a changing environment.

In biological neuronal systems, this is possible by altering existing
synapses (for example by exchanging the receptors on the surface of
dendrites), or by creating and abandoning existing synapses (i.e.
connecting the axon terminals of a neuron to different neurons). There
are several known cellular mechanisms for that, among them LTP, LTD,
and PTP\cite{BermudezFederico2007}. Strengthening of the synaptic
link (that occurs within minutes and remains after hours and up to
weeks in the hippocampus of mammals) is called long-term potentiation
(LTP), while its weakening is called long-term depression (LTD). LTP
is induced by associativity of connected neurons, that means, when
a neuron contributes to the depolarization in a directly connected
neuron, the efficiency of that connection will be strengthened. The
molecular mechanisms responsible for this phenomenon are not yet completely
understood. It is known that they differ between brain regions, and
also between types of synapses in the same brain region. The cellular
mechanisms controlling these processes, and their interplay in larger
neuron ensembles are a field of active research \cite{BermudezFederico2007}.

An unproven hypothesis is that learning is local\index{locality of Hebbian learning rule},
which means that changes at a synapse only depend on the directly
connected neurons, but not on other distantly-connected neurons. This
type of local learning is called \emph{Hebbian learning}\index{Hebbian learning}.

The learning in biological neuronal systems happens seemingly automatically,
for example, migratory birds learn and remember travel paths around
the globe, without an apparent teacher. Ultimately the goals of learning
are determined by an interplay of evolution and the environment.

\subsection{Back-propagation for Training Artificial Neural Networks\label{subsec:Back-propagation}}

The nearest analog to learning in biological neural networks is \emph{training
}artificial neural networks\index{training of artificial neural networks}.
Here, the goal is explicitly set by humans by providing training data
sets. One training sample consists of a vector of real numbers called
\emph{input patterns}\index{input pattern} and a corresponding vector
of real numbers of desired \emph{output patterns}\index{output pattern},
also called \emph{labels}\index{label}. For every input pattern in
the training data set an output pattern is defined that the learner
should compute from the input pattern. Learning is hereby facilitated
by changing the parameters of the artificial neural network.

Back-propagation is a supervised training procedure for artificial
neural networks \cite{RumelhartWilliams1988}. It learns from labeled
training samples. The parameters of the network, the weights and biases,
are adapted using \emph{gradient descent}\index{gradient descent}.
The basic idea is to set the neurons in the input layer to the input
pattern, compute the activations of neurons in the network, compute
the total error observed in the output layer using the difference
between actual and desired output, then determine how much each neuron
was ``responsible'' for the total error in the network, and finally
use it to adapt the weights and biases. This procedure is then repeated
until the network is fully trained.

Let the supervised training patterns be indexed by $p$, $x_{i,p}$
the activation of a neuron $i$ in the input layer for training pattern
$p$, and $y_{k,p}$ the desired activation of neuron $k$ in the
output layer for this training pattern.

\paragraph{Forward Pass}

The supervised training procedure first performs the \emph{forward
pass}\index{forward pass}: it sets activations $o_{i}$ in the input
layer according to the input pattern $x_{i,p}$ to be learned, computes
the activations $o_{j}$ of the hidden layers and the output layer
$o_{k}$.

(In the following, index $k$ is used for a neuron in the output layer,
and indices $j$ and $i$ for a neuron in a hidden layer or the output
layer. If the network has only one hidden layer, then index $i$ refers
to a neuron in the input layer.)

Each input neuron's output $o_{i}$ is set to a training input:
\begin{equation}
o_{i}=x_{i,p}.
\end{equation}
The input $v_{j}$ to a hidden or output neuron is computed from the
sum of the connected neurons' outputs in the layer above (see section
\ref{subsec:Artificial-Neurons-as-Simple-Models-of-Biological-Neurons}
):
\begin{equation}
v_{j}=b_{j}+\sum_{i\in\mathbf{c_{j}}}o_{i}w_{ji},
\end{equation}
 where $b_{j}$ is the bias, $o_{i}$ is the output of a neuron in
the layer above, and $w_{ji}$ is the weight of the connection from
neuron $i$ to neuron $j$. The input $v_{j}$ is then scaled by the
sigmoid function to produce a neuron's output $o_{j}$:

\begin{equation}
o_{j}=\sigma(v_{j})=\frac{1}{1+\exp(-v_{j})}.\label{eq:sigmoid-function-2}
\end{equation}

\paragraph{Backward Pass for the Output Layer}

While in the forward pass the information flowed from input to output
layer, in the \emph{backward pass}\index{backward pass}, the information
flows from output to input layer, adjusting the weights and biases
on the way. See figure \ref{fig:Data-flow-in-back-propagation} for
an illustration of the two data flow directions.

\begin{figure}
\begin{centering}
\includegraphics[width=0.34\columnwidth]{images/introduction-backpropagation-data-flow}
\par\end{centering}
\caption[Data flow in training a neural network using back-propagation.]{\label{fig:Data-flow-in-back-propagation}Data flow in training a
neural network with three layers using back-propagation. The input
layer is at the top, the hidden layer in the middle, and the output
layer at the bottom. The black arrows denote information flow during
the forward pass. Each black arrow is associated with a weight from
tail to head. The grey arrows denote information flow during the backward
pass, which propagates errors in the reverse direction.}
\end{figure}
The training procedure computes the total error $E_{total}$ of the
network, which is defined as the squared sum of differences between
actual output $o_{k}$ and desired output $y_{k}$ over all training
patterns:
\begin{eqnarray}
E_{total} & = & \sum_{p}\sum_{k}\frac{1}{2}(o_{k,p}-y_{k,p})^{2}\\
 & = & \sum_{p}E,\nonumber
\end{eqnarray}
 where $E$ is the error of one training pattern. We further split
$E$ into a sum of errors of individual neurons $E_{k}$:
\begin{eqnarray}
E & = & \sum_{k}\frac{1}{2}(o_{k}-y_{k})^{2}\\
 & = & \sum_{k}E_{k},\nonumber
\end{eqnarray}
where $E_{k}=\frac{1}{2}(o_{k}-y_{k})^{2}$. To compute the contribution
of weight $w_{kj}$ to the error, the error $E$ is then differentiated
with respect to a weight $w_{kj}$ for a connection from neuron $j$
in the last hidden layer to neuron $k$ in the output layer:
\begin{eqnarray}
\frac{\partial E}{\partial w_{kj}} & = & \frac{\partial E}{\partial o_{k}}\cdot\frac{\partial o_{k}}{\partial v_{k}}\cdot\frac{\partial v_{k}}{\partial w_{kj}}\nonumber \\
 & = & \frac{\partial\sum_{k}E_{k}}{\partial o_{k}}\cdot\frac{\partial o_{k}}{\partial v_{k}}\cdot\frac{\partial v_{k}}{\partial w_{kj}}\nonumber \\
 & = & \frac{\partial E_{k}}{\partial o_{k}}\cdot\frac{\partial o_{k}}{\partial v_{k}}\cdot\frac{\partial v_{k}}{\partial w_{kj}}\label{eq:backpropagation-error-wrt-weight-output-layer}\\
 & = & (o_{k}-y_{k})\cdot o_{k}(1-o_{k})\cdot o_{j},\nonumber
\end{eqnarray}
 which uses that the derivative of the sigmoid function $o_{k}=\frac{1}{1+\exp(-v_{k})}$
is $\frac{\partial o_{k}}{\partial v_{k}}=o_{k}(1-o_{k})$. The derivative
of the error with respect to $b_{k}$ is
\begin{equation}
\frac{\partial E}{\partial b_{k}}=\frac{\partial E}{\partial o_{k}}\cdot\frac{\partial o_{k}}{\partial v_{k}}\cdot\frac{\partial v_{k}}{\partial b_{k}}=(o_{k}-y_{k})\cdot o_{k}(1-o_{k})\cdot1.
\end{equation}

\paragraph{Backward Pass for the Other Layers}

To perform gradient descent, we also need to update the weights and
biases for the remaining connections between hidden layers, and from
the input layer to the first hidden layer. The derivative of the error
with respect to the weight $w_{ji}$ of the connection from neuron
$i$ in a layer to neuron $j$ in the layer below (for example, from
neuron $i$ in the second last hidden layer to neuron $j$ in the
last hidden layer) is
\begin{eqnarray}
\frac{\partial E}{\partial w_{ji}} & = & \frac{\partial E}{\partial o_{j}}\cdot\frac{\partial o_{j}}{\partial v_{j}}\cdot\frac{\partial v_{j}}{\partial w_{ji}}\label{eq:backpropagation-error-wrt-weight-other-layers}\\
 & = & \frac{\partial E}{\partial o_{j}}\cdot o_{j}(1-o_{j})\cdot o_{i},\nonumber
\end{eqnarray}
 where
\begin{eqnarray}
\frac{\partial E}{\partial o_{j}} & = & \frac{\partial}{\partial o_{j}}\sum_{k}E_{k}\nonumber \\
 & = & \sum_{k}\frac{\partial}{\partial o_{j}}E_{k}\nonumber \\
 & = & \sum_{k}\frac{\partial E_{k}}{\partial o_{k}}\frac{\partial o_{k}}{\partial v_{k}}\cdot\frac{\partial v_{k}}{\partial o_{j}}\nonumber \\
 & = & \sum_{k}\frac{\partial E_{k}}{\partial o_{k}}\frac{\partial o_{k}}{\partial v_{k}}\cdot w_{kj},\label{eq:backpropagation-error-wrt-output-other-layers}
\end{eqnarray}
and neuron $k$ is in the layer below the layer that neuron $j$ is
in (in our example neuron $k$ is in the output layer). We take the
value for $\frac{\partial E_{k}}{\partial o_{k}}\frac{\partial o_{k}}{\partial v_{k}}$
in equation \ref{eq:backpropagation-error-wrt-output-other-layers}
above from equation \ref{eq:backpropagation-error-wrt-weight-output-layer}
when neuron $k$ is in the output layer or equation \ref{eq:backpropagation-error-wrt-weight-other-layers}
when neuron $k$ is in a hidden layer. (Neuron $k$ is named $j$
in equation \ref{eq:backpropagation-error-wrt-weight-other-layers}.)
In our example (for neuron $j$ in the last hidden layer), we take
$\frac{\partial E_{k}}{\partial o_{k}}\frac{\partial o_{k}}{\partial v_{k}}$
from the output layer:
\begin{eqnarray}
\frac{\partial E}{\partial o_{j}} & = & \sum_{k}\frac{\partial E_{k}}{\partial o_{k}}\cdot\frac{\partial o_{k}}{\partial v_{k}}\cdot w_{kj}\\
 & = & \sum_{k}(o_{k}-y_{k})\cdot o_{k}(1-o_{k})\cdot w_{kj}.\nonumber
\end{eqnarray}
Analogously, the derivative of $E$ with respect to $b_{j}$ is
\begin{eqnarray}
\frac{\partial E}{\partial b_{j}} & = & \frac{\partial E}{\partial o_{k}}\frac{\partial o_{k}}{\partial v_{k}}\cdot\frac{\partial v_{k}}{\partial o_{j}}\cdot\frac{\partial o_{j}}{\partial v_{j}}\cdot\frac{\partial v_{j}}{\partial b_{j}}\\
 & = & \sum_{k}\frac{\partial E_{k}}{\partial o_{k}}\frac{\partial o_{k}}{\partial v_{k}}w_{kj}\cdot o_{j}(1-o_{j})\cdot1.\nonumber
\end{eqnarray}

The error for each node $\frac{\partial E_{k}}{\partial o_{k}}\cdot\frac{\partial o_{k}}{\partial v_{k}}$
is thus back-propagated in reverse input direction through the hidden
layers, until all derivatives have been determined.

\subparagraph{Updating rule}

After having computed the derivatives of the error with respect to
the parameters of the network, we can perform gradient descent and
integrate these derivatives over the training patterns $p$. We update
each weight $w$ and bias $b$ using the learning rate\index{learning rate}
$\epsilon$, a small positive number:
\begin{eqnarray}
\Delta w & = & -\epsilon\sum_{p}\frac{\partial E}{\partial w}^{(p)}\\
\Delta b & = & -\epsilon\sum_{p}\frac{\partial E}{\partial b}^{(p)},\nonumber
\end{eqnarray}
where $\frac{\partial E}{\partial w}^{(p)}$ or $\frac{\partial E}{\partial b}^{(p)}$
are the derivatives of the error with respect to a weight or bias,
when the input layer was set to training pattern $p$.

Alternatingly performing forward and backward pass and updating the
weights and biases, until the error over all training patterns $E_{total}$
is small enough, forms the complete back-propagation training procedure
of a neural network.

\paragraph{Limits of Backpropagation}

The purpose of error back-propagation is to adjust the weights of
the artificial neural network following the gradient, such that when
the current input pattern pair is presented to the network, its computed
output pattern gets closer and closer to the desired output pattern.
However, back-propagation is not able to train networks with more
than one or two hidden layers, because it is a gradient descent method
and can get stuck in poor local optima, and the error surface gets
more rugged the more hidden neurons and layers there are \cite{GoriTesi1992}.
Having artificial neural networks with more than one hidden layer
is desirable, because they can perform the same computation with less
total number of hidden nodes compared to a network with less hidden
layers. A network with one hidden layer less needs up to an exponential
factor more hidden nodes \cite{Hastad1987}.

\section{Introduction to Machine Learning}

\subsection{Supervised and Unsupervised Machine Learning}

In machine learning, there are two major types of learning: \emph{supervised}\index{supervised machine learning}
and \emph{unsupervised learning}\index{unsupervised machine learning}
\cite{Barber2012}. Both methods process training data sets that are
in matrix form: for example, in expression data, the rows usually
denote different genes or transcripts, and each column represents
an independently measured sample. (Note that in the general machine
learning literature, usually the data matrix is transposed: the columns
denote the features, and the rows the samples.) Samples usually are
tissue, blood samples, or cell line, and differ in their biological
background (e.g. cell type, gene knock-out or knock-in, cell cycle
phase) or treatment (e.g. drugs applied).

In supervised learning, for every input pattern in the data set an
output pattern is defined that the learner should compute from the
input pattern. Herein, both input and output pattern can be one- or
multi-dimensional vectors. The goal of supervised learning is to infer
a function that maps from the space of input patterns to the space
of output patterns. The output patterns are also called \emph{labels}\index{labels}.
(The fact that for every input pattern there is a defined output pattern
is termed ``the input data is labeled'').

In unsupervised learning, there is only an input data set and the
goal is to find its compact description. The output of an unsupervised
learning algorithm is the underlying structure of the data according
to the algorithm's objective function. The objective of an unsupervised
learning algorithm can range from dimensionality reduction to data
re-representation to latent variable modelling.

Examples of supervised learning algorithms are (linear or logistic)
regression, k-Nearest Neighbor (k-NN) regression, support vector machines
(SVMs), and backpropagation neural networks. Examples for unsupervised
learning algorithms are (hierarchical or kNN) clustering, self-organising
maps (SOMs), principal component analysis (PCA), and Restricted Boltzmann
Machines (RBMs).

A goal of both supervised and unsupervised learning is that the learned
rules should generalize well, i.e. previously unseen data should be
characterized correctly. The samples are therefore split into training,
validation and test data sets. The \emph{training data set}\index{training data set}
is used to train a machine learning algorithm. Some machine learning
algorithms have meta-parameters, i.e. parameters that are needed for
the algorithm, but that we are not really interested in. (An example
is the number of hidden neurons in an artificial neural network.)
The meta-parameters are optimized using the \emph{validation data
set}\index{validation data set}. At the end of training, the performance
of the machine learning algorithm must be evaluated on previously
unseen samples, the \emph{testing data set}\index{testing data set}.

\subsection{Semi-supervised Machine Learning}

An intermediate form between supervised and unsupervised machine learning
is semi-supervised learning\index{semi-supervised learning}. In contrast
to supervised machine learning, which has for every input pattern
a target output pattern, semi-supervised learning does not need a
target output pattern for every input pattern. However, in contrast
to fully unsupervised machine learning, it does need some labeled
input data sets. A common objective of semi-supervised machine learning
algorithms is to find underlying structure in all input data sets
and then use the known labels to assign labels to unlabeled input
samples. This assumes that samples close in the (high-dimensional)
input space have the same label. Another assumption is that samples
distant to each other have different labels. An example of a possible
improvement of semi-supervised classification over supervised classification
is given in figure \ref{fig:Illustration-of-semi-supervised-learning}
(adapted from \cite{Joachims1999a}). In the figure, the two-dimensional
samples are either labeled orange or blue, or unlabeled, and the task
is to (1) find a straight line that separates samples with the two
colors and (2) color the unlabeled samples. Supervised learning alone
does not take into account the unlabeled samples, while semi-supervised
learning recognizes that there are two clouds of samples, separated
by a gap, where the labeled samples from each color are on different
sides of the gap. It then draws the separating line in the middle
of the gap, and colors the unlabeled samples on the side with the
orange samples orange, and the unlabed samples on the other side blue.

\begin{figure}
\begin{centering}
A\includegraphics[width=0.3\columnwidth]{images/semi-supervised-advantage1}
B\includegraphics[width=0.3\columnwidth]{images/semi-supervised-advantage2}
C\includegraphics[width=0.3\columnwidth]{images/semi-supervised-advantage3}
\par\end{centering}
\caption[Illustration of semi-supervised learning in two dimensions.]{\label{fig:Illustration-of-semi-supervised-learning}Illustration
of semi-supervised learning in two dimensions. Each axis is a dimension.
Circles are samples; filled circles are labeled samples; unfilled
circles are unlabeled samples. The blue circles are samples with label
1, the orange circles are samples with label 2. A: Supervised SVM
learning produces a maximum-margin classifier. B: Supervised learning
ignores and probably mis-classifies some unlabeled samples (the red
crossed-out samples). C: Semi-supervised learning regards densities
of unlabeled samples and may give better results than supervised learning
on the labeled samples alone.}
\end{figure}

There are two types of semi-supervised learning: transductive and
inductive semi-supervised learning. The goal of transductive semi-supervised
learning\index{transductive semi-supervised learning} is to predict
the class labels of a pre-specified list of unlabeled input patterns,
while the goal of inductive\index{inductive semi-supervised learning}
semi-supervised learning is to find a universal rule mapping from
the space of input patterns to class labels, which could be applied
to classify unknown, future input patterns. In case unknown, future
input patterns are to be classified using transductive semi-supervised
machine learning, the whole model may have to be re-evaluated.

\subsubsection{Example Scenarios for Semi-supervised Learning}

An advantage of semi-supervised learning over supervised learning
is that it does not need labels for all input patterns, because labels
are often time-consuming or costly to acquire. For example, the Gene
Expression Omnibus data base (GEO) \cite{BarrettSoboleva2013} contains
41,379 expression data sets that were uploaded between Jan 1st, 2000,
and August 31st, 2013. Many of these are potentially usable as unlabeled
data sets in semi-supervised learning.

However in practice, many machine learning algorithms require samples
to be independently and identically distributed\index{independetly and identically distributed}
(iid\index{iid}). In an ideal world, GEO samples could be assumed
to be identically distributed within a data set. Unfortunately even
within the same GEO data set there often is systematic variation between
samples, called the \emph{batch-effect}\index{batch-effect}, caused,
for example, by different sample handling or conditions at measurement
time. Hence one either has to use samples from one data set only,
or, if one wants to use samples from different GEO data sets simultaneously,
correct for a possible batch-effect manually, or use an algorithm
that has some built-in mechanism to make such a correction.

One machine learning algorithm with such a built-in mechanism is\emph{
deep learning}\index{deep learning}, as employed in \emph{Deep Belief
Networks}\index{Deep Belief Network}\emph{ (DBNs}\index{DBN}\emph{)}
\cite{HintonTeh2006}. This machine-learning algorithm (which is unsupervised,
but can be used for supervised and semi-supervised learning) can learn
from images of objects or faces, where the objects or faces are in
different lighting conditions or are viewed from different angles
(\cite{HintonSalakhutdinov2006,KrizhevskyHinton2012,KarpathyFei-Fei2014}).
In this setting, the batch effect would be the lighting condition
or viewing angle. Such results seem to imply that DBNs are able to
abstract the images, which are given as vectors of pixels, into encodings
of relevant features and compute a classifier on these abstract features.
This can be seen as a form of batch-effect correction.

An extreme form of batch-effect correction is when all training data
are in batch 1 and all test data in batch 2. Are neural networks able
to handle such a situation? For example, suppose that batch 1 are
all face portraits (frontal view), batch 2 are half profile faces
(30 degree angle view) of \textendash{} not necessarily the same \textendash{}
people, and the task is to match faces seen from both viewing angles
to the same person. During training, semi-supervised learning would
have access to the unlabeled faces from both viewing angles, but all
the half profiles would be unlabeled. We are not aware of a study
of that setting using artificial neural networks, but a similar setting
was studied in cognitive psychology, where humans were the learners
\cite{BobakBate2015}. The participants of the study were asked to
match a given half profile to one out of 10 portraits, or indicate
that there is no matching face. The accuracy of average people was
81\%, while people who have extraordinary face recognition abilities,
so-called ``super-recognizers'', had 94\% accuracy. When there was
no matching face, average people correctly rejected the face in 65\%
of all cases, and super-recognizers did so in 92\%. For humans, the
unsupervised training would consist in seeing people's faces from
different angles during normal day-to-day activity. However, one could
argue that this training is unfair, because humans had access to the
label of many half profiles, since they often have seen a person's
portrait just moments before seeing the half profiles.

\subsection{Deep Learning}

Deep learning\index{deep learning} is a term used in artificial neural
networks with several hidden layers. The advantage of a deep network
over a network with just one hidden layer is that it can model a problem
more compactly using less hidden neurons in total, because it has
more than one intermediate computation step.

\emph{Autoencoders} and \emph{Deep Belief Networks}\index{Deep Belief Network}\emph{
}(\emph{DBNs}\index{DBN}) overcame the limitation of only a few hidden
layers \cite{BengioLarochelle2007,HintonTeh2006}. Here we give brief
overviews over both types of artificial neural network.

\paragraph{Autoencoder}

An autoencoder is a network with more than one hidden layer, whose
training is unsupervised and its objective is to reconstruct the input
in the output layer. An autoencoder starts as a three-layer network
composed of input layer, hidden layer, and output layer. This network
is trained using back-propagation. The middle hidden layer is then
copied and new hidden layer is inserted between both copies, forming
a five-layer-network. The newly added parameters of this network are
the same in number compared to a three-layer network, which can be
trained using back-propagation. Hence, training the five-layer-network
using back-propagation also works, and adding a new hidden layer in
the middle can be repeated.

\paragraph{Deep Belief Network}

Training a Deep Belief Network is separated into a pre-training phase
and a fine-tuning phase. The pre-training phase is unsupervised. It
starts with a network consisting of one input layer and a hidden layer.
This pair of  layers is called a \emph{Restricted Boltzmann Machine}\index{Restricted Boltzmann Machine}
(RBM\index{RBM}). There is an accompanying unsupervised training
procedure called \emph{contrastive divergence}\index{contrastive divergence}
which finds weights between these layers. After training the RBM,
hidden layers are iteratively added on top of the RBM, and the new
weights between layers are initialized and pre-trained using contrastive
divergence. While the pre-training phase is unsupervised, the fine-tuning
phase can be unsupervised as well as supervised. After training, the
multiple-hidden-layer-network forms a generative artificial neural
network called a Deep Belief Network.

\section{Overview of Own and Related Work}

We used autoencoders, Deep Belief Networks, and Transductive Support
Vector Machines on expression data to predict whether breast cancer
patients will show pathologic complete response to chemotherapy or
residual disease. The expression data are high-dimensional ($\approx22,000$
genes) and we use a relatively large data set ($\approx500$ patients).

\subsection{Motivations for Using Deep Belief Networks on Transcriptomic Data}

The motivations for using Deep Belief Networks on transcriptomic data
come from those networks' successes when used on image data. In the
hand-written digit classification and graphical object recognition
data sets on which the deep artificial neural networks were developed,
they are among the best-performing predictors.

\subsubsection{The ImageNet Large Scale Visual Recognition Challenge}

An example for the success of deep neural network is image classification
in the ImageNet Large Scale Visual Recognition Challenge \cite{RussakovskyFeiFei2015}.
It is a yearly contest, wherein participants are given around 1.2
million training images. Each training image is labeled with one of
$1,000$ possible object categories describing the main object appearing
in the image, for example ``trumpet'' or ``butterfly''. After
training an image classification algorithm, each contestant must compute
up to $5$ labels for each of $100,000$ test images. Each test image
has a single label, which is kept hidden by the contest organizer.
A test image is scored as correctly classified if the correct label
appears in the (up to $5$) labels submitted by the contestant. (Up
to 5 labels may be submitted because for example a street scene may
contain, besides the correct ``car'' label also street signs and
drivers.) Finally, the accuracy of a contestant is computed as the
average fraction of correctly classified test images.

There have been notable improvements in the accuracy of the winning
contestant, starting in 2012. In the last years, all top contestants
have moved to using deep neural networks. See table \ref{tab:ILSVRC-Test-set-accuracies}
for the winning contestants between 2010 and 2014. Significant differences
before and after 2012 are the usage of neural networks directly on
the image pixel data, and not using pre-computed image features in
a supervised learning algorithm like a Support Vector Machine.

\begin{table}
\begin{centering}
\begin{tabular}{|c|c|c|c|}
\hline
Year & Winner & Accuracy & Technique\tabularnewline
\hline
\hline
2010 & NEC & 71.8\% & SIFT and LBP image features classified by SVM\tabularnewline
\hline
2011 & XRCE & 74.2\% & image signatures classified by one-vs-all SVMs\tabularnewline
\hline
2012 & SuperVision & 83.6\% & deep convolutional neural network\tabularnewline
\hline
2013 & Clarifai & 88.3\% & deep convolutional neural networks averaged\tabularnewline
\hline
2014 & GoogLeNet & 93.3\% & deep convolutional neural network\tabularnewline
\hline
\end{tabular}
\par\end{centering}
\caption[Test set accuracies and techniques of winning contestants in the ImageNet
Large Scale Visual Recognition Challenge from 2010-2014.]{\label{tab:ILSVRC-Test-set-accuracies}Test set accuracies and techniques
of winning contestants in the ImageNet Large Scale Visual Recognition
Challenge from 2010-2014. Column ``Year'' is the year of the contest,
``Winner'' the team name of the winner of this year, ``Accuracy''
the test set accuracy of the winner's submission, and ``Technique''
a summary of the winner's algorithm technique. SVM, support vector
machine.}
\end{table}

\subsubsection{Highly Correlated Inputs}

We will now discuss similarities between image classification and
expression data classification.

Both underlying distributions \textendash{} of images and of expression
data \textendash{} have many correlated features. For images, adjacent
pixels often display the same object and have therefore correlated
values. In some face recognition tasks for example, the faces are
scaled and translated so that the centers of both eyes and mouth are
aligned in different faces. There will be highly correlated pixels
for areas of the image where the cheeks and lips usually are. If you
use the pixels of the whole image as input to the neural network,
the corresponding input nodes will be highly correlated as well.

For transcriptomic data, one almost always observes many correlated
genes. The correlations can be due to many genes being regulated by
the same transcription factor \cite{TornowMewes2003,KlebanovYakovlev2007}.

\subsubsection{Deep Belief Networks Find Correlated Nodes}

Deep Belief Networks can group correlated input nodes. The network
can do this by increasing the weights from the correlated group to
a single hidden node, and decreasing the weights from the group to
all other hidden nodes. The hidden node becomes the representative
of the correlated group of input nodes. This is a form of abstraction
and dimensionality reduction. The hidden node will only be active
if many of its highly-weighted input nodes are active and only few
of its negatively-weighted input nodes are active. Repeated application
of this principle of abstraction in deeper and deeper hidden layers
allows the Deep Belief Network to form more and more abstract representations
of its input.

In face recognition for example, an abstract representation might
have a single value for the size of the lips. In expression data,
a single node in an abstract representation might encode the activity
of a gene module.

\subsubsection{Transductive Support Vector Machine}

We compare the artificial neural network approach with another, older,
and established semi-supervised method, the Transductive Support Vector
Machine (TSVM) \cite{Joachims1999}. Despite the name, it supports
transductive as well as inductive learning. A standard Support Vector
Machine searches for a decision boundary such that the margin between
samples with differing labels is maximal (see figure \ref{fig:Illustration-of-semi-supervised-learning}).
The TSVM seeks a labeling of the unlabeled samples so that the decision
boundary has the maximal margin between all samples with differing
classes.

\subsection{Previous Work: Gene Expression Inference With Deep Learning\label{subsec:Previous-Work:Gene-Expression-Inference-with-Deep-Learning}}

Very recently, \cite{ChenXie2015} published work on compressing expression
data into fewer dimensions on a large scale using deep learning. The
motivation for this work was that principal component analysis found
that $943$ ``landmark'' genes can capture about 80\% of the information
in the CMAP data set. This prompted the development of the ``L1000
Luminex bead technology'', which measures the expression of these
$943$ genes at a low cost, and computationally infers the remaining
$\approx21,000$ genes. \cite{ChenXie2015} worked on improving this
computational inference step.

Input data were all $\approx111,000$ genome-wide expression profiles
from the GEO database of Affymetrix microarrays, which were partitioned
into training, validation, and testing data sets. For each sample,
the same subset of $943$ landmark genes was chosen and $9,520$ other
genes were predicted from the landmark genes. This is different from
our work, because we classified breast cancer samples using gene expression
levels, while \cite{ChenXie2015} did regression of gene expression
levels using other gene expression levels.

\cite{ChenXie2015}'s artificial neural network architecture had between
$1$ and $3$ hidden layers with either $3,000$, $6,000$, or $9,000$
nodes. It had 943 input expression values (one for each landmark gene),
and a total of $9,520$ output expression values (one for each gene
to be predicted). In addition to the (non-linear) neural network,
they evaluated linear regression with no regularization, L1-, and
L2-regularization.

\cite{ChenXie2015} also evaluated k-Nearest Neighbor \index{k-Nearest Neighbor}(kNN\index{kNN}).
During training, they determined a number, $k$, of landmark genes
with expression value closest to each target gene $i$ (let's call
this set of genes $knn_{i}$) in the training data set. During testing,
they predicted the expression value of the target gene $i$ as the
average of the gene's $knn_{i}$ expression values in the testing
data set. The optimal $k$ (number of genes to average over) was chosen
based on a validation data set.

The input values were quantile normalized expression values between
4 and 15. The models were ranked according to the average prediction
errors over all 9,520 target genes.

k-Nearest Neighbor performed worst, with an average prediction error
of $0.5866$. Linear regression without regularization and with L2-regularization
both had an average prediction error of $0.3784$. Linear regression
with L1-regularization had an average prediction error of $0.3782$.
As the three linear regression models performed about equally well,
regularization did not improve linear regression. The neural network-based
average prediction errors were between $0.3421$ and $0.3204$, with
the network having $3$ hidden layers of size $9,000$ and $10\%$
dropout rate performing best. (\emph{Dropout }is a regularization
technique for neural networks, explained in section \ref{subsec:Dropout}.)
Because the input expression values were between $4$ and $15$, an
average prediction error of $0.3204$ implies an average error of
about $3\%$ on the GEO dataset.

In another dataset, \cite{ChenXie2015} again predicted $9,520$ genes
from the $943$ landmark genes, but used the GEO dataset for training,
the $1,000$ Genomes data for validation \cite{LappalainenPedro2013},
and GTEx data for testing \cite{ArdlieLek2015}. Learning in this
data set is harder since the training, validation, and testing data
sets are measured using different expression measurement technologies,
and therefore prone to the batch-effect. Nevertheless, the performance
ranking of the methods was the same, but worse than the data set without
batch-effect. KNN scored worst, with an average prediction error of
$0.6520$. Linear regression with L1-regularization had an average
prediction error of $0.5667$. Linear regression without regularization
and with L2-regularization both had an average prediction error of
$0.4702$. The artificial neural networks all scored consistently
better than KNN and linear regression, with the artificial neural
network with $2$ hidden layers of size $9,000$, and $25\%$ dropout
rate having the lowest prediction error of $0.4393$ (which is equivalent
to a relative error of $4\%$). On the validation data set, the average
prediction error was $0.7467$, which is a relative error of $6.8\%$.
This shows that artificial neural networks are capable of processing
input from multiple sources, with an acceptable gain in error. We
had a similar result, in section \ref{sec:Unsupervised-Reconstruction-of-Expression-Values}.


\section{Lecture 16}
In (1+1)D there are two types of anomaly.
Chiral anomaly $U(1)$, characterised by some integer, the Chern number.
Gravitational anomaly, characterised by Chiral central charge.
They are different,
but the minimal example of a theory with a chiral anomaly is a 1D complex
fermion.
A chiral fermion will have chiral anomaly.
When they say chiral anomaly,
they mean two different things.
If you have a left and right mover,
there is a symmetry corresponding to the left and right,
sometimes called the axial current.
The fact there's a chiral anomaly here
means that if you take two copies,
that anomaly will persist.
If you take a single chiral fermion,
and it has this gravitational anomaly.

If you have a general interacting CFT,
these don't have to be the same thing.

Hopefully I'll put up another homework assignment to cover what we've done since
last homework assignment.
We've been talking about topological insulators,
discovered by Charlie Kane who's here today.

Let me clarify some things I didn't explain,
then we'll go to 3D.

\section{2+1D case}
Here we have a topological insulator.
The fermionic symmetric group was
\begin{align}
    G_F &= U(1)^f \rtimes \mathbb{Z}_4^{T,f} / \mathbb{Z}_2
\end{align}
$U(1)$ is the same as fermion parity.
Time reversal also square the fermion parity.
This is class AII.

Then then are matrix elements of the time-reversal operator.
\begin{align}
    w_{m,n}(k) &=
    \bra{U_m(k)} \tau \ket{U_n(-k)}
\end{align}
and there are 4 time-reversal invariant momenta (TRIM) labelled $\Lambda_a$.

So we have $(0,0)$, $(\pi ,0)$, $(\pi, \pi)$, $(0, \pi)$.
Then we have $\mathbb{Z}_2$ topological band invariant.
\begin{align}
    \left( -1 \right)^{\nu} &= \prod_{a=1}^{4} \delta_a
    \prod
\end{align}
where
\begin{align}
    \delta_a &=
    \frac{\Pf(w(\Lambda_a))}{\sqrt{\det(w(\Lambda_a))}}
\end{align}
and we need to pick Bloch wave functions $\ket{u_n(k)}$ such that
$\sqrt{\det(w(k))}$ is continuous.

For 1D systems,
time-reversal polarization
\begin{align}
    (-1)^{P_T} &=
    \prod_{a=1}^{2}\delta_a = \delta_1 \delta_2
    = \delta_` \delta_2
\end{align}
When the time-reversal polarization is odd,
it has a Kramers degeneracy on the boundary,
and none if it's even.
A 1D system is not a well-defined thing,
because putting an electron at the end will ad a Kramers degeneracy.
When you do a gauge transformation the Bloch wave functions,
you can actually change $P_T$ from $0$ to $1$.

$P_T=0,1$ tells us if the boundary of a 1D system has Kramers degeneracy,
but there is a gauge transformation taking
$\ket{u_n(k)}\to e^{i\theta_k}\ket{u_n(k)}$.
So there is some phase change of the Bloch wave function that will change $P_T$
from $0$ to $1$.

There is an integer ambiguity of the charge polarization in 1D.
When we talk about Chern number,
the Chern number tells us how polarization changes affects the 1D changes.
If we had a 1D system,
if we insert flux,
the polarization as we insert flux changes from 0 to 1,
means we pump change from one end to another.
Adiabatic changes to polarization around a cycle are well defined and define
a topological pump.
Whether it's 0 to 1 in a given 1D system is not useful,
however if you think of a 2D system this way,
when you insert flux, or tune $k_y$,
the fact that the time-reversal polarization is changing between 0 and $\pi$
is the basic effect that describes what's going on here.
In equations,
this invariant is the same as
\begin{align}
    (-1)^{\nu} = (-1)^{P_T(k_y=0) - P_T(k_y = \pi)}
\end{align}
And it could be a plus.

What does it mean for the surface stats.
If $\nu$ is odd,
we have helical edge modes with counter propagating complex fermions as I said
last time.

Now, I want to explain the connection to the surface states a little more
clearly.

\section{Connection to Surface States}
The equality I didn't derive last time
\begin{align}
    (-1)^{\nu} = (-1)^{P_T(k_y=0) - P_T(k_y = \pi)}
\end{align}
I didn't describe,
so I might put that in your homework.
What does it actually mean?

We conclude that we have to have gapless surface states if the symmetry remains
intact.
Taking $k_y$ from 0 to $\pi$.
Remember $k_y + A_y$ enters the equation when we insert flux.
So $k_y + A_y$ is what enters the Hamiltonian.
So
\begin{align}
    A_y &= \Phi/L_y
\end{align}
Taking $A_y$ from $0$ to $\pi$ is equivalent to taking
$\Phi$ from $0$ to $\pi L_y$.
The cylinder is along $x$.

$2\pi$ flux is equivalent to no flux,
so this is just $\pi$ flux if $L_y$ is odd.
So take $L_y$ odd and consider inserting $\pi$ flux.
$\phi=0\to \pi$.

If we start with a Kramers singlet on a boundary,
we need to get to a Kramers doublet.
What this tells us is that if we start off with a non-degenerate ground sate,
we adiabatically insert flux.
If $\nu$ is odd,
we better get to a Kramers doublet.
That immediately tells us it's gapless?

Why?
Suppose it's gapped.
Then inserting flux can only change the ground state energy by an amount
$\Delta E \propto e^{-L_y/\xi}$.
So that means if it starts off in a Kramers singlet,
there's no way it could become a Kramers doublet.
It has to remain a Kramers singlet.

So if the boundary is gapped,
there are 2 possibilities.
Either this $\nu=0$ is trivial,
so a singlet stays singlet.
Or, we break time-reversal symmetry,
so it doesn't make sense to talk about Kramers doublet or singlet anymore.

If we break $U(1)$ charge conservation,
we have asuperconductor on the boundary,
inserting flux is well defined anymroe if you want to stay on the ground state
manifold.
If I insert flux, it gives lare enrgy cost.

We can conlude that if $\nu=1$ mod 2,
then either
\begin{enumerate}
    \item The boundary is gapped and symmetry-breaking.
    \item The boundary is gapless, so it can preserve the symmetry.
\end{enumerate}
If it's gapless it actually may or may not preserve the symmetry,
but the point is that if we want to preserve the symmetry,
the boundary must be gapless.
Remember this picture where I drew $E$ as a function of $k_y$,
where we had one bad down here and one bad up there.

At $k_y=\pi$,
they had to be Kramers degenerate,
and these Kramers pairs split at $k_y=0$.
There are two ways they can connect going from $k_y=0\to \pi$.
Either they can switch partners like this [picture].

Or they don't switch partners.
In the case of no switching pairs,
there is symmetry,
and if it's gapped, it's in conflict,
so it must be the pair switching case if $\nu$ is odd,
and not switching if $\nu$ is even.

This establishes a connection betwen the bulk nivariatn $\nu$ and the way the
edge modes connect on to the boundary,
which guarantees these robust edge states.

You could go further.
Suppose you had one of these situations,
can you convinceyouself that the insertion of $\pi$ flux would change the bands.

Now I'm going to draw the bnads from $-\pi$ to $\pi$.
Suppose your chemical potential is exactly at this point,
so htis state which isa Kramers doublet,
and bothsates in the doublet are occuppied.
And then all the states below that chemical potential are also occupied.
So then you can insert the $\pi$ flux,
which shifts all tese states over to hte right by $\pi/L_y$.
And we get a picture where one of these states goes up a little,
and hte other states shifts down.
But here there's an empty state.
So this guy becomes empty and this guy becomes filled.
Under $\pI$ flux,
these states hsfit over,
and you have occupied and unoccpied states.
This start is Kramers doublet because there's an exact anaolgous state with a
Kramers degeneracy with a degeneracy.
You can draw these diagrams in all scenarios and convince yourself that iserting
$\pi$ flux changes a state from an even number of doublets to an odd number of
doublets, and vice versa.
That makes the surface and bulk correspondnce even more explicit.

I wanted to fill in this hole last time.
I didn't make a connecino with suraace staes.

THis (1+1)D helical boudnary,
in the Chern number case,
we had a U(1) anomaly.
Something has ot cancel out the anomaly,
which si the bulk.
Here the anomaly is a mix between $U(1)$ and time-reversal.
If we want to preserve both of them so these gapless modes are robust,
that's a signature that we hav ea mixed anomaly between U(1) and TR symmetry.
The fact you insert $\pI$ flux and the Kramers degeneracy of the ground state
chagnes,
you can think of it as a symptom of the anomaly.
So the ground sate went from beinga Karmers singlet to Kramers doublet.

If you take this helical theroy,
noe way to see hwat's wrong is the study it on a spacteim that iss Klein bottle.
It's useful to put sapceteime on a non-orientable manifold,
it turns out you can also in that case see more things wrong and inconsistent
with the theory.
Just like the chiral fermion, thei nconsistency wwas curretn conservation breka
down.

Seeing anomalies wiht discrete ymmetries is a muc hmore complicated game poeple
only started to understand 10 years ago.
Studying on a Klein bottle is one thing you could do to se the anomaly.
But I'm not going to go down that path here.

So now let's go to (3+1)D.
We're still in Class AII and TI still squares to -1.
And here we have 8 TRIM,
which I write as
\begin{align}
    \Lambda_a=(n_1,n_2,n_3)
    =
    \frac{1}{2}n_1 \vec{b}_1
    + \frac{1}{2}n_2 \vec{b}_2
    + \frac{1}{2}n_3 \vec{b}_3
\end{align}
where the $b$s are primitive reciprocal lattice vectors and $n$s are iether 0 or
1.
This defines a 3D topological insulator.
The topological invariant is
\begin{align}
    (-1)^\nu &= \prod_{a=1j^{8}}\delta_a.
\end{align}
Consider a cubic lattice.
Consider a surface in the $z$ direction.
We can look at hte surface states.
At the time-reversal invaritn mometna,
at a stte in this TRIM,
we have Kramers degenercy,
but as soon as we go away,
the bands split.
So that means we have a Dirac node,
when we locally have some state at some energy.

So surface states at $(k_x,k_y)=(0,0)$, $(0,\pi)$, $(\pi, 0)$, $(\pi, \pi)$
come in Kramers pairs,
and that means we have Dirac cones every time we have states with these momenta.

We can draw a picture.
We have these different points here,
these TRIM,
and wheneter you have some state that comes iwth degeneracy,
there's a bunch of dirac cones when we hvae some staes at these moemnta.
For exmplae
at this momenta,
we have Dirac cones at vairous energis.
And just like this dimension,
just like here,
it was a question of how the states connect up.
Similarly here,
it's a uestion of how these diract cones all connect up with each other.
This index $\nu$ ultimately gives informatino about the way these cones connect
up to each other.

The fermi level could be anywhere.

The question is how do these cones connect up.
And basically hte answer is that if $\nu=1$.
Let me mention nadi mportant property of Dirac cones,
I hsould put on the homework.
If you calculate the Bery phase of going around a dirac cone,
you get a $\pI$ Berry phase,
it's the most important property of a Dirac cone.

When $\nu=1$,
the cones connect up in such a way that wheterver your chemical potential is,
if you lok tahte Fermi suaces,
and claculate the Berry phase to go aorund all your fermi surfaces,
you get a Berry phase of $\pi$.
If $\nu=1$,
the cones connect up sothat the Fermi surface encloses an odd numbero f Dirac
cones,
or more precisely there is a $\pi$ Berry phase.

For example, suppose you had $k_x$ and $k_y$.
At each $\Lambda_a$, define
\begin{align}
    \pi_a = \delta_{\Lambda_a} \delta_{\Lambda_a + \pi \hat{z}} = \pm 1
\end{align}
Depending on the sign of this guy,
what you can argue is that every time you want to go from a filled to an empty
circle,
youhave to cross the Fermi circle.
Every time you went from 0 to $\pi$ we had to cross the band so that the Fermi
surface lies here.
It's anlogous in higher dimension,
that if you want to gor from one poitn ot another,
whether you hae to cross a Fermi surface is determined by this number here.

So far I've just told you what happens,
but I haven't explaine dhtis very much,
but I won't explain this to you,
just too much band theory.
I hope you got the idea.
It's imlar to 1D.
You want to go from one side to another,
whether you cross the band depends on some invariants,
and in this case happens to be this one.
It's more confusing to think of this 3D case.
If you are interested in understanding this point in detail,
I can recommend references.

If $\nu=1$ mod 2,
then the surface can be modelled by an odd number of Dirac cones at some
chemical potnetial nd the minimal surface model would be having just 1 Dirac
cone.

So we've writtne Dirac cones before in class, for example
\begin{align}
    H &=
    \int d^2x\,
    \left[
    \Psi^\dagger\left(
        i\sigma^x D_x - i\sigma^y D_y
    \right)\Psi
    - \mu \Psi^\dagger \Psi
    \right].
\end{align}
Can the Fermi surface be gapped?

In this model,
time-reversla symmetry can e implmented by
\begin{align}
    \mathcal{T} &= i\sigma^y K
\end{align}
where $K$ is complex conjugatino.
In 1D it's either gapped and breaks the smmetry
or it's not gapped and preserves the symmetry.
Same thing here.

A surface is either
\begin{enumerate}
    \item Trivially gapped and symmetry-breaking.
        The things you can dd to the Hamiltonian to gap out the Fermi surface
        would be a mass term
        \begin{align}
            \delta H_1 &= m \Psi^\dagger \sigma^z \Psi
        \end{align}
        which breaks TR symmetry, but conserves $U(1)$.
        If you ahvea single massive dirac fermion,
        integrate out the fermion,
        look tah te electrmagnetic response,
        you geta Chern-Simons term in the background gauge field with a value of
        half.
        so this leads to a $\frac{1}{2}$ integer Hall conductivity,
        meaning hte Hall conducatnce is
        \begin{align}
            \sigma_H &= \frac{1}{2} \frac{e^2}{h}
        \end{align}
        So breaknig TR symmetyr leads to Hall condcityity that is half.
        But if there is an integer ambiguoulity more generally,
        it's
        \begin{align}
            \sigma_n &= \left( n + \frac{1}{2} \right) \frac{e^2}{h}
        \end{align}
        the half shift of the Landau levels is related to the Dirac cone.
        You will never see the surface in a pure 2D system,
        because in a 2D system,
        you only have an even numbero f Dirac cones.
        Having an odd number of Dirac cones gives that 1/2 conduance,
        and is non-trivial.

        Another term you could add to theHamiltonain to gap the Fermi surface is
        \begin{align}
            \delta H_2 &=
            \Delta \Psi^\dagger i\sigma^y \Psi + \mathrm{h.c.}
            \propto
            \Delta \Psi_1\Psi_2 + \mathrm{h.c.}
        \end{align}
        This term by itsefl opens a gap at the Fermi surface.
        Superconducting terms usually produced gapped at the surface.
        there's no way of introducing a gap trivially,
        by trivally I mean adding a Fermion bilinear and gapping hte surface.
        These are really the only two terms you could add.
    \item The other possiblity is that you don't hvae these terms and the
        surface is symmetry-preseriving.
        But now you have in general an odd number of Dirac cones,
        with this symmetry and time-reversal,
        and this has a ``parity anomaly''.
        And in our situtiona,
        this is really a mixed anomaly between U(1) and time-reversal.
        What this mixed anomaly means is that if you take a theory of a single
        Dirac cone,
        make it well defined,
        then you run into problems and you need to regularize it,
        and any way of regualirazting necessarily breaks U(1) or T symmetry.
        For examle, compue the harge of the U(1) symmetry,
        but you need to do a sum you need to regularize,
        but ther's no way of regularlizing wihtout brekaign T.
        You need to introduce an extra dimensino.
        Alternatively,
        you could try to get teh sign of the path integral real to be
        consistentwtih TR symetry without messing the bulk.
        This discussion is probably a lecture of its own.
        There's a really nice review by Ed Witten from 5 or 6 years ago where he
        does it and he releates it to TSE index theorems,
        which I encourage you to look up.
    \item This is new to this dimensino and wasn't discovered until after 5 or 6
        years after topological insulators were discovered.
        This is the possbility that the surface is gapped and
        symmetry-preserving,
        but the surface has a non-invertible topological order.
        Somtimes people call this ``intrinsic toplogical order''.
        This means anyonic excitaitons,
        particles iwth topological nontrivial faactional statsitics,
        and the particles act on the anyons in anomalous ways that show upin a
        2+1D system.
        We haven't discussed non-invertible toolgoical ordre yet,
        so we're not in a position to discuss this third possibility,
        there there is such a possibility.
        The ground state on a nontrivial surface is degenerate,
        in fact a topological degeneracy.
        For example, if the surface of your system isa torus,
        you would havea ground sate degenarcy coming from hte boundaries,
        likethe Majoranas.
        In 1 you also have a ground state degenearcy,
        but it is trivial.
\end{enumerate}

\begin{question}
    When it says we have to enclose an odd numbero f Dirac coenes.
    Say I sweep the fermi level,
    that means it should come in paris?
    If I sweep up the Fermi level,
    and I need to retain an odd number of Dirac cones,
    then i have to pass 2 Dirac cones at a time?
\end{question}
Just ignore ``Fermi surface enclosed odd number of Dirac cones''.
What matters is that when you go around the Dirac cone,
you have a $\pi$ Berry phase.

You have to think pretty carefully about what happens in 2D, 3D, but 3D is much
more complicated.
It's analysis you have to go through.

Those are just a few words bout hte surface.o
Bulk response theory.
I assert what the bulk response theory is .

\begin{question}
    Trivial gapping is being introduced by fermion bilinears,
    by non-trivial you mean higher order terms?
\end{question}
Let me be more specific.
Trivially gapped means that the surface does not have intrinsic topological
order.
Now,
if the surface does not have intrinsic topological order,
usually those kinds of gapped surfaces which don'th ave intrinsic gapped order
can always be cancleeleld by adding fermion bilinears.
But if you add a four-fermion term,
that could lead ot expectation values for two-fermion blinears.
That's what happesn in superconductivity.
what I really mean is more naunced.

\begin{question}
    Given a material,
    would you be able to tell from this band diagram?
\end{question}
You have to find whether the symmetry is preserved or not,
which you can convince experiennatally.

\begin{question}
    Possiblity 4 with gappless and symmetry breaking?
\end{question}
That's kind of trivial.
It could be gapless and symmetry breaking.
The reason I emphasies symmetry preserving is because that's usually the
possiblity that occurs if you preserve the symmetry.

Once you break the symmetries,
you could think of what's hapening as another manifestatino of hte anomaly.
Symmetry brekaing ofa system with an anolay is an intersting subject itself.
Usulaly it means various defects have nontrivial properties because of the
anomaly.

One thing I didn't mention.
This superconductivity is superconducting in a single fermion band.
Ify ou inteoduced supercondcutivty in a single band systme,
you expect ot et Majorana zero modes.
The vortices of this superconector $\delta H_2$ carry Majorana zero modes.
This gave the first example beyond
which doesn't have a non-trivial pairing symmetry,
you can have an odd numberof so vortices carry MZMs.

It's really a single band,
there's no spin degenearcy.

\section{Electromagnetic response}
I want to discuss the electromagnetic response.
The electromagneti response has an effective action that is topological
\begin{align}
    S_{3+1}^{top}[\theta, A] &= \theta\cdot P
\end{align}
this is the so-called $\theta$ term in gauge theory.
This is called the instanton number.
\begin{align}
    P &= \frac{1}{32\pi^2} \int d^3x\, dt\,
    \epsilon^{\mu\nu\rho\sigma} F_{\mu\nu} F_{\rho\sigma}
    =
    \frac{1}{4\pi^2} \int d^3x\, dt\, \vec{E}\cdot\vec{B}
\end{align}
Let me define the non-abelian Berry conncetion,
where we take Bloch states from different bands.
\begin{align}
    a_i^{\alpha\beta} &=
    -i \bra{\alpha,k}
    \frac{\partial}{\partial k_i}
    \ket{\beta,k}
\end{align}
Before we were taking Blcoh staes from the same band,
but now we're taking them from different bands.
the non-abelian field strength
\begin{align}
    f_{ij}^{\alpha\beta} &=
    \partial_i a_j^{\alpha\beta}
    - \partial_j a_i^{\alpha\beta}
    + i[a_i, a_j]^{\alpha\beta}
\end{align}
Then the $\theta$ term is
\begin{align}
    \theta &=
    \frac{1}{2\pi}\int d^3k \, \epsilon^{ijk}\Tr\left(
    \left( f_ij - \frac{1}{3}[a_i, a_j] \right)a_k
    \right)
\end{align}
The trace is over occupied bands.
This is called the ``theta term'',
the ``topological magnetoelectric effect''
or the magnetoelectric polarizability.

Note that $F\wedge F$ is a total derivative because
\begin{align}
    \epsilon^{\mu\nu\rho\sigma} F_{\mu\nu} F_{\rho\sigma}
    \propto
    \epsilon^{\mu\nu\rho\sigma} \partial_{\mu}\left(
    A_\nu \partial_\rho A_\sigma
    \right)
\end{align}
One way of thinking about this is either a surface effect,
and you can think of it as a property that does something to monopoles in your
system.
We see both of those.

Just like the Chern number arises from integrating fields strength over a
surface.
But here, w'ere integrating $F\wedge F$ over a closed $4$-manifold.

$P$ is an integer if space-time is closed.
And because $P$ is an integer and spacetime is closed,
our path integral,
or effective parition function looks like
\begin{align}
    e^{i\theta P}
\end{align}
and because $P$ is an integer,
changing $\theta$ to $\theta+2\pi$ doesn't change anything,
so in the bulk it doens't change anything
$\theta \sim \theta + 2\pi$.
We're just trying to undertand the bulk of hte sytem by putting it on a closed
sacetime manifodl,
and ther's no effect changing by $2\pi$.
Then you can ask what time-versal does.c
It doesn't do anytign to $\vec{E}$,
but it flips $\vec{B}$.
So effeively time-reversal is like flipping $\theta$.

So, you could say a time-versed system would have a $-\theta$ response,
which means that $T:\theta \to -\theta$.

And because $\theta=0$ or $\pi$,
but are consistent with timer-reversal.

\begin{question}
    The anlog of $\theat$ in 2+1 dimensisno is anot integer,
    but now we can define something up to an integer?
\end{question}
There's a fundamental distinction betwen even and odd idmensions.
In odd dmidension,
youhave $\theat$ terms.
If you have a insulator wiht no particular symemtries,
$\theat$ can be anything.
But now we add a symmetry and we quatnzie it.
The coefficient in the CS theory was integer.
The short answer is there is a fundamental distcintion betwene odd and even
dimensions.
There's a long answer how to relate 2+1 and 3+1 dimensinos,
but I can't go into that at the moment.

As an aside,
let me say that if you're intersted in the TQFT aspect of it,
you might also be interested in not only turning on a non-trival spacteime,
but also be interested in putting it on a non-trivial spacetime so it has
curvature.
So if you include the gravatiational response,
here we have not just $F\wedge F$,
but also $R\wedge R$ where $R$ is the Riemann tensor.
So
\begin{align}
    S[A, g] &=
    \theta\cdot P
    -
    \frac{1}{48} \int \frac{1}{\left( 2\pi \right)^2} \Tr R\wedge R
\end{align}
Let me mention some physical consequences of this magnetoelectric term.

\begin{enumerate}
    \item Surface quantum Hall effect.
        Suppose that we have a domain wall of $\theta$ as a function of $z$.
        And $\theta$ goes from $0$ at $z=-\infty$ to $\pi$ at $z=+\infty$.
        So then
        $\theta(z)=\pi H(z)$ as a step function and its derivatve is
        $\theta'(z) = \pi \delta(z)$.
        Then
        \begin{align}
            \frac{1}{8\pi^2}\int\theta(z)
            \epsilon^{\mu\nu\lambda}\partial_z\left(
            A_\mu \partial_\nu A_\lambda
            \right)
            &= -\frac{1}{8\pi}\int \partial_z\theta\cdot
            \epsilon^{\mu\nu\lambda} A_\mu \partial_\nu A_\lambda\\
            &= -\frac{1}{2} \frac{1}{4\pi}\int_{z>0}
            A_\mu \partial_\nu A_\lambda \epsilon^{\mu\nu\lambda}
        \end{align}
        and so we get the Hall conductance
        \begin{align}
            \sigma_H &= -\frac{1}{2}
        \end{align}
    \item Witten effect.
        Suppose we have a bulk cube with boundaries.
        Suppose $\theta$ goes from $\pi$ to $0$ as discussed.
        Imagine a monopole form the outside and bring it inside.
        The basic effect is that monopole carries half charge.
        One quick way of seeing that is tht the surface has half-integer QHE,
        I take the monompole iwth strnght $2\pi$,
        and this monopole feels flux $2\pI$,
        so it changes by $2\pi$.
        Becasue of the half integer efefct,
        it has to bind the half charge,
        so I get a half-charge at the surface and that needs to be compensated
        somewhere else,
        and that's in the monopole.
        That's a quick way to see the half-integer hall effect.
        You could do a few lines of algebra and actually derive it a little more
        concretely.
        Let me derive it quickly.
        \begin{align}
            j^{\mu} &=
            \frac{\delta S}{\delta A^{\mu}} =
            \frac{1}{2\pi} \epsilon^{\mu\nu\lambda\rho}
            \partial_\nu \theta
            \partial_\lambda A_\rho
        \end{align}
        Suppose $\theta$ is uniform in spce,
        but time-varying
        \begin{align}
            \partial_i j^i &=
            \frac{1}{2\pi}
            \epsilon^{itjk} \partial_t \theta
            \partial_i \partial_j A_k\\
            &= -\frac{1}{2\pi} \partial_t \theta \partial_i B^i
        \end{align}
        and this is the same as the time-derivativeof the charnge
        \begin{align}
            \partial_t \rho &=
            \frac{1}{2\pi} \partial_t \theta \partial_i B^i
        \end{align}
        So then
        \begin{align}
            \rho &= \frac{1}{2\pi}\theta\nabla\cdot\vec{B}
        \end{align}
        and so tthe electric charge is half the monopole charge
        \begin{align}
            Q_e &= \frac{1}{2} Q_m
        \end{align}
\end{enumerate}

Ther'es one other consequence I want to discuss next time,
then I'll do dimensinoal reduction from 4+1D Chern insulator.

Let me just take a few minutes and do one another thing,
which is closely replated to the Hall conductivity called the Witten effect.