\clearpage\chapter{The Type Inferencing Model}

Three sections of this dissertation are devoted to type inferencing:
two chapters and an appendix. This chapter develops a theoretical
model of type inferencing for Icon. For simplicity, it ignores some
features of the language. This chapter presents intuitive arguments
for the correctness of the formal model. Chapter 19 describes the
actual implementation of type inferencing in the Icon compiler. The
implementation handles the full Icon language and, for pragmatic
reasons, differs from the theoretical model in some details.

This chapter starts with the motivation for performing type
inferencing. It then describes the concept of \textit{abstract
interpretation}. This concept is used as a tool in this chapter to
develop a type inferencing system from Icon's semantics. This chapter
gives an intuitive presentation of this development process before
presenting the formal models of abstract semantics for Icon. The most
abstract of the formal models is the type inferencing system.

\section{Motivation}

Variables in the Icon programming language are untyped. That is, a
variable may take on values of different types as the execution of a
program proceeds. In the following example, \texttt{x} contains a
string after the read (if the read succeeds), but it is then assigned
an integer or real, provided the string can be converted to a numeric
type.

\goodbreak
\begin{iconcode}
x := read()\\
if numeric(x) then x +:= 4\\
\end{iconcode}

In general, it is impossible to know the type of an operator's
operands at translation time, so some type checking must be done at
run time. This type checking may result in type conversions, run-time
errors, or the selection among polymorphous operations (for example,
the selection of integer versus real addition). In the Icon
interpreter system, all operators check all of their operands at run
time. This incurs significant overhead.

Much of this run-time type checking is unnecessary. An examination of
typical Icon programs reveals that the types of most variables remain
consistent throughout execution (except for the initial null value)
and that these types can often be determined by inspection. Consider

\goodbreak
\begin{iconcode}
if x := read() then\\
\>y := x || ";"\\
\end{iconcode}

Clearly both operands of {\textbar}{\textbar} are strings so no
checking or conversion is needed.

The goal of a type inferencing system is to determine what types
variables may take on during the execution of a program. It associates
with each variable usage a set of the possible types of values that
variable might have when execution reaches the usage. This set may be
a conservative estimate (overestimate) of the actual set of possible
types that a variable may take on because the actual set may not be
computable, or because an analysis to compute the actual set may be
too expensive. However, a good type inferencing system operating on
realistic programs can determine the exact set of types for most
operands and the majority of these sets in fact contain single types,
which is the information needed to generate code without type
checking. The Icon compiler has an effective type inferencing system
based on data flow analysis techniques.

\section{Abstract Interpretation }

Data flow analysis can be viewed as a form of abstract interpretation
[.absintrp.]. This can be particularly useful for understanding type
inferencing. A ``concrete'' interpreter for a language implements the
standard (operational) semantics of the language, producing a sequence
of states, where a state consists of an execution point, bindings of
program variables to values, and so forth. An abstract interpreter
does not implement the semantics, but rather computes information
related to the semantics. For example, an abstract interpretation may
compute the sign of an arithmetic expression rather than its
value. Often it computes a ``conservative'' estimate for the property
of interest rather than computing exact information. Data flow
analysis is simply a form of abstract interpretation that is
guaranteed to terminate. This chapter presents a sequence of
approximations to Icon semantics, culminating in one suitable for type
inferencing.

Consider a simplified operational semantics for Icon, consisting only
of program points (with the current execution point maintained in a
program counter) and variable bindings (maintained in an
environment). As an example of these semantics, consider the following
program. Four program points are annotated with numbers using comments
(there are numerous intermediate points not annotated).

\goodbreak
\begin{iconcode}
procedure main()\\
local s, n\\
\\
\>\# 1:\\
\>s := read()\\
\>\# 2:\\
\>every n := 1 to 2 do \{\\
\>\>\# 3:\\
\>\>write(s[n])\\
\>\>\}\\
\>\# 4:\\
end\\
\end{iconcode}

If the program is executed with an input of \texttt{abc}, the following states
are included in the execution sequence (only the annotated points are
listed). States are expressed in the form \textit{program point}:
\textit{environment}.

\goodbreak
\begin{specialcode}{}
\>\>1: [\texttt{s} = null, \texttt{n} = null]\\
\>\>2: [\texttt{s} = "\texttt{abc}", \texttt{n} = null]\\
\>\>3: [\texttt{s} = "\texttt{abc}", \texttt{n} = 1]\\
\>\>3: [\texttt{s} = "\texttt{abc}", \texttt{n} = 2]\\
\>\>4: [\texttt{s} = "\texttt{abc}", \texttt{n} = 2]\\
\end{specialcode}

It is customary to use the \textit{collecting semantics} of a language
as the first abstraction (approximation) to the standard semantics of
the language. The collecting semantics of a program is defined in
Cousot and Cousot [.absintrp.]  (they use the term \textit{static
semantics}) to be an association between program points and the sets
of environments that can occur at those points during all possible
executions of the program.

Once again, consider the previous example. In general, the input to
the program is unknown, so the read function is assumed to be capable
of producing any string. Representing this general case, the set of
environments (once again showing only variable bindings) that can
occur at point 3 is

\goodbreak
\begin{specialcode}{}
\>\>[\texttt{s} = "", \texttt{n} = 1],\\
\>\>[\texttt{s} = "", \texttt{n} = 2],\\
\>\>[\texttt{s} = "a", \texttt{n} = 1],\\
\>\>[\texttt{s} = "a", \texttt{n} = 2],\\
\>\>\>\ \ldots\\
\>\>[\texttt{s} = "\texttt{abcd}", \texttt{n} = 1],\\
\>\>[\texttt{s} = "\texttt{abcd}", \texttt{n} = 2],\\
\>\>\> \ldots\\
\end{specialcode}

A type inferencing abstraction further approximates this information,
producing an association between each variable and a type at each
program point. The actual type system chosen for this abstraction must
be based on the language and the use to which the information is
put. The type system used here is based on Icon's run-time type
system. For structure types, the system used retains more information
than a simple use of Icon's type system would retain; this is
explained in detail later. For atomic types, Icon's type system is
used as is. For point 3 in the preceding example the associations
between variables and types are

\begin{specialcode}{}
\>\>[\texttt{s} = string, \texttt{n} = integer]\\
\end{specialcode}

The type inferencing system presented in this chapter is best
understood as the culmination of a sequence of abstractions to the
semantics of Icon, where each abstraction discards certain
information. For example, the collecting semantics discards sequencing
information among states; in the preceding program, collecting
semantics determine that, at point 3, states may occur with \texttt{n}
equal to 1 and with \texttt{n} equal to 2, but does not determine the
order in which they must occur. This sequencing information is
discarded because desired type information is a static property of the
program.

The first abstraction beyond the collecting semantics discards dynamic
control flow information for goal directed evaluation. The second
abstraction collects, for each variable, the value associated with the
variable in each environment. It discards information such as,
``\texttt{x} has the value 3 when \texttt{y} has the value 7'',
replacing it with ``\texttt{x} may have the value 3 sometime
and \texttt{y} may have the value 7 sometime.''. It effectively
decouples associations between variables.

This second abstraction associates a set of values with a variable,
but this set may be any of an infinite number of sets and it may
contain an infinite number of values. In general, this precludes
either a finite computation of the sets or a finite representation of
them. The third abstraction defines a type system that has a finite
representation.  This abstraction discards information by increasing
the set associated with a variable (that is, making the set less
precise) until it matches a type. This third model can be implemented
with standard iterative data flow analysis techniques.

This chapter assumes that an Icon program consists of a single
procedure and that all invocations are to built-in functions. It also
assumes that there are no co-expressions beyond the main
co-expression. See Chapter 19 for information on how to extend the
abstractions to multiple procedures and multiple co-expressions.

\section{Collecting Semantics}

The collecting semantics of an Icon program is defined in terms of a
\textit{flow graph} of the program. A flow graph is a directed graph
used to represent the flow of control in a program. Nodes in the graph
represent the executable primitives in the program. An edge exists
from node \textbf{A} to node \textbf{B} if it is possible for
execution to pass directly from the primitive represented by node
\textbf{A} to the primitive represented by node \textbf{B}. Cousot and
Cousot [.absintrp.] prove that the collecting semantics of a program
can be represented as the least fixed point of a set of equations
defined over the edges of the program's flow graph. These equations
operate on sets of environments.

For an example of a flow graph, consider the Icon program

\goodbreak
\begin{iconcode}
procedure main()\\
\>every write(1 to 3)\\
end\\
\end{iconcode}

\noindent
The diagram below on the left shows the abstract syntax tree for this
procedure, including the implicit \texttt{fail} at the end of the
procedure. The invoke node in the syntax tree represents procedure
invocation. Its first argument must evaluate to the procedure to be
invoked; in this case the first argument is the global variable
\texttt{write}. The rest of the arguments are used as the arguments to the
procedure. \texttt{pfail} represents procedure failure (as opposed to
expression failure within a procedure). Nodes corresponding to
operations that produce values are numbered for purposes explained
below.

A flow graph can be derived from the syntax tree. This is shown on the right.

\begin{figure}[htb]
%% \begin{picture}(400,250)
%% %\put(0,0){\graphpaper{40}{25}}
%% \put(0,0){\includegraphics[width=2.4in,height=3.4in]{kw/figure3-1.png}}
%% \put(200,20){\includegraphics[width=2.2in,height=3.1in]{kw/figure3-2.png}}
%% \end{picture}
\begin{center}
\begin{tikzpicture} [draw,thick,font=\small\tt,
% define a "curved box" style
   level distance=12mm,
   cbox/.style={rounded rectangle, rounded rectangle arc length=90, minimum width=10mm, draw},
   every label/.style={font=\small\it}
   ]

\node (lhs) [draw,shape=rounded rectangle, rounded rectangle arc length=180] {procedure main}
  child { node [cbox] {;}
      child {node [cbox] {every}
          child {node[cbox,label=left:1] {invoke}
              child { node[cbox, label=left:2] {write}}
              child[missing]
              child { node[cbox, label=left:3] {to}
                  child {node[cbox, label=left:4] {1}}
                  child {node[cbox, label=left:5] {3}}
              }
          }
      }
      child[missing]
      child {node [cbox] {pfail}}
  };

\begin{scope}[>=latex, edge from parent/.style={draw,arrows=-latex}]
\node(rhs) [right=5cm, draw,shape=rounded rectangle, rounded rectangle arc length=180,
] {procedure main}
  child
  { node [cbox,label=left:2] {write}
      child {node[cbox,label=left:4] {1}
         child {node[cbox,label=left:5] {3}
            child {node (to) [cbox, label=left:3] {to}
              child [missing]
              child[missing]
              child { node (invoke) [cbox, label=left:1] {invoke}}
              child [missing]
              child { node [cbox] {pfail}}
            }
         }
      }
  };
%\draw[blue] (current bounding box.south west) rectangle (current bounding box.north east);
\draw[->] (invoke.south) to[out=225, in=135,distance=3cm] (to.north west);
%\draw[red] (current bounding box.south west) rectangle (current bounding box.north east);
\end{scope}
\end{tikzpicture}
\vspace*{-2cm} % The last draw command's control points extend the bounding box too far.
\end{center}
\caption{Syntax tree and flow graph for a simple procedure}
\end{figure}

The node labeled procedure main is the \textit{start node} for the
procedure; it performs any necessary initializations to establish the
execution environment for the procedure. The edge from \texttt{invoke}
to \texttt{to} is a resumption path induced by the control structure
\texttt{every}. The path from \texttt{to} to \texttt{pfail} is the
failure path for \texttt{to}. It is a forward execution path rather
than a resumption path because the compound expression (indicated by
\texttt{;}) limits backtracking out of its left-hand sub-expression.
Chapter 19 describes how to determine the edges of the flow graph for
an Icon program.

Both the standard semantics and the abstract semantics must deal with
the intermediate results of expression evaluation.  A
temporary-variable model is used because it is more convenient for
this analysis than a stack model. This decision is unrelated to the
use of a temporary-variable model in the compiler. This analysis uses
a trivial assignment of temporary variables to intermediate
results. Temporary variables are not reused. Each node that produces a
result is assigned some temporary variable \textit{ri} in the
environment. Assuming that temporary variables are assigned to the
example according to the node numbering, the \texttt{to} operation has
the effect of

\iconline{\>\>\ttit{r3} := \ttit{r4} to \ttit{r5} }

\noindent
Expressions that represent alternate computations must be assigned the
same temporary variable, as in the following example for the
subexpression \texttt{x := ("a" {\textbar} "b")}. The syntax tree
below on the left and the flow graph are shown on the right.

%-% \begin{picture}(400,180)
%-% %\put(0,0){\graphpaper{40}{18}}
%-% \put(0,0){\includegraphics[width=2.2492in,height=2.2398in]{kw/figure3-3.png}}
%-% \put(200,0){\includegraphics[width=2.4154in,height=2.3in]{kw/figure3-4.png}}
%-% \end{picture}

\begin{figure}[htb]
\begin{center}
\begin{tikzpicture} [draw,thick,font=\small\tt,
   level distance=12mm,
% define a "curved box" style
   cbox/.style={rounded rectangle, rounded rectangle arc length=90, minimum width=10mm, draw},
   every label/.style={font=\small\it}
   ]

\node (lhs) {}
    child
    { node [cbox,label=left:1] {:=}
        child { node[cbox,label=left:2] {x}}
        child[missing]
        child { node[cbox] {alt}
               child { node[cbox, label=left:3] {"a"}}
               child [label=right:lab] { node[cbox, label=left:3] (bottom) {"b"}}
              }
    };

% The flow graph part of the diagram is best drawn "bottom up"
\begin{scope}[edge from parent/.style={draw,arrows=latex-}, grow'=up]
\node(rhs) at ($(bottom.south) + (5cm,-1cm)$) {}
  child
  { node [cbox,label=left:1] {:=}
      child {
              node[cbox,label=left:3] {"a"}
              child {node[cbox, label=left:2] {x} child{}}
            }
      child[missing]
      child { node[cbox,label=left:3] {"b"} child {}}
  };
\end{scope}
%\draw (current bounding box.south west) rectangle (current bounding box.north east);
\end{tikzpicture}
\end{center}
\caption{syntax tree and flow graph for \texttt{x := ("a" {\textbar} "b")}}
\end{figure}

The \texttt{if} and \texttt{case} control structures are handled
similarly. In addition to temporary variables for intermediate
results, some generators may need additional temporary variables to
hold internal states during suspension. It is easy to devise a scheme
to allocate them where they are needed; details are not presented
here. The syntax tree is kept during abstract interpretation and used
to determine the temporary variables associated with an operation and
its operands.

The equations that determine the collecting semantics of the program
are derived directly from the standard semantics of the language. The
set of environments on an edge of the flow graph is related to the
sets of environments on edges coming into the node at the head of this
edge. This relationship is derived by applying the meaning of the node
(in the standard semantics) to each of the incoming environments.

It requires a rather complex environment to capture the full
operational semantics (and collecting semantics) of a language like
Icon. For example, the environment needs to include a representation
of the external file system.  However, later abstractions only use the
fact that the function \texttt{read} produces strings. This discussion
assumes that it is possible to represent the file system in the
environment, but does not give a representation. Other complexities of
the environment are discussed later. For the moment, examples only
show the bindings of variables to unstructured (atomic) values.

As an example of environments associated with the edges of a flow
graph, consider the assignment at the end of the following code
fragment. The comments in the if expression are assertions that are
assumed to hold at those points in the example.

\goodbreak
\begin{iconcode}
\>if x = 7 then \{\\
\>\>...\\
\>\>\# x is 7 and y is 3\\
\>\>\}\\
\>else \{\\
\>\>...\\
\>\>\# (x is null and y is 1) or (x is "abc" and y is 2)\\
\>\>\}\\
\>x := y + 2\\
\end{iconcode}

Because of the preceding \texttt{if} expression, there are two paths
reaching the assignment. The diagram below shows the flow graph and
accompanying environments for the expression; the diagram ignores the
fact that the assignment expression requires several primitive
operations to implement.

 %-% \includegraphics[width=6.0in,height=2.1in]{kw/figure3-5.png}
\begin{figure}[htb]
\begin{center}
% This tree is best drawn upside down
\begin{tikzpicture}[draw, edge from parent/.style={draw,arrows=latex-}, grow'=up,
                    thick,font=\small\tt,
                    level distance=2cm
                    ]
\node (root) {}
  child
  {
    node [rounded rectangle, draw] {x := y + 2}
    { child { edge from parent coordinate (inL)}
      child[missing]
      child { edge from parent coordinate (inR)}
    }
    edge from parent coordinate (out)
  };
\node[anchor=east] at ([xshift=-0.3cm]inL) {\{~[x=null,y=1]~,~[x="abc",y=2]~\}};
\node[anchor=west] at ([xshift=0.3cm]inR) {\{~[x=7,y=3]~\}};
\node[anchor=east] at ([xshift=-0.3cm]out) {\{~[x=3,y=1]~,~[x=4,y=2]~,~[x=5,y=3]~\}};
\end{tikzpicture}
\end{center}
\caption{A flow graph for \texttt{x := y + 2}}
\end{figure}

For a conditional expression, an incoming environment is propagated to
the path that it would cause execution to take in the standard
semantics. This requires distinguishing the paths to be taken on
failure (backtracking paths) from those to be taken on success. The
following diagram shows an example of this.

%-%{\centering  \includegraphics[width=5.3992in,height=2.2299in]{kw/figure3-6.png} \par}

\begin{figure}[htb]
\begin{center}
\begin{tikzpicture}[
    draw, edge from parent/.style={draw,arrows=-latex},
    thick,font=\small\tt, level distance=2cm,
    cbox/.style={rounded rectangle, rounded rectangle arc length=90, minimum width=10mm, draw}
   ]
\node (root) {}
  child
  {
    node [cbox] {x < y}
    { child { edge from parent coordinate (outL)}
      child[missing]
      child { edge from parent coordinate (outR)}
    }
    edge from parent coordinate (in)
  };
\node[anchor=east] at ([xshift=-0.3cm]in) {\{~[x=1,y=2]~,~[x=1,y=1]~,~[x=3,y=100]~\}};
\node[anchor=east] at ([xshift=-0.5cm,yshift=-0.2cm]outL) {\{~[x=1,y=2]~,~[x=3,y=100]~\}};
\node[anchor=east] at ([yshift=0.5cm]outL) {\it success};
\node[anchor=west] at ([xshift=0.5cm,yshift=-0.2cm]outR) {\{~[x=1,y=1]~\}};
\node[anchor=west] at ([yshift=0.5cm]outR) {\it failure};

\end{tikzpicture}
\end{center}
\caption{A flow graph for a conditional expression}
\end{figure}

In general there may be several possible backtracking paths. The
environments in the standard and collecting semantics need to include
a stack of current backtracking points and control flow information,
and the flow graph needs instructions to maintain this stack. The Icon
interpreter system described in Part I is an example of how this
information can be maintained. However, the first abstraction to the
collecting semantics eliminates the need for this information, so the
information is not presented in detail here.

\section*{Model 1: Eliminating Control Flow Information}
\addcontentsline{toc}{section}{Model 1: Eliminating Control Flow Information}

The first abstraction involves taking the union of the environments
propagated along all the failure paths from a node in the collecting
semantics and propagating that union along each of the failure paths
in the new abstraction. This abstraction eliminates the stack of
backtracking points from the environment.

A more formal definition for this model requires taking a closer look
at Icon data values, especially those values with internal
structure. In order to handle Icon data objects with pointer
semantics, an environment needs more than variable bindings. This fact
is important to type inferencing. The problem is handled by including
two components in the environment. The first is the \textit{store},
which maps variables to values. Variables include \textit{named}
variables, \textit{temporary} variables, and \textit{structure}
variables. Named variables correspond to program
identifiers. Temporary variables hold intermediate results as
discussed above. Structure variables are elements of structures such
as lists. Note that the sets of named variables and temporary
variables are each finite (based on the assumption that a program
consists of a single non-recursive procedure; as mentioned earlier,
this assumption is removed in Chapter 19), but for some
non-terminating programs, the set of structure variables may be
infinite.  \textit{Program} variables include both named variables and
structure variables but not temporary variables.

Values include atomic data values such as integers, csets, and
strings. They also include \textit{pointers} that reference objects
with pointer semantics. In addition to the values just described,
temporary variables may contain references to program variables. These
\textit{variable references} may be used by assignments to update the
store or they may be dereferenced by other operations to obtain the
values stored in the variables.

The second part of the environment is the \textit{heap}. It maps
pointers to the corresponding data objects (this differs from the heap
in the Icon implementation in that that heap also contains some data
objects that do not have pointer semantics). For simplicity, the only
data type with pointer semantics included in this discussion is the
list.  A list is a partial mapping from integers to
variables. Representing other data types with pointer semantics is
straightforward; this is discussed in Chapter 19.

The first abstraction is called Model 1. The notations envir$_{[n]}$,
store$_{[n]}$, and heap$_{[n]}$ refer to the sets of possible
environments, stores, and heaps respectively in model $n$. For
example, envir$_{[1]}$ is the set of possible environments in the
first abstraction. In the following set of definitions, $X \times Y$
is the set of ordered pairs where the first value in the pair is from
$X$ and the second value is from $Y$. $ X \rightarrow Y$ is the set of
partial functions from $X$ to $Y$. The definition of the set of
possible environments for model 1 is

\goodbreak
\begin{specialcode}{}
\>envir$_{[1]}$ = store$_{[1]}$ $\times$ heap$_{[1]}$\\
\>store$_{[1]}$ = variables $\rightarrow$ values\\
\>values       = integers $\cup$ strings $\cup$ \ldots $\cup$ pointers $\cup$ variables\\
\>heap$_{[1]}$  = pointers \textrm{${\rightarrow}$} lists,~%
                 where lists = integers $\rightarrow$ variables\\
\end{specialcode}

For example, the expression

\iconline{ \>a := ["abc"] }

\noindent creates a list of one element whose value is the string \texttt{abc}
and assigns the list to the variable \texttt{a}. Let $p_1$ be the
pointer to the list and let $v_1$ be the (anonymous) variable within
the list. The resulting environment, e $\in$ envir$_{[1]}$, might be

\goodbreak
\begin{specialcode}{}
\>e = $(s,h)$, where $s \in $ store$_{[1]}$, $h \in $ heap$_{[1]}$\\
\>$s($\texttt{a}$) = p_1$\\
\>$s(v_1) = $ "\texttt{abc}"\\
\\
\>$h(p_1) = L_1$, where $L_1 \in$ lists\\
\\
\>$L_1(1) = v_1$\\
\end{specialcode}

\noindent
If the statement

\iconline{ \>a[1] := "xyz" }

\noindent is executed, the subscripting operation dereferences \texttt{a}
producing $p_1$, then uses the heap to find $L_1$, which it applies to
1 to produce the result $v_1$. The only change in the environment at
this point is to temporary variables that are not shown. The
assignment then updates the store, producing

\goodbreak
\begin{iconcode}
\>$e_1 = (s_1 , h)$\\
\>$s_1$(\texttt{a})$ = p_1$\\
\>$s_1(v_1) =$ "xyz"\\
\end{iconcode}

\noindent
Assignment does not change the heap. On the other hand, the expression

\iconline{ \ \ put(a, "xyz") }

\noindent adds the string \texttt{xyz} to the end of the list; if it is
executed in the environment $e$, it alters the heap along with adding a
new variable to the store.

\goodbreak
\begin{specialcode}{}
\>$e_1 = (s_1 , h_1$)\\
\>$s_1($\texttt{a}$) = p_1$\\
\>$s_1(v_1) =$ "\texttt{abc}"\\
\>$s_1(v_2) =$ "\texttt{xyz}"\\
\>$h_1(p_1) = L_2$\\
\>$L_2(1) = v_1$\\
\>$L_2(2) = v_2$\\
\end{specialcode}

If a formal model were developed for the collecting semantics, it
would have an environment similar to the one in Model 1. However, it
would need a third component with which to represent the backtracking
stack.

\section*{Model 2: Decoupling Variables}
\addcontentsline{toc}{section}{Model 2: Decoupling Variables}

The next approximation to Icon semantics, Model 2, takes all the
values that a variable might have at a given program point and gathers
them together. In general, a variable may have the same value in many
environments, so this, in some sense, reduces the amount of space
required to store the information (though the space may still be
unbounded). The ``cost'' of this reduction of storage is that any
information about relationship of values between variables is lost.

Model 2 is also defined in terms of environments, stores, and heaps,
although they are different from those of Model 1.  A store in Model 2
maps sets of variables to sets of values; each resulting set contains
the values associated with the corresponding variables in environments
in Model 1. Similarly, a heap in Model 2 maps sets of pointers to sets
of lists; each of these sets contains the lists associated with the
corresponding pointers in environments in Model 1. An environment in
Model 2 contains a store and a heap, but unlike in Model 1, there is
only one of these environments associated with each program point. The
environment is constructed so that it effectively ``contains'' the
environments in the set associated with the point in Model 1.

The definition of Model 2 is

\goodbreak
\begin{specialcode}{}
\>envir$_{[2]}$  =  store$_{[2]} \times $ heap$_{[2]}$\\
\>store$_{[2]}$  =  $2^{\textrm{variables}} \rightarrow 2^{\textrm{values}}$ \\
\>heap$_{[2]}$   =  $2^{\textrm{pointers}} \rightarrow 2^{\textrm{lists}}$ \\
\end{specialcode}

In Model 1, operations produce elements from the set
\textit{values}. In Model 2, operations produce subsets of this
set. It is in this model that \texttt{read} is taken to produce the
set of all strings and that the existence of an external file system
can be ignored.

Suppose a program point is annotated with the set containing the
following two environments from Model 1.

\goodbreak
\begin{iconcode}
\> $e_1,e_2 \in$ envir$_{[1]}$\\
\> $e_1 = (s_1, h_1)$\\
\> $s_1($\texttt{x}$) = 1$\\
\> $s_1($\texttt{y}$) = p_1$\\
\> $h_1(p_1) = L_1$\\
\\
\> $e_2 = (s_2, h_2)$\\
\> $s_2($\texttt{x}$) = 2$\\
\> $s_2($\texttt{y}$) = p_1$\\
\> $h_2(p_1) = L_2$\\
\end{iconcode}

\noindent
Under Model 2 the program point is annotated with the single
environment $\hat{e} {\in}$ envir$_{[2]}$, where

\goodbreak
\begin{iconcode}
\> $\hat{e} = (\hat{s},\hat{h})$\\
\> $\hat{s}(\{$\texttt{x}\}$) = \{1,2\}$\\
\> $\hat{s}(\{$\texttt{y}\}$) = \{p_1\}$\\
\> $\hat{s}(\{$\texttt{x}, \texttt{y}\}$) = \{1, 2, p_1\}$\\
\> $\hat{h}(\{p_1\}) = \{L_1, L_2\}$\\
\end{iconcode}

\noindent
Note that a store in Model 2 is distributive over union. That is,

$\hat{s}(X \cup Y) = \hat{s}(X) \cup \hat{s}(Y)$

\noindent
so listing the result of $\hat{s}(\{$\texttt{x}, \texttt{y}$\})$ is
redundant. A heap in Model 2 also is distributive over union.

In going to Model 2 information is lost. In the last example, the fact
that \texttt{x = 1} is paired with $p_1 =L_1$ and \texttt{x = 2} is
paired with $p_1 = L_2$ is not represented in Model 2.

Just as \texttt{read} is extended to produce a set of values, so are
all other operations. These "extended" operations are then used to set
up the equations whose solution formally defines Model 2. This
extension is straightforward. For example, the result of applying a
unary operator to a set is the set obtained by applying the operator
to each of the elements in the operand. The result of applying a
binary operator to two sets is the set obtained by applying the
operator to all pairs of elements from the two operands. Operations
with more operands are treated similarly. For example

\begin{eqnarray*}
\{1, 3, 5\} + \{2, 4\} & = & \{1 + 2, 1 + 4, 3 + 2, 3 + 4, 5 + 2, 5 + 4\}\\
                       & = & \{3, 5, 5, 7, 7, 9\}\\
                       & = & \{3, 5, 7, 9\}\\
\end{eqnarray*}

The loss of information mentioned above affects the calculation of
environments in Model 2. Suppose the addition in the last example is
from

\iconline{\>z := x + y }

\noindent and that Model 1 has the following three environments at the
point before the calculation

\goodbreak
\begin{iconcode}
\>[x = 1, y = 2, z = 0]\\
\>[x = 3, y = 2, z = 0]\\
\>[x = 5, y = 4, z = 0]\\
\end{iconcode}

After the calculation the three environments will be

\goodbreak
\begin{iconcode}
\>[x = 1, y = 2, z = 3]\\
\>[x = 3, y = 2, z = 5]\\
\>[x = 5, y = 4, z = 9]\\
\end{iconcode}

If these latter three environments are translated into an environment
of Model 2, the result is

\iconline{ \>[x = \{1, 3, 5\}, y = \{2, 4\}, z = \{3, 5, 9\}] }

However, when doing the computation using the semantics of + in Model
2, the value for \texttt{z} is \texttt{\{3, 5, 7, 9\}}. The solution
to the equations in Model 2 overestimates (that is, gives a
conservative estimate for) the values obtained by computing a solution
using Model 1 and translating it into the domain of Model 2.

Consider the following code with respect to the semantics of
assignment in Model 2. (Assume that the code is executed once, so only
one list is created.)

\goodbreak
\begin{iconcode}
\>x := [10, 20]\\
\>i := if read() then 1 else 2\\
\>x[i] := 30\\
\end{iconcode}

After the first two assignments, the store maps \texttt{x} to a set containing
one pointer and maps \texttt{i} to a set containing 1 and 2. The third
assignment is not as straightforward. Its left operand evaluates to
two variables; the most that can be said about one of these variables
after the assignment is that it might have been assigned 30. If $(s, h)$
is the environment after the third assignment then

\goodbreak
\begin{specialcode}{}
\>$s(\{$\texttt{x}$\}) = \{ p_1 \}$\\
\>$s(\{$\texttt{i}$\}) = \{1, 2\}$\\
\>$s(\{v_1\}) = \{10, 30\}$\\
\>$s(\{v_2\}) = \{20, 30\}$\\
\\
\>$h(\{p_1\}) = \{L_1\}$\\
\\
\>$L_1(1) = v_1$\\
\>$L_1(2) = v_2$\\
\end{specialcode}

Clearly all assignments could be treated as \textit{weak updates}
[.pntstr.], where a weak update is an update that may or may not take
place. However, this would involve discarding too much information;
assignments would only add to the values associated with variables and
not replace the values. Therefore assignments where the left hand side
evaluates to a set containing a single variable are treated as special
cases. These are implemented as \textit{strong updates}.

\section*{Model 3: A Finite Type System}
\addcontentsline{toc}{section}{Model 3: A Finite Type System}

The environments in Model 2 can contain infinite amounts of
information, as in the program

\goodbreak
\begin{iconcode}
\>x := 1\\
\>repeat x +:= 1\\
\end{iconcode}

\noindent where the set of values associated with x in the loop
consists of all the counting numbers. Because equations in Model 2 can
involve arbitrary arithmetic, no algorithm can find the least fixed
point of an arbitrary set of these equations.

The final step is to impose a finitely representable type system on
values. A type is a (possibly infinite) set of values. The type system
presented here includes three classifications of basic types. The
first classification consists of the Icon types without pointer
semantics: integers, strings, csets, etc. The second classification
groups pointers together according to the lexical point of their
creation. This is similar to the method used to handle recursive data
structures in Jones and Muchnick [.analrcsv.]. Consider the code

\iconline{ \>every insert(x, [1 to 5]) }

If this code is executed once, five lists are created, but they are
all created at the same point in the program, so they all belong to
the same type. The intuition behind this choice of types is that
structures created at the same point in a program are likely to have
components of the same type, while structures created at different
points in a program may have components of different types.

The third classification of basic types handles variable
references. Each named variable and temporary variable is given a type
to itself. Therefore, if \texttt{a} is a named variable,
\texttt{\{a\}} is a type. Structure variables are grouped into types
according to the program point where the pointer to the structure is
created. This is not necessarily the point where the variable is
created; in the following code, a pointer to a list is created at one
program point, but variables are added to the list at different points

\goodbreak
\begin{iconcode}
\ \ x := []\\
\ \ push(x, 1)\\
\ \ push(x ,2)\\
\end{iconcode}

References to these variables are grouped into a type associated with
the program point for \texttt{[]}, not the point for the corresponding
push.

If a program contains k non-structure variables and there are n
locations where pointers can be created, then the basic types for the
program are integer, string, ..., P\TextSubscript{1}, ...,
P\TextSubscript{n}, V\TextSubscript{1}, ..., V\TextSubscript{n},
\{v\TextSubscript{1}\}, ..., \{v\TextSubscript{k}\} where
P\TextSubscript{i} is the pointer type created at location i,
V\TextSubscript{i} is the variable type associated with
P\TextSubscript{i}, and v\TextSubscript{i} is a named variable or a
temporary variable. Because programs are lexically finite they each
have a finite number of basic types. The set of all types for a
program is the smallest set that is closed under union and contains
the empty set along with the basic types:

\begin{specialcode}{}
\>types = \{\{\}, integers, strings,...,~%
    (integers ${\cup}$ \ strings),...,~%
    (integers ${\cup}$ strings ${\cup}$ ... ${\cup}$ \{v\TextSubscript{k}\})\}
\end{specialcode}

Model 3 replaces the arbitrary sets of values of Model 2 by
types. This replacement reduces the precision of the information, but
allows for a finite representation and allows the information to be
computed in finite time.

In Model 3, both the store and the heap map types to types. This store
is referred to as the \textit{type store}. The domain of type store is
\textit{variable types}, that is, those types whose only values are
variable references.  Similarly, the domain of the heap is
\textit{pointer types}. Its range is the set types containing only
structure variables. A set of values from Model 2 is converted to a
type in Model 3 by mapping that set to the smallest type containing
it. For example, the set

\iconline{ \>\{1, 4, 5, "23", "0"\} }

\noindent is mapped to

\begin{specialcode}{}
\>integer ${\cup}$ string\\
\end{specialcode}

\noindent
The definition of envir$_{[3]}$ is

\goodbreak
\begin{specialcode}{}
\>envir$_{[3]} = $ store$_{[3]} \times $ heap$_{[3]}$\\
\>store$_{[3]} = $ variable-types $\rightarrow $ types\\
\>heap$_{[3]} = $ pointer-types $\rightarrow $ structure-variable-types\\
\>types ${\subseteq}$ 2\textsuperscript{values}\\
\>variable-types ${\subseteq}$ types\\
\>structure-variable-types ${\subseteq}$ variable-types\\
\>pointer-types ${\subseteq}$ types\\
\end{specialcode}

There is exactly one variable type for each pointer type in this
model. The heap simply consists of this one-to-one mapping; the heap
is of the form

{\ttfamily\mdseries
\ \ \ $h$( P\TextSubscript{i} ) = V\TextSubscript{i}}

This mapping is invariant over a given program. Therefore, the type
equations for a program can be defined over store$_{[3]}$ rather than
envir$_{[3]}$ with the heap embedded within the type equations.

Suppose an environment from Model 2 is

\goodbreak
\begin{specialcode}{}
\> $e \in $ envir$_{[2]}$\\
\> $e = (s, h)$\\
\\
\> $s(\{$\texttt{a}$\}) = \{ p_1 , p_2\}$\\
\> $s(\{v_1\}) = \{1, 2\}$\\
\> $s(\{v_2\}) = \{1\}$\\
\> $s(\{v_3\}) = \{12.03\}$\\
\\
\> $h(\{p_1\}) = \{L_1, L_2\}$\\
\> $h(\{p_2\}) = \{L_3\}$\\
\\
\> $L_1(1) = v_1$\\
\\
\> $L_2(1) = v_1$\\
\> $L_2(2) = v_2$\\
\\
\> $L_3(1) = v_3$\\
\end{specialcode}

Suppose the pointers p\TextSubscript{1} and p\TextSubscript{2} are
both created at program point 1. Then the associated pointer type is
P\TextSubscript{1} and the associated variable type is
V\TextSubscript{1}. The corresponding environment in Model 3 is

\goodbreak
\begin{specialcode}{}
\>$\hat{e} \in $ envir$_{[3]}$\\
\>$\hat{e} = (\hat{s},\hat{h})$\\

\>$\hat{s}(\{$\texttt{a}$\}) =  $ P$_1$\\
\>$\hat{s}($V$_1) = $ integer  $\cup$ real\\

\>$\hat{h}($P$_1) = $ V$_1$\\
\end{specialcode}

The collecting semantics of a program establishes a set of (possibly)
recursive equations between the sets of environments on the edges of
the program's flow graph. The collecting semantics of the program is
the least fixed point of these equations in which the set on the edge
entering the start state contains all possible initial environments.
Similarly, type inferencing establishes a set of recursive equations
between the type stores on the edges of the flow graph. The least
fixed point of these type inferencing equations is computable using
iterative methods. This is discussed in Chapter 19. The fact that
these equations have solutions is due to the fact that the equations
in the collecting semantics have a solution and the fact the each
abstraction maintains the ``structure'' of the problem, simply
discarding some details.

Chapter 19 also extends type inferencing to handle the entire Icon
language. Chapter 22 uses the information from type inferencing to
optimize the generated code.


\doublespacing

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/static/adjuster_workflow.pdf}
	\caption[Batch adjustment justification and steps]{\textbf{Batch adjustment justification and steps.}
	\begin{enumerate*}[(a)]
		\item When measurements are collected from a sample ($X$), systemic effects ($H$) may affect the measurements.
		\item If data from the same sample $X$ is measured under two different conditions, $H_1$ and $H_2$, we may obtain slightly different measurements.
		\item In order to normalize batches of data relative to one another, we first estimate the effect of the hidden variables based on differences in measurements between batches.
		\item Second, we remove the estimated effects in order to normalize the batches relative to one another.
	\end{enumerate*}
	}
	\label{fig:workflow}
\end{figure}

\section{Methods} \label{sec:methods}

An open-source implementation of Confounded is publicly available at \url{https://github.com/jdayton3/Confounded}. Materials for generating this manuscript, including scripts and data, are available at \url{https://github.com/jdayton3/confounded-paper} and \url{https://osf.io/b76ch/}, respectively.
%TODO: We may be able to remove the OSF repo now...but let's wait...

\subsection{Neural network structure}

We developed an adversarial autoencoder network to model and remove confounding effects.
We structured this network in two parts: a variational autoencoder \cite{louizos_variational_2015} to replicate the input (expression) data and a discriminator to detect remaining confounding effects in the autoencoder's output.
By penalizing the autoencoder for the discriminator's success, the autoencoder subnetwork learns over the course of training to output the expression data with confounding effects minimized.
We implemented the neural network in TensorFlow 1.11.0 \cite{abadi_tensorflow_2015} with Python 3.6 \cite{python_software_foundation_python_2019}.
All layers in the network are fully connected, and all activation functions are Rectified Linear Units (ReLUs) \cite{agarap_deep_2018}, except the final layers in the autoencoder and the discriminator, which use the sigmoid function.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/static/network.pdf}
	\caption[Network architecture of Confounded]{\textbf{Network architecture of Confounded.}
	Data with batch effects (represented by different colors) are input into an autoencoder.
	The output of the autoencoder is classified by a discriminator network based on batch.
	The autoencoder is then penalized based on the success of the discriminator.
	Over time, the autoencoder learns to output a faithful representation of the data without signal due to batch.}
	\label{fig:network}
\end{figure}

\subsubsection{Autoencoder}

%% ToDo: This is where we would have to change the default values in the confounded :
%% for DR Piccolo to Change :
%%
%%
We implemented the variational autoencoder architecture \citep{louizos_variational_2015} described by \citet[Chapter 15]{geron_hands-machine_2017}.
This network has 2 hidden encoding layers and 2 decoding layers, each of size 500. % Is this sample size? I'm not sure where the 500 comes from
The code size is 200. %was: 20.
Each hidden layer is activated with the Exponential Linear Unit (ELU) function \citep{clevert_fast_2015} and trained with the Adam optimizer \citep{kingma_adam_2014} on reconstruction loss (sigmoid cross entropy) combined with latent loss (KL divergence \citep{kullback_information_1951}).

\subsubsection{Discriminator}

We trained the discriminator to predict the original batches based on the autoencoder's output.
The discriminator subnetwork consists of an input layer; four fully connected hidden layers of sizes 1024, 512, 512, and 128, respectively; and an output layer sized based on the number of batches (i.e. if there are 4 distinct batches in the data, the output layer will be size 4).
To combat overfitting and improve training, we also added 50\%-probability dropout \cite{srivastava_dropout_2014} (which prevents overfitting by dropping a random subset of layer inputs in each training iteration) and batch normalization \cite{ioffe_batch_2015} (which helps training by smoothing out the optimization landscape \citep{santurkar_how_2018}) to each layer of the discriminator.
These additions seem to reduce overfitting in the discriminator.

\subsubsection{Loss functions}

To evaluate the network's performance, we used three loss functions.
First, we calculated the autoencoder's loss ($L_A$) by summing the reconstruction loss (sigmoid cross entropy between the autoencoder's input and output) and the latent loss (Kullback-Leibler (KL) divergence \cite{kullback_information_1951} of the code layer).
Second, we calculated the discriminator's loss ($L_D$) as sigmoid cross entropy between its output and a one-hot encoding of the samples' batch labels.
Finally, we trained the autoencoder layers on a combination of the two previous losses,

\begin{equation}
	\label{dual_loss}
	L_{dual} = L_A - \lambda{}(L_D)
\end{equation}

The $\lambda$ value represents a tradeoff parameter for tuning the network's tendency to more faithfully replicate the input or to more completely remove confounding effects.
A higher $\lambda$ value indicates that the network should remove confounding effects more aggressively, whereas a lower value indicates that the network should instead favor faithfully reconstructing the input data.
We did not optimize $L_A$ directly; instead we trained the autoencoder by optimizing $L_{dual}$.

\subsubsection{Training}
%% also another place to change the default values
In all cases, we trained the network using the Adam Optimizer \citep{kingma_adam_2014} with a training rate of 0.0001 for 10,000 iterations on mini-batches of size 256. % was: 100.
In each iteration, we optimized on both $L_D$ and $L_{dual}$.
When optimizing $L_D$ (i.e. training the discriminator), we froze the autoencoder's weights, and vice versa.
We trained the network on a 2017 Dell XPS 15 9560 with an 8-core Intel i7-7700HQ central processing unit and 16 GB of random access memory.
For each dataset, training took roughly 30 minutes to complete, including time to load the input into memory and save the output to disk. % We should probably re-time it to make sure this is still accurate

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/training_loss.pdf}
	\caption[Autoencoder and discriminator loss]{\textbf{Autoencoder and discriminator loss} over time for one run of Confounded on the MNIST dataset (which we used to validate the network).
	Over the course of training, the autoencoder more faithfully replicates the input data.
	The autoencoder also seems to introduce noise (see the red dashed line near iteration 3100) in response to the discriminator's slight improvements.}
	\label{fig:training_loss}
\end{figure}

\subsection{Comparison to other methods}

We compared Confounded against two other methods that can be used for batch correction. The first was a simple scaling adjuster,
which we implemented in the R programming language (version 3.6.0) \citep{r_core_team_r_2014} using RStudio version 1.2.1194 \citep{rstudio_team_rstudio_2018}.
It adjusts the data by linearly expanding or contracting the values for each variable so that all variables have the same range across the batches.
We also compared against ComBat \citep{johnson_adjusting_2007} using an implementation from the sva package \citep{leek_sva_2017} with some modifications to allow it to work on columns without variance.

We initially intended to test against the SVA \citep{leek_capturing_2007} method but concluded that SVA is more suited for producing surrogate variables for further statistical research rather than removing those variables from the data.
A number of other methods for batch adjustment are available. Some of these use deep neural networks \citep{shaham_removal_2017,shaham_batch_2018,upadhyay_removal_2019}, while others do not\citep{espin-perez_comparison_2018}.
Unfortunately, the datasets we tested did not meet the requirements of these methods (e.g., two equal-sized batches, large sample sizes).

% ----- Brief explanation of the other methods -----
% 1.
% The Shaham lab has two method that we reference up here^, one from 2017 , and one from 2018.
% The 2017 one is the distribution-matching residual net.
% It arbitrarily designates one batch as the "target" and the other batch as the "source".
% It calculates MMD with 3 kernels and averages them, and uses them as the loss function.
% Over training, the source's distribution (according to MMD) should be aligned to the target's distribution.
% In our testing, this was SUPER memory-intensive.
% I think this is the practical reason they reduced their dataset to the first 37 principal components, but their explanation in the paper is that the people who generated the data said the first 37 components contain most of the variation.
% 2.
% The 2018 Shaham paper is about using an adversarial autoencoder to make a batch-free encoding of expression data.
% They have a variational autoencoder with a shared encoder, but separate decoders for each of the 2 batches.
% They say that the decoders output data that has batch info in it but that the encoding layer doesn't have batch info.
% Their encoding layer is only size 15-20, so this is essentially a dimensionality reduction technique--the "batch free encoding" they're producing doesn't have values for each gene.
% 3.
% I don't know how worth-mentioning the Upadhyay paper is, but they basically replicate the DMResNet paper and replace the MMD loss with a discriminator loss like we do.
% They use the Shaham datasets, and I think their code, which is either not publicly available or is just really hard to find, probably has the same caveats as the Shaham code.
% Most particularly, they only account for 2 batches.
% I might have misunderstood their approach though because the English is a little difficult to interpret.

\subsection{Datasets}

We tested each method on four datasets that varied in size and the type of data measured.

\begin{table}
	\centering
	\caption[Dataset information]{
		\textbf{Information} about each dataset used to compare algorithm performance.
	}
	\input{tables/static/datasets.tex}
	\label{tab:datasets}
\end{table}

%% I keep forgetting whether or not we are leaving the MNIST dataset and just not doing anything, or if we are deleting it out.
%% maybe put in the simulated expression data and information here

%\subsection{Simulated Expression}
%We simulated a balanced two class two batch dataset that contains 200 samples of 1000 features each that allowed us to simulate a large dataset to optimize the parameters of the confounded method.
%Initially, we used make_classification from the sklearn classification library \url{}, but combat always outperformed confounded because the clustering made by the make_classification algorithm made it possible for linear models to accurately classify the true classes.
%As we moved away from clustering and added nonlinear patterns to a randomly generated normal distribution set, we saw confounded begin to perform better than combat.
%When the true class and batch signals are nonlinear and small, combat is not as able to acuratly classify the classes or remove the batches, but confounded is able to identify the classes and effectively remove the batches.

\subsubsection{MNIST}

Initially, we used an image dataset so we could visually assess how well the batch-correction methods preserve true signal (in this case, the shape and definition of each handwritten digit image).
The MNIST dataset \cite{lecun_mnist_nodate} contains images of handwritten digits that are size-normalized and centered.
We limited our analysis to the 10,000 test images.
We flattened each 28-by-28 image into a 1D vector of size 784 and put each in a CSV file along with the accompanying digit (class) information.
Although convolutional layers are typically used when working with image data, we used fully connected layers to maintain consistency with what would be used on gene-expression data.
However, the autoencoder was still able to represent spatial relationships without explicitly defining them in the model.

\paragraph{Synthetic batch effects}

Because there is no batch information in the MNIST digits dataset, we simulated nonlinear confonding effects.
We applied these effects by iteratively realizing vectors of normally distributed values, multiplying and adding these vectors to the ``expression'' vectors, and applying nonlinearity to the adjusted vector.[@Be more specific in the last part. How did you apply nonlinearity? Or maybe this last part of the sentence is unnecessary.@]
We split the images into two balanced batches (5,000 images for each batch) and included the same number of each digit in each batch.
We applied the same random vectors to each image in a given batch.
Finally, we added random noise to each image to prevent images in a batch from being overly similar to each other.

\subsubsection{Bladderbatch data}

The bladderbatch dataset is a microarray gene-expression dataset from a study of patients with bladder cancer \cite{dyrskjot_gene_2004}.
It is available as an R package \cite{leek_bladderbatch_2017}.
It contains expression values for 57 tissue samples with and without bladder cancer across 5 unbalanced batches.
The dataset has a cancer status (cancerous vs. normal tissue) variable, which we used as the ``true class'' in our validations; it also has a batch variable.
Because bladderbatch is small relative to datasets typically used for deep learning, it helped us to evaluate whether our network overfits on gene-expression datasets of a size typical of applied research.

\subsubsection{Prostate cancer data}

The GSE37199 dataset was profiled using Affymetrix microarrays from patients with advanced castration-resistant prostate cancer \cite{olmos_prognostic_2012}.
We used a curated version of this dataset from \url{http://doi.org/10.17605/OSF.IO/SSK3T}\cite{golightly_curated_2018}.
It contains expression values for 93 tissue samples categorized as either ``Advanced castration resistant'' or ``good prognosis.''
We used these categories for the ``true class'' labels.
This dataset has two types of batch variables---``plate'' and ``centre''---which indicate the microarray plate and research lab that processed the data, respectively.
We adjusted against the ``plate'' variable because it is more granular in nature (with counts of $\{43, 50\}$).

\subsubsection{Pan-cancer data}

The Cancer Genome Atlas (TCGA) produced RNA-Sequencing data for tumors across many cancer types \cite{the_cancer_genome_atlas_research_network_cancer_2013}.
In a previous study \cite{dayton_classifying_2017-1}, we classified this dataset based on the presence or absence of mutations in several known cancer genes.
In that study, we observed that cancer type had a strong confounding effect on our analysis, in which sought to identify patterns spanning many cancer types. We adjusted for this effect using ComBat.
However, we found that a strong nonlinear signal could still be identified by the Random Forests algorithm after adjustment.
Here, we used the same version of TCGA dataset (available at \url{https://osf.io/7xjdn/}),
which has expression values for 9,365 samples across 25 cancer types.
%TODO: Change this to 8744 samples (after averaging across replicates and limiting to patients that had all types of data), 23,386 genes and 24 cancer types?
%TODO: Mention that we used all genes rather than 300?

\subsection{Statistics and metrics}

\subsubsection{Mean squared error}

Mean squared error (MSE) is a measure of how much two vectors or matrices deviate from one another.
It is commonly used to quantify loss when the objective (as with autoencoders) is to minimize the difference between input and output values.

\subsubsection{Maximum mean discrepancy} \label{section:mmd}

In an earlier paper, \citet{shaham_removal_2017} used neural networks to remove batch effects.
%WAS: In a recent paper, \citet{shaham_removal_2017} used neural networks to remove batch effects.
Instead of constraining the autoencoder to remove batch effects based on a discriminator, these researchers trained their network to minimize maximum mean discrepancy (MMD) between batches in an embedded layer of their network.
We calculated MMD using the same formula to determine whether batches appear to come from the same distribution after adjustment.
We used the Gaussian kernel as implemented in \texttt{sklearn.metrics.pairwise.rbf\_kernel} \cite{pedregosa_scikit-learn_2011}.
In cases where there were more than two batches, we averaged all pairwise MMD values to calculate an overall MMD.

\subsubsection{Classification accuracy}

To determine
a) whether batch could still be identified post-adjustment and
b) how well class-related signal was maintained after adjustment,
we used classification algorithms to predict either batch or the ``true class`` labels based on the expression data and used classification accuracy as a metric.
Table \ref{tab:datasets} details which columns were used for these analyses.

%TODO: Update the algorithms listed here.
We used four classification algorithms from the scikit-learn (version 0.19.1) library \cite{pedregosa_scikit-learn_2011}: Random Forests \citep{tin_kam_ho_random_1995}, k-Nearest Neighbors \citep{fix_discriminatory_1951}, and Support Vector Machines (SVM) \citep{cortes_support-vector_1995} with a radial-basis kernel.
%We used four classification algorithms from the scikit-learn (version 0.19.1) library \cite{pedregosa_scikit-learn_2011}: Naive Bayes \citep{maron_automatic_1961}, Random Forests \citep{tin_kam_ho_random_1995}, k-Nearest Neighbors \citep{fix_discriminatory_1951}, and Support Vector Machines (SVM) \citep{cortes_support-vector_1995} with a radial-basis kernel.
In most cases, we used the default hyperparameters, with the exceptions of using Random Forests with \texttt{n\_estimators=10} and using SVM with \texttt{kernel="rbf"}.
[@The defaults in version 0.19.1 of sklearn are n\_estimators=10 for RF and kernel="rbf" for SVM. Did you use these defaults or something different?@] %we used those parameters

We calculated the average classification accuracy across three cross-validation folds, repeated ten times.
%We calculated the average classification accuracy across four cross-validation folds, repeated three times.
We interpreted lower accuracy for batch classification to indicate that batch signal was removed more effectively.
We interpreted higher true-class accuracy to indicate that biologically relevant signal was retained during the batch-adjustment process.
Therefore, given output data from an ideal batch adjuster, batch classification accuracy would be no better than random chance, and true-class accuracy would be equal to what would be obtained for the unadjusted data.

\section{Results} \label{sec:results}

In this study, we created Confounded, an adversarial, variational autoencoder, neural network, to remove nonlinear confounding effects from gene-expression data that may not be accounted for by linear methods.
We compared Confounded to a scaling method and to ComBat \citep{johnson_adjusting_2007} and used qualitative and quantitative techniques to compare the performance of these methods.
Overall, the scaling method performed consistently worse than the other methods;
ComBat and Confounded each excelled in different scenarios, which we describe below.

\subsection{Confounded removes nonlinear confounding effects that other adjusters miss}

[@I also think we should have t-SNE graphs for all three datasets. - Do you think this should be in the main text or just in the supplement? - The supplement.@]

[@Please structure this so that the MNIST results are described first. I think the bladderbatch results should come next because that is the closest thing we have to a reference set for gene expression. Then GSE37199. Then pan-cancer. Please restructure the results and wording to reflect this order. It's OK to mix and match. But the first set of results should be in this order.@]

We used principal component analysis (PCA) and t-distributed Stochastic Neighbor Embedding (t-SNE) \citep{maaten_visualizing_2008} plots to visualize the data before and after batch adjustment.
Before any adjustment, batches (microarray plates) in the GSE37199 dataset showed distinct expression patterns for each batch.
After adjustment with each of the three methods, the batches were less distinct (see \figurename{s} \ref{fig:pca} and \ref{fig:tsne}).
In the PCA plot, Confounded appears to maintain a similar distribution to the unadjusted data, indicating that the underlying distribution has been faithfully reproduced by the networks.
The t-SNE plot shows that the data post-adjustment by Confounded and ComBat appear to cluster less tightly by batch than the unadjusted and scale-adjusted data.
This may indicate an effective removal of nonlinear effects in both cases.
%However, previous research has shown that these plots are not completely trustworthy in representing nonlinear effects \cite{dayton_classifying_2017-1}.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/pca.pdf}
	\caption[Principal components analysis (PCA)]{\textbf{Principal components analysis} (PCA) of the GSE37199 dataset before and after batch adjustment with various adjusters.
	None of the datasets appear to be linearly separable.
	Confounded appears to maintain the same distribution of data overall as the unadjusted data while perhaps aligning the batches' distributions.}
	\label{fig:pca}
\end{figure}
\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/tsne.pdf}
	\caption[T-distributed Stochastic Neighbor Embedding (t-SNE)]{\textbf{T-distributed Stochastic Neighbor Embedding (t-SNE)} plot for the GSE37199 dataset before and after adjustment with several algorithms.
	The data seem to cluster less by batch for both Confounded and ComBat, indicating that both adjusters may be removing nonlinear effects in this dataset.}
	\label{fig:tsne}
\end{figure}

%%%To compare Confounded against the other batch adjustment methods, we compared PCA and t-SNE plots along with MSE, MMD, and classifier batch prediction accuracy (using various classifiers).
Confounded shows mixed success with the MSE and MMD metrics.
With MSE, Confounded outperformed the scale adjuster in 3 of the 4 datasets but scored drastically worse on the MNIST dataset, with scores listed in Table \ref{tab:mse} (see also \figurename{} \ref{fig:mse}).
With MMD, Confounded outperformed the scale adjuster again in 3 of the 4 datasets and tied the scale adjuster on the TCGA dataset, with scores listed in Table \ref{tab:mmd} (see also \figurename{} \ref{fig:mmd}).
With both metrics, Confounded consistently performed somewhat worse than ComBat.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/mse.pdf}
	\caption[Mean squared error (MSE)]{
		\textbf{Mean squared error} (MSE) between the data prior to and after adjustment with various algorithms.
		Lower MSE represents that the adjuster has more faithfully reproduced the input data.
		MSE for unadjusted data will always be 0 because the input data is identical to the output data.
		Confounded usually performs better than the scale adjuster and somewhat worse than ComBat when measuring MSE.
		See also Table \ref{tab:mse}.
	}
	\label{fig:mse}
\end{figure}
\begin{table}
	\input{tables/static/mse.tex}
	\caption[Mean squared error (MSE)]{
		\textbf{Mean squared error (MSE)} of the unadjusted input data compared to the data output by the given adjusters.
		Lower MSE indicates that the output has changed less from the input.
		See also \figurename{} \ref{fig:mse}.
	}
	\label{tab:mse}
\end{table}
\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/mmd.pdf}
	\caption[Maximum mean discrepancy (MMD)]{
		\textbf{Maximum mean discrepancy} (MMD) between different batches.
		Lower MMD indicates that the distributions of the different batches are more similar.
		In cases with more than two batches, MMD is computed pairwise between each batch and averaged.
		In each case, Confounded usually performs better than the scale adjuster and somewhat worse than ComBat when measuring MSE.
		See also Table \ref{tab:mmd}.
	}
	\label{fig:mmd}
\end{figure}
\begin{table}
	\centering
	\input{tables/static/mmd.tex}
	\caption[Maximum mean discrepancy (MMD)]{
		\textbf{Maximum mean discrepancy} (MMD) comparing the distributions of the batches to each other after a given adjustment.
		Lower MMD indicates that the distributions of the different batches are more similar.
		In cases with more than two batches, MMD is computed pairwise between each batch and averaged.
		See also \figurename{} \ref{fig:mmd}.
	}
	\label{tab:mmd}
\end{table}

We would expect that after batch adjustment by an ideal adjuster, batch would no longer be detectable by any machine learning classifier.
Using the batch classification accuracy metric, Confounded seems to outperform other adjusters on larger datasets, whereas ComBat and Confounded seem to perform about the same on smaller datasets (see \figurename{} \ref{fig:batch}).
With both the bladderbatch and GSE37199 datasets, batch classification accuracy decreases well below baseline after batch adjustment with ComBat for all classifiers we tested (see Table \ref{tab:batch}).
%Interestingly, batch accuracy also decreases drastically for the MNIST and TCGA datasets, but only for the Naive Bayes classifier.
%This may be due to two factors: both ComBat and Naive Bayes use Bayesian methods, so ComBat may specifically remove the effects that Naive Bayes identifies; and Naive Bayes does not find patterns based on interactions between variables.
%Although Naive Bayes is no longer able to identify confounding effects in the data after ComBat-adjustment, Random Forests (which does use interactions between variables) still has a very high accuracy for MNIST and an increased accuracy for TCGA.
In contrast, after adjustment by Confounded, the Random Forests algorithm's accuracy decreases more than with any other adjuster for both the MNIST and TCGA datasets.
This indicates that while ComBat's performance may work at least as well as Confounded for smaller expression datasets, Confounded may work better with larger datasets.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/batch_accuracy.pdf}
	\caption[Batch classification accuracy]{\textbf{Batch classification accuracy} from 3-fold cross-validation repeated 10 times for several classifiers.
	Lower batch accuracy indicates that more batch-related signal has been removed and therefore indicates better performance.
	Confounded's performance is similar to ComBat's for the smaller datasets and is improved for the larger datasets.
	See also Table \ref{tab:batch}.}
	\label{fig:batch}
\end{figure}
\begin{table}
	\centering
	\input{tables/static/batch.tex}
	\caption[Batch classification accuracy]{
		\textbf{Batch classification accuracy} for several datasets and adjusters.
		The ideal batch adjuster would completely remove all signal due to batch and would therefore \textit{decrease} batch classification accuracy to around the baseline for all classifiers.
		See also \figurename{} \ref{fig:batch}.
	}
	\label{tab:batch}
\end{table}

With the larger datasets in particular, Confounded outperforms the other adjusters.
On the MNIST dataset, Random Forests is able to detect batch with perfect or near-perfect accuracy after adjustment with the scale adjuster and ComBat, but the highest batch classification accuracy after adjustment by Confounded is Naive Bayes, with an accuracy of 68.8\%.
With TCGA, both the scale adjuster and ComBat drastically increase Random Forests' batch classification accuracy from 95.0\% to 98.5\% and 99.9\% respectively, whereas Confounded decreases the accuracy to 77.8\%.
%With TCGA, both the scale adjuster and ComBat drastically increase Random Forests' batch classification accuracy from 87.6\% to 96.3\% and 97.1\% respectively, whereas Confounded decreases the accuracy to 8.8\%.

\subsection{Class-related signal is still detectable after adjustment by Confounded}

%What is the true class information for Bladderbatch?
With the smaller datasets, Confounded seems to keep true class information roughly as well as ComBat, (with Random Forests, Bladderbatch: 74.3 for ComBat\% and 72.1\% for Confounded, GSE37199: 68.7\% for ComBat and 68.8\% for Confounded; see \figurename{} \ref{fig:true_class} and Table \ref{tab:true_class}).
%With the smaller datasets, Confounded seems to keep true class information roughly as well as ComBat, (with Random Forests, Bladderbatch: 74.3 for ComBat\% and 72.1\% for Confounded, GSE37199: 60.4\% for ComBat and 69.0\% for Confounded; see \figurename{} \ref{fig:true_class} and Table \ref{tab:true_class}).
For the Bladderbatch dataset, true class accuracy is much lower after adjusting with any algorithm, indicating that cancer status and batch may not be independent.

With the larger datasets, Confounded's true class accuracy consistently decreases below the accuracy of other adjusters.
%%This isn't accurate anymore. Our fine tuning made it better.
%%This may be a good place to include simulated expression results.
The Confounded's accuracy with Random Forest on our simulated dataset is still much higher than the baseline(70.5\% vs. 50.0\%). %Not sure if that's the right wording
A look at the TCGA data before and after adjustment shows that even with larger datasets, the accuracy of Confounded never decreases below the baseline (72.6\% vs. 69.8\%).
% compare TCGA datasets here to show that the size of datasets affects the performance.
%To compare the effect that the size of the data has on Confounded in relation to the other methods, we created two sub-datasets from the TCGA data, a medium and a small and compared their performance. On Random Forest, the ability for machine learning to identify smaples with TP53_Mutated genes as the data size increased changed in Scaled: 71.2 \% to 71.9\% to 80.1\%, in Combat: 71.0\% to 75.2\% to 81.0\%, and Confounded: 67.9\% to 71.0\% to 72.6\%.
%The ability for machine learning classification algorithms to detect the CancerType also increased with increase in size of the dataset. The Random Forest accuracy increased in each adjuster with Scaled: 7.2\% to 89.0\% to 98.5\%, in Combat: 96.3\% to 99.9\% to 99.9\%, and Confounded: 65.8\% to 66.5\% to 78.0\%.
%Though the ability for Confounded to replicate the origional expression data is a little lower than that of the other adjusters, the ability for the Confounded to remove the patterns that allow machine learning to detect CancerType is sgnificantly better than that of the other adjusters which have very high, and sometimes higher accuracy after adjustment than before any kind of adjustment is made. See table?
%A look at the MNIST digits before and after adjustment (see \figurename{} \ref{fig:mnist}) shows that Confounded's output is often blurry, as is common with the output of variational autoencoders \cite{hou_deep_2016}.
%With MNIST, Confounded's accuracy with Random Forests is still much higher than baseline (84.8\% vs. 11.4\%), but with TCGA, the accuracy decreases below baseline (66.5\% vs. 69.8\%) while the other adjusters' accuracies remain above baseline.
%However, the particular set of parameters that we used in Confounded are likely not optimal for every dataset.
Additional tuning may improve the performance metrics.

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/true_class_accuracy.pdf}
	\caption[True class classification accuracy]{\textbf{True class classification accuracy} for several datasets and adjusters with 4-fold cross-validation repeated 3 times. A higher accuracy after adjustment is desired because it represents that the adjuster has not destroyed the true class signal.
	See also Table \ref{tab:true_class}.}
	\label{fig:true_class}
\end{figure}
\begin{table}
	\centering
	\input{tables/static/true_class.tex}
	\caption[True class classification accuracy]{
		\textbf{True class classification accuracy} for several datasets and adjusters.
		After adjustment by the ideal batch adjuster, all true class signal should be preserved, and all classifiers should therefore have the same accuracy in predicting true class before and after adjustment.
		See also \figurename{} \ref{fig:true_class}.
	}
	\label{tab:true_class}
\end{table}

\begin{figure}
	\centering
	\includegraphics[width=\columnwidth]{figures/final/mnist.png}
	\caption[MNIST handwritten digits]{\textbf{MNIST handwritten digits}
	\begin{enumerate*}[(a)]
		\item before any adjustment,
		\item with artificial noise added,
		\item adjusted for noise by the scale adjuster,
		\item adjusted for noise by ComBat, and
		\item adjusted for noise by Confounded.
	\end{enumerate*}
	Although Confounded seems to remove more noise from the background, it struggles in some cases to accurately replicate the input data.}
	\label{fig:mnist}
\end{figure}

\section{Discussion} \label{sec:discussion}

\textbf{Why should I use a neural network for batch adjustment?}
The process of measuring data typically leaves confounding effects, especially when the data or measurement process involves living things.
Imagine an experiment where two researchers measure the length of caterpillars.
If one researcher is left-eye dominant and the other is right-eye dominant, they may each see lengths as being a millimeter different.
If both researchers use the same wooden ruler at different relative humidities, the measurements may come out differently due to the wood swelling or due to the caterpillars reactions to humidity.
These same types of problems exist when measuring expression levels, but to more of an extreme---in addition to the target values and measuring equipment changing slightly due to systemic factors, the target values also change in response to each other.
Each of the 20,000 transcript levels may influence or be influenced by other transcript levels.
These cascading network-like effects are extremely likely to have nonlinear components, and our research shows that these nonlinear effects do indeed exist in gene expression data, thus rendering linear batch correction methods insufficient.
Therefore, some form of nonlinear adjustment must be used in order to correct for real-world confounding effects.
%It may just be me, but this senstence is confusing because of the length and the number of commas
Rather than individually model each of infinitely many possible nonlinear interactions to see which represent confounders, we can use a neural network to both approximate and remove the confounders, since neural networks are proven to be universal function approximators \citep{csaji_approximation_2001}.

\textbf{How can I tell how well batch adjustment worked?}
Although the metrics and figures that have been used in the past to validate batch adjustment (such as PCA, MSE, and MMD) represent how well linear effects have been removed, they cannot completely display whether two batches are distinguishable from one another.
Machine learning algorithms are designed specifically to tease out patterns in data that may distinguish one group from another.
We suggest to users of batch correction software that they use machine learning classification accuracy before and after correction in order to determine the degree of batch removal.
We also suggest to researchers in the field of batch correction that classification accuracy be used as a metric in validating their software.
Specifically, the Random Forests algorithm \citep{tin_kam_ho_random_1995} seems to work very well and runs relatively quickly on gene expression data.

\textbf{Which batch adjuster should I use?}
In our testing, ComBat did very well with small ($n < 100$) datasets, even with removing any identifiable nonlinear effects.
However, Confounded outperformed ComBat on the larger datasets according to the batch classification accuracy metric.
In addition to dataset size, researchers selecting a batch adjustment algorithm should consider how important it is for them to accurately replicate their input data.
Such researchers can adjust Confounded's $\lambda$ parameter in order to balance the tradeoff of removing batch and matching the inputs.

\textbf{What limitations does Confounded have?}
\begin{enumerate*}[(a)]
	\item Confounded uses a variational autoencoder, which are known for often outputting a blurry version of the input data (as can be seen in \figurename{} \ref{fig:mnist}).
	However, work in 2016 has identified modifications that may be made to the basic VAE structure to make output images sharper and more realistic \citep{hou_deep_2016}
%	However, recent work has identified modifications that may be made to the basic VAE structure to make output images sharper and more realistic \citep{hou_deep_2016}.
	Similar research with variational autoencoders and gene expression data may yield improved reconstruction losses and decrease the blurring effect.
	\item Confounded takes a long time to run in comparison with ComBat and other linear adjusters.
	Although we acknowledge this as a downfall of many types of machine learning and of neural networks in particular, we believe that 30-60 minutes is a reasonable amount of time for a step that will be run only once per pipeline and that can greatly improve data quality.
	\item It can be difficult to identify the optimal network structure and parameter set for a neural network.
	Though this is the case for many applications of neural networks, we feel that Confounded's default structure worked well in our testing and that it will suffice for most batch correction applications.
	% I wrote that in simulated expression that we optimized the parameters, so we could say that we optimized parameters that worked well across several different datasets and should be good for most applications, but can further be optimized to perform better on individual datasets.
	\item Neural networks usually perform better when given large amounts of data, and traditional batch datasets typically have very few samples.
	We did find that ComBat may outperform Confounded on many smaller, more traditional datasets, but that Confounded performs better on larger datasets.
	However, Confounded did perform reasonably well with the smaller datasets and appeared to avoid overfitting.
	In cases where ComBat is unable to completely remove confounding effects in a small dataset, Confounded may be a viable replacement method.
\end{enumerate*}

\textbf{What else might Confounded be used for?}
At its root, batch correction is a data integration problem:
data from multiple batches must have batch-specific confounding effects removed in order to be treated as one dataset.
Confounded shows promise in removing traditional batch effects from microarray expression data in the Bladderbatch and GSE37199 datasets.
It also effectively decreased artificial batch effects in image data and cancer-type-specific confounding effects in RNA-Seq data.
Confounded may be effective in other data integration problems, such as combining microarray with RNA-Seq datasets, or merging several large datasets measured under different conditions.

Confounded, and adversarial autoencoders in general, show promise as a valuable way to remove confounding biases from expression datasets.
Such methods will enable researchers access to larger datasets, therefore increasing the scope of analyses and furthering science as a whole.

\newpage
\phantomsection
\addcontentsline{toc}{section}{References}
\bibliography{references}

\end{document}