\defmodule {bbattery}

This module contains predefined batteries of statistical tests for
sources of random bits or sequences of uniform random numbers in
the interval $[0, 1)$.
\hpierre{The choice of tests and parameters in these batteries is
        subject to change in the near future,
        following further experimentations.}
To test a RNG for general use, one could first apply the
small and fast battery {\tt SmallCrush}.
If it passes, one could then apply the more stringent battery
{\tt Crush}, and finally the yet more time-consuming
battery {\tt BigCrush}.
The batteries {\tt Alphabit} and {\tt Rabbit} can be applied on a binary
file considered as a source of random bits. They can also be applied
on a programmed generator.  {\tt Alphabit} has been defined primarily to test
{\it hardware} random bits generators.
The battery {PseudoDIEHARD} applies most of the tests in
the well-known {\it DIEHARD\/} suite of Marsaglia \cite{rMAR96b}.
The battery {\tt FIPS\_140\_2} implements the small suite of tests
of the {\it FIPS-140-2} standard from NIST.

The batteries described in this module  will write the results of each test
(on standard output) with a standard level of details (assuming that the
 boolean switches of module {\tt swrite} have their default values),
followed by a summary  report of the suspect $p$-values obtained from the
 specific tests included in the batteries.
It is also possible to get only the summary report in the output,
with no detailed output from the tests,
by setting the boolean switch {\tt swrite\_Basic} to {\tt FALSE}.

Some of the tests compute more than one statistic using the same stream of
random numbers and these statistics are thus not independent.
\emph{That is why the number of statistics in the summary reports is larger than
the number of tests in the description of the batteries.}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bigskip
\hrule
\code\hide
/* bbattery.h for ANSI C */
#ifndef BBATTERY_H
#define BBATTERY_H
\endhide
#include "unif01.h"

extern int bbattery_NTests;
\endcode
  \tab The maximum number of $p$-values in the array {\tt bbattery\_pVal}.
  For small sample size, some of the tests in the battery may not be done.
  Furthermore, some of the tests computes more than one statistic and
  its $p$-value, so {\tt bbattery\_NTests} will usually be larger than
  the number of tests in the battery.
  \endtab
\code

extern double bbattery_pVal[];
\endcode
  \tab This array keeps the $p$-values resulting from the  battery of tests
  that is currently applied (or the last one that has been called). It is
  used by any battery in this module. The $p$-value of the $j$-th test in
  the battery is kept in {\tt bbattery\_pVal[$j-1$]}, for $1\le j\le $
  {\tt bbattery\_NTests}.
%% When a particular test in the battery
%%   is not done (because the sample size is too small), then the
%%  corresponding $p$-value is set to $-1$.
 \endtab
\code

extern char *bbattery_TestNames[];
\endcode
  \tab This array keeps the names of each test from  the
  battery that is currently applied (or the last one that has
  been called). It is used by any battery in this module.
  The name of the $j$-th test in the battery is
   kept in {\tt bbattery\_TestNames[$j-1$]},
  for $1\le j\le $ {\tt bbattery\_NTests}.
 \endtab

\guisec{The batteries of tests}

\code

void bbattery_SmallCrush (unif01_Gen *gen);

void bbattery_SmallCrushFile (char *filename);
\endcode
  \tab  Both functions applies {\tt SmallCrush}, a small and fast battery
   of tests, to a RNG. The function {\tt bbattery\_Small\-CrushFile} applies
  {\tt SmallCrush} to a RNG given as a text file of floating-point numbers in
  $[0, 1)$; the file requires slightly less than 51320000 random numbers.
   \label{bat:SmallCrush}
   The file will be rewound to the beginning before each test.
  \index{SmallCrush}\index{battery of tests!SmallCrush}%
  Thus  {\tt bbattery\_SmallCrush} applies the tests on one unbroken
  stream of successive numbers, while {\tt bbattery\_Small\-CrushFile}
  applies each test on the same sequence of numbers.
  Some of these tests assume that {\tt gen} returns at least 30 bits of
  resolution; if this is not the case, then the generator is most likely to
  fail these particular tests.

  The following tests are applied:
  \endtab
  \begin{enumerate}
  \item {\tt smarsa\_BirthdaySpacings} with $N=1$,  $n=5*10^6$,  $r=0$,
   $d = 2^{30}$, $t=2$, $p=1$.

  \item {\tt sknuth\_Collision} with $N=1$,  $n=5*10^6$,  $r=0$,
   $d = 2^{16}$, $t=2$.

  \item {\tt sknuth\_Gap}  with $N=1$,  $n=2*10^5$,  $r=22$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/256$.

  \item {\tt sknuth\_SimpPoker}  with $N=1$,  $n=4*10^5$,  $r=24$,
   $d = 64$, $k=64$.

  \item {\tt sknuth\_CouponCollector}  with $N=1$, $n=5*10^5$, $r=26$,
   $d=16$.

  \item {\tt  sknuth\_MaxOft}  with $N=1$,  $n=2*10^6$,  $r=0$,
   $d = 10^5$, $t=6$.

  \item {\tt  svaria\_WeightDistrib} with $N=1$, $n=2*10^5$, $r=27$,
   $k=256$, {\tt Alpha} $=0$,  {\tt Beta} $=1/8$.

  \item {\tt smarsa\_MatrixRank} with $N=1$,  $n=20000$,  $r=20$, $s=10$,
   $L = k=60$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=5*10^5$,  $r=20$, $s=10$,
   $L = 300$, $d = 0$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^6$,  $r=0$, $s=30$,
   $L_0 = 150$, $L_1=150$.

  \end{enumerate}

\code

void bbattery_RepeatSmallCrush (unif01_Gen *gen, int rep[]);
\endcode
  \tab This function applies specific tests of {\tt SmallCrush} on
   generator {\tt gen}. Test numbered $i$ in the enumeration above will
  be applied {\tt rep[$i$]} times successively on  {\tt gen}. Those tests with
   {\tt rep[$i$]} = 0 will not be applied. This is useful when a test in
    {\tt SmallCrush} had a suspect $p$-value, and one wants to reapply the
  test a few more times to find out whether the generator failed
  the test or whether the suspect $p$-value was a statistical fluke.
  Restriction: Array {\tt rep} must have one more element than the
  number of tests in {\tt SmallCrush}.
 \endtab
\bigskip

\hrule
\code

void bbattery_Crush (unif01_Gen *gen);
\endcode
  \tab
  Applies the battery {\tt Crush}, a suite of stringent statistical
  tests, to the generator {\tt gen}.
  \index{Crush}\index{battery of tests!Crush}%
  The battery includes the classical tests described in Knuth \cite{rKNU98a}
  as well as many other tests. Some of these tests assume that
  {\tt gen} returns at least 30 bits of resolution; if that
  is not the case, then the generator will certainly
  fail these particular tests. One test requires 31 bits of resolution:
  the {\tt BirthdaySpacings} test with $t=2$.
  On a PC with an AMD Athlon 64 Processor 4000+
  of clock speed 2400 MHz running with Red Hat Linux, Crush
  will require around 1 hour of CPU time. {\tt Crush} uses approximately
  $2^{35}$ random numbers.
%%  Some of the tests will require as much as 350 Megs of RAM memory.
  The following tests are applied: \label{bat:Crush}
  \endtab

  \begin{enumerate}
  \item {\tt smarsa\_SerialOver} with $N=1$,  $n=5*10^8$,  $r=0$,
  $d = 2^{12}$, $t=2$.

  \item {\tt smarsa\_SerialOver} with $N=1$,  $n=3*10^8$,  $r=0$,
  $d = 2^6$, $t=4$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=0$,
  $d = 2^{20}$, $t=2$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=10$,
  $d = 2^{20}$,  $t=2$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=0$,
  $d = 2^{10}$, $t=4$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=20$,
  $d = 2^{10}$,  $t=4$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=0$,
  $d = 32$, $t=8$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=25$,
  $d = 32$,  $t=8$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=0$,
  $d = 4$, $t=20$.

  \item {\tt smarsa\_CollisionOver} with $N=10$,  $n=10^7$,  $r=28$,
  $d = 4$,  $t=20$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=5$,  $n=2*10^7$, $r=0$,
   $d = 2^{31}$, $t=2$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=5$,  $n=2*10^7$, $r=0$,
   $d = 2^{21}$, $t=3$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=5$,  $n=2*10^7$, $r=0$,
   $d = 2^{16}$, $t=4$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=3$,  $n=2*10^7$, $r=0$,
   $d = 2^{9}$, $t=7$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=3$,  $n=2*10^7$, $r=7$,
   $d = 2^{9}$, $t=7$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=3$,  $n=2*10^7$, $r=14$,
   $d = 2^{8}$, $t=8$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=3$,  $n=2*10^7$, $r=22$,
   $d = 2^{8}$, $t=8$, $p=1$.

  \item {\tt snpair\_ClosePairs}  with $N=10$,  $n=2*10^6$, $r=0$, $t=2$,
   $p=0$,  $m=30$.

  \item {\tt snpair\_ClosePairs}  with $N=10$,  $n=2*10^6$, $r=0$, $t=3$,
   $p=0$,  $m=30$.

  \item {\tt snpair\_ClosePairs}  with $N=5$,  $n=2*10^6$, $r=0$, $t=7$,
   $p=0$,  $m=30$.

  \item {\tt snpair\_ClosePairsBitMatch}  with $N=4$,  $n=4*10^6$,
   $r=0$, $t=2$.

  \item {\tt snpair\_ClosePairsBitMatch}  with $N=2$,  $n=4*10^6$,
   $r=0$, $t=4$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=4*10^7$,  $r=0$, $d=16$, $k=16$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=4*10^7$,  $r=26$, $d=16$, $k=16$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=10^7$,  $r=0$, $d=64$, $k=64$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=10^7$,  $r=24$, $d=64$, $k=64$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=4*10^7$, $r=0$, $d=4$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=4*10^7$, $r=28$, $d=4$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=10^7$, $r=0$, $d=16$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=10^7$, $r=26$, $d=16$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=10^8$, $r=0$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/8$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=10^8$, $r=27$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/8$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=5*10^6$, $r=0$, {\tt Alpha} $=0$,
   {\tt Beta} $=1/256$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=5*10^6$, $r=22$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/256$.

  \item {\tt sknuth\_Run}  with $N=1$, $n=5*10^8$, $r=0$, {\tt Up = TRUE}.

  \item {\tt sknuth\_Run}  with $N=1$, $n=5*10^8$, $r=15$, {\tt Up = FALSE}.

  \item {\tt sknuth\_Permutation} with $N=1$, $n=5*10^7$, $r=0$, $t=10$.

  \item {\tt sknuth\_Permutation} with $N=1$, $n=5*10^7$, $r=15$, $t=10$.

  \item {\tt sknuth\_CollisionPermut} with $N=5$, $n=10^7$, $r=0$, $t=13$.

  \item {\tt sknuth\_CollisionPermut} with $N=5$, $n=10^7$, $r=15$, $t=13$.

  \item {\tt sknuth\_MaxOft} with $N=10$, $n=10^7$, $r=0$, $d=10^5$, $t=5$.

  \item {\tt sknuth\_MaxOft} with $N=5$, $n=10^7$, $r=0$, $d=10^5$, $t=10$.

  \item {\tt sknuth\_MaxOft} with $N=1$, $n=10^7$, $r=0$, $d=10^5$, $t=20$.

  \item {\tt sknuth\_MaxOft} with $N=1$, $n=10^7$, $r=0$, $d=10^5$, $t=30$.

  \item {\tt svaria\_SampleProd} with $N=1$, $n=10^7$, $r=0$, $t=10$.

  \item {\tt svaria\_SampleProd} with $N=1$, $n=10^7$, $r=0$, $t=30$.

  \item {\tt svaria\_SampleMean} with $N=10^7$, $n=20$, $r=0$.

  \item {\tt svaria\_SampleCorr} with $N=1$, $n=5*10^8$, $r=0$, $k=1$.

  \item {\tt svaria\_AppearanceSpacings} with $N=1$,  $Q=10^7$, $K=4*10^8$,
   $r=0$,  $s=30$, $L=15$.

  \item {\tt svaria\_AppearanceSpacings} with $N=1$,  $Q=10^7$, $K=10^8$,
   $r=20$,  $s=10$, $L=15$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^6$, $r=0$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/8$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^6$, $r=8$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/8$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^6$, $r=16$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/8$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^6$, $r=24$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/8$.

  \item {\tt svaria\_SumCollector} with $N=1$, $n=2*10^7$, $r=0$,  $g=10$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=10^6$, $r=0$,
   $s=30$, $L=k=60$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=10^6$, $r=20$,
   $s=10$, $L=k=60$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=50000$, $r=0$,
   $s=30$, $L=k=300$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=50000$, $r=20$,
   $s=10$, $L=k=300$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=2000$, $r=0$,
   $s=30$, $L=k=1200$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=2000$, $r=20$,
   $s=10$, $L=k=1200$.

  \item {\tt smarsa\_Savir2} with $N=1$, $n=2*10^7$, $r=0$, $m=2^{20}$, $t=30$.

  \item {\tt smarsa\_GCD} with $N=1$, $n=10^8$, $r=0$, $s=30$.

  \item {\tt smarsa\_GCD} with $N=1$, $n=4*10^7$, $r=10$, $s=20$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=5*10^7$, $r=0$,
   $s=30$, $L_0=L_1=90$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^7$, $r=20$,
   $s=10$, $L_0=L_1=90$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=5*10^6$, $r=0$,
   $s=30$, $L_0=L_1=1000$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^6$, $r=20$,
   $s=10$, $L_0=L_1=1000$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=5*10^5$, $r=0$,
   $s=30$, $L_0=L_1=10000$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^5$, $r=20$,
   $s=10$, $L_0=L_1=10000$.

  \item {\tt scomp\_LinearComp} with $N=1$, $n=120000$, $r=0$, $s=1$.

  \item {\tt scomp\_LinearComp} with $N=1$, $n=120000$, $r=29$, $s=1$.

  \item {\tt scomp\_LempelZiv} with $N=10$, $k=25$, $r=0$, $s=30$.

  \item {\tt sspectral\_Fourier3} with $N=50000$, $k=14$, $r=0$, $s=30$.

  \item {\tt sspectral\_Fourier3} with $N=50000$, $k=14$, $r=20$, $s=10$.

  \item {\tt sstring\_LongestHeadRun} with $N=1$, $n=1000$, $r=0$,
   $s=30$, $L=10^7$.

  \item {\tt sstring\_LongestHeadRun} with $N=1$, $n=300$, $r=20$,
   $s=10$, $L=10^7$.

  \item {\tt sstring\_PeriodsInStrings} with $N=1$, $n=3*10^8$, $r=0$,
   $s=30$.

  \item {\tt sstring\_PeriodsInStrings} with $N=1$, $n=3*10^8$, $r=15$,
   $s=15$.

  \item {\tt sstring\_HammingWeight2} with $N=100$, $n=10^8$, $r=0$,
   $s=30$, $L=10^6$.

  \item {\tt sstring\_HammingWeight2} with $N=30$, $n=10^8$, $r=20$,
   $s=10$, $L=10^6$.

  \item {\tt sstring\_HammingCorr} with $N=1$, $n=5*10^8$, $r=0$,
   $s=30$, $L=30$.

  \item {\tt sstring\_HammingCorr} with $N=1$, $n=5*10^7$, $r=0$,
   $s=30$, $L=300$.

  \item {\tt sstring\_HammingCorr} with $N=1$, $n=10^7$, $r=0$,
   $s=30$, $L=1200$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=3*10^8$, $r=0$,
   $s=30$, $L=30$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=10^8$, $r=20$,
   $s=10$, $L=30$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=3*10^7$, $r=0$,
   $s=30$, $L=300$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=10^7$, $r=20$,
   $s=10$, $L=300$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=10^7$, $r=0$,
   $s=30$, $L=1200$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=10^6$, $r=20$,
   $s=10$, $L=1200$, $d=0$.

  \item {\tt sstring\_Run} with $N=1$, $n=10^9$, $r=0$, $s=30$.

  \item {\tt sstring\_Run} with $N=1$, $n=10^9$, $r=20$, $s=10$.

  \item {\tt sstring\_AutoCor} with $N=10$, $n=10^9$, $r=0$, $s=30$,
  $d=1$.

  \item {\tt sstring\_AutoCor} with $N=5$, $n=10^9$, $r=20$, $s=10$,
  $d=1$.

  \item {\tt sstring\_AutoCor} with $N=10$, $n=10^9$, $r=0$, $s=30$,
  $d=30$.

  \item {\tt sstring\_AutoCor} with $N=5$, $n=10^9$, $r=20$, $s=10$,
  $d=10$.

\end{enumerate}

\code

void bbattery_RepeatCrush (unif01_Gen *gen, int rep[]);
\endcode
  \tab Similar to {\tt bbattery\_RepeatSmallCrush} above but applied on
  {\tt Crush}.
  \endtab
\bigskip

\hrule
\code

void bbattery_BigCrush (unif01_Gen *gen);
\endcode
  \tab
  Applies the battery {\tt BigCrush}, a suite of very stringent statistical
  tests, to the generator {\tt gen}.
  \index{BigCrush}\index{battery of tests!BigCrush}%
  Some of these tests assume that {\tt gen} returns at least 30 bits of
  resolution; if that is not the case, then the generator will certainly
  fail these particular tests. One test requires 31 bits of resolution:
  the {\tt BirthdaySpacings} test with $t=2$.
  On a PC with an AMD Athlon 64 Processor 4000+
  of clock speed 2400 MHz running with Linux, BigCrush
  will take around 8 hours of CPU time. {\tt BigCrush} uses close to
  $2^{38}$ random numbers.
  The following tests are applied:\label{bat:BigCrush}
  \endtab

  \begin{enumerate}
  \item {\tt smarsa\_SerialOver} with $N=1$,  $n=10^9$,  $r=0$,
  $d = 2^{8}$, $t=3$.

  \item {\tt smarsa\_SerialOver} with $N=1$,  $n=10^9$,  $r=22$,
  $d = 2^{8}$, $t=3$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=0$,
  $d = 2^{21}$, $t=2$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=9$,
  $d = 2^{21}$,  $t=2$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=0$,
  $d = 2^{14}$, $t=3$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=16$,
  $d = 2^{14}$,  $t=3$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=0$,
  $d = 64$, $t=7$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=24$,
  $d = 64$,  $t=7$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=0$,
  $d = 8$, $t=14$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=27$,
  $d = 8$,  $t=14$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=0$,
  $d =4$,  $t=21$.

  \item {\tt smarsa\_CollisionOver} with $N=30$,  $n=2*10^7$,  $r=28$,
  $d =4$,  $t=21$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=100$,  $n=10^7$, $r=0$,
   $d = 2^{31}$, $t=2$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=2*10^7$, $r=0$,
   $d = 2^{21}$, $t=3$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=3*10^7$, $r=14$,
   $d = 2^{16}$, $t=4$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=2*10^7$, $r=0$,
   $d = 2^{9}$, $t=7$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=2*10^7$, $r=7$,
   $d = 2^{9}$, $t=7$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=3*10^7$, $r=14$,
   $d = 2^{8}$, $t=8$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=3*10^7$, $r=22$,
   $d = 2^{8}$, $t=8$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=3*10^7$, $r=0$,
   $d = 2^{4}$, $t=16$, $p=1$.

  \item {\tt smarsa\_BirthdaySpacings}  with $N=20$,  $n=3*10^7$, $r=26$,
   $d = 2^{4}$, $t=16$, $p=1$.

  \item {\tt snpair\_ClosePairs}  with $N=30$,  $n=6*10^6$, $r=0$, $t=3$,
   $p=0$,  $m=30$.

  \item {\tt snpair\_ClosePairs}  with $N=20$,  $n=4*10^6$, $r=0$, $t=5$,
   $p=0$,  $m=30$.

  \item {\tt snpair\_ClosePairs}  with $N=10$,  $n=3*10^6$, $r=0$, $t=9$,
   $p=0$,  $m=30$.

  \item {\tt snpair\_ClosePairs}  with $N=5$,  $n=2*10^6$, $r=0$, $t=16$,
   $p=0$,  $m=30$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=4*10^8$,  $r=0$, $d=8$, $k=8$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=4*10^8$,  $r=27$, $d=8$, $k=8$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=10^8$,  $r=0$, $d=32$, $k=32$.

  \item {\tt sknuth\_SimpPoker} with $N=1$,  $n=10^8$,  $r=25$, $d=32$, $k=32$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=2*10^8$, $r=0$, $d=8$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=2*10^8$, $r=10$, $d=8$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=2*10^8$, $r=20$, $d=8$.

  \item {\tt sknuth\_CouponCollector} with $N=1$, $n=2*10^8$, $r=27$, $d=8$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=5*10^8$, $r=0$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/16$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=3*10^8$, $r=25$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/32$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=10^8$, $r=0$, {\tt Alpha} $=0$,
    {\tt Beta} $=1/128$.

  \item {\tt sknuth\_Gap} with $N=1$, $n=10^7$, $r=20$, {\tt Alpha} $=0$,
   {\tt Beta} $=1/1024$.

  \item {\tt sknuth\_Run}  with $N=5$, $n=10^9$, $r=0$, {\tt Up = FALSE}.

  \item {\tt sknuth\_Run}  with $N=5$, $n=10^9$, $r=15$, {\tt Up = TRUE}.

  \item {\tt sknuth\_Permutation} with $N=1$, $n=10^9$, $r=0$, $t=3$.

  \item {\tt sknuth\_Permutation} with $N=1$, $n=10^9$, $r=0$, $t=5$.

  \item {\tt sknuth\_Permutation} with $N=1$, $n=5*10^8$, $r=0$, $t=7$.

  \item {\tt sknuth\_Permutation} with $N=1$, $n=5*10^8$, $r=10$, $t=10$.

  \item {\tt sknuth\_CollisionPermut} with $N=20$, $n=2*10^7$, $r=0$, $t=14$.

  \item {\tt sknuth\_CollisionPermut} with $N=20$, $n=2*10^7$, $r=10$, $t=14$.

  \item {\tt sknuth\_MaxOft} with $N=40$, $n=10^7$, $r=0$, $d=10^5$, $t=8$.

  \item {\tt sknuth\_MaxOft} with $N=30$, $n=10^7$, $r=0$, $d=10^5$, $t=16$.

  \item {\tt sknuth\_MaxOft} with $N=20$, $n=10^7$, $r=0$, $d=10^5$, $t=24$.

  \item {\tt sknuth\_MaxOft} with $N=20$, $n=10^7$, $r=0$, $d=10^5$, $t=32$.

  \item {\tt svaria\_SampleProd} with $N=40$, $n=10^7$, $r=0$, $t=8$.

  \item {\tt svaria\_SampleProd} with $N=20$, $n=10^7$, $r=0$, $t=16$.

  \item {\tt svaria\_SampleProd} with $N=20$, $n=10^7$, $r=0$, $t=24$.

  \item {\tt svaria\_SampleMean} with $N=2*10^7$, $n=30$, $r=0$.

  \item {\tt svaria\_SampleMean} with $N=2*10^7$, $n=30$, $r=10$.

  \item {\tt svaria\_SampleCorr} with $N=1$, $n=2*10^9$, $r=0$, $k=1$.

  \item {\tt svaria\_SampleCorr} with $N=1$, $n=2*10^9$, $r=0$, $k=2$.

  \item {\tt svaria\_AppearanceSpacings} with $N=1$,  $Q=10^7$, $K=10^9$,
   $r=0$,  $s=3$, $L=15$.

  \item {\tt svaria\_AppearanceSpacings} with $N=1$,  $Q=10^7$, $K=10^9$,
   $r=27$,  $s=3$, $L=15$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^7$, $r=0$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/4$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^7$, $r=20$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/4$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^7$, $r=28$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/4$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^7$, $r=0$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/16$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^7$, $r=10$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/16$.

  \item {\tt svaria\_WeightDistrib} with $N=1$, $n=2*10^7$, $r=26$, $k=256$,
   {\tt Alpha} $=0$,  {\tt Beta} $=1/16$.

  \item {\tt svaria\_SumCollector} with $N=1$, $n=5*10^8$, $r=0$,  $g=10$.

  \item {\tt smarsa\_MatrixRank} with $N=10$, $n=10^6$, $r=0$,
   $s=5$, $L=k=30$.

  \item {\tt smarsa\_MatrixRank} with $N=10$, $n=10^6$, $r=25$,
   $s=5$, $L=k=30$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=5000$, $r=0$,
   $s=4$, $L=k=1000$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=5000$, $r=26$,
   $s=4$, $L=k=1000$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=80$, $r=15$,
   $s=15$, $L=k=5000$.

  \item {\tt smarsa\_MatrixRank} with $N=1$, $n=80$, $r=0$,
   $s=30$, $L=k=5000$.

  \item {\tt smarsa\_Savir2} with $N=10$, $n=10^7$, $r=10$, $m=2^{20}$, $t=30$.

  \item {\tt smarsa\_GCD} with $N=10$, $n=5*10^7$, $r=0$, $s=30$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^8$, $r=0$,
   $s=5$, $L_0=L_1=50$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^8$, $r=25$,
   $s=5$, $L_0=L_1=50$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^7$, $r=0$,
   $s=10$, $L_0=L_1=1000$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^7$, $r=20$,
   $s=10$, $L_0=L_1=1000$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^6$, $r=0$,
   $s=15$, $L_0=L_1=10000$.

  \item {\tt swalk\_RandomWalk1} with $N=1$, $n=10^6$, $r=15$,
   $s=15$, $L_0=L_1=10000$.

  \item {\tt scomp\_LinearComp} with $N=1$, $n=400000$, $r=0$, $s=1$.

  \item {\tt scomp\_LinearComp} with $N=1$, $n=400000$, $r=29$, $s=1$.

  \item {\tt scomp\_LempelZiv} with $N=10$, $k=27$, $r=0$, $s=30$.

  \item {\tt scomp\_LempelZiv} with $N=10$, $k=27$, $r=15$, $s=15$.

  \item {\tt sspectral\_Fourier3} with $N=100000$, $r=0$, $s=3$, $k=14$.

  \item {\tt sspectral\_Fourier3} with $N=100000$, $r=27$, $s=3$, $k=14$.

  \item {\tt sstring\_LongestHeadRun} with $N=1$, $n=1000$, $r=0$,
   $s=3$, $L=10^7$.

  \item {\tt sstring\_LongestHeadRun} with $N=1$, $n=1000$, $r=27$,
   $s=3$, $L=10^7$.

  \item {\tt sstring\_PeriodsInStrings} with $N=10$, $n=5*10^8$, $r=0$,
   $s=10$.

  \item {\tt sstring\_PeriodsInStrings} with $N=10$, $n=5*10^8$, $r=20$,
   $s=10$.

  \item {\tt sstring\_HammingWeight2} with $N=10$, $n=10^9$, $r=0$,
   $s=3$, $L=10^6$.

  \item {\tt sstring\_HammingWeight2} with $N=10$, $n=10^9$, $r=27$,
   $s=3$, $L=10^6$.

  \item {\tt sstring\_HammingCorr} with $N=1$, $n=10^9$, $r=10$,
   $s=10$, $L=30$.

  \item {\tt sstring\_HammingCorr} with $N=1$, $n=10^8$, $r=10$,
   $s=10$, $L=300$.

  \item {\tt sstring\_HammingCorr} with $N=1$, $n=10^8$, $r=10$,
   $s=10$, $L=1200$.

  \item {\tt sstring\_HammingIndep} with $N=10$, $n=3*10^7$, $r=0$,
   $s=3$, $L=30$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=10$, $n=3*10^7$, $r=27$,
   $s=3$, $L=30$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=3*10^7$, $r=0$,
   $s=4$, $L=300$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=3*10^7$, $r=26$,
   $s=4$, $L=300$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=10^7$, $r=0$,
   $s=5$, $L=1200$, $d=0$.

  \item {\tt sstring\_HammingIndep} with $N=1$, $n=10^7$, $r=25$,
   $s=5$, $L=1200$, $d=0$.

  \item {\tt sstring\_Run}  with $N=1$, $n=2*10^9$, $r=0$, $s=3$.

  \item {\tt sstring\_Run}  with $N=1$, $n=2*10^9$, $r=27$, $s=3$.

  \item {\tt sstring\_AutoCor} with $N=10$, $n=10^9$, $r=0$, $s=3$,
   $d=1$.

  \item {\tt sstring\_AutoCor} with $N=10$, $n=10^9$, $r=0$, $s=3$,
   $d=3$.

  \item {\tt sstring\_AutoCor} with $N=10$, $n=10^9$, $r=27$, $s=3$,
   $d=1$.

  \item {\tt sstring\_AutoCor} with $N=10$, $n=10^9$, $r=27$, $s=3$,
   $d=3$.

\end{enumerate}

\code

void bbattery_RepeatBigCrush (unif01_Gen *gen, int rep[]);
\endcode
  \tab Similar to {\tt bbattery\_RepeatSmallCrush} above but applied on
  {\tt BigCrush}.
  \endtab
\bigskip

\bigskip
\hrule
\code

void bbattery_Rabbit (unif01_Gen *gen, double nb);
\endcode
  \tab Applies the {\tt Rabbit} battery of tests to the generator {\tt gen}
   using at most {\tt nb} bits for each test. See the description of the
   tests in {\tt bbattery\_RabbitFile}.
  \endtab
\code

void bbattery_RabbitFile (char *filename, double nb);
\endcode
  \tab Applies the {\tt Rabbit} battery of tests to the first {\tt nb}
   bits (or less, if {\tt nb} is too large) of the binary file
   {\tt filename}.
  \index{Rabbit}\index{battery of tests!Rabbit}%
%  All the following tests are applied on the same sequence of
%  bits of the file.
  For each test, the file is reset and the test is applied to the bit
  stream starting at the beginning of the file. The bits themselves are
  processed in nearly all the tests as blocks of 32 bits (unsigned
  integers); the two exceptions are {\tt svaria\_AppearanceSpacings},
  which uses blocks of 30 bits (and discards the last 2 bits out of
  each block of 32),
  and {\tt sstring\_PeriodsInStrings} which uses blocks of 31 bits (and
  discards 1 bit out of 32).
  The parameters of each test are chosen automatically as a function of
  {\tt nb}, in order to make the test reasonably sensitive.
% Constraints are also set on the parameter $n$, because of memory limitations,
% in such a way that no test should normally require more than 420 Megs of memory.
%  Thus  when {\tt nb} is not too large, the number of replications $N$ is
%   chosen equal to 1, and the sample size $n$ is determined as a function of
%   {\tt nb}.
  On a PC with an Athlon processor of clock speed
  1733 MHz running under Linux,  {\tt Rabbit} will take about 5 seconds to
  test a stream of $2^{20}$ bits, 90 seconds to test a stream of
  $2^{25}$ bits, and 28 minutes to test a stream of
  $2^{30}$ bits.
  Restriction: {\tt nb} $\ge 500$.
  \endtab

\begin{enumerate}
  \item {\tt smultin\_MultinomialBitsOver}
  \item {\tt snpair\_ClosePairsBitMatch} in $t=2$ dimensions.
  \item {\tt snpair\_ClosePairsBitMatch} in $t=4$ dimensions.
  \item {\tt svaria\_AppearanceSpacings}
  \item {\tt scomp\_LinearComp}
  \item {\tt scomp\_LempelZiv}
  \item {\tt sspectral\_Fourier1}
  \item {\tt sspectral\_Fourier3}
  \item {\tt sstring\_LongestHeadRun}
  \item {\tt sstring\_PeriodsInStrings}
  \item {\tt sstring\_HammingWeight} with blocks of $L = 32$ bits.
  \item {\tt sstring\_HammingCorr} with blocks of $L = 32$ bits.
  \item {\tt sstring\_HammingCorr} with blocks of $L = 64$ bits.
  \item {\tt sstring\_HammingCorr} with blocks of $L = 128$ bits.
  \item {\tt sstring\_HammingIndep} with blocks of $L = 16$ bits.
  \item {\tt sstring\_HammingIndep} with blocks of $L = 32$ bits.
  \item {\tt sstring\_HammingIndep} with blocks of $L = 64$ bits.
  \item {\tt sstring\_AutoCor} with a lag $d = 1$.
  \item {\tt sstring\_AutoCor} with a lag $d = 2$.
  \item {\tt sstring\_Run}
  \item {\tt smarsa\_MatrixRank} with $32 \times 32$ matrices.
  \item {\tt smarsa\_MatrixRank} with $320 \times 320$ matrices.
  \item {\tt smarsa\_MatrixRank} with $1024 \times 1024$ matrices.
  \item {\tt swalk\_RandomWalk1} with walks of length $L = 128$.
  \item {\tt swalk\_RandomWalk1} with walks of length $L = 1024$.
  \item {\tt swalk\_RandomWalk1} with walks of length $L = 10016$.
\end{enumerate}

\code

void bbattery_RepeatRabbit (unif01_Gen *gen, double nb, int rep[]);
\endcode
  \tab Similar to {\tt bbattery\_RepeatSmallCrush} above but applied on
 {\tt Rabbit}.
  \endtab

\bigskip
\hrule
\code

void bbattery_Alphabit (unif01_Gen *gen, double nb, int r, int s);
\endcode
  \tab Applies the {\tt Alphabit} battery of tests to the generator {\tt gen}
   using at most {\tt nb} bits for each test. The bits themselves are
  processed as blocks of 32 bits (unsigned integers). For each block of
  32 bits, the $r$ most significant bits are dropped, and the test is
  applied on the $s$ following bits. If one wants to test all bits of
  the stream, one should set $r=0$ and $s=32$. If one wants to test only
  1 bit out of 32, one should set $s=1$.
   See the description of the tests in {\tt bbattery\_AlphabitFile}.
  \endtab
\code

void bbattery_AlphabitFile (char *filename, double nb);
\endcode
  \tab Applies the {\tt Alphabit} battery of tests to the first {\tt nb}
   bits (or less, if {\tt nb} is too large) of the binary file
   {\tt filename}. Unlike the {\tt bbattery\_Alphabit} function above,
  \index{Alphabit}\index{battery of tests!Alphabit}%
%%   all the following tests are applied on the same sequence of
%%  bits of the file, as the binary file is rewound
  for each test, the file is rewound and the test is applied to the bit
  stream starting at the beginning of the file.
  On a PC with an Athlon processor of clock speed 1733 MHz running under
  Linux,  {\tt Alphabit} takes about 4.2 seconds to test a file of
  $2^{25}$ bits, and 2.3 minutes to test a file of $2^{30}$ bits.

  {\tt Alphabit} and {\tt AlphabitFile} have been designed primarily to test
   {\it hardware} random bits generators. The four {\tt MultinomialBitsOver}
  tests should detect correlations between successive bits by
   applying a {\tt SerialOver} test to overlapping blocks of $2$, $4$,
   $8$ and $16$ bits. The {\tt Hamming} tests should detect correlations
  between the successive bits of overlapping blocks of $16$ and $32$ bits,
  and the {\tt RandomWalk} tests consider blocks of 64 and 320 bits.

%  Restriction: {\tt nb} $\ge 500$.
  \endtab

\begin{enumerate}
  \item {\tt smultin\_MultinomialBitsOver} with $L=2$.
  \item {\tt smultin\_MultinomialBitsOver} with $L=4$.
  \item {\tt smultin\_MultinomialBitsOver} with $L=8$.
  \item {\tt smultin\_MultinomialBitsOver} with $L=16$.
  \item {\tt sstring\_HammingIndep} with blocks of $L = 16$ bits.
  \item {\tt sstring\_HammingIndep} with blocks of $L = 32$ bits.
  \item {\tt sstring\_HammingCorr} with blocks of $L = 32$ bits.
  \item {\tt swalk\_RandomWalk1} with walks of length $L = 64$.
  \item {\tt swalk\_RandomWalk1} with walks of length $L = 320$.
\end{enumerate}

\code

void bbattery_RepeatAlphabit (unif01_Gen *gen, double nb, int r, int s,
                              int rep[]);
\endcode
  \tab Similar to {\tt bbattery\_RepeatSmallCrush} above but applied on
  {\tt Alphabit}.
  \endtab
\code

void bbattery_BlockAlphabit (unif01_Gen *gen, double nb, int r, int s);
void bbattery_BlockAlphabitFile (char *filename, double nb);
\endcode
 \tab Apply the {\tt Alphabit} battery of tests repeatedly to the generator
  {\tt gen} or to the binary file {\tt filename} after reordering the bits
  as described in the filter {\tt unif01\_CreateBitBlockGen}.
  {\tt Alphabit} will be applied for the different values of
   $w \in \{1, 2, 4, 8, 16, 32\}$. If $s <32$, only values of $w \le s$ will
   be used. Each test uses at most {\tt nb} bits.
   See the description of the tests in {\tt bbattery\_AlphabitFile}.
  \endtab
\code

void bbattery_RepeatBlockAlphabit (unif01_Gen *gen, double nb, int r, int s,
                                   int rep[], int w);
\endcode
  \tab Similar to {\tt bbattery\_RepeatSmallCrush} above but applied on
   {\tt BlockAlphabit}. The parameter $w$ is the one described in
    {\tt bbattery\_BlockAlphabit}.  Restrictions:
   $w \in \{1, 2, 4, 8, 16, 32\}$ and $w \le s$.
  \endtab

\guisec {Other Tests Suites}
\code

void bbattery_pseudoDIEHARD (unif01_Gen *gen);
\endcode
  \tab
 \index{DIEHARD}\index{battery of tests!PseudoDIEHARD}%
 \index{battery of tests!DIEHARD}%
  Applies the battery PseudoDIEHARD, which implements most of
  the tests in the popular battery DIEHARD \cite{rMAR96b}
  or, in some cases, close approximations to them. \textbf{We do not recommend
  this battery as it is not very stringent} (we do not know of any generator
  that passes the batteries \textsc{Crush} and  \textsc{BigCrush}, and
  fails PseudoDIEHARD, while we have seen the converse for several defective
  generators). It is included here only for convenience to the user.
% Some of the tests in DIEHARD however are not implemented in TestU01.
  The DIEHARD tests and the corresponding tests in  PseudoDIEHARD are:
  \endtab

\begin{enumerate}

\item The {\bf Birthday Spacings} test. This corresponds
  to {\tt smarsa\_BirthdaySpacings} with $n=512$, $d=2^{24}$, $t=1$ and
  $r=0, 1, 2, 3, 4, 5, 6, 7, 8, 9$ successively. The test with each
 value of $r$ is repeated 500 times and a chi-square test is then applied.

\item The {\bf Overlapping 5-Permutation} test. This test is not
implemented in TestU01.

\item The {\bf Binary Rank Tests for Matrices}. This corresponds
 to {\tt smarsa\_MatrixRank}.

\item The {\bf Bitstream} test.  Closely related to
 {\tt smultin\_MultinomialBitsOver} with  {\tt Delta} $= -1$,
 $n=2^{21}$, $L=20$.

\item The {\bf OPSO} test. This corresponds to {\tt smarsa\_CollisionOver}
 with $n=2^{21}$, $d=1024$, $t=2$ and all values of $r$ from 0 to 22.

\item The {\bf OQSO} test. This corresponds to {\tt smarsa\_CollisionOver}
 with $n=2^{21}$, $d=32$, $t=4$ and all values of $r$ from 0 to 27.

\item The {\bf DNA} test. This corresponds to {\tt smarsa\_CollisionOver}
 with $n=2^{21}$, $d=4$, $t=10$ and all values of $r$ from 0 to 30.

\item The {\bf Count-the-1's} test is not implemented in TestU01. It is
a 5-dimensional overlapping version of {\tt sstring\_HammingIndep}.

\item The {\bf Parking Lot} test is not implemented in TestU01.

\item The {\bf Minimum Distance} test. Closely related
  to {\tt snpair\_ClosePairs}  with
 $N=100$, $n=8000$, $t=2$, $p=2$, $m=1$.

\item The {\bf 3-D Spheres} test. Closely related
  to {\tt snpair\_ClosePairs}  with
 $N=20$, $n=4000$, $t=3$, $p=2$, $m=1$.

\item The {\bf Squeeze} test.  Closely related
  to {\tt smarsa\_Savir2}.

\item The {\bf Overlapping Sums} test  is not implemented in TestU01.

\item The {\bf Runs} test.  This corresponds to  {\tt sknuth\_Run}.

\item The {\bf Craps} test  is not implemented in TestU01.

\end{enumerate}

\bigskip
\hrule

\paragraph{The NIST test suite}

The NIST (National Institute of Standards and Technology) of the U.S.
federal government has proposed a statistical test suite \cite{rRUK01a}
for use in the evaluation of the randomness of bitstreams
produced by cryptographic random number generators.
\index{NIST tests suite}\index{battery of tests!NIST}%
%% A close equivalent to the NIST test suite is the battery
%%  {\tt sbattery\_myNIST}. It will apply on a generator the tests in the
%%  NIST test suite that are available in TestU01.
The test parameters are not predetermined.
The NIST tests and the equivalent tests in TestU01 are:

\begin{enumerate}
\item The {\bf Monobit} test. This corresponds to
 {\tt sstring\_HammingWeight2} with $L = n$.
% NIST uses a half normal distribution of the random walk W where bit 0 is
% replaced by -1 and bit 1 by +1. W = s1 + s2 + ...sn.
% TestU01 uses a chi-square distribution with one degree of freedom
% for the number of 1 bits.

\item The {\bf Frequency test within a  Block}. Corresponds to
 {\tt sstring\_HammingWeight2}.

\item The {\bf Runs} test.   Is implemented as {\tt sstring\_Run}.

\item The test for the {\bf Longest Run of Ones in a Block}.
 Is implemented as the test
 {\tt sstring\_LongestHeadRun}.

\item The {\bf Binary Matrix rank} test.  Is implemented as
 {\tt smarsa\_MatrixRank}.

\item The {\bf Discrete Fourier Transform} test.  Is implemented as
 {\tt sspectral\_Fourier1}.

\item The {\bf Non-overlapping Template Matching} test. Is implemented
  as the test {\tt smarsa\_CATBits}.

\item The {\bf Overlapping Template Matching} test. This test does not
exist as such in TestU01, but a similar and more powerful test is
{\tt smultin\_MultinomialBitsOver}.

\item {\bf Maurer's Universal Statistical} test. This test is implemented as
 {\tt svaria\_Appear\-anceSpacings}.

\item The {\bf Lempel-Ziv Compression} test.  Is implemented as
 {\tt scomp\_LempelZiv}.

\item The {\bf Linear Complexity} test. Is implemented as part of
 {\tt scomp\_LinearComp}.

\item The {\bf Serial} test.  Corresponds to
 {\tt smultin\_MultinomialBitsOver} with  {\tt Delta} = 1.

\item The {\bf Approximate Entropy} test.  Corresponds to
 {\tt smultin\_MultinomialBitsOver} with  {\tt Delta} = 0, and to
 {\tt sentrop\_EntropyDiscOver} or  {\tt sentrop\_EntropyDiscOver2}.

\item The {\bf Cumulative Sums} test. This test is  closely related to the $M$
 statistic in  {\tt swalk\_RandomWalk1}.

\item The {\bf Random Excursions} test. This test does not
exist in TestU01, but closely related tests are in
  {\tt swalk\_RandomWalk1}.

\item The {\bf Random Excursions Variant} test.  This test does not
exist in TestU01, but a closely related test is based on the $R$ statistic in
  {\tt swalk\_RandomWalk1}.

\end{enumerate}

\bigskip
\hrule
\code

void bbattery_FIPS_140_2 (unif01_Gen *gen);
void bbattery_FIPS_140_2File (char *filename);
\endcode
\tab
\index{FIPS-140-2}\index{battery of tests!FIPS-140-2}%
 These functions apply the four tests described in the NIST document
 {\sl FIPS PUB 140-2, Security Requirements for Cryptographic Modules},
 page 35, with exactly the same parameters (see the WEB page at
  \url{http://csrc.nist.gov/rng/rng6_3.html}).  They report the values
 of the test statistics and their $p$-values (except for the runs test)
 and indicate which values fall outside the intervals specified by
 FIPS-140-2. The first function applies the tests on a generator {\tt gen},
 and the second applies them on the file of bits {\tt filename}. First,
 20000 bits are generated and put in an array, then the tests are applied
 upon these. The tests applied are:
\endtab

\begin{enumerate}
\item The {\bf Monobit} test. This corresponds to
% {\tt sstring\_HammingWeight2} with $L = n$.
 {\tt smultin\_MultinomialBits} with $s=32$, $L=1$, $n=20000$.

\item The ``{\bf poker}'' test, which is in fact equivalent to
 {\tt smultin\_MultinomialBits} with $s=32$, $L=4$, $n=5000$.

\item The {\bf Runs} test, which is related to {\tt sstring\_Run}.

\item The test for the {\bf Longest Run of Ones in a Block},
 which is implemented as {\tt sstring\_LongestHeadRun}.
\end{enumerate}

\code
\hide
#endif
\endhide
\endcode


\section{Separation of Signal from Background Using Matrix Elements}
\label{separation-ME}

\subsection{Method Overiew}
\subsection{Calculation of the Event Probability Density Functions}
\subsubsection{Differential Cross Section}
\subsubsection{Cross Section}
\subsubsection{Treatment of Combinatorial Background}
\subsection{Single Top Discriminant}
\subsubsection{One-Dimensional Discriminants}
\subsubsection{Performance}
\subsubsection{Two-Dimensional Discrimiannts}

\subsection{Method Description}
The matrix element analysis uses leading order matrix elements to separate
single top quark events from background events. The matrix
element analysis calculates the probability for an event to be a signal top event,
either $s$-channel or $t$-channel, and a probability for one of several $W$+jets
backgrounds. This probability is defined in Eq.~\ref{prob} as

\begin{equation}
\label{prob}
P(\vec{x}) = \frac{1}{\sigma} \sum_{i,j} \int f_{i}(q_{1}, Q^{2})dq_{1}
\times
f_{j}(q_{2}, Q^{2})dq_{2} \times \pderiv{\sigma_{hs,ij}(\vec{y})}{\vec{y}}
\times W(\vec{x},\vec{y})d\vec{y}
\end{equation}

\noindent Each term in the Eq.~\ref{prob} is described below.

\begin{itemize}
\item $\sum_{i,j}$ is a sum of initial parton flavors in the hard scatter
collision. For example, an $s$-channel collision can occur from a $u\bar{d}$,
$c\bar{s}$, $d\bar{u}$, or $s\bar{c}$.
\item $f_{i}(q_{1}, Q^{2})$ is the parton distribution function for parton, $i$.
\item $\pderiv{\sigma_{hs,ij}(\vec{y})}{\vec{y}}$ is the differential cross
section for the hard scatter collision.
\item $W(\vec{x},\vec{y})$ is called the transfer function which relates observed
objects in the detector, $\vec{x}$, to the final state objects from the matrix
element, $\vec{y}$.
\item $\int d\vec{y}$ is an integration over the matrix element phase space.
\item $\frac{1}{\sigma}$ is the normalization constant and is described in
Eq.~\ref{cross}.
\end{itemize}

\begin{equation}
\label{cross}
\sigma = \sum_{i,j} \int f_{i}(x_{1}, Q^{2})dx_{1}
\times
f_{j}(x_{2}, Q^{2})dx_{2} \times \pderiv{\sigma_{hs,ij}(\vec{y})}{\vec{y}}
\times W(\vec{x},\vec{y})d\vec{y}d\vec{x} \times \varepsilon(\vec{x}) \times
\Theta_{\rm{cuts}}(\vec{x})
\end{equation}

\noindent This analysis uses CTEQ 6.1 LO~\cite{Pumplin:2005rh} parton distribution
functions access via LHAPDF~\cite{LHAPDF}. Matrix elements in
this analysis were taken from the Madgraph~\cite{Maltoni:2002qb} leading
order matrix element generator. The multidimensional integrals were calculated
using the GNU~\cite{GNU} Scientific Library's version of the VEGAS Monte Carlo integration technique ~\cite{Lepage:1980dq}.

The transfer functions, $W(\vec{x},\vec{y})$, relate the
observed state in the detector to the final state at the parton level. These
functions are determined using Monte Carlo where the final state flavor and
four-vector is known. A transfer function is determined for each jet flavor and
for several regions of the calorimeter. The functions used in this analysis as the same
as the ones used in the top mass analysis~\cite{JetTF}. An example plot of the
average energy difference between partons and jets for each jet flavor is shown
in Fig~\ref{FlavorTF}.

\vspace{0.1in}
\begin{figure}[!h!tbp]
\begin{center}
\includegraphics[width=0.48\textwidth]{figures/transfer/tfs.eps}
\end{center}
\vspace{-0.1in}
\caption[2jets]{Energy difference between a jet a match parton for three types
of jets.}
\label{FlavorTF}
\end{figure}

The muon transfer functions were determined for muons with and without
an SMT hit and are parametrized as a function of
$\large{\frac{1}{p_{T}}}$. The functions used in this analysis were
again the same as the ones used in the top mass
analysis~\cite{MuonTF,MuonTF2}. Lei Wang provided us the electron
resolution as a function of $\eta$ and $E$, out of which we made
Gaussian transfer functions.~\cite{ElectronTF}.

The event probabilities shown in Eq.~\ref{prob} assume a particle
assignment of a jet to a parton in the final state. In practice, this assignment
is not known and we must sum over all possible assignments to ensure the correct
one is chosen. This analysis uses
information from the neural network $b$ tagger to weight event where a b-Jet is
assigned to a $b$ quark and an untagged jet is assigned to a light flavor quark. The factor
$\varepsilon(\vec{x})$ is included in the probability calculation as the efficiency
for this $b$-tagging assignment. This efficiency is described in more detail in
appendix~\ref{JetPartonAssigmnent}.

For two-jet events, the event probability is then re-written as a sum over
assignment possibilities as shown in Eq.~\ref{prob2jet}.

\begin{equation}
\label{prob2jet}
P(\ell, \nu, j_{1}, j_{2}) = P(\ell, \nu, \rm{j}_{1} \rightarrow p_{1}, j_{2}
\rightarrow p_{2}) + P(\ell, \nu, \rm{j}_{1} \rightarrow p_{2}, j_{2} \rightarrow p_{1})
\end{equation}

%\noindent and the new event
%probability for three-jet events is shown in Eq.~\ref{prob3jet}.
%\begin{eqnarray}
%\label{prob3jet}
%\nonumber
%P(\ell, \nu, j_{1}, j_{2}, j_{3}) = P(\ell, \nu, \rm{j}_{1} \rightarrow p_{1}, j_{2} \rightarrow p_{2}, j_{3} \rightarrow p_{3}) + P(\ell, \nu, \rm{j}_{1} \rightarrow p_{1}, j_{2} \rightarrow p_{3}, j_{3} \rightarrow p_{2}) + \\
%\nonumber
%P(\ell, \nu, \rm{j}_{1} \rightarrow p_{2}, j_{2} \rightarrow p_{1}, j_{3} \rightarrow p_{3}) + P(\ell, \nu, \rm{j}_{1} \rightarrow p_{2}, j_{2} \rightarrow p_{3}, j_{3} \rightarrow p_{1}) + \\
%P(\ell, \nu, \rm{j}_{1} \rightarrow p_{3}, j_{2} \rightarrow p_{2}, j_{3} \rightarrow p_{1}) + P(\ell, \nu, \rm{j}_{1} \rightarrow p_{3}, j_{2} \rightarrow p_{1}, j_{3} \rightarrow p_{2})
%\end{eqnarray}

\subsection{Matrix Elements}

The matrix element analysis is named so because the differential cross section for the hard scatter interaction is proportional to the leading order matrix element~\cite{PDG} as shown in Eq.~\ref{me}

\begin{equation}
\label{me}
d\sigma_{hs} = \frac{(2\pi)^4}{4} \frac{{|\cal M|}^{2}}{\sqrt{(q_{1}q_{2})^2 - m_{1}^2
m_{2}^2}} d\Phi_{n}(y)
\end{equation}

\noindent For this analysis, we calculate five probabilities for two-jet events and
three probabilities for three-jet events. For two-jet events events, we
calculate the probability for $s$-channel single top ($u\bar{d}$ $\rightarrow$
$t\bar{b}$), $t$-channel single top ($ub$ $\rightarrow$ $td$), Wbb production
($u\bar{d}$ $\rightarrow$ $Wb\bar{b}$), Wcg production ($sg$ $\rightarrow$
$W\bar{c}g$), and Wgg production ($u\bar{d}$ $\rightarrow$ $Wgg$). The leading order diagrams for these channels are shown in Fig~\ref{2jets}.

\vspace{0.1in}
\begin{figure}[!h!tbp]
\begin{center}
\subfigure[]{
	\includegraphics[width=0.24\textwidth]{figures/feynman/tb.eps}
}
\subfigure[]{
	\includegraphics[width=0.24\textwidth]{figures/feynman/tq.eps}
}
\subfigure[]{
	\includegraphics[width=0.24\textwidth]{figures/feynman/wbb.eps}
}
\subfigure[]{
	\includegraphics[width=0.24\textwidth]{figures/feynman/wcg.eps}
}
\subfigure[]{
	\includegraphics[width=0.24\textwidth]{figures/feynman/wgg.eps}
}
\end{center}
\vspace{-0.1in}
\caption[2jets]{The leading order matrix elements used for event probability
calculation for events with two jets. (a) $u\bar{d}$ $\rightarrow$ $t\bar{b}$,
(b) $ub$ $\rightarrow$ $td$, (c) $u\bar{d}$ $\rightarrow$ $Wb\bar{b}$, (d) $sg$
$\rightarrow$ $W\bar{c}g$, and (e) $u\bar{d}$ $\rightarrow$ $Wgg$}
\label{2jets}
\end{figure}

%For events with three jets, we use four matrix elements, $s$-channel with gluon
%radiation ($u\bar{d}$ $\rightarrow$ $t\bar{b}g$), $t$-channel with associated $b$
%quark ($ug$ $\rightarrow$ $t\bar{b}d$), Wbbg ($u\bar{d}$ $\rightarrow$
%$Wb\bar{b}g$), and Wugg ($g\bar{d}$ $\rightarrow$
%$W\bar{u}gg$. The leading order Feynman diagrams are shown in Fig~\ref{3jets}.

%\vspace{0.1in}
%\begin{figure}[!h!tbp]
%\begin{center}
%\subfigure[]{
%	\includegraphics[width=0.24\textwidth]{figures/feynman/tbg.eps}
%}
%\subfigure[]{
%	\includegraphics[width=0.24\textwidth]{figures/feynman/tqb.eps}
%}
%\subfigure[]{
%	\includegraphics[width=0.24\textwidth]{figures/feynman/wbbg.eps}
%}
%\subfigure[]{
%	\includegraphics[width=0.24\textwidth]{figures/feynman/wugg.eps}
%}
%\end{center}
%\vspace{-0.1in}
%\caption[3jets]{The leading order matrix elements used for event probability
%calculation for events with two jets.  (a) $u\bar{d}$ $\rightarrow$ $t\bar{b}g$,
%(b) $ug$ $\rightarrow$ $t\bar{b}d$, (c) $u\bar{d}$ $\rightarrow$ $Wb\bar{b}g$,
%and (d) $g\bar{d}$ $\rightarrow$ $W\bar{u}gg$}
%\label{3jets}
%\end{figure}

\subsection{Cross Sections}

To obtain a properly normalized probability density value for each event it is
necessary to calculate the signal or background cross section as defined in
Eq.~\ref{cross}. Summarized in table~\ref{crosssections} are the cross
sections used for each signal and background process.

All cross sections were calculated with these parton level cuts

\begin{itemize}
\item Parton Isolation: $\Delta$R($q_{i}$, $q_{j}$) $> 0.5$
\item Min Parton Energy: $q_{i}$ $P_{T}$ $> 6$ GeV
\item Max Parton Angle: $q_{i}$ $\eta <$ 3.5
\item No cuts were applied to the lepton or neutrino
\end{itemize}

and the following selection cuts on reconstructed objects

\begin{itemize}
\item Lepton $P_{T}$ $>$ 15.0 GeV
\item Electron(Muon) $|\eta|$ $<$ 1.1(2.0)
\item Missing $E_{T}$ $>$ 15.0 GeV
\item Leading Jet $P_{T}$ $>$ 25.0 GeV
\item Leading Jet $|\eta|$ $<$ 2.5
\item Second Jet $P_{T}$ $>$ 20.0 GeV
\item Second Jet $|\eta|$ $<$ 3.5
%\item Third Jet $P_{T}$ $>$ 20.0 GeV (if 3 jet event)
%\item Third Jet $|\eta|$ $<$ 3.5 (if 3 jet event)
\end{itemize}

All $W$+jets cross sections used $M_{W}^{2} + \sum_{jets}({M_{i}^2 + P_{i,T}^2})$ as
the factorization and renormalization scales while the single top cross sections
used $M_{\rm{top}}^{2}$. All cross sections have less than 1$\%$ error from the
Monte Carlo integration.

\vspace{0.05in}
\begin{table}[!h!tbp]
\begin{center}
\begin{minipage}{4.5 in}
\begin{ruledtabular}
\begin{tabular}{l||ccc}
\multicolumn{4}{c} {\hspace{0.5in}\underline{Cross Section $\times$ Branching Ratio For Each Source}}\vspace{0.1in} \\
Source					&	Lepton		&	Number of $b$ tags	&
	$\sigma \times B$ [fb]		\\
\hline
$p\bar{p}$ $\rightarrow$ $tq$		&	Electron	&	1	&
	19.6			\\
$p\bar{p}$ $\rightarrow$ $tq$		&	Electron	&	2	&
	0.27			\\
$p\bar{p}$ $\rightarrow$ $tq$		&	Muon	&	1	&
	26.8			\\
$p\bar{p}$ $\rightarrow$ $tq$		&	Muon	&	2	&
	0.38			\\
\hline
%$p\bar{p}$ $\rightarrow$ $tqb$		&	Electron	&	1	&
%	6.34			\\
%$p\bar{p}$ $\rightarrow$ $tqb$		&	Electron	&	2	&
%	5.40			\\
%$p\bar{p}$ $\rightarrow$ $tqb$		&	Muon	&	1	&
%	8.56			\\
%$p\bar{p}$ $\rightarrow$ $tqb$		&	Muon	&	2	&
%	7.40			\\
%\hline
$p\bar{p}$ $\rightarrow$ $t\bar{b}$	&	Electron	&	1	&
	8.07			\\
$p\bar{p}$ $\rightarrow$ $t\bar{b}$	&	Electron	&	2	&
	6.90			\\
$p\bar{p}$ $\rightarrow$ $t\bar{b}$	&	Muon	&	1	&
	10.4			\\
$p\bar{p}$ $\rightarrow$ $t\bar{b}$	&	Muon	&	2	&
	8.90			\\
\hline
%$p\bar{p}$ $\rightarrow$ $tbg$		&	Electron	&	1	&
%	6.02			\\
%$p\bar{p}$ $\rightarrow$ $tbg$		&	Electron	&	2	&
%	5.22			\\
%$p\bar{p}$ $\rightarrow$ $tbg$		&	Muon	&	1	&
%	7.64			\\
%$p\bar{p}$ $\rightarrow$ $tbg$		&	Muon	&	2	&
%	6.66			\\
%\hline
$p\bar{p}$ $\rightarrow$ $Wb\bar{b}$	&	Electron	&	1	&
	29.5			\\
$p\bar{p}$ $\rightarrow$ $Wb\bar{b}$	&	Electron	&	2	&
	24.6			\\
$p\bar{p}$ $\rightarrow$ $Wb\bar{b}$	&	Muon	&	1	&
	41.9			\\
$p\bar{p}$ $\rightarrow$ $Wb\bar{b}$	&	Muon	&	2	&
	34.7			\\
\hline
%$p\bar{p}$ $\rightarrow$ $Wb\bar{b}g$	&	Electron	&	1	&
%	16.5			\\
%$p\bar{p}$ $\rightarrow$ $Wb\bar{b}g$	&	Electron	&	2	&
%	14.3			\\
%$p\bar{p}$ $\rightarrow$ $Wb\bar{b}g$	&	Muon	&	1	&
%	23.1			\\
%$p\bar{p}$ $\rightarrow$ $Wb\bar{b}g$	&	Muon	&	2	&
%	19.9			\\
\hline
$p\bar{p}$ $\rightarrow$ $W\bar{c}g$	&	Electron	&	1	&
	36.4			\\
$p\bar{p}$ $\rightarrow$ $W\bar{c}g$	&	Electron	&	2	&
	0.33			\\
$p\bar{p}$ $\rightarrow$ $W\bar{c}g$	&	Muon	&	1	&
	54.0			\\
$p\bar{p}$ $\rightarrow$ $W\bar{c}g$	&	Muon	&	2	&
	0.61			\\
\hline
$p\bar{p}$ $\rightarrow$ $Wgg$		&	Electron	&	1	&
	52.3			\\
$p\bar{p}$ $\rightarrow$ $Wgg$		&	Electron	&	2	&
	0.33			\\
$p\bar{p}$ $\rightarrow$ $Wgg$		&	Muon	&	1	&
	74.5			\\
$p\bar{p}$ $\rightarrow$ $Wgg$		&	Muon	&	2	&
	0.47			\\
\hline
\end{tabular}
\end{ruledtabular}
\vspace{-0.1 in}
\caption[obs_me]{Cross sections in femtobarns for each channel, lepton, and number of $b$ tags.}
\label{crosssections}
\end{minipage}
\end{center}
\end{table}

\clearpage
\subsection{Discriminant Definition}

After calculating event probabilities we combine them into a discriminant output variable, D, defined in Eq.~\ref{disc}

\begin{equation}
\label{disc}
D(\vec{x}) = \frac{P_{\rm{signal}}(\vec{x})}{P_{\rm{signal}}(\vec{x}) + P_{\rm{background}}(\vec{x})}
\end{equation}

\noindent For events with two jets the discriminant for either $s$-channel or $t$-channel as signal is defined as

\begin{equation}
D(\vec{x})_{\rm{s|t}} = \frac{P_{\rm{s|t}}(\vec{x})}{P_{\rm{s|t}}(\vec{x}) +
c_{wbb}P_{\rm{wbb}}(\vec{x}) + c_{wcg}P_{\rm{wcg}}(\vec{x}) + c_{wgg}P_{\rm{wgg}}(\vec{x})}
\end{equation}

\noindent where $c_{wbb}$, $c_{wcg}$, and $c_{wgg}$ are, in principle, the relative
background fractions for each background in the data; however, these background
fractions are optimized for the analysis.

%Events with three jets the discriminant for $s$-channel and $t$-channel is defined as
%
%\begin{equation}
%D(\vec{x})_{\rm{s|t}} = \frac{P_{\rm{s|t}}(\vec{x})}{P_{\rm{s|t}}(\vec{x}) + P_{\rm{wbbg}}(\vec{x})}
%\end{equation}

\noindent The background fractions for the two-jet discriminant were found by a grid
search of each point ($c_{wcg}$, $c_{wb\bar{b}}$, $c_{wgg}$) and calculating the expected
Bayes ratio defined as the maximum of the two dimensional posterior divided by
the value of the posterior at zero signal cross section. This procedure was used
by each analysis in the single top group to optimize the final discriminant
variable. The values of the background fractions are summarized in table~\ref{frac}.

\vspace{0.05in}
\begin{table}[!h!tbp]
\begin{center}
\begin{minipage}{4.5 in}
\begin{ruledtabular}
\begin{tabular}{l||cccc}
\multicolumn{5}{c} {\hspace{0.5in}\underline{Background Fractions For Each Channel}}\vspace{0.1in} \\
Lepton		&	Number of $b$ tags	&	$C_{Wbb}$	&	$C_{Wcg}$	&	$C_{Wgg}$\\
\hline
Electron	&		1		&	$\frac{1}{5}$	&	$\frac{2}{5}$	&	$\frac{2}{5}$	\\
Electron	&		2		&	$\frac{2}{3}$	&	0		&	$\frac{1}{3}$	\\
Muon		&		1		&	$\frac{2}{5}$	&	$\frac{2}{5}$	&	$\frac{2}{5}$	\\
Muon		&		2		&	1		&	0		&	0		\\
\end{tabular}
\end{ruledtabular}
\vspace{-0.1 in}
\caption[frac]{Background fractions for each lepton and number of $b$ tags channel.}
\label{frac}
\end{minipage}
\end{center}
\end{table}


% ========== Synopsis

\chapter {Abstract}

\section{Abbreviated introduction and plan}

My dissertation is focused on engineering direct cortical stimulation (DCS) in humans implanted with electrocorticography (ECoG) and deep brain stimulation (DBS) electrodes. Specifically, my work aims to understand the spread of DCS through cortex for future targeting and modeling of DCS (Aim 1), to develop algorithms and share human stimulation data sets (Aim 1), to characterize the behavioral and cortical response to DCS relative and in addition to natural peripheral feedback (Aim 2), and to investigate the ability to enhance neural plasticity through DCS (Aim 3). Through a comprehensive understanding of both the physical, neural, and behavioral effects of stimulation, we can better engineer devices to treat disorders of the nervous system and restore lost function.

\section{Aims overview}

\subsection{Aim 1: Modeling cortical stimulation and interpreting its effects on neural dynamics}

In this aim, we aim to better understand the resistivities of different cortical layers to inform future modeling approaches, to compare finite element method (FEM) models to analytic models, as well as to use principled mathematical techniques to extract neural activity in the context of stimulation. The first element of engineering electrical stimulation in humans is to understand where the applied currents flow. The subsequent step is to disentangle the neural response to this stimulation from the ongoing physical propagation of the stimulation. Prior efforts have successfully created patient specific models and sophisticated FEM \cite{Kim2015}, as well as demonstrated methods that putatively extract the neural data during stimulation. However, numerous fundamental, core elements of cortical stimulation critical to effective model development, including a precise characterization of charge transmission through the brain, still require substantial exploration. Further, there is no clear consensus, nor clearly documented and published code and data, on the best method for extracting the neural response to cortical stimulation \cite{Zhou2018}. What are the simple, core elements of cortical stimulation that we can model and understand to gain intuition into how neural stimulation propagates through cortex? How can we develop algorithms to extract the neural response to cortical stimulation?

\subsubsection{Aim 1.1: Creation of interpretable, analytic models that explain the resistivity of various cortical layers}

The first part of this aim was to see how a one-layer apparent resistivity model would fit our data, working along with Larry Sorensen at the University of Washington and collaborators at the University of Utah, Northeastern University, and University of Freiburg. We found that across 8 subjects, a one-layer model inspired by work in geophysics and the semiconductor industry \cite{Miccoli2015} reliably fit our data and suggested that many of the published values for the resistivities of grey, white, and cerebrospinal fluid (CSF) may be incorrect. A one-layer, apparent resistivity model captures all of the variability in different layers as a single value. The correct modeling of cortical stimulation, and accurate resistivity values for different cortical layers is necessary not only for targeted DCS, but also for source modeling and understanding how the signals we record on the surface are affected by the layers surrounding the neurons generating our observed signals.

Future work will involve extending our work to a circular electrode, 3-layer analytic model \cite{Schumann1969a} to understand the potential contributions of the different thicknesses and resistivities of cortical layers (CSF, gray, and white matter) and finite-sized electrodes. Through this, we will established physiologically relevant parameters of layer thickness and resistivities that best fit experimentally acquired data across subjects. We developed algorithms to better extract and quantify the steady state voltage during stimulation, which we will use to better calculate the apparent resistivity in the lower signal to noise parts of our data. We will align all of our data on a common grid to average across subjects and minimize variability due to any single subject.

An additional avenue will be careful characterization of the clinical ECoG grids and DBS electrodes in a saline solution to better understand and model our stimulation. By analyzing the applied stimulation curves and subsequent electrical measurements on other channels, we can extract information about the capacitance and resistance of the electrode-tissue interface, as well as the resistance of the tissue. We can further compare our results to those of our collaborators at the University of Utah and Northeastern University who are developing patient-specific finite element models, in order to better understand the impact of patient specific geometry.

\subsubsection{Aim 1.2: Extraction of neural signals during concurrent DCS, and open source software and data to aid replication}

This work is aimed at extracting the underlying neural activity and neural responses to DCS from electrical artifacts, with Bingni Brunton and Nathan Kutz. The measured voltages in Aim 1.2, which are our signal of interest for understanding the spread of current through the cortex, are often three orders of magnitude greater than the underlying neural activity of interest.

We are developing methods that perform better than the commonly used simple linear interpolation and channel-wise averaging \cite{Zhou2018}, and offer speedups relative to methods that require signal realignment and tissue electrode interface modeling. Currently, we create a dictionary of potential stimulation artifact templates for each channel across all measured trials, against which the best matching artifact for any given stimulation pulse can be removed. In this way, we seek to account for variable stimulation magnitudes that can arise during stimulation. Following stimulation, we quantify the reduction in artifact as the root mean square (RMS) decrease in average signal amplitude during a trial, as well as the reduction in power at the stimulation frequencies, along with maintenance of the physiologic frequencies of interest.

Future validation will include explorations of stimulation parameters under which this method may be applicable, including sampling rates, relative pulse widths, and recording modalities (DBS, ECoG). Further, a synthetic data set using clustered artifact templates will be generated and tested against to best validate our method. To assess the preservation of power in frequency bands of interest, we will further use techniques such as the Lomb-Scargle periodogram \cite{Muller2017} to estimate the average power in different frequencies across trials, and quantify the difference in average power between pre- and post-processed trials. In order to assess cluster viability, we plan to quantify the within cluster distances and between cluster distances to establish if meaningful clusters are extracted. Additionally, we plan to take sections (folds) of the data, and test the performance of the algorithm on individual folds. We will take artifacts that were not part of the clusters, and see how well they map onto the discovered clusters.

An additional further avenue to develop improved methods of artifact removal is upsampling of each artifact period, and employing a variant of principal component regression \cite{OShea2017} in which we regress the artifact in each channel against the components from nearby channels which have the same stimulation polarity during each phase of stimulation. A potential alternative to pursue would be the implementation of a Dirichlet Process Mixture Model \cite{WhyeTeh2004,Teh2010} in order to use a non-parametric model to cluster the data, and potentially include time dependencies.

To aid in reproducibility and further algorithm development in the field, we seek to publish our algorithm and example data openly.

\subsection{Aim 2: Neural and behavioral effects of cortical stimulation in somatosensory cortex }

In this aim, we will determine how humans behaviorally respond to DCS applied to cortex contrasting native stimuli with artificial feedback. A secondary aim is to better understand neural dynamics in the context of stimulation. These aims are motivated in part by the notion that, in a real world neuroprosthetic, stimulation will be delivered during ongoing behavior and neural activity. In a closed-loop system, the latency between various brain computer interface (BCI) components needs to be accounted for in order to achieve the desired performance. What are the latencies for sensory restoration in sensory cortex, what are the neural dynamics relative to natural feedback, and can we engineer stimulation to modify these latencies? How do humans respond to both engineered stimulation and natural haptic feedback which overlap spatially and temporally?

\subsubsection{Aim 2.1: Slower response times to trains of DCS relative to natural haptic feedback in the same spatial location}

A first step in understanding the behavioral effects of DCS is to compare the behavioral performance of DCS to natural sensory input. We did this by designing a response timing task where epilepsy patients with in-dwelling macroscale ECoG grids were asked to respond as quickly as possible to DCS trains of differing lengths applied to primary somatosensory cortex (S1), as well as to natural, haptic stimuli. We observed significantly faster reaction times to haptic stimuli relative to DCS, regardless of DCS train length and despite the fact that DCS circumvents ascending sensory pathways. This matches previous NHP work of ICMS stimulation of S1 \cite{Godlove2014a}. Collectively, the increased latency in responding to DCS relative to haptic stimuli would need to be accounted for in any future neuroprosthetic device.

This is completed work in the process of submission.

\subsubsection{Aim 2.2: Comparisons between temporal dynamics and time frequency responses to haptic stimuli and DCS in somatosensory cortex}

Motivated by the results in Aim 2.1, we seek to understand the neural differences between natural haptic stimuli and DCS stimulation of sensory cortex. We will use the neural data acquired during Aim 2.1 and the methods of Aim 1.2 to compare the neural response to DCS relative to that of haptic stimuli, in order to better understand the cortical processing of DCS to S1.

We will compare broadband high gamma signals and event related potentials (ERPs) following stimulation onset, as well as differences in the time frequency maps between the two conditions. Preliminary analysis reveals some similarities in both the time series and time frequency maps in select, adjacent electrodes immediately surrounding sensory cortex, but additional responses unique to the natural haptic stimuli (i.e. absent in DCS trials) in somatosensory association areas.

Additional analysis will include characterizing canonical oscillatory band (theta, 4-8 Hz, alpha, 8-12 Hz) activity and whether specific frequency bands track differences in response times in different trials. Particular metrics will include phase-locking to analyze phase relationships between different channels in relation to conscious awareness of DCS, as well as event related phase amplitude coupling between theta and high gamma regimes \cite{Voytek2013}.

For future experiments, we have designed waveforms with initial high pulses, followed by trains of lower stimulation amplitude, which may better represent the cortical response to natural peripheral stimuli. Preliminary analysis shows a trend towards faster reaction times with initial high pulses. Future experiments will involve ensuring charge balanced waveforms are compared, in order to better understand the parameters of the waveform important for response.

Preliminary work has been completed (2 subjects), and experiments are ongoing.

\subsubsection{Aim 2.3: The behavioral and neural effects of temporally simultaneous and spatially overlapping haptic stimuli and DCS}

In dynamic, closed-loop neuroprosthetics that provide feedback to the user, both natural and artificial feedback will be arriving concurrently. To better understand this, we developed a temporal order judgement task (TOJ), in which haptic stimuli and DCS are delivered with varying degrees of stimulus onset asynchrony (SOA) \cite{Miyazaki2016}.

Preliminary analysis suggests that the response time delay to DCS relative to haptic stimuli predicts a region of overlap where haptic stimuli and DCS will be judged as potentially occurring simultaneously. We also demonstrate no specific masking of perception of either stimulus, suggesting that S1 cortex is able to process and separate spatiotemporally overlapping sensory modalities.

Our future work will involve continuing to collect data on this task, and investigating the functional neuroanatomy reported in fMRI studies during temporal order judgement tasks of tactile and visuo-tactile stimuli. Areas of potential interest include left ventral, bilateral dorsal premotor, and left posterior parietal cortices \cite{Miyazaki2016,Takahashi2013}.

Preliminary work has been completed (2 subjects), and experiments are ongoing.

\subsection{Aim 3: Engineered DCS protocols to induce changes in cortical connectivity, plasticity, and output mapping }

The optimal engineering of stimulation for clinical applications requires that the myriad of different protocols, developed theoretically and in animal models, will need to be validated in humans. What neural engineering applications from animal studies will best translate to human patients? Current experimental and patient constraints prevent a full exploration of established protocols. However, we can establish baseline forays into the efficacy of cycle triggered \cite{Zanos2013}, paired pulse \cite{Seeman2017}, and EMG triggered \cite{Lucas2013} stimulation protocols. These protocols are focused on enhancing plasticity in cortex, with the ultimate goal of restoring function to damaged cortex. Additionally, we seek to validate other output metrics for plasticity and cortical excitability in humans to assess the rehabilitative potential of different paradigms. These metrics include motor evoked potential (MEP) measurements such as motor thresholds (MT), electromyographic (EMG) amplitudes, as well as the overall motor pattern elicited by motor cortex stimulation. These protocols are focused on enhancing plasticity in cortex, with the ultimate goal of restoring function to damaged cortex. Lessons from these experiments would be important for clinical trials in human patients.

\subsubsection{Aim 3.1: Beta-oscillation triggered stimulation paradigm to enhance connectivity of beta-synchronous regions}

By stimulating cortical areas during periods of heightened activity and excitability, we may be able to induce plasticity in cortex. This a collaborative effort with Jared Olson and Jeremiah Wander, inspired by primate work from Stavros Zanos and Eberhard Fetz at the University of Washington \cite{Zanos2013}. Specifically, beta (12-25 Hz) oscillations, which are thought to represent coordinated synaptic drive to excitatory pyramidal cells in cortical areas such as motor cortex \cite{Sherman2016}, have been shown in primate research to hold potential for closed-loop enhancement of plasticity. By stimulating during consistent phases of heightened beta power in the oscillatory cycle, we may be able to induce plasticity in cortex which could help with future rehabilitative applications. Evoked potentials (EPs) in adjacent areas of cortex serve as a proxy for connectivity, and changes in evoked potential size may be markers of local plasticity. By quantifying the changes in EPs, we can assess how plasticity may be affected by stimulation paradigms.

Thus far, we have observed a dose dependent effect on EPs during closed-loop, real-time beta oscillation triggered stimulation across seven subjects, measured by the change in EP magnitude in either the channel we recorded our beta oscillations from, or in an adjacent channel. The phase of stimulation does not seem to be a large contributing factor, but the greatest degree of enhancement may be when stimuli are consistently delivered during the hyperpolarizing surface potential. In one subject, we demonstrated a smaller increase, or no statistically significant increase in EP magnitude on adjacent beta-recording channels during a playback control condition. This work demonstrates the potential utility of closed-loop, oscillation based stimulation in humans. Future work will assess changes in resting state connectivity, and the correspondence between accuracy of phase of delivery and changes in EP magnitude.

For this experiment, data from 7 subjects have been collected, and analysis is ongoing.

\subsubsection{Aim 3.2:  Paired pulse stimulation to enhance connectivity between two cortical sites}

In order to validate other potentially clinically useful stimulation paradigms in humans, we will implement a paired pulse stimulation paradigm inspired by previous NHP and rodent work \cite{Seeman2017,Rebesco2010}. Importantly, this modality does not require real time recording from cortex concurrently with stimulation, enabling simpler hardware for implanted devices. This is inspired by the work of Stephanie Seeman, Brian Mogen, and Steve Perlmutter at the University of Washington. Here, in DBS patients undergoing DBS electrode placement, we will intraoperatively stimulate two connected regions of the cortex with fixed lags previously demonstrated to maximize facilitation and inhibition of cortical connectivity. Previous research has demonstrated an increase in connectivity between two regions as assessed through EP magnitude by consistently stimulating two connected regions with a latency determined to maximize long term potentiation (LTP),  \cite{Seeman2017}. Our main outcome variables will be motor evoked potential (MEP) amplitude as assessed through electromyography (EMG) recordings, motor thresholds (MT), and EP magnitude before and after conditioning. We hypothesize that we will observe an increase in MEP amplitude, EP size, and a decrease in MT with stimulation at the presynaptic site leading the postsynaptic site by 25 ms. However, if we use a lag of 100 ms between presynaptic and postsynaptic stimulation (A before B), or in a further experimental condition stimulate site B by 25 before site A (B before A, postsynaptic before presynaptic), we anticipate no changes in the above mentioned output metrics, or the opposite direction of changes.

This experiment is already programmed, and data has been collected on one subject.

\subsubsection{Aim 3.3: EMG triggered stimulation to remap motor cortex output}

Another potentially clinically viable stimulation paradigm is the use of EMG triggered stimulation as a marker for increased cortical activity in motor cortical regions \cite{Lucas2013}. This is informed by prior work from Timothy Lucas and Eberhard Fetz at the University of Washington. Following arm movements, motor cortex demonstrates a heightened period of activity that is greater than the window for LTP following spontaneous cortical spiking \cite{Edwardson2013,Crammond2000}. Critically, this modality does not require real time recording from cortex concurrently with stimulation, enabling simpler hardware.

We will implement this paradigm in patients with in-dwelling ECoG electrodes as they are recovering in the hospital. We will record EMG activity from one muscle group (biceps, for example), and trigger stimulation in an adjacent motor cortical region (electrodes covering deltoid muscle motor cortex, for example) when the EMG amplitude exceeds a threshold. Our control condition will be the playback of the recorded stimulation pattern, irrespective of the underlying EMG activity. Similar to Aim 3.2, we will use changes in MEP amplitude and MT, as well as EP magnitude, to assess the effects of conditioning on both site A and site B.

We expect that after a short conditioning period (on the order of 20-30 minutes), we will observe a facilitation of the connection between the two cortical regions of these two muscle groups, and that upon stimulation of site B, we will see muscles recruited that were originally more prominent for site A. For cortical connectivity as assessed by EP testing, we expect to see greater EP magnitudes following test pulse stimulation, indicating a strengthened connection between site A and site B. Additionally, we expect to see a decreased MT for site B. Upon MEP stimulation of the site responsible for the recorded muscle group. During the control condition, we expect to see no change, or the opposite of the changes described above (decreased connectivity)

This experiment will be programmed and tested in patients with in-dwelling ECoG electrodes later this year and early next year.

\subsection{Related publications and presentations}

\noindent Caldwell DJ, Cronin JA, Rao RPN, Ko AL, Ojemann JG, Sorensen LB, “What is the resistivity of the human brain? Insights from direct electrical stimulation, electrocorticographic recordings of the human cortex, and analytic models”, BMC Neuroscience 2018, CNS 2018 - Aim 1.1
\medskip

\noindent Caldwell DJ, Cronin JA, Rao RPN, Ko AL, Ojemann JG, Sorensen LB, “What is the resistivity of the human brain? Insights from direct electrical stimulation, electrocorticographic recordings of the human cortex, and analytic models”, Computational Neuroscience Meeting 2018, Seattle, WA, July 2018 - Aim 1.1
\medskip

\noindent Caldwell DJ*, Cronin JA*, Wu J, Weaver K, Ko AL, Rao RPN, Ojemann JG, “Cortical stimulation results in slower reaction times compared to peripheral touch in humans” *These authors contributed equally - in resubmission - Aim 2.1
\medskip

\noindent Olson JD*, Caldwell DJ*, Wander JD,  Zanos S, Sarma D, Su D, Cronin JA, Collins K, Wu J, Casimo K, Johnson L, Buckley R, Richardson A, Weaver K, Fetz E, Rao, RPN, Ojemann JG, “Dose dependent enhancement of Cortico-Cortical Evoked Potentials during Beta-Oscillation Triggered Direct Electrical Stimulation” - in preparation - Aim 3.1
\medskip

\noindent Caldwell DJ, “Understanding the Neural Response to Direct Cortical Stimulation”, UW Data Science Summit, April 2018 - Aim 1.2, 2.2
\medskip

\noindent Caldwell DJ, "Engineering direct cortical stimulation in humans", UW Neural Computation and Engineering Connection, January 2018 - Aim 1.2, 2.2
\medskip

\noindent Caldwell DJ, Cronin JA, Wu J, Kutz JN, Brunton BW, Weaver K, Rao RPN, Ojemann JG, “Spectrotemporal analysis of direct cortical stimulation compared to haptic stimulation in a response timing task in humans”, Society for Neuroscience – Annual Meeting, Washington DC, District of Columbia, November 2017 - Aim 1.2, 2.2
\medskip

\noindent Caldwell DJ, Cronin JA, Wu J, Weaver K, Rao RPN, Ojemann JG, “Direct Cortical Stimulation Results in Slower Reaction Times Compared to Peripheral Touch in Humans”, OHBM 2017, Vancouver, Canada, June 2017 - AIm 1.2, 2.2
\medskip

\noindent Caldwell DJ, “Behavioral and neural differences between haptic stimulation and direct cortical stimulation in humans: implications for neuroprosthetics”, 7th International BCI Meeting, Workshop: Perception of Sensation Restored through Neural Interfaces, Asilomar, CA, May 2018 - Aim 1.2, 2.2, 2.3
\medskip

\noindent Caldwell DJ, Olson JD, Wander JD,  Zanos S, Sarma D, Su D, Cronin J, Collins K, Wu J, Johnson L, Weaver K,  Casimo K, Fetz E, Rao RPN, Ojemann JG, “Exploration of the phase and dose dependence of cortico-cortical evoked potentials during beta-oscillation triggered direct electrical stimulation in humans”, Society for Neuroscience – Annual Meeting, San Diego, CA, November 2016 - Aim 3.1
\medskip

\noindent Caldwell DJ, Olson JD, Wander JD,  Zanos S, Sarma D, Su D, Cronin J, Collins K, Wu J, Johnson L, Weaver K, Fetz E, Rao RPN, Ojemann JG, “Enhancement of Cortico-Cortical Evoked Potentials by Beta-Oscillation Triggered Direct Electrical Stimulation in Humans”, NANS-NIC Meeting, Baltimore, MD, June 2016 - Aim 3.1
\medskip

\noindent Caldwell DJ, Olson JD, Wander JD, Zanos S, Sarma D, Su D, Cronin J, Collins K, Johnson L, Weaver K, Fetz E, Rao RPN, Ojemann JG, “Effect of distance on the magnitude and timing of Cortico-Cortical Evoked Potentials in oscillation triggered direct electrical stimulation in humans”, Neurofutures Meeting, Seattle, WA, June 2016 - Aim 3.1
\medskip

\noindent \textbf{Additional publications}
\medskip

\noindent Caldwell DJ*, Wu J*, Casimo K, Ojemann JG, Rao RPN, “Interactive Web Application for Exploring Matrices of Brain Connectivity”, Int. IEEE/EMBS Conf. Neural Eng. NER, pp. 42–45, 2017 , doi:10.1109/NER.2017.8008287, *These authors contributed equally
\medskip

\noindent Wu J, Casimo K, Caldwell DJ, Rao RPN, Ojemann JG, “Electrocorticographic Dynamics Predict Visually Guided Motor Imagery of Grasp Shaping “,Int. IEEE/EMBS Conf. Neural Eng. NER, pp. 199-202, 2017, doi:10.1109/NER.2017.8008325
\medskip


\chapter{Introduction}

\paragraph*{Dependent type theory.}

Dependent types are an important part of Martin-L\"of type theory \cite{mart84}, which
was conceived as a foundation for constructive mathematics.

Through the proofs-as-programs paradigm, they are fundamental to proof assistants
like Coq \cite{coq}, Lego \cite{pollack94theory} and Twelf \cite{pfenning99system}.
As programming languages receive more and more powerful type systems, there have been renewed efforts to use dependent types in programming languages: Cayenne \cite{augustsson98cayenne}, Agda \cite{norell:thesis} and Epigram \cite{epigram} strive to be seen more as practical programming languages rather than logical frameworks.

\paragraph*{Inductive families.}
Dependent type theory\footnote{for an introductory text, the reader is referred to \cite{NPS:promlt}}
 is often described as an open theory: new constants can be added to the system.
As a programming language, there is the need to define inductive types like lists and trees.
For logical purposes, inductive predicates have to be defined.

Inductive families \cite{dybjer94inductive} offer a powerful general scheme for defining inductive types.
A nice example for programming is matrix multiplication.
One could define a inductive family of types \verb+Mat+ that is \emph{indexed} by two natural numbers.
The type for matrix multiplication could then be given as
\begin{quote}
\begin{verbatim}
mmult : Mat k m -> Mat m n -> Mat k n
\end{verbatim}
\end{quote}
which is not possible with a simple type system.
\paragraph*{Pattern Matching.}
In traditional type theory, so-called elimination constants for each data type provide primitive recursion on objects.
As an alternative, it was suggested in \cite{coquand92pattern} to allow definitions by pattern matching, a well-known concept from functional languages \cite{DBLP:conf/fpca/Augustsson85}.

Pattern matching does allow for more readable and intuitive definitions, which is essential for type theory \footnote{Pattern matching is actually not a conservative extension to a theory with elimination constants.
Pattern matching definitions can only be translated to a traditional type theory by adding the so-called \emph{K Axiom} \cite{hofmann95groupoid,GoguHMcBrCM2006}.
} to be useful as a programming languages.
It has to be ensured that the pattern clauses cover all possible cases.
\paragraph*{Termination}
Furthermore, pattern matching allows unrestricted recursion, and functions with non-terminating computations have to be disallowed for the following reasons:
\begin{itemize}
\item
Proofs as programs need to be total functions to be correct.
\item
As types can depend on terms, computation is performed during type checking.
Thus, for type checking to be decidable, non-terminating computation cannot be allowed.
\end{itemize}
To be more concrete, let us look at a simple recursive definition by pattern matching on a list in the functional programming Haskell:
\begin{verbatim}
length :: [a] -> Int
length [] = 0
length (x:xs) = 1 + length xs
\end{verbatim}
The only recursive call is \texttt{length xs}.
The argument \texttt{xs} is a structural part of the input argument on the left hand side \texttt{(x:xs)}.
We conclude, if the list data type is well founded (i.e there are only finite lists), then the program \texttt{length} is guaranteed to terminate on all input lists.

\paragraph*{Term-based termination.}
We note that we looked only at the defining clauses of the function to decide termination, its type declaration was irrelevant. Such methods are called \emph{term-based} to differentiate them from the \emph{type-based} methods discussed below.

When pattern matching for dependent types was introduced in  \cite{coquand92pattern},
the test that was outlined above was suggested as the criterion for termination:
at least one argument needs to get \emph{structurally smaller} in each recursive call.
But not all defined functions have that clear \emph{structurally recursive} form.

In \cite{abelAltenkirch:predStRec}, a decision procedure for a simply typed language was given that also handles lexicographic orderings on arguments and mutual definitions.
The \emph{size change principle} \cite{lee01sizechange} subsumes this effort and also handles functions with so-called permuted arguments.

\paragraph*{Type-based termination.}
The above methods require that a declaration does not only need to be accepted by the \emph{type-checker}, but also pass a separate syntactic \emph{termination check}.
In contrast, \emph{type-based termination}\footnote{for an introduction, see \cite{abel:PhD}} refers to methods where the type-checker itself is in charge of checking termination: It is ensured by the typing rules.

The \emph{sized types} approach is an instance of type-based termination:
Inductive types are decorated with size annotations that denote an upper bound on the height of the objects inhabiting them.
It is then checked that this size is getting smaller during recursive calls.

In comparison with term-based termination, sized types are more robust to syntactical changes to the program and the type system is able to use the information that certain functions are \emph{size-preserving}.
Some effort \cite{bgp:lpar06} has been made to automatically infer these size annotations.

\paragraph{Infinite objects.}
Often there arises the need for potentially infinite objects, for example to model a continuous stream of network packages.
Streams, infinite sequences of elements, are the prime example of \emph{coinductive} types, where inhabitants can have
\emph{infinite height}.

While \emph{corecursive definitions} that create or manipulate infinite objects are inherently non-terminating, they still should be \emph{productive}. For productive definitions, looking at a finite portion of the object is well-defined.
As for termination, there are syntactic tests to guarantee productivity \cite{coquand-infinite}, but here again sized types offer a worthwhile alternative.

\paragraph*{Mugda.}
Our work started with an investigation of how a sized type approach could be added to a system like Agda.
The current version of Agda supports inductive families and mutual recursive function declarations by pattern matching.
The Agda termination checker was, at that time, based on the work of \cite{abelAltenkirch:predStRec}.

In the description above, sized types sound like a very different approach compared to term-based termination.
But, with inductive families and pattern matching, these approaches can be naturally combined.

As a result, the system \mugda was developed.
It supports inductive families and pattern matching.
As special features, the language supports coinductive definitions and also offers a built-in \emph{Size type} to form size annotations.
We outline here how termination and productivity checking of definitions is handled by \mugda:
\begin{itemize}
\item
a syntactic termination checker based on the size-change principle is employed.
\item
in addition, the Size type can be used to create sized data types, enabling advantages of the
sized type approach.
\item
productivity of corecursive functions is also guaranteed by the Size type.
\item
the usage of the Size type is controlled by the type checker.
\end{itemize}
\section{Overview}
We introduce the syntax and semantics of the \mugda language in the next chapter, and show some example programs.
The language described is missing the Size type that is later added in chapter 5.

Type-Checking for $\mugda$ is presented in chapter 3.
We cannot just focus on termination-checking because a termination-checker can easily be tricked by ill-typed declarations.

In the fourth chapter, a termination criterion for inductive function declarations based on the size-change principle is presented. Then we motivate and present an extension that increases its strength for definitions with nested patterns.

In chapter 5, the Size type is added to the language, which enables the formation of sized data types. It is shown how sized data types help to allow more recursive definitions pass the termination check of the third chapter.
It also enables the declaration of a broad range of productive corecursive definitions to pass this check.
Finally, we first motivate why the usage of size annotations needs to be restricted, and then give an \emph{admissibility criterion} to achieve this.

In the conclusion, we take a look back and list ideas for future work. The appendix contains a description of the implementation of $\mugda$.