\documentclass[11pt]{article}
\usepackage{reduce}
\title{EXCALC: A System for Doing Calculations in the Calculus of Modern
Differential Geometry}
\author{Eberhard Schr\"{u}fer \\
Institute SCAI.Alg   \\
German National Research Center for Information Technology (GMD) \\
Schloss Birlinghoven \\
D-53754 Sankt Augustin \\
Germany       \\[0.05in]
Email: eschruefer@ca-musings.de}
\begin{document}
\maketitle

\index{EXCALC package}

\section*{Acknowledgments}

This program was developed over several years. I would like to express
my deep gratitude to Dr.\ Anthony Hearn for his continuous interest in
this work, and especially for his hospitality and support during a
visit in 1984/85 at the RAND Corporation, where substantial progress
on this package could be achieved. The Heinrich Hertz-Stiftung
supported this visit. Many thanks are also due to Drs. F.W. Hehl,
University of Cologne, and J.D. McCrea, University College Dublin, for
their suggestions and work on testing this program.

\section{Introduction}

\index{differential geometry}
{\bf EXCALC} is designed for easy use by all who are familiar with the
calculus of Modern Differential Geometry.  Its syntax is kept as close
as possible to standard textbook notations.  Therefore, no great
experience in writing computer algebra programs is required.  It is
almost possible to input to the computer the same as what would have
been written down for a hand-calculation.  For example, the statement
\begin{verbatim}
                       f*x^y + u _| (y^z^x)
\end{verbatim}
\index{exterior calculus}
would be recognized by the program as a formula involving exterior
products and an inner product.  The program is currently able to
handle scalar-valued exterior forms, vectors and operations between
them, as well as non-scalar valued forms (indexed forms).  With this,
it should be an ideal tool for studying differential equations,
doing calculations in general relativity and field theories, or doing
such simple things as calculating the Laplacian of a tensor field for
an arbitrary given frame.  With the increasing popularity of this
calculus, this program should have an application in almost any field
of physics and mathematics.

Since the program is completely embedded in {\REDUCE}, all features and
facilities of {\REDUCE} are available in a calculation.  Even for those
who are not quite comfortable in this calculus, there is a good chance
of learning it by just playing with the program.

This is the last release of version 2. A much extended differential
geometry package (which includes complete symbolic index simplification,
tensors, mappings, bundles and others) is under development.

Complaints and comments are appreciated and should be send to the author.
If the use of this program leads to a publication, this document should
be cited, and a copy of the article to the above address would be
welcome.

\section{Declarations}

Geometrical objects like exterior forms or vectors are introduced to the
system by declaration commands.  The declarations can appear anywhere in
a program, but must, of course, be made prior to the use of the object.
Everything that has no declaration is treated as a constant; therefore
zero-forms must also be declared.

An exterior form is introduced by\label{PFORM} \index{PFORM statement}
\index{exterior form ! declaration}

\hspace*{2em} \k{PFORM} \s{declaration$_1$}, \s{declaration$_2$}, \ldots;

where
\begin{tabbing}
\s{declaration} ::= \s{name} $\mid$ \s{list of names}=\s{number} $\mid$  \s{identifier} $\mid$ \\
\s{expression} \\
\s{name} ::= \s{identifier} $\mid$ \s{identifier}(\s{arguments})
\end{tabbing}

For example
\begin{verbatim}
     pform u=k,v=4,f=0,w=dim-1;
\end{verbatim}
declares \texttt{U} to be an exterior form of degree \texttt{K}, \texttt{V} to be a
form of degree 4, \texttt{F} to be a form of degree 0 (a function), and \texttt{W}
to be a form of degree \texttt{DIM}-1.

If the exterior form should have indices, the declaration would be
\index{exterior form ! with indices}
\begin{verbatim}
     pform curv(a,b)=2,chris(a,b)=1;
\end{verbatim}
The names of the indices are arbitrary.

Exterior forms of the same degree can be grouped into lists to save typing.
\begin{verbatim}
     pform {x,y,z}=0,{rho(k,l),u,v(k)}=1;
\end{verbatim}
The declaration of vectors is similar. The command \texttt{TVECTOR}\label{TVECTOR}
takes a list of names. \index{TVECTOR command} \index{exterior form ! vector}

\hspace*{2em} \k{TVECTOR} \s{name$_1$}, \s{name$_2$}, \ldots;

For example, to declare \texttt{X} as a vector and \texttt{COMM} as a vector with
two indices, one would say
\begin{verbatim}
     tvector x,comm(a,b);
\end{verbatim}
If a declaration of an already existing name is made, the old
declaration is removed, and the new one is taken.

The exterior degree of a symbol or a general expression can be obtained
with the function \label{EXDEGREE} \index{EXDEGREE command}

\hspace*{2em} \k{EXDEGREE} \s{expression};

Example:
\begin{verbatim}
     exdegree(u + 3*chris(k,-k));

     1
\end{verbatim}

\section{Exterior Multiplication}

\index{"\^{} ! exterior multiplication} \index{exterior product}
Exterior multiplication between exterior forms is carried out with the
nary infix operator \^{ } (wedge)\label{wedge}.  Factors are ordered
according to the usual ordering in {\REDUCE} using the commutation
rule for exterior products.

\example\index{EXCALC package ! example}

\begin{verbatim}
     pform u=1,v=1,w=k;

     u^v;

     U^V

     v^u;

     - U^V

     u^u;

     0

     w^u^v;

           K
     ( - 1) *U^V^W

     (3*u-a*w)^(w+5*v)^u;

     A*(5*U^V^W - U^W^W)
\end{verbatim}

It is possible to declare the dimension of the underlying space
by\label{SPACEDIM} \index{SPACEDIM command} \index{dimension}

\hspace*{2em} \k{SPACEDIM} \s{number} $\mid$ \s{identifier};

If an exterior product has a degree higher than the dimension of the
space, it is replaced by 0:

\begin{verbatim}
     spacedim 4;

     pform u=2,v=3;

     u^v;

     0
\end{verbatim}

\section{Partial Differentiation}

Partial differentiation is denoted by the operator \texttt{@}\label{at}.  Its
capability is the same as the {\REDUCE} \texttt{DF} operator.
\index{"@ operator} \index{partial differentiation}
\index{differentiation ! partial}

\example\index{EXCALC package ! example}

\begin{verbatim}
     @(sin x,x);

     COS(X)

     @(f,x);

     0
\end{verbatim}

An identifier can be declared to be a function of certain variables.
\index{FDOMAIN command}
This is done with the command \texttt{FDOMAIN}\label{FDOMAIN}.  The
following would tell the partial differentiation operator that \texttt{F}
is a function of the variables \texttt{X} and \texttt{Y} and that \texttt{H} is
a function of \texttt{X}.

\begin{verbatim}
     fdomain f=f(x,y),h=h(x);
\end{verbatim}

Applying \texttt{@} to \texttt{F} and \texttt{H} would result in

\begin{verbatim}
     @(x*f,x);

     F + X*@  F
            X

     @(h,y);

     0
\end{verbatim}

\index{tangent vector}
The partial derivative symbol can also be an operator with a single
argument.  It then represents a natural base element of a tangent
vector\label{at1}.

\example\index{EXCALC package ! example}

\begin{verbatim}
     a*@ x + b*@ y;

     A*@  + B*@
        X      Y
\end{verbatim}

\section{Exterior Differentiation}
\index{exterior differentiation}
Exterior differentiation of exterior forms is carried out by the
operator \texttt{d}\label{d}.  Products are normally differentiated out,
\emph{i.e.}

\begin{verbatim}
     pform x=0,y=k,z=m;

     d(x * y);

     X*d Y + d X^Y

     d(r*y);

     R*d Y

     d(x*y^z);

           K
     ( - 1) *X*Y^d Z  + X*d Y^Z + d X^Y^Z
\end{verbatim}

This expansion can be suppressed by the command \texttt{NOXPND D}\label{NOXPNDD}.
\index{NOXPND ! D}

\begin{verbatim}
     noxpnd d;

     d(y^z);

     d(Y^Z)
\end{verbatim}

To obtain a canonical form for an exterior product when the expansion
is switched off, the operator \texttt{D} is shifted to the right if it
appears in the leftmost place.

\begin{verbatim}
     d y ^ z;

             K
     - ( - 1) *Y^d Z + d(Y^Z)
\end{verbatim}

Expansion is performed again when the command \texttt{XPND D}\label{XPNDD}
is executed. \index{XPND ! D}

Functions which are implicitly defined by the \texttt{FDOMAIN} command are
expanded into partial derivatives:

\begin{verbatim}
     pform x=0,y=0,z=0,f=0;

     fdomain f=f(x,y);

     d f;

     @  F*d X + @  F*d Y
      X          Y
\end{verbatim}

If an argument of an implicitly defined function has further
dependencies the chain rule will be applied \emph{e.g.} \index{chain rule}

\begin{verbatim}
     fdomain y=y(z);

     d f;

     @  F*d X + @  F*@  Y*d Z
      X          Y    Z
\end{verbatim}

Expansion into partial derivatives can be inhibited by
\texttt{NOXPND @}\label{NOXPNDA}
and enabled again by \texttt{XPND @}\label{XPNDA}.
\index{NOXPND ! "@} \index{XPND ! "@}

The operator is of course aware of the rules that a repeated
application always leads to zero and that there is no exterior form of
higher degree than the dimension of the space.

\begin{verbatim}
     d d x;

     0

     pform u=k;
     spacedim k;

     d u;

     0
\end{verbatim}
\section{Inner Product}
\index{inner product ! exterior form}
The inner product between a vector and an exterior form is represented
by the diphthong \texttt{\_\textbar} \label{innerp} (underscore or-bar), which is the
notation of many textbooks.  If the exterior form is an exterior
product, the inner product is carried through any factor.
\index{\_\textbar operator}

\example\index{EXCALC package ! example}

\begin{verbatim}
     pform x=0,y=k,z=m;

     tvector u,v;

     u _| (x*y^z);

              K
     X*(( - 1) *Y^U _| Z + U _| Y^Z)
\end{verbatim}

In repeated applications of the inner product to the same exterior
form the vector arguments are ordered \emph{e.g.}

\begin{verbatim}
     (u+x*v) _| (u _| (3*z));

     - 3*U _| V _| Z
\end{verbatim}

The duality of natural base elements is also known by the system, \emph{i.e.}

\begin{verbatim}
     pform {x,y}=0;

     (a*@ x+b*@(y)) _| (3*d x-d y);

     3*A - B
\end{verbatim}

\section{Lie Derivative}

\index{Lie Derivative}
The Lie derivative can be taken between a vector and an exterior form
or between two vectors.  It is represented by the infix operator \texttt{\textbar\_}
\label{lie}.  In the case of Lie differentiating, an exterior form by
a vector, the Lie derivative is expressed through inner products and
exterior differentiations, \emph{i.e.} \index{\textbar\_ operator}

\begin{verbatim}
     pform z=k;

     tvector u;

     u |_ z;

     U _| d Z + d(U _| Z)
\end{verbatim}

If the arguments of the Lie derivative are vectors, the vectors are
ordered using the anticommutivity property, and functions (zero forms)
are differentiated out.

\example\index{EXCALC package ! example}

\begin{verbatim}
     tvector u,v;

     v |_ u;

     - U |_ V

     pform x=0,y=0;

     (x*u) |_ (y*v);

     - U*Y*V _| d X + V*X*U _| d Y + X*Y*U |_ V
\end{verbatim}

\section{Hodge-* Duality Operator}

\index{Hodge-* duality operator} \index{"\# ! Hodge-* operator}
The Hodge-*\label{hodge} duality operator maps an exterior form of degree
\texttt{K} to an exterior form of degree \texttt{N-K}, where \texttt{N} is the
dimension of the space.  The double application of the operator must
lead back to the original exterior form up to a factor. The following
example shows how the factor is chosen here

\begin{verbatim}
     spacedim n;
     pform x=k;

     # # x;

             2
           (K  + K*N)
     ( - 1)          *X*SGN
\end{verbatim}
\pagebreak
\index{SGN ! indeterminate sign} \index{coframe}
The indeterminate SGN in the above example denotes the sign of the
determinant of the metric. It can be assigned a value or will be
automatically set if more of the metric structure is specified (via
COFRAME), \emph{i.e.} it is then set to $g/|g|$, where $g$ is the
determinant of the metric.  If the Hodge-* operator appears in an
exterior product of maximal degree as the leftmost factor, the Hodge-*
is shifted to the right according to

\begin{verbatim}
     pform {x,y}=k;

     # x ^ y;

             2
           (K  + K*N)
     ( - 1)          *X^# Y
\end{verbatim}

More simplifications are performed if a coframe is defined.

\section{Variational Derivative}

\index{derivative ! variational} \index{variational derivative}
\ttindex{VARDF}
The function \texttt{VARDF}\label{VARDF} returns as its value the
variation of a given Lagrangian n-form with respect to a specified
exterior form (a field of the Lagrangian).  In the shared variable
\ttindex{BNDEQ"!*}
\texttt{BNDEQ!*}, the expression is stored that has to yield zero if
integrated over the boundary.

Syntax:

\hspace*{2em} \k{VARDF}(\s{Lagrangian n-form},\s{exterior form})

\example\index{EXCALC package ! example}

\begin{verbatim}
  spacedim 4;

  pform l=4,a=1,j=3;

  l:=-1/2*d a ^ # d a - a^# j$  %Lagrangian of the e.m. field

  vardf(l,a);

  - (# J + d # d A)             %Maxwell's equations

  bndeq!*;

  - 'A^# d A                    %Equation at the boundary
\end{verbatim}
Restrictions:

In the current implementation, the Lagrangian must be built up by the
fields and the operations \texttt{d}, \texttt{\#}, and \texttt{@}. Variation
with respect to indexed quantities is currently not allowed.

For the calculation of the conserved currents induced by symmetry
operators (vector fields), the function \texttt{NOETHER}\label{NOETHER}
\index{NOETHER function}
is provided.  It has the syntax:

\hspace*{2em}
\k{NOETHER}(\s{Lagrangian n-form},\s{field},\s{symmetry generator})

\example\index{EXCALC package ! example}

\begin{verbatim}
  pform l=4,a=1,f=2;

  spacedim 4;

  l := -1/2*d a^#d a;   %Free Maxwell field;

  tvector x;        %An unspecified generator;

  noether(l,a,x);

    - 2*d(x _| a)^# d a + d a^x _| # d a - x _| d a^# d a
   --------------------------------------------------------
                           2

\end{verbatim}

The above expression would be the canonical energy
momentum 3-forms of the Maxwell field, if X is interpreted
as a translation;

\section{Handling of Indices}
\index{exterior form ! with indices}
Exterior forms and vectors may have indices.  On input, the indices
are given as arguments of the object.  A positive argument denotes a
superscript and a negative argument a subscript.  On output, the
indexed quantity is displayed two dimensionally if \texttt{NAT} is on.
\index{NAT flag}
Indices may be identifiers or numbers.

\example\index{EXCALC package ! example}

\begin{verbatim}
     pform om(k,l)=m,e(k)=1;

     e(k)^e(-l);

      K
     E ^E
         L

     om(4,-2);

       4
     OM
        2
\end{verbatim}

In the current release, full simplification is performed only if an
index range is specified.  It is hoped that this restriction can be
removed soon.  If the index range (the values that the indices can
obtain) is specified, the given expression is evaluated for all
possible index values, and the summation convention is understood.

\example\label{INDEXRANGE}\index{EXCALC package ! example}

\begin{verbatim}
     indexrange t,r,ph,z;

     pform e(k)=1,s(k,l)=2;

     w := e(k)*e(-k);

              T       R        PH       Z
     W := E *E  + E *E  + E  *E   + E *E
           T       R       PH        Z

     s(k,l):=e(k)^e(l);

      T T
     S    := 0

      R T       T  R
     S    := - E ^E

      PH T       T  PH
     S     := - E ^E

       .
       .
       .

\end{verbatim}

If the expression to be evaluated is not an assignment, the values of
the expression are displayed as an assignment to an indexed variable
with name \texttt{NS}.  This is done only on output, \emph{i.e.} no actual
binding to the variable NS occurs.
\index{NS dummy variable}

\begin{verbatim}
     e(k)^e(l);

       T T
     NS    := 0

       R T       T  R
     NS    := - E ^E
       .
       .
       .
\end{verbatim}

It should be noted, however, that the index positions on the variable
NS can sometimes not be uniquely determined by the system (because of
possible reorderings in the expression). Generally it is advisable to
use assignments to display complicated expressions.

A range can also be assigned to individual index-names. For example,
the declaration

\begin{verbatim}
    indexrange {k,l}={x,y,z},{u,v,w}={1,2};
\end{verbatim}

would assign to the index identifiers k,l the range values x,y,z and
to the index identifiers u,v,w the range values 1,2. The use of an
index identifier not listed in previous indexrange statements has the
range of the union of all given index ranges.

With the above example of an indexrange statement, the following
index evaluations would take place

\begin{verbatim}
     pform w n=0;

     w(k)*w(-k);

         X       Y       Z
     W *W  + W *W  + W *W
      X       Y       Z

     w(u)*w(-u);

         1       2
     W *W  + W *W
      1       2

     w(r)*w(-r);

         1       2       X       Y       Z
     W *W  + W *W  + W *W  + W *W  + W *W
      1       2       X       Y       Z
\end{verbatim}

In certain cases, one would like to inhibit the summation over
specified index names, or at all.  For this the command

\index{NOSUM command}
\hspace*{2em} \k{NOSUM} \s{indexname$_1$}, \ldots;\label{NOSUM}

and the switch \texttt{NOSUM} are \index{NOSUM switch}
available.  The command \texttt{NOSUM} has the effect that summation is
not performed over those indices which had been listed.  The command
\texttt{RENOSUM}\label{RENOSUM} enables summation again.  The switch \texttt{NOSUM},
if on, inhibits any summation. \index{RENOSUM command}

\label{INDEXSYMMETRIES} \index{INDEXSYMMETRIES command}
It is possible to declare symmetry properties for an indexed quantity by
the command \texttt{INDEX\_SYMMETRIES}. A prototypical example is as
follows

\begin{verbatim}

    index_symmetries u(k,l,m,n): symmetric     in {k,l},{m,n}
                                 antisymmetric in {{k,l},{m,n}},
                     g(k,l),h(k,l): symmetric;

\end{verbatim}

It declares the object \texttt{u} symmetric in the first two and last
two indices and antisymmetric with respect to commutation of the given
index pairs. If an object is completely symmetric or antisymmetric,
the indices need not to be given after the corresponding keyword as
shown above for \texttt{g} and \texttt{h}.

If applicable, this command should
be issued, since great savings in memory and execution time result.
Only strict components are printed.

The commands symmetric and antisymmetric of earlier releases have no
effect.

\section{Metric Structures}

\index{metric structure} \index{coframe}
A metric structure is defined in {\bf EXCALC} by specifying a set of
basis one-forms (the coframe) together with the metric.

Syntax:\label{COFRAME}

\begin{tabbing}
\hspace*{2em} \k{COFRAME} \=
\s{identifier}\s{(index$_1$)}=\s{expression$_1$}, \\
\> \s{identifier}\s{(index$_2$)}=\s{expression$_2$}, \\
\> . \\
\> . \\
\> . \\
\> \s{identifier}\s{(index$_n$)}=\s{expression$_n$} \\
\> \hspace{1em} \k{WITH} \k{METRIC} \s{name}=\s{expression}; \\
\end{tabbing}

\index{euclidean metric} \index{COFRAME ! WITH METRIC}
This statement automatically sets the dimension of the space and the
index range. The clause \texttt{WITH METRIC} can be omitted if the metric
\index{COFRAME ! WITH SIGNATURE}
is Euclidean and the shorthand \texttt{WITH SIGNATURE \s{diagonal elements}}
\label{SIGNATURE} can be used in the case of a pseudo-Euclidean metric. The
splitting of a metric structure in its metric tensor coefficients and
basis one-forms is completely arbitrary including the extremes of an
orthonormal frame and a coordinate frame.

\example\index{EXCALC package ! example}

\begin{verbatim}
 coframe e r=d r, e(ph)=r*d ph
   with metric g=e(r)*e(r)+e(ph)*e(ph);    %Polar coframe

 coframe e(r)=d r,e(ph)=r*d(ph);           %Same as before

 coframe o(t)=d t, o x=d x
   with signature -1,1;                    %A Lorentz coframe

 coframe b(xi)=d xi, b(eta)=d eta          %A lightcone coframe
   with metric w=-1/2*(b(xi)*b(eta)+b(eta)*b(xi));

 coframe e r=d r, e ph=d ph                %Polar coordinate
   with metric g=e r*e r+r**2*e ph*e ph;   %basis

\end{verbatim}

Individual elements of the metric can be accessed just by calling them
with the desired indices. The value of the determinant of the
\index{determinant ! in DETM"!*} \ttindex{DETM"!*}
covariant metric is stored in the variable \texttt{DETM!*}.  The metric
is not needed for lowering or raising of indices as the system
performs this automatically, \emph{i.e.} no matter in what index
position values were assigned to an indexed quantity, the values can
be retrieved for any index position just by writing the indexed
quantity with the desired indices.

\example\index{EXCALC package ! example}

\begin{verbatim}
     coframe e t=d t,e x=d x,e y=d y
      with signature -1,1,1;

     pform f(k,l)=0;

     index_symmetries f(k,l): antisymmetric;

     f(k,l) := 0$
     f(-t,-x):=ex$ f(-x,-y):=b$
     on nero;

     f(k,-l);

       X
     NS     := - EX
         T

       T
     NS    := - EX
         X

       Y
     NS     := - B
         X

       X
     NS      := B
         Y
\end{verbatim}

Any expression containing differentials of the coordinate functions will
be transformed into an expression of the basis one-forms.The system also
knows how to take the exterior derivative of the basis one-forms.

\index{spherical coordinates}
\example (Spherical coordinates)\index{EXCALC package ! example}

\begin{verbatim}
     coframe e(r)=d(r), e(th)=r*d(th), e(ph)=r*sin(th)*d(ph);

     d r^d th;

       R  TH
     (E ^E  )/R

     d(e(th));

       R  TH
     (E ^E  )/R

     pform f=0;

     fdomain f=f(r,th,ph);

     factor e;

     on rat;

     d f;       %The "gradient" of F in spherical coordinates;

      R          TH              PH
     E *@  F + (E  *@   F)/R + (E  *@   F)/(R*SIN(TH))
         R           TH              PH
\end{verbatim}

The frame dual to the frame defined by the \texttt{COFRAME} command can
be introduced by \k{FRAME} command. \index{FRAME command}

\hspace*{2em} \k{FRAME} \s{identifier};\label{FRAME}

This command causes the
dual property to be recognized, and the tangent vectors of the
coordinate functions are replaced by the frame basis vectors.

\example\index{EXCALC package ! example}

\begin{verbatim}
   coframe b r=d r,b ph=r*d ph,e z=d z; %Cylindrical coframe;

   frame x;

   on nero;

   x(-k) _| b(l);

       R
   NS    := 1
     R

        PH
   NS      := 1
     PH

       Z
   NS    := 1
     Z

   x(-k) |_ x(-l);       %The commutator of the dual frame;

   NS     := X  /R
     PH R     PH

   NS     := ( - X  )/R  %i.e. it is not a coordinate base;
     R PH         PH

\end{verbatim}

\index{DISPLAYFRAME command} \index{tracing ! EXCALC}
As a convenience, the frames can be displayed at any point in a program
by the command \texttt{DISPLAYFRAME;}\label{DISPLAYFRAME}.

\index{Hodge-* duality operator}
The Hodge-* duality operator returns the explicitly constructed dual
element if applied to coframe base elements. The metric is properly
taken into account.

\index{Levi-Cevita tensor} \ttindex{EPS}
The total antisymmetric Levi-Cevita tensor \texttt{EPS}\label{EPS} is
also available.  The value of \texttt{EPS} with an even permutation of the
indices in a covariant position is taken to be +1.

\section{Riemannian Connections}

\index{Riemannian Connections}
The command \texttt{RIEMANNCONX} is provided for calculating the
\index{RIEMANNCONX command} \label{RIEMANNCONX}
connection 1 forms.  The values are stored on the name given to \texttt{RIEMANNCONX}.
This command is far more efficient than calculating the
connection from the differential of the basis one-forms and using
inner products.

\example (Calculate the connection 1-form and curvature 2-form on S(2))
\index{EXCALC package ! example}

\begin{verbatim}
   coframe e th=r*d th,e ph=r*sin(th)*d ph;

   riemannconx om;

   om(k,-l);                 %Display the connection forms;

     TH
   NS      := 0
        TH

     PH         PH
   NS      := (E  *COS(TH))/(SIN(TH)*R)
        TH

     TH            PH
   NS      := ( - E  *COS(TH))/(SIN(TH)*R)
        PH

     PH
   NS      := 0
        PH

   pform curv(k,l)=2;

   curv(k,-l):=d om(k,-l) + om(k,-m)^om(m-l);
                 %The curvature forms

       TH
   CURV      := 0
          TH

       PH            TH  PH   2
   CURV      := ( - E  ^E  )/R
          TH               %Of course it was a sphere with
                           %radius R.

       TH         TH  PH   2
   CURV      := (E  ^E  )/R
          PH

       PH
   CURV      := 0
          PH
\end{verbatim}

\section{Killing Vectors}

\index{Killing Vectors}
The command \texttt{KILLING\_VECTOR} is provided for calculating the
\index{KILLING_VECTOR command} \label{KILLING_VECTOR}
determining system of partial differential equations of Killing vectors for
a given metric structure provided by the coframe statement.  The result is
a list where the first entry is a vector constructed from the identifier
given to the command and the second entry consists of a list of partial
differential equations for the coefficients of this vector.

\example (Calculate the determining pde's for a Killing vector of S(2))
\index{EXCALC package ! example}
\begin{verbatim}

coframe e th = d th,e ph = sin th*d ph;

killing_vector u;

      ph        th
{@  *u   + @  *u  ,
  ph        th

       th
 {@  (u  ),
   th

       ph         2        th
  @  (u  )*sin(th)  + @  (u  ),
   th                  ph

           th        ph
  cos(th)*u   + @  (u  )*sin(th)}}
                 ph

\end{verbatim}

\section{Ordering and Structuring}

\index{ordering ! exterior form} \index{FORDER command}
The ordering of an exterior form or vector can be changed by the
command \texttt{FORDER}.\label{FORDER}  In an expression, the first
identifier or kernel in the arguments of \texttt{FORDER} is ordered ahead
of the second, and so on, and ordered ahead of all not appearing as
arguments.  This ordering is done on the internal level and not only
on output.  The execution of this statement can therefore have
tremendous effects on computation time and memory requirements.
\texttt{REMFORDER}\label{REMFORDER} brings back standard ordering for those
elements that are listed as arguments. \index{REMFORDER command}

An expression can be put in a more structured form by renaming a
subexpression.  This is done with the command \texttt{KEEP} which
has the syntax \index{KEEP command}\label{KEEP}

\hspace*{2em} \k{KEEP}
\s{name$_1$}=\s{expression$_1$},\s{name$_2$}=\s{expression$_2$}, \ldots

The effect is that rules are set up for simplifying \s{name} without
introducing its definition in an expression. In an expression the system
also tries by reordering to generate as many instances of \s{name} as
possible.

\example\index{EXCALC package ! example}

\begin{verbatim}
     pform x=0,y=0,z=0,f=0,j=3;

     keep j=d x^d y^d z;

     j;

     J

     d j;

     0

     j^d x;

     0

     fdomain f=f(x);

     d f^d y^d z;

     @  F*J
      X
\end{verbatim}

\index{exterior product}
The capabilities of \texttt{KEEP} are currently very limited.  Only exterior
products should occur as righthand sides in \texttt{KEEP}.

\section{Summary of Operators and Commands}
Table~\ref{EXCALC:sum} summarizes EXCALC commands and the page number they are
defined on.

\begin{table}
\begin{tabular}{l l r}
\index{"\^{} ! exterior multiplication} \index{wedge}
\texttt{\^{ }}  &  Exterior Multiplication & \pageref{wedge} \\
\index{"@ ! partial differentiation}
\texttt{@}  & Partial Differentiation & \pageref{at}  \\
\index{"@ ! tangent vector}
\texttt{@}  & Tangent Vector  & \pageref{at1}  \\
\index{"\# ! Hodge-* operator}
\texttt{\#}  & Hodge-* Operator & \pageref{hodge} \\
\index{\_\textbar operator}
\texttt{\_\textbar}  & Inner Product  & \pageref{innerp} \\
\index{\textbar\_ operator}
\texttt{\textbar\_}  & Lie Derivative  & \pageref{lie}  \\
\index{COFRAME command}
\texttt{COFRAME} & Declaration of a coframe & \pageref{COFRAME} \\
\index{d ! exterior differentiation}
\texttt{d} &  Exterior differentiation & \pageref{d} \\
\index{DISPLAYFRAME command}
\texttt{DISPLAYFRAME} & Displays the frame & \pageref{DISPLAYFRAME}\\
\index{EPS ! Levi-Civita tensor}
\texttt{EPS} & Levi-Civita tensor  & \pageref{EPS}  \\
\index{EXDEGREE}
\texttt{EXDEGREE} & Calculates the exterior degree of an expression & \pageref{EXDEGREE}  \\
\index{FDOMAIN command}
\texttt{FDOMAIN} & Declaration of implicit dependencies &\pageref{FDOMAIN} \\
\index{FORDER command}
\texttt{FORDER} & Ordering command  & \pageref{FORDER} \\
\index{FRAME command}
\texttt{FRAME} & Declares the frame dual to the coframe & \pageref{FRAME} \\
\index{INDEXRANGE command}
\texttt{INDEXRANGE} & Declaration of indices & \pageref{INDEXRANGE} \\
\index{INDEX\_SYMMETRIES command}
\texttt{INDEX\_SYMMETRIES} & Declares arbitrary index symmetry properties  & \pageref{INDEXSYMMETRIES} \\
\index{KEEP command}
\texttt{KEEP} & Structuring command  & \pageref{KEEP} \\
\index{KILLING_VECTOR command}
\texttt{KILLING\_VECTOR} & Structuring command  & \pageref{KILLING_VECTOR} \\
\index{METRIC command}
\texttt{METRIC} & Clause of COFRAME to specify a metric & \pageref{COFRAME} \\
\index{NOETHER function}
\texttt{NOETHER} & Calculates the Noether current & \pageref{NOETHER} \\
\index{NOSUM command}
\texttt{NOSUM} & Inhibits summation convention & \pageref{NOSUM} \\
\index{NOXPND command}
\texttt{NOXPND d} & Inhibits the use of product rule for d &
\pageref{NOXPNDD} \\
\index{NOXPND "@ command}
\texttt{NOXPND @} & Inhibits expansion into partial derivatives &
\pageref{NOXPNDA} \\
\index{PFORM command}
\texttt{PFORM} & Declaration of exterior forms & \pageref{PFORM} \\
\index{REMFORDER command}
\texttt{REMFORDER} & Clears ordering  & \pageref{REMFORDER} \\
\index{RENOSUM command}
\texttt{RENOSUM} & Enables summation convention & \pageref{RENOSUM} \\
\index{RIEMANNCONX command}
\texttt{RIEMANNCONX} & Calculation of a Riemannian Connection &
\pageref{RIEMANNCONX} \\
\index{SIGNATURE command}
\texttt{SIGNATURE} & Clause of COFRAME to specify a pseudo- & \pageref{SIGNATURE} \\
  & Euclidean metric &   \\
\index{SPACEDIM command}
\texttt{SPACEDIM} & Command to set the dimension of a space &
\pageref{SPACEDIM} \\
\index{TVECTOR command}
\texttt{TVECTOR} & Declaration of vectors  & \pageref{TVECTOR} \\
\ttindex{VARDF}
\texttt{VARDF} & Variational derivative  & \pageref{VARDF} \\
\index{XPND command}
\texttt{XPND d} & Enables the use of product rule for d & \pageref{XPNDD} \\
  & (default)  &   \\
\index{XPND ! "@}
\texttt{XPND @} & Enables expansion into partial derivatives & \pageref{XPNDA} \\
  & (default)
\end{tabular}
\caption{EXCALC Command Summary}\label{EXCALC:sum}
\end{table}
\newpage
\section{Examples}

The following examples should illustrate the use of {\bf EXCALC}. It is not
intended to show the most efficient or most elegant way of stating the
problems; rather the variety of syntactic constructs are exemplified.
The examples are on a test file distributed with {\bf EXCALC}.
\index{EXCALC package ! example}
{\small
\begin{verbatim}

% Problem: Calculate the PDE's for the isovector of the heat equation.
% --------
%         (c.f. B.K. Harrison, f.B. Estabrook, "Geometric Approach...",
%          J. Math. Phys. 12, 653, 1971)

% The heat equation @   psi = @  psi is equivalent to the set of exterior
%                    xx        t

% equations (with u=@ psi, y=@ psi):
%                    T        x

pform {psi,u,x,y,t}=0,a=1,{da,b}=2;

a := d psi - u*d t - y*d x;

da := - d u^d t - d y^d x;

b := u*d x^d t - d y^d t;

% Now calculate the PDE's for the isovector.

tvector v;

pform {vpsi,vt,vu,vx,vy}=0;
fdomain vpsi=vpsi(psi,t,u,x,y),vt=vt(psi,t,u,x,y),vu=vu(psi,t,u,x,y),
                               vx=vx(psi,t,u,x,y),vy=vy(psi,t,u,x,y);

v := vpsi*@ psi + vt*@ t + vu*@ u + vx*@ x + vy*@ y;

factor d;
on rat;

i1 := v |_ a - l*a;

pform o=1;

o := ot*d t + ox*d x + ou*d u + oy*d y;

fdomain f=f(psi,t,u,x,y);

i11 := v _| d a - l*a + d f;

let vx=-@(f,y),vt=-@(f,u),vu=@(f,t)+u*@(f,psi),vy=@(f,x)+y*@(f,psi),
    vpsi=f-u*@(f,u)-y*@(f,y);

factor ^;

i2 := v |_ b - xi*b - o^a + zeta*da;

let ou=0,oy=@(f,u,psi),ox=-u*@(f,u,psi),
    ot=@(f,x,psi)+u*@(f,y,psi)+y*@(f,psi,psi);

i2;

let zeta=-@(f,u,x)-@(f,u,y)*u-@(f,u,psi)*y;

i2;

let xi=-@(f,t,u)-u*@(f,u,psi)+@(f,x,y)+u*@(f,y,y)+y*@(f,y,psi)+@(f,psi);

i2;

let @(f,u,u)=0;

i2;      % These PDE's have to be solved.

clear a,da,b,v,i1,i11,o,i2,xi,t;
remfdomain f,vpsi,vt,vu,vx,vy;
clear @(f,u,u);

% Problem:
% --------
% Calculate the integrability conditions for the system of PDE's:
% (c.f. B.F. Schutz, "Geometrical Methods of Mathematical Physics"
% Cambridge University Press, 1984, p. 156)

% @ z /@ x + a1*z  + b1*z  = c1
%    1           1       2

% @ z /@ y + a2*z  + b2*z  = c2
%    1           1       2

% @ z /@ x + f1*z  + g1*z  = h1
%    2           1       2

% @ z /@ y + f2*z  + g2*z  = h2
%    2           1       2      ;

pform w(k)=1,integ(k)=4,{z(k),x,y}=0,{a,b,c,f,g,h}=1,
      {a1,a2,b1,b2,c1,c2,f1,f2,g1,g2,h1,h2}=0;

fdomain  a1=a1(x,y),a2=a2(x,y),b1=b1(x,y),b2=b2(x,y),
         c1=c1(x,y),c2=c2(x,y),f1=f1(x,y),f2=f2(x,y),
         g1=g1(x,y),g2=g2(x,y),h1=h1(x,y),h2=h2(x,y);

a:=a1*d x+a2*d y$
b:=b1*d x+b2*d y$
c:=c1*d x+c2*d y$
f:=f1*d x+f2*d y$
g:=g1*d x+g2*d y$
h:=h1*d x+h2*d y$

% The equivalent exterior system:
factor d;
w(1) := d z(-1) + z(-1)*a + z(-2)*b - c;
w(2) := d z(-2) + z(-1)*f + z(-2)*g - h;
indexrange 1,2;
factor z;
% The integrability conditions:

integ(k) := d w(k) ^ w(1) ^ w(2);

clear a,b,c,f,g,h,x,y,w(k),integ(k),z(k);
remfdomain a1,a2,b1,c1,c2,f1,f2,g1,g2,h1,h2;

% Problem:
% --------
% Calculate the PDE's for the generators of the d-theta symmetries of
% the Lagrangian system of the planar Kepler problem.
% c.f. W.Sarlet, F.Cantrijn, Siam Review 23, 467, 1981
% Verify that time translation is a d-theta symmetry and calculate the
% corresponding integral.

pform {t,q(k),v(k),lam(k),tau,xi(k),eta(k)}=0,theta=1,f=0,
      {l,glq(k),glv(k),glt}=0;

tvector gam,y;

indexrange 1,2;

fdomain tau=tau(t,q(k),v(k)),xi=xi(t,q(k),v(k)),f=f(t,q(k),v(k));

l := 1/2*(v(1)**2 + v(2)**2) + m/r$      % The Lagrangian.

pform r=0;
fdomain r=r(q(k));
let @(r,q 1)=q(1)/r,@(r,q 2)=q(2)/r,q(1)**2+q(2)**2=r**2;

lam(k) := -m*q(k)/r;                                % The force.

gam := @ t + v(k)*@(q(k)) + lam(k)*@(v(k))$

eta(k) := gam _| d xi(k) - v(k)*gam _| d tau$

y  := tau*@ t + xi(k)*@(q(k)) + eta(k)*@(v(k))$     % Symmetry generator.

theta := l*d t + @(l,v(k))*(d q(k) - v(k)*d t)$

factor @;

s := y |_ theta - d f$

glq(k) := @(q k) _| s;
glv(k) := @(v k) _| s;
glt := @(t) _| s;

% Translation in time must generate a symmetry.
xi(k) := 0;
tau := 1;

glq k := glq k;
glv k := glv k;
glt;

% The corresponding integral is of course the energy.
integ := - y _| theta;

clear l,lam k,gam,eta k,y,theta,s,glq k,glv k,glt,t,q k,v k,tau,xi k;
remfdomain r,f,tau,xi;

% Problem:
% --------
% Calculate the "gradient" and "Laplacian" of a function and the "curl"
% and "divergence" of a one-form in elliptic coordinates.

coframe e u = sqrt(cosh(v)**2 - sin(u)**2)*d u,
        e v = sqrt(cosh(v)**2 - sin(u)**2)*d v,
        e phi = cos u*sinh v*d phi;

pform f=0;

fdomain f=f(u,v,phi);

factor e,^;
on rat,gcd;
order cosh v, sin u;
% The gradient:
d f;

factor @;
% The Laplacian:
# d # d f;

% Another way of calculating the Laplacian:
-#vardf(1/2*d f^#d f,f);

remfac @;

% Now calculate the "curl" and the "divergence" of a one-form.

pform w=1,a(k)=0;

fdomain a=a(u,v,phi);

w := a(-k)*e k;
% The curl:
x := # d w;

factor @;
% The divergence:
y := # d # w;

remfac @;
clear x,y,w,u,v,phi,e k,a k;
remfdomain a,f;

% Problem:
% --------
% Calculate in a spherical coordinate system the Navier Stokes equations.

coframe e r=d r, e theta =r*d theta, e phi = r*sin theta *d phi;
frame x;

fdomain v=v(t,r,theta,phi),p=p(r,theta,phi);

pform v(k)=0,p=0,w=1;

% We first calculate the convective derivative.

w := v(-k)*e(k)$

factor e; on rat;

cdv := @(w,t) + (v(k)*x(-k)) |_ w - 1/2*d(v(k)*v(-k));

%next we calculate the viscous terms;

visc := nu*(d#d# w - #d#d w) + mu*d#d# w;

% Finally we add the pressure term and print the components of the
% whole equation.

pform nasteq=1,nast(k)=0;

nasteq := cdv - visc + 1/rho*d p$

factor @;

nast(-k) := x(-k) _| nasteq;

remfac @,e;

clear v k,x k,nast k,cdv,visc,p,w,nasteq,e k;
remfdomain p,v;

% Problem:
% --------
% Calculate from the Lagrangian of a vibrating rod the equation of
% motion and show that the invariance under time translation leads
% to a conserved current.

pform {y,x,t,q,j}=0,lagr=2;

fdomain y=y(x,t),q=q(x),j=j(x);

factor ^;

lagr := 1/2*(rho*q*@(y,t)**2 - e*j*@(y,x,x)**2)*d x^d t;

vardf(lagr,y);

% The Lagrangian does not explicitly depend on time; therefore the
% vector field @ t generates a symmetry. The conserved current is

pform c=1;
factor d;

c := noether(lagr,y,@ t);

% The exterior derivative of this must be zero or a multiple of the
% equation of motion (weak conservation law) to be a conserved current.

remfac d;

d c;

% i.e. it is a multiple of the equation of motion.

clear lagr,c,j,y,q;
remfdomain y,q,j;

% Problem:
% --------
% Show that the metric structure given by Eguchi and Hanson induces a
% self-dual curvature.
% c.f. T. Eguchi, P.B. Gilkey, A.J. Hanson, "Gravitation, Gauge Theories
% and Differential Geometry", Physics Reports 66, 213, 1980

for all x let cos(x)**2=1-sin(x)**2;

pform f=0,g=0;
fdomain f=f(r), g=g(r);

coframe   o(r) = f*d r,
      o(theta) = (r/2)*(sin(psi)*d theta - sin(theta)*cos(psi)*d phi),
        o(phi) = (r/2)*(-cos(psi)*d theta - sin(theta)*sin(psi)*d phi),
        o(psi) = (r/2)*g*(d psi + cos(theta)*d phi);

frame e;

pform gamma(a,b)=1,curv2(a,b)=2;
index_symmetries gamma(a,b),curv2(a,b): antisymmetric;

factor o;

gamma(-a,-b) := -(1/2)*( e(-a) _| (e(-c) _| (d o(-b)))
		        -e(-b) _| (e(-a) _| (d o(-c)))
		        +e(-c) _| (e(-b) _| (d o(-a))) )*o(c)$

curv2(-a,b) := d gamma(-a,b) + gamma(-c,b)^gamma(-a,c)$

let f=1/g,g=sqrt(1-(a/r)**4);

pform chck(k,l)=2;
index_symmetries chck(k,l): antisymmetric;

% The following has to be zero for a self-dual curvature.

chck(k,l) := 1/2*eps(k,l,m,n)*curv2(-m,-n) + curv2(k,l);

clear gamma(a,b),curv2(a,b),f,g,chck(a,b),o(k),e(k),r,phi,psi;
remfdomain f,g;

% Example: 6-dimensional FRW model with quadratic curvature terms in
% -------
% the Lagrangian (Lanczos and Gauss-Bonnet terms).
% cf. Henriques, Nuclear Physics, B277, 621 (1986)

for all x let cos(x)**2+sin(x)**2=1;

pform {r,s}=0;
fdomain r=r(t),s=s(t);

coframe o(t) = d t,
        o(1) = r*d u/(1 + k*(u**2)/4),
        o(2) = r*u*d theta/(1 + k*(u**2)/4),
        o(3) = r*u*sin(theta)*d phi/(1 + k*(u**2)/4),
        o(4) = s*d v1,
        o(5) = s*sin(v1)*d v2
 with metric g =-o(t)*o(t)+o(1)*o(1)+o(2)*o(2)+o(3)*o(3)
                +o(4)*o(4)+o(5)*o(5);

frame e;

on nero; factor o,^;

riemannconx om;

pform curv(k,l)=2,{riemann(a,b,c,d),ricci(a,b),riccisc}=0;

index_symmetries curv(k,l): antisymmetric,
                 riemann(k,l,m,n): antisymmetric in {k,l},{m,n}
                                   symmetric in {{k,l},{m,n}},
                 ricci(k,l): symmetric;

curv(k,l) := d om(k,l) + om(k,-m)^om(m,l);

riemann(a,b,c,d) := e(d) _| (e (c) _| curv(a,b));

% The rest is done in the Ricci calculus language,

ricci(-a,-b) := riemann(c,-a,-d,-b)*g(-c,d);

riccisc := ricci(-a,-b)*g(a,b);

pform {laglanc,inv1,inv2} = 0;

index_symmetries riemc3(k,l),riemri(k,l),
                 hlang(k,l),einst(k,l): symmetric;

pform {riemc3(i,j),riemri(i,j)}=0;

riemc3(-i,-j) := riemann(-i,-k,-l,-m)*riemann(-j,k,l,m)$
inv1 := riemc3(-i,-j)*g(i,j);
riemri(-i,-j) := 2*riemann(-i,-k,-j,-l)*ricci(k,l)$
inv2 := ricci(-a,-b)*ricci(a,b);
laglanc := (1/2)*(inv1 - 4*inv2 + riccisc**2);

pform {einst(a,b),hlang(a,b)}=0;

hlang(-i,-j) := 2*(riemc3(-i,-j) - riemri(-i,-j) -
		   2*ricci(-i,-k)*ricci(-j,K) +
		   riccisc*ricci(-i,-j) - (1/2)*laglanc*g(-i,-j));

% The complete Einstein tensor:

einst(-i,-j) := (ricci(-i,-j) - (1/2)*riccisc*g(-i,-j))*alp1 +
		hlang(-i,-j)*alp2$

alp1 := 1$
factor alp2;

einst(-i,-j) := einst(-i,-j);

clear o(k),e(k),riemc3(i,j),riemri(i,j),curv(k,l),riemann(a,b,c,d),
      ricci(a,b),riccisc,t,u,v1,v2,theta,phi,r,om(k,l),einst(a,b),
      hlang(a,b);

remfdomain r,s;

% Problem:
% --------
% Calculate for a given coframe and given torsion the Riemannian part and
% the torsion induced part of the connection. Calculate the curvature.

% For a more elaborate example see E.Schruefer, F.W. Hehl, J.D. McCrea,
% "Application of the REDUCE package EXCALC to the Poincare gauge field
% theory of gravity", GRG Journal, vol. 19, (1988)  197--218

pform {ff, gg}=0;

fdomain ff=ff(r), gg=gg(r);

coframe o(4) = d u + 2*b0*cos(theta)*d phi,
        o(1) = ff*(d u + 2*b0*cos(theta)*d phi) + d r,
        o(2) = gg*d theta,
        o(3) = gg*sin(theta)*d phi
 with metric g = -o(4)*o(1)-o(4)*o(1)+o(2)*o(2)+o(3)*o(3);

frame e;

pform {tor(a),gwt(a)}=2,gamma(a,b)=1,
      {u1,u3,u5}=0;

index_symmetries gamma(a,b): antisymmetric;

fdomain u1=u1(r),u3=u3(r),u5=u5(r);

tor(4) := 0$

tor(1) := -u5*o(4)^o(1) - 2*u3*o(2)^o(3)$

tor(2) := u1*o(4)^o(2) + u3*o(4)^o(3)$

tor(3) := u1*o(4)^o(3) - u3*o(4)^o(2)$

gwt(-a) := d o(-a) - tor(-a)$

% The following is the combined connection.
% The Riemannian part could have equally well been calculated by the
% RIEMANNCONX statement.

gamma(-a,-b) := (1/2)*( e(-b) _| (e(-c) _| gwt(-a))
	               +e(-c) _| (e(-a) _| gwt(-b))
                       -e(-a) _| (e(-b) _| gwt(-c)) )*o(c);

pform curv(a,b)=2;
index_symmetries curv(a,b): antisymmetric;
factor ^;

curv(-a,b) := d gamma(-a,b) + gamma(-c,b)^gamma(-a,c);

clear o(k),e(k),curv(a,b),gamma(a,b),theta,phi,x,y,z,r,s,t,u,v,p,q,c,cs;
remfdomain u1,u3,u5,ff,gg;

showtime;
end;

\end{verbatim}
}
\end{document}


\section{Fourier analysis}


    \filetitle{sumofcoeff}{Doan et al sum-of-coefficient prior dummy observations for BVARs}{BVAR/sumofcoeff}

	\paragraph{Syntax}\label{syntax}

\begin{verbatim}
O = BVAR.sumofcoeff(Mu)
\end{verbatim}

\paragraph{Input arguments}\label{input-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{Mu} {[} numeric {]} - Weight on the dummy observations.
\end{itemize}

\paragraph{Output arguments}\label{output-arguments}

\begin{itemize}
\itemsep1pt\parskip0pt\parsep0pt
\item
  \texttt{O} {[} bvarobj {]} - BVAR object that can be passed into the
  \href{VAR/estimate}{\texttt{VAR/estimate}} function.
\end{itemize}

\paragraph{Description}\label{description}

See \href{BVAR/Contents}{the section explaining the weights on prior
dummies}, i.e.~the input argument \texttt{Mu}.

\paragraph{Example}\label{example}


\chapter{Multiblock Orthogonal Projections to Latent Structures}

\begin{quote}
{\it
  I see NIPALS as an open ended array of models with unlimited complexity
  in the combined use of several devices.}
\\\\
 -- Herman Wold
\end{quote}

\section{Introduction}

\begin{doublespace}
The method of nonlinear iterative partial least squares (NIPALS) has firmly
entrenched itself in the field of chemometrics. Implementations of principal
component analysis (PCA) and projections to latent structures (PLS) that
utilize NIPALS-type algorithms benefit from its numerical stability, as well
as its flexibility and simplicity
\cite{andersson:jchemo2009,hoskuldsson:jchemo1988,wold:cils2001}.
Only a few subroutines from level 2 of the basic linear algebra subprograms
(BLAS) specification are required to construct a complete NIPALS-type
algorithm \cite{golub2012}, making it an attractive means of constructing
PCA and PLS models of high-dimensional spectroscopic datasets.
\\\\
One particularly recent addition to the NIPALS family of algorithms, called
orthogonal projections to latent structures (OPLS), integrates an orthogonal
signal correction (OSC) filter into NIPALS PLS
\cite{trygg:jchemo2002,boulet:cils2012}. By extracting variation from its
computed PLS components that is uncorrelated (orthogonal) to the responses,
OPLS produces a more interpretable regression model compared to PLS. In fact,
when trained on the same data and responses, and OPLS model and a PLS model
with the same total number of components will show no difference in predictive
ability \cite{verron:jchemo2004}. Despite its relative novelty to the field,
the enhanced interpretability of OPLS over PLS has made it a popular method
in exploratory studies of spectroscopic datasets of complex chemical mixtures.
\\\\
Extensions of NIPALS PCA and PLS to incorporate blocking information that
partitions the set of measured variables into multiple ``blocks'' of data have
recently gained attention in the field, as more experimental designs involve
the collection of data from multiple analytical platforms per sample. In such
experiments, referred to as ``class II'' multiblock schemes by Smilde et al.
\cite{smilde:jchemo2003}, correlated consensus directions are sought from the
blocks that maximally capture block variation and (optionally) maximally
predict a set of responses. Of the available extensions of NIPALS to
multiblock modeling, a class of methods exists that bears attractive
computational qualities, namely computability from single-block bilinear
factorizations. When both super weights and block loadings are normalized in
consensus PCA (i.e. CPCA-W), the obtained super scores are equivalent to those
obtained from PCA of the concatenated matrix of blocks
\cite{westerhuis:jchemo1998}. Likewise, scores obtained from PLS of the
concatenated matrix are equivalent to super scores from multiblock PLS
(MB-PLS) when super scores are used in the deflation step
\cite{westerhuis:jchemo1997,westerhuis:jchemo1998}. As a result, these
multiblock bilinear factorizations inherit many of the useful properties
of their single-block equivalents.
\\\\
A second class of multiblock methods exists in which every block is predicted
in a regression model by every other block. In the first of such methods,
known as nPLS, the MAXDIFF criterion \cite{tenberge:psych1988} is optimized
one component at a time (i.e. sequentially) to yield a set of predictive
weight vectors for each block \cite{lofstedt:jchemo2011}. The recently
described OnPLS algorithm also falls within this class. OnPLS extends O2PLS
to three or more matrices and may be considered a prefixing of nPLS with an
OSC filtering step. OnPLS deflates non-globally predictive variation that
may or may not be orthogonal to all blocks from each matrix, and then computes
an nPLS model from the filtered result \cite{lofstedt:jchemo2011}. While fully
symmetric OnPLS is a powerful and general addition to the existing set of
multiblock modeling frameworks, it is arguably an over-complication when
the regression of a single response matrix on multiple data blocks
(i.e. MB-PLS) is sought. For such situations, a novel algorithm termed
MB-OPLS for multiblock orthogonal projections to latent structures is
introduced that embeds an OSC filter within NIPALS MB-PLS, thus solving an
inherently different problem from OnPLS. It will be shown that MB-OPLS, in
analogy to CPCA-W and MB-PLS, is computable from a single-block OPLS model
of the matrix of concatenated data blocks. Thus, MB-OPLS forms a bridge
between this special class of consensus component methods and the highly
general symmetric regression framework of OnPLS.
\end{doublespace}

\section{Theory}

\begin{doublespace}
MB-OPLS belongs to a set of multiblock methods that exhibit a computability
from their single-block equivalents. A short discussion of these methods
follows, in which the optimization criterion of each method is shown to
belong to the MAXBET family of objective functions. This is contrasted to
nPLS and OnPLS, which have been shown to optimize a MAXDIFF objective.
Finally, the equivalence of MB-OPLS and OPLS is demonstrated, and final
mentions are made to differences between MB-OPLS and OnPLS.
\\\\
In all following discussions, it will be understood that there exist $n$
data matrices $\mathbf{X}_1$ to $\mathbf{X}_n$, each having $N$ rows
(observations) and $K_i$ columns (variables). The matrix
$\mathbf{X} = [\mathbf{X}_1\mid\cdots\mid\mathbf{X}_n]$ of all concatenated
blocks will be used in cases of single-block modeling. Finally, a response
matrix $\mathbf{Y}$ having $N$ rows and $M$ columns will be assumed to exist
for the purposes of regression.
\end{doublespace}

\subsection{nPLS and OnPLS}

\begin{doublespace}
In their initial description of the OnPLS modeling framework
\cite{lofstedt:jchemo2011}, L\"{o}fstedt and Trygg introduced nPLS as a
generalization of PLS regression to cases where $n>2$, and a model is
sought in which each matrix $\mathbf{X}_i$ is predicted by all other
matrices $\mathbf{X}_{j \ne i}$. The nPLS solution involves identifying
a set of weight vectors $\mathbf{w}_i$ that simultaneously maximize the
covariances between each pair of resulting scores
$\mathbf{t}_i = \mathbf{X}_i \mathbf{w}_i$ via the following objective
function:
\begin{equation}
\sum_{\substack{i,j=1\\ i \ne j}}^n
 {\mathbf{t}_i}^T \mathbf{t}_j =
\sum_{\substack{i,j=1\\ i \ne j}}^n
 {\mathbf{w}_i}^T {\mathbf{X}_i}^T \mathbf{X}_j \mathbf{w}_j
\end{equation}
subject to the constraints $\|\mathbf{w}_i\|=1$. This objective was recognized
to be a member of the MAXDIFF family of functions, whose solution is obtainable
using a general algorithm from Hanafi and Kiers \cite{hanafi:csda2006}. After
the identification of a set of weight vectors, the scores
\begin{equation*}
\mathbf{t}_i = \mathbf{X}_i \mathbf{w}_i
\end{equation*}
and loadings
\begin{equation*}
\mathbf{p}_i = \frac{{\mathbf{X}_i}^T \mathbf{t}_i}
                    {{\mathbf{t}_i}^T \mathbf{t}_i}
\end{equation*}
may be computed for each matrix, which is then deflated prior to the
computation of subsequent component weights:
\begin{equation}
\mathbf{X}_i \gets \mathbf{X}_i - \mathbf{t}_i {\mathbf{p}_i}^T =
 \left( \mathbf{I} - \frac{\mathbf{t}_i {\mathbf{t}_i}^T}
                          {{\mathbf{t}_i}^T \mathbf{t}_i}
 \right) \mathbf{X}_i
\end{equation}

This deflation scheme follows the precedent set by two-block PLS regression.
Because their described approach used a distinct deflation scheme from
single-component (sequential) MAXDIFF, it was given the name ``nPLS'' by
the authors to distinguish it from MAXDIFF
\cite{lofstedt:jchemo2011,lofstedt2012}.
\\\\
OnPLS extends nPLS by decomposing each matrix into a globally predictive
part and a non-globally predictive (orthogonal) part using an orthogonal
projection. By removing orthogonal variation from each block prior to
constructing an nPLS model, OnPLS optimizes the following MAXDIFF-type
objective function:
\begin{equation}
\sum_{\substack{i,j=1\\ i \ne j}}^n
 {\mathbf{t}_i}^T \mathbf{t}_j =
\sum_{\substack{i,j=1\\ i \ne j}}^n
 {\mathbf{w}_i}^T {\mathbf{X}_i}^T \mathbf{Z}_i
  \mathbf{Z}_j \mathbf{X}_j \mathbf{w}_j
\end{equation}
where $\mathbf{Z}_i$ represents the orthogonal projector identified by
OnPLS for matrix $i$:
\begin{equation*}
\mathbf{Z}_i = \mathbf{I} - \mathbf{T_o}_i \left(
  {\mathbf{T_o}_i}^T \mathbf{T_o}_i
 \right)^{-1} {\mathbf{T_o}_i}^T
\end{equation*}
where
$\mathbf{T_o}_i = [\mathbf{t_o}_{i,1}\mid\cdots\mid\mathbf{t_o}_{i,A_o}]$,
the concatenation of all orthogonal score vectors for the block, and
$\mathbf{t_o}_{i,a} = \mathbf{X}_i \mathbf{w_o}_{i,a}$. In OnPLS, each
orthogonal weight $\mathbf{w_o}_{i,a}$ is chosen such that its score
$\mathbf{t_o}_{i,a}$ contains maximal covariance with the variation in
$\mathbf{X}_{j \ne i}$ that is not jointly predictive of $\mathbf{X}_i$.
The OnPLS framework provides a powerful set of methods for unsupervised
data mining and path modeling \cite{lofstedt:jchemo2011,lofstedt:jchemo2012,
  lofstedt:cils2012,lofstedt:aca2013}.
\end{doublespace}

\subsection{CPCA-W and MB-PLS}

\begin{doublespace}
The consensus PCA method, introduced by Wold et al. as CPCA and modified by
Westerhuis et al. into CPCA-W, identifies a set of weights $\mathbf{p}_i$
that maximally capture the within-block variances and between-block
covariances of a set of $n$ matrices \cite{westerhuis:jchemo1998}. It was
further proven by Westerhuis, Kourti and MacGregor that the results of CPCA-W
computed on matrices $\mathbf{X}_1$ to $\mathbf{X}_n$ are identical to those
from PCA of the concatenated matrix $[\mathbf{X}_1\mid\cdots\mid\mathbf{X}_n]$.
It immediately follows from this equivalence that the CPCA-W algorithm
optimizes the following objective function:
\begin{equation}
\mathbf{t}^T \mathbf{t} =
\mathbf{p}^T \mathbf{X}^T \mathbf{X} \mathbf{p} =
\sum_{i,j=1}^n {\mathbf{t}_i}^T \mathbf{t}_j =
\sum_{i,j=1}^n {\mathbf{p}_i}^T {\mathbf{X}_i}^T \mathbf{X}_j \mathbf{p}_j
\end{equation}
subject to the constraint $\|\mathbf{p}\|=1$, where
$\mathbf{p}^T=[{\mathbf{p}_1}^T\mid\cdots\mid{\mathbf{p}_n}^T]$. Maximizing
the above function yields a set of super scores $\mathbf{t}$ that relate the
$N$ observations in $\mathbf{X}$ to each other based on the extracted
consensus in $\mathbf{p}$, as well as block scores $\mathbf{t}_i$ and loadings
$\mathbf{p}_i$ that describe each block. This objective function is of the
MAXBET variety, in contrast to the MAXDIFF objective of nPLS and OnPLS. As
a result, the CPCA-W NIPALS algorithm may be considered a special case of
the general algorithm from Hanafi and Kiers \cite{hanafi:csda2006}.
\\\\
The multiblock PLS (MB-PLS) method, when deflation is performed using super
scores \cite{westerhuis:jchemo1997}, shares an equivalence with single-block
PLS as proven by Westerhuis et al. \cite{westerhuis:jchemo1998}. Therefore,
the MB-PLS objective takes on a similar form as in CPCA-W, with the addition
of a weighting matrix:
\begin{equation}
\mathbf{t}^T \mathbf{Y} \mathbf{Y}^T \mathbf{t} =
\mathbf{w}^T \mathbf{X}^T \mathbf{Y} \mathbf{Y}^T \mathbf{X} \mathbf{w} =
\sum_{i,j=1}^n
 {\mathbf{t}_i}^T \mathbf{Y} \mathbf{Y}^T \mathbf{t}_j =
\sum_{i,j=1}^n
 {\mathbf{w}_i}^T {\mathbf{X}_i}^T
  \mathbf{Y} \mathbf{Y}^T
 \mathbf{X}_j \mathbf{w}_j
\end{equation}
where once again $\|\mathbf{w}\|$ is constrained to unity. In analogy to
H\"{o}skuldsson's interpretation of PLS as a regression on orthogonal
components, where $\mathbf{Y}\mathbf{Y}^T$ is used to weight the covariance
matrix, the above function corresponds to a MAXBET objective with an inner
weighting of $\mathbf{Y}\mathbf{Y}^T$ \cite{hoskuldsson:jchemo1988}.
Alternatively, equation 9.5 could be interpreted as a MAXBET computed on the
$n$ cross-covariance matrices $\mathbf{Y}^T \mathbf{X}_1$ to
$\mathbf{Y}^T \mathbf{X}_n$.
\end{doublespace}

\subsection{MB-OPLS}

\begin{doublespace}
Extension of prior multiblock NIPALS algorithms to incorporate an OSC filter
rests on the observation that, in both the cases of CPCA-W and MB-PLS,
deflation of each computed component is accomplished using super scores.
For any super score deflation method, a loading vector is computed for each
block:
\begin{equation*}
\mathbf{p}_i = \frac{{\mathbf{X}_i}^T \mathbf{t}}
                    {\mathbf{t}^T \mathbf{t}}
\end{equation*}
and the super scores $\mathbf{t}$ and block loadings are then used to deflate
their respective block:
\begin{equation}
\mathbf{X}_i \gets \mathbf{X}_i - \mathbf{t} {\mathbf{p}_i}^T =
 \left( \mathbf{I} - \frac{\mathbf{t} \mathbf{t}^T}
                          {\mathbf{t}^T \mathbf{t}}
 \right) \mathbf{X}_i
\end{equation}

Equation 9.6 differs from equation 9.2 used in nPLS and OnPLS, which uses
block-specific scores and loadings during deflation. This method of super
score deflation ensures that the super scores become an orthogonal basis,
while allowing scores and loadings to become slightly correlated at the
block level, and is a necessary condition for the equivalences between
CPCA-W and MB-OPLS and their single-block counterparts
\cite{westerhuis:jchemo1998}. This condition shall be employed in MB-OPLS
by deflating each matrix by a set of orthogonal super scores $\mathbf{T_o}$,
which shall be shown to be equal to the orthogonal scores obtained from
single-block OPLS. By constructing an MB-PLS model on the set of matrices
after deflation by $\mathbf{T_o}$, we effectively arrive at another MAXBET
objective:
\begin{equation}
\mathbf{t}^T \mathbf{Y} \mathbf{Y}^T \mathbf{t} =
 \mathbf{w}^T \mathbf{X}^T \mathbf{Z} \mathbf{Y}
 \mathbf{Y}^T \mathbf{Z} \mathbf{X} \mathbf{w} =
\sum_{i,j=1}^n
 {\mathbf{t}_i}^T \mathbf{Y} \mathbf{Y}^T \mathbf{t}_j =
\sum_{i,j=1}^n
 {\mathbf{w}_i}^T {\mathbf{X}_i}^T \mathbf{Z}
  \mathbf{Y} \mathbf{Y}^T
 \mathbf{Z} \mathbf{X}_j \mathbf{w}_j
\end{equation}
where $\mathbf{w}$ is constrained to unit $\ell_2$-norm and $\mathbf{Z}$ is
the orthogonal projector for the super scores $\mathbf{T_o}$.
\end{doublespace}

\subsubsection{The MB-OPLS Model}

\begin{doublespace}
MB-OPLS constructs an OPLS model for each matrix $\mathbf{X}_i$, where the
predictive and orthogonal loadings for each matrix are interrelated by a
set of predictive and orthogonal super scores, respectively:
\begin{equation}
\mathbf{X}_i =
 \underbrace{\mathbf{T} {\mathbf{P}_i}^T}_{\mathbf{X_p}_i} +
 \underbrace{\mathbf{T_o} {\mathbf{P_o}_i}^T}_{\mathbf{X_o}_i} +
 \mathbf{E}_i
\end{equation}

Concatenation of all block-level matrices together in equation 9.8 results
in a top-level consensus model, which is in fact equivalent to an OPLS model
trained on the partitioned data matrix $\mathbf{X}$:
\begin{equation}
\mathbf{X} = [\mathbf{X}_1\mid\cdots\mid\mathbf{X}_n] =
 \underbrace{\mathbf{T} [{\mathbf{P}_1}^T\mid\cdots\mid{\mathbf{P}_n}^T]}
  _{\mathbf{X_p}} +
 \underbrace{\mathbf{T_o} [{\mathbf{P_o}_1}^T\mid\cdots\mid{\mathbf{P_o}_n}^T]}
  _{\mathbf{X_o}} +
 \underbrace{[\mathbf{E}_1\mid\cdots\mid\mathbf{E}_n]}_{\mathbf{E}}
\end{equation}

Like PLS, OPLS and MB-PLS, an MB-OPLS model contains a second equation that
relates the predictive super scores and responses:
\begin{equation}
\mathbf{Y} = \mathbf{T} \mathbf{C}^T + \mathbf{F}
\end{equation}
\end{doublespace}

\subsubsection{The MB-OPLS Algorithm}

\begin{doublespace}
The MB-OPLS algorithm was introduced in
\hyperlink{subsection.3.5.6}{Chapter 3}, but will be decomposed in more
detail as sub-algorithms here in order to relate it with the OPLS NIPALS
algorithm. For simplicity, procedures relating to cross-validation and
automatic component count identification will be stripped from this
discussion of MB-OPLS. MB-OPLS admits a matrix of responses $\mathbf{Y}$,
but also supports vector-$\mathbf{y}$ modeling as a special case. Direct
and normed assignment will be indicated by ``$\gets$'' and ``$\propto$'',
respectively.
\end{doublespace}

\begin{algorithm}[H]
\caption{Core Algorithm for MB-OPLS}
\label{algorithm.9.1}
\begin{algorithmic}[1]
\REQUIRE $\{\mathbf{X}_i \in \mathbb{R}^{N \times K_i}\}_{i=1}^n$,%
         $\:\mathbf{Y} \in \mathbb{R}^{N \times M}$
\STATE $\mathbf{V}_i \gets \text{SUBSPACE}(\mathbf{X}_i, \mathbf{Y})
        \quad \forall i \in \{1, \dots, n\}$
\STATE $\{\mathbf{w}_i\}_{i=1}^n,
        \{\mathbf{t}_i\}_{i=1}^n,
        \{\mathbf{p}_i\}_{i=1}^n,
        \mathbf{t}, \mathbf{c}, \mathbf{u} \gets
        \text{PREDCMP}(\{\mathbf{X}_i\}_{i=1}^n, \mathbf{Y})$
\STATE{
  To compute an orthogonal component, continue to step (4). \\
  Otherwise, proceed to step (7).
}
\STATE $\{\mathbf{w_o}_i\}_{i=1}^n,
        \{\mathbf{t_o}_i\}_{i=1}^n,
        \{\mathbf{p_o}_i\}_{i=1}^n,
        \mathbf{t_o} \gets
        \text{ORTHCMP}(\{\mathbf{X}_i\}_{i=1}^n,
                       \{\mathbf{V}_i\}_{i=1}^n,
                       \{\mathbf{p}_i\}_{i=1}^n)$
\STATE $\mathbf{X}_i \gets \mathbf{X}_i - \mathbf{t_o} {\mathbf{p_o}_i}^T
        \quad \forall i \in \{1, \dots, n\}$
\STATE{Return to step (2).}
\STATE $\mathbf{X}_i \gets \mathbf{X}_i - \mathbf{t} \mathbf{p}_i^T
        \quad \forall i \in \{1, \dots, n\}$
\STATE{
  To compute another predictive component, return to step (2). \\
  Otherwise, end.
}
\end{algorithmic}
\end{algorithm}

\begin{doublespace}
The SUBSPACE method computes the $\mathbf{Y}$-predictive subspace
for each data matrix $\mathbf{X}_i$, and follows directly from
the single-block OPLS algorithm described by Trygg and Wold
\cite{trygg:jchemo2002}:
\end{doublespace}

\begin{algorithm}[H]
\caption{Predictive Subspace Identification for MB-OPLS}
\label{algorithm.9.2}
\begin{algorithmic}[1]
\REQUIRE $\mathbf{X} \in \mathbb{R}^{N \times K}$,%
       $\:\mathbf{Y} \in \mathbb{R}^{N \times M}$
\FORALL{$m \in \{1, \dots, M\}$}
  \STATE $\mathbf{v}_m \gets \mathbf{X}^T \mathbf{y}_m
          \cdot \left( \mathbf{y}_m^T \mathbf{y}_m \right)^{-1}$
  \STATE $\mathbf{V} \gets [\mathbf{V} \mid \mathbf{v}_m]$
\ENDFOR
\end{algorithmic}
\end{algorithm}

\begin{doublespace}
Predictive MB-OPLS components identified by the PREDCMP method are, in fact,
MB-PLS components:
\end{doublespace}
\newpage

\begin{algorithm}[H]
\caption{Predictive Component Computation for MB-OPLS}
\label{algorithm.9.3}
\begin{algorithmic}[1]
\REQUIRE $\{\mathbf{X}_i \in \mathbb{R}^{N \times K_i}\}_{i=1}^n$,%
         $\:\mathbf{Y} \in \mathbb{R}^{N \times M}$
\STATE{Initialize $\mathbf{u}$ to a column of $\mathbf{Y}$}
\REPEAT
  \STATE $\mathbf{w}_i \propto {\mathbf{X}_i}^T \mathbf{u}
          \quad \forall i \in \{1, \dots, n\}$
  \STATE $\mathbf{t}_i \gets \mathbf{X}_i \mathbf{w}_i
          \quad \forall i \in \{1, \dots, n\}$
  \STATE $\mathbf{R} \gets [\mathbf{t}_1 \mid\cdots\mid \mathbf{t}_n]$
  \STATE $\mathbf{w_T} \propto \mathbf{R}^T \mathbf{u}$
  \STATE $\mathbf{t} \gets \mathbf{R} \mathbf{w_T}$
  \STATE $\mathbf{c} \gets
          \left( \mathbf{Y}^T \mathbf{t} \right) \cdot
          \left( \mathbf{t}^T \mathbf{t} \right)^{-1}$
  \STATE $\mathbf{u} \gets
          \left( \mathbf{Y} \mathbf{c} \right) \cdot
          \left( \mathbf{c}^T \mathbf{c} \right)^{-1}$
\UNTIL{
  $\|\mathbf{u} - \mathbf{u}_{old}\|
   \cdot \|\mathbf{u}_{old}\|^{-1} < \varepsilon$
}
\STATE $\mathbf{p}_i \gets
        \left( {\mathbf{X}_i}^T \mathbf{t} \right) \cdot
        \left( \mathbf{t}^T \mathbf{t} \right)^{-1}
        \quad \forall i \in \{1, \dots, n\}$
\end{algorithmic}
\end{algorithm}

\begin{doublespace}
In the above method, the value of $\varepsilon$ is set to a very small
number, such as $10^{-9}$. Once a predictive component has been computed,
MB-OPLS uses the ORTHCMP method to extract a new orthogonal component:
\end{doublespace}

\begin{algorithm}[H]
\caption{Orthogonal Component Computation for MB-OPLS}
\label{algorithm.9.4}
\begin{algorithmic}[1]
\REQUIRE $\{\mathbf{X}_i \in \mathbb{R}^{N \times K_i}\}_{i=1}^n$,%
       $\:\{\mathbf{V}_i \in \mathbb{R}^{M \times K_i}\}_{i=1}^n$,%
       $\:\{\mathbf{p}_i \in \mathbb{R}^{K_i}\}_{i=1}^n$
\STATE $\mathbf{w_o}_i \gets \mathbf{p}_i
        \quad \forall i \in \{1, \dots, n\}$
\FORALL{$m \in \{1, \dots, M\}$}
  \STATE $\phi \gets
      \left( \sum_{i=1}^n {\mathbf{v}_{i,m}}^T \mathbf{w_o}_i \right) \cdot
      \left( \sum_{i=1}^n {\mathbf{v}_{i,m}}^T \mathbf{v}_{i,m} \right)^{-1}$
  \STATE $\mathbf{w_o}_i \gets \mathbf{w_o}_i - \phi \mathbf{v}_{i,m}
          \quad \forall i \in \{1, \dots, n\}$
\ENDFOR
\STATE $\alpha \gets \left(
         \sum_{i=1}^n {\mathbf{w_o}_i}^T \mathbf{w_o}_i
        \right)^{-1/2}$
\STATE $\mathbf{w_o}_i \gets \alpha \mathbf{w_o}_i
        \quad \forall i \in \{1, \dots, n\}$
\STATE $\mathbf{t_o}_i \gets \mathbf{X}_i \mathbf{w_o}_i
        \quad \forall i \in \{1, \dots, n\}$
\STATE $\mathbf{t_o} \gets \sum_{i=1}^n \mathbf{t_o}_i$
\STATE $\mathbf{p_o}_i \gets
        \left( {\mathbf{X}_i}^T \mathbf{t_o} \right) \cdot
        \left( \mathbf{t_o}^T \mathbf{t_o} \right)^{-1}
        \quad \forall i \in \{1, \dots, n\}$
\end{algorithmic}
\end{algorithm}

\begin{doublespace}
For each predictive component in the model, a set of orthogonal components is
extracted. After the computation of a new orthogonal component, the current
predictive component is updated to reflect the removal of orthogonal variation
from the matrices $\mathbf{X}_i$. The MB-OPLS algorithm closely follows the
matrix-$\mathbf{Y}$ OPLS algorithm presented by Trygg and Wold
\cite{trygg:jchemo2002}, but replaces the standard PLS computation
(steps 4--10 in OPLS) with an MB-PLS computation.
\end{doublespace}

\subsubsection{Equivalence to OPLS}

\begin{doublespace}
In both the vector-$\mathbf{y}$ and matrix-$\mathbf{Y}$ OPLS algorithms
proposed by Trygg and Wold \cite{trygg:jchemo2002}, a basis $\mathbf{V}$
for the response-correlated variation in $\mathbf{X}$ is constructed by
regressing the data onto each column of responses:
\begin{equation}
\mathbf{v}_m \gets \frac{\mathbf{X}^T \mathbf{y}_m}
                        {{\mathbf{y}_m}^T \mathbf{y}_m}
 \quad \forall m \in \{1, \dots, M\}
\end{equation}
where $\mathbf{y}_m$ and $\mathbf{v}_m$ denote the $m$-th columns of
$\mathbf{Y}$ and $\mathbf{V}$, respectively. When $\mathbf{X}$ is partitioned
into multiple blocks, the computed basis also bears the same partitioning,
i.e. $\mathbf{V}^T=[{\mathbf{V}_1}^T\mid\cdots\mid{\mathbf{V}_n}^T]$, where
each of the $n$ submatrices corresponds to the regression of its respective
block $\mathbf{X}_i$ onto the responses:
\begin{equation}
\mathbf{v}_{i,m} \gets \frac{{\mathbf{X}_i}^T \mathbf{y}_m}
                            {{\mathbf{y}_m}^T \mathbf{y}_m}
 \quad \forall m \in \{1, \dots, M\}
\end{equation}
where $\mathbf{v}_{i,m}$ is the $m$-th column of $\mathbf{V}_i$. Therefore,
the bases of response-correlated variation identified by OPLS and MB-OPLS
are equal. Given a single-block PLS loading vector $\mathbf{p}$, the OPLS
algorithm computes an orthogonal weight $\mathbf{w_o}$ by orthogonalizing
$\mathbf{p}$ to the columns of $\mathbf{V}$:
\begin{equation}
\mathbf{w_o} \gets \mathbf{w_o} - \left(
 \frac{{\mathbf{v}_m}^T \mathbf{w_o}}
      {{\mathbf{v}_m}^T \mathbf{v}_m}
 \right) \cdot \mathbf{v}_m
 \quad \forall m \in \{1, \dots, M\}
\end{equation}
after $\mathbf{w_o}$ has been initialized from $\mathbf{p}$. From the proof
of Westerhuis et al. \cite{westerhuis:jchemo1998}, it is known that the
single-block PLS loading $\mathbf{p}$ equals the concatenation of all block
loadings from MB-PLS, i.e. that
$\mathbf{p}^T=[{\mathbf{p}_1}^T\mid\cdots\mid{\mathbf{p}_n}^T]$. Expansion
of all vector terms in the above equation into their partitioned forms
results in the following new assignment rule:
\begin{equation}
\mathbf{w_o}_i \gets \mathbf{w_o}_i - \left(
 \frac{\sum_{i=1}^n {\mathbf{v}_{i,m}}^T \mathbf{w_o}_i}
      {\sum_{i=1}^n {\mathbf{v}_{i,m}}^T \mathbf{v}_{i,m}}
 \right) \cdot \mathbf{v}_{i,m}
 \quad \forall m \in \{1, \dots, M\}
\end{equation}

The scalar term in equation 9.14 should be recognized as $\phi$ in the
ORTHCMP method. By the same reasoning, steps 6 and 7 in ORTHCMP are
equivalent to scaling $\mathbf{w_o}$ to unit norm. Therefore, because
$\mathbf{w_o}$ is the column-wise concatenation of all weights
$\mathbf{w_o}_i$, it is then apparent that the orthogonal super scores
extracted by MB-OPLS are identical to those from OPLS of the concatenated
matrix $\mathbf{X}$:
\begin{equation}
\mathbf{t_o} = \mathbf{X} \mathbf{w_o} =
 [\mathbf{X}_1 \mid\cdots\mid \mathbf{X}_n]
 \begin{bmatrix}
  \mathbf{w_o}_1 \\
  \vdots \\
  \mathbf{w_o}_n
 \end{bmatrix}
 = \sum_{i=1}^n \mathbf{X}_i \mathbf{w_o}_i
 = \sum_{i=1}^n \mathbf{t_o}_i
\end{equation}

From this equivalence, and the fact that PREDCMP in MB-OPLS constitutes an
MB-PLS computation, we arrive at the equivalence between MB-OPLS and OPLS.
Thus, orthogonality between the responses and orthogonal super scores
$\mathbf{t_o}$ computed by MB-OPLS is also ensured. However, because the
computation of orthogonal weights involves all blocks, the resulting
orthogonal block scores $\mathbf{t_o}_i$ are not guaranteed to be orthogonal
to the responses.
\end{doublespace}

\subsubsection{Computation from an OPLS Model}

\begin{doublespace}
The equivalence between MB-OPLS super scores and OPLS scores may be leveraged
to generate an MB-OPLS model from an existing OPLS model of a partitioned
data matrix, saving computation time during cross-validated model training.
Algorithm 3.7 in \hyperlink{subsection.3.5.6}{Chapter 3} details the
extraction of MB-OPLS block scores and loadings from an OPLS model.
\\\\
The keen observer will recognize the similarity between Algorithm 3.7 and the
procedure outlined by Westerhuis et al. for extracting MB-PLS block components
from a PLS model \cite{westerhuis:jchemo1998}. By using this algorithm to
compute MB-OPLS models, the analyst avoid the unnecessary computation of
block components during cross-validated model training.
\end{doublespace}

\section{Datasets}

\begin{doublespace}
Two datasets will be described to illustrate how MB-OPLS effectively
integrates an OSC filter into an MB-PLS decomposition of a set of $n$
matrices. The first synthetic dataset contrasts the mixing of predictive
information in MB-PLS with its separation in MB-OPLS using a contrived
three-block regression example similar to that introduced by L\"{o}fstedt
and Trygg \cite{lofstedt:jchemo2011}. The second dataset, a joint set of
NMR and MS observations introduced in \hyperlink{section.4.3}{Chapter 4},
is used to demonstrate the enhanced interpretability of MB-OPLS models
over MB-PLS in a real example of discriminant analysis. All modeling and
validation were performed using routines available in the MVAPACK
chemometrics toolbox \cite{worley:acscb2014}.
\end{doublespace}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/mbopls/01-synthetic.png}
\caption
      [Synthetic Three-block Example Dataset.]{
  {\bf Synthetic Three-block Example Dataset.}
  \\
  Block loadings in the synthetic multiblock example dataset.
  ({\bf A}) True predictive loadings (solid) and orthogonal loadings (dashed)
  used to construct the three-block dataset. First, second and third block
  loadings are colored in red, green and blue, respectively.
  ({\bf B}) First component (solid) and second component (dashed) loadings
  identified by MB-PLS modeling of the three data blocks.
  ({\bf C}) Predictive (solid) and orthogonal (dashed) block loadings
  identified by MB-OPLS, illustrating the separation of
  $\mathbf{y}$-uncorrelated variation accomplished by the integrated
  OSC filter.
}
\label{figure.9.1}
\end{figure}

\subsection{Synthetic Example}

\begin{doublespace}
In the first dataset, three matrices (all having 100 rows and 200 columns)
were constructed to hold one $\mathbf{y}$-predictive component
($\mathbf{t} {\mathbf{p}_i}^T$) and one $\mathbf{y}$-orthogonal component
($\mathbf{t_o} {\mathbf{p_o}_i}^T$). The score vectors were non-overlapping
(orthogonal) Gaussian density functions, and all block loading vectors were
mutually overlapping Gaussian density or square step functions. The true
synthetic block loadings are illustrated in \figref{9.1}{Figure 9.1A}.
A two-component MB-PLS-R model was trained on the synthetic three-block
example dataset, as well as a $1+1$ (one predictive, one orthogonal)
component MB-OPLS-R model. Block loadings extracted by MB-PLS-R and
MB-OPLS-R and shown in \figref{9.1}{Figures 9.1B} and
\figref{9.1}{9.1C}, respectively.
\end{doublespace}

\subsection{Joint \hnmr{} NMR and DI-ESI-MS Datasets}

\begin{doublespace}
The second dataset is a pair of processed and treated data matrices, collected
on 29 samples of metabolite extracts from human dopaminergic neuroblastoma
cells treated with various neurotoxic agents \cite{lei:acscb2014}. Details
about the collection, processing and treatment of this dataset may be found
in \hyperlink{section.4.3}{Chapter 4}, but will be summarized here. The
first matrix, collected using \hnmr{} NMR spectroscopy, contains 16,138
columns and the second, collected using direct injection electrospray
ionization mass spectrometry (DI-ESI-MS), contains 2,095 columns. Prior
to all modeling, block weighting was applied after Pareto scaling to ensure
equal contribution of each block to the models \cite{smilde:jchemo2003}.
\\\\
In a previously published analysis of this dataset \cite{marshall:metab2015},
a two-component, two-class (vector-$\mathbf{y}$) MB-PLS-DA model was trained
on the dataset in order to discriminate between untreated and
neurotoxin-treated cell samples. To highlight the improved interpretability
of MB-OPLS over MB-PLS, a $1+1$ MB-OPLS-DA model was trained on the data
using an identical vector of class labels. Block components were extracted
from an OPLS-DA model of the concatenated matrix
$\mathbf{X}=[\mathbf{X}_{NMR}\mid\mathbf{X}_{MS}]$ using Algorithm 3.7 in
\hyperlink{subsection.3.5.6}{Chapter 3}. For both models, fifty rounds of
Monte Carlo seven-fold cross-validation \cite{shao:jasa1993,xu:cils2001}
were performed to compute per-component \qsq{} statistics
\cite{wold:cils2001}, in addition to the \rsq{} statistics available
from model training. CV-ANOVA significance testing was also applied
to further assess model reliability \cite{eriksson:jchemo2008}.
\end{doublespace}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/mbopls/02-scores.png}
\caption
      [Super Scores of Joint Spectroscopic Data.]{
  {\bf Super Scores of Joint Spectroscopic Data.}
  \\
  Super scores identified by ({\bf A}) MB-PLS and ({\bf B}) MB-OPLS modeling
  of the joint \hnmr{} NMR and DI-ESI-MS data matrices. Extraction of
  $\mathbf{y}$-orthogonal variation from the first PLS component is
  clear in the MB-OPLS scores. Ellipses represent the 95\% confidence
  regions for each sub-class of observations, assuming normal distributions.
  Colors indicate membership to the untreated (yellow), 6-hydroxydopamine
  (red), 1-methyl-4-phenylpyridinium (green) and paraquat (violet)
  sub-classes. Cross-validated super scores are shown in Figure 9.3.
}
\label{figure.9.2}
\end{figure}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/mbopls/03-cvscores.png}
\caption
      [Cross-validated Super Scores of Joint Spectroscopic Data.]{
  {\bf Cross-validated Super Scores of Joint Spectroscopic Data.}
  \\
  Cross-validation estimated super scores from ({\bf A}) MB-PLS and ({\bf B})
  MB-OPLS modeling of the joint \hnmr{} NMR and DI-ESI-MS data matrices.
  Points indicate mean values for each observation, and filled regions
  represent the union of all observations' confidence intervals from
  Monte Carlo iterations. Class colors are identical to those in
  Figure 9.2.
}
\label{figure.9.3}
\end{figure}

\section{Results and Discussion}

\begin{doublespace}
In both the contrived dataset and the real spectroscopic dataset, the
interpretative advantage offered by MB-OPLS over MB-PLS is strikingly
apparent. In the synthetic example, MB-OPLS capably identifies the true
predictive and orthogonal loadings in the presence of $\mathbf{y}$-orthogonal
variation that clouds the interpretation of MB-PLS loadings
(\figref{9.1}{Figure 9.1}). By design, this comparison between
MB-OPLS and MB-PLS is highly similar to the first example
presented by L\"{o}fstedt and Trygg to compare nPLS and OnPLS
for general data discovery \cite{lofstedt:jchemo2011}. However, as is
evidenced by the differences between equations 9.3 and 9.7 above, MB-OPLS
solves an inherently distinct problem from OnPLS: the identification of
consensus variation in multiple blocks of data that predicts a single set
of responses.
\\\\
The ability of MB-OPLS to separate predictive and orthogonal variation from
multiple data matrices is further exemplified in the discriminant analysis
of the real spectroscopic dataset. From the rotated discrimination axis in
the MB-PLS-DA scores (\figref{9.2}{Figure 9.2A}), it is clear that predictive
and orthogonal variation have become mixed in the corresponding block loadings
(\figref{4.11}{Figure 4.11}). Integration of an OSC filter into the multiblock
model in the form of MB-OPLS-DA achieved the expected rotation of super scores
to place more predictive variation into the first component
(\figref{9.2}{Figure 9.2B}). As a consequence of this rotation, spectral
information that separates paraquat treatment from other neurotoxin treatments
is also moved into the orthogonal component. For example, strong loadings from
citrate in the \hnmr{} NMR MB-PLS block loadings
(\figref{4.11}{Figure 4.11A}, 2.6 ppm) are substantially diminished
in the predictive block loadings from MB-OPLS
(\figref{4.12}{Figure 4.12A}), as separation between paraquat and
other treatments has been isolated along the orthogonal component in super
scores. Inspection of the orthogonal block loadings from MB-OPLS
(\figref{4.13}{Figure 4.13}) will reveal, as expected, that citrate
contributes more to separation between neurotoxin treatments than to
separation between treatments and controls.
\\\\
The partial correlation of both predictive and orthogonal block scores in
MB-OPLS is readily observed in the comparison of block scores from MB-PLS
and MB-OPLS (\figref{9.4}{Figures 9.4} and \figref{9.5}{9.5}). While the
super scores in \figref{9.2}{Figure 9.2B} are rotated to separate predictive
and orthogonal variation, block scores in \figref{9.4}{Figures 9.4B} and
\figref{9.5}{9.5B} have slightly rotated back into alignment with
the MB-PLS block scores. This partial correlation and re-mixing of
predictive and orthogonal variation in MB-OPLS block scores is a consequence
of the use of super score deflation in the presented algorithm. When all
matrices contain similar patterns of orthogonal variation, their MB-OPLS
block scores will reflect this by retaining the OSC-induced rotation captured
at the consensus level by the super scores.
\\\\
Because the MB-OPLS-DA model of the real spectral data matrices was trained
using the single-block OPLS routine already present in MVAPACK, all standard
cross-validation metrics were available in the model without further
computational expenditure. Monte Carlo cross-validation of the MB-PLS model
produced cumulative \rsqy{} and \qsq{} statistics of 0.988 and
0.901$\pm$0.015, respectively, and validation of the MB-OPLS model resulted
in statistics of 0.903 and 0.706$\pm$0.028, respectively. In addition, MB-OPLS
modeling yielded \rsqxp{} and \rsqxo{} statistics of 0.378 and 0.245 for the
first block, and 0.236 and 0.083 for the second block. Monte Carlo
cross-validated super scores from MB-PLS and MB-OPLS are depicted in
\figref{9.2}{Figure 9.2}. Compared to MB-PLS scores in
\figref{9.2}{Figure 9.2A}, MB-OPLS scores
(\figref{9.2}{Figure 9.2B}) exhibit an increased uncertainty due to the
coupled nature of predictive and orthogonal components in OPLS models.
Further validation of the MB-OPLS-DA model via CV-ANOVA produced
a $p$ value equal to $2.88 \times 10^{-6}$, indicating a sufficiently
reliable model.
\\\\
It is worthy of final mention that the objective solved by MB-OPLS is but a
single member of a super-family of methods introduced in detail by Hanafi
and Kiers \cite{hanafi:csda2006}. In the first family, nPLS and OPLS maximally
capture the between-matrix covariances before and after orthogonal signal
correction, respectively, and thus serve to regress a set of matrices
against each other. Methods in the second family capture \emph{both}
within-matrix variances and between-matrix covariances of a set of matrices
(CPCA-W), a set of response-weighted matrices (MB-PLS), and a set of
response-weighted OSC-filtered matrices (MB-OPLS). By casting these methods
in the light of MAXDIFF and MAXBET, we obtain an informative picture of their
characteristics, commonalities, and differences. For example, nPLS and OnPLS
force an equal contribution of each matrix to the solution through the
constraint $\|\mathbf{w}_i\|=1$, while CPCA-W, MB-PLS and MB-OPLS allow
contributions to float based on the ``importance'' of each matrix to the
modeling problem at hand. This super weight approach necessitates a block
scaling procedure to avoid highly weighting any given matrix due to size
alone \cite{smilde:jchemo2003,westerhuis:jchemo1998}.
\end{doublespace}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/mbopls/04-blkscores-1.png}
\caption
      [First-block Scores of Joint Spectroscopic Data.]{
  {\bf First-block Scores of Joint Spectroscopic Data.}
  \\
  Block scores from ({\bf A}) MB-PLS and ({\bf B}) MB-OPLS modeling of the
  \hnmr{} NMR data matrix. Ellipses and class colors are identical to those
  in Figure 9.2.
}
\label{figure.9.4}
\end{figure}

\begin{figure}[ht!]
\includegraphics[width=6in]{figs/mbopls/05-blkscores-2.png}
\caption
      [Second-block Scores of Joint Spectroscopic Data.]{
  {\bf Second-block Scores of Joint Spectroscopic Data.}
  \\
  Block scores from ({\bf A}) MB-PLS and ({\bf B}) MB-OPLS modeling of the
  DI-ESI-MS data matrix. Ellipses and class colors are identical to those
  in Figure 9.2.
}
\label{figure.9.5}
\end{figure}

\section{Conclusions}

\begin{doublespace}
The MB-OPLS method described in this chapter is a versatile extension of
MB-PLS to include an OSC filter, and belongs to a family of MAXBET optimizers
that share an equivalence with their single-block factorizations. By removing
consensus response-uncorrelated variation from a set of $n$ data matrices,
MB-OPLS expands the scope and benefits of OPLS to cases where blocking
information is available. The ability of MB-OPLS to separate predictive and
orthogonal variation from multiple blocks of data has been demonstrated on
both synthetic and real spectral data, both in cases of vector-$\mathbf{y}$
regression and discriminant analysis. The described algorithm admits either
a vector or a matrix as responses, and is implemented in the latest version
of the open-source MVAPACK chemometrics toolbox \cite{worley:acscb2014}.
\end{doublespace}

\bibliographystyle{abbrv}
\bibliography{bworley}