\section{Disentangling Adversarial Robustness\\[2px]\hspace*{-6px}and Generalization}
\label{sec:main}

To clarify the relationship between adversarial robustness and generalization, we explicitly distinguish between regular and on-manifold adversarial examples, as illustrated in \figref{fig:introduction}. Then, the hypothesis \cite{TsiprasARXIV2018,SuARXV2018} that robustness and generalization are contradicting goals is challenged in four arguments: regular \red{unconstrained} adversarial examples leave the manifold; adversarial examples constrained to the manifold exist; robustness against on-manifold adversarial examples is essentially generalization; \red{robustness against regular adversarial examples is not influenced by generalization when controlled through the amount of training data}. Altogether, our results imply that adversarial robustness and generalization are not opposing objectives \red{and both robust and accurate models are possible} but require higher sample complexity.

\vskip 0px
\subsection{Experimental Setup}

\myparagraph{Datasets:} We use \MNIST \cite{CohenARXIV2017}, F(ashion)-MNIST \cite{XiaoARXIV2017} and \Celeb \cite{LiuICCV2015} for our experiments ($240\text{k}/40\text{k}$, $60\text{k}/10\text{k}$ and $182\text{k}/20\text{k}$ training/test images); \Celeb has been re-sized to $56{\times}48$ and we classify ``Male'' \vs ``Female''. Our synthetic dataset, \Fonts, consists of letters ``A'' to ``J'' of $1000$ Google Fonts randomly transformed (uniformly over translation, shear, scale, rotation in $[-0.2,0.2]$, $[-0.5,0.5]$, $[0.75,1.15]$, $[-\nicefrac{\pi}{2},\nicefrac{\pi}{2}]$) using a spatial transformer network \cite{JaderbergNIPS2015} such that the generation process is completely differentiable. The latent variables correspond to the transformation parameters, font and class. We generated $960\text{k}/40\text{k}$ (balanced) training/test images of size $28{\times}28$.

We consider classifiers with three (four on \Celeb) convolutional layers ($4\times4$ kernels; stride $2$; $16$, $32$, $64$ channels), each followed by ReLU activations and batch normalization \cite{IoffeICML2015}, and two fully connected layers. The networks are trained using ADAM \cite{KingmaICLR2015}, with learning rate $0.01$ (decayed by $0.95$ per epoch), weight decay $0.0001$ and batch size $100$, for $20$ epochs. Most importantly, to control their generalization performance, we use $N$ training images, with $N$ between $250$ and $40\text{k}$; for each $N$, we train $5$ models with random weight initialization \cite{GlorotAISTATS2010} an report averages.

We learn class-specific \VAEGANs, similar to \cite{LarsenICML2016,RoscaARXIV2017}, to approximate the underlying manifold; we refer to the supplementary material for details.

\myparagraph{Attack:} Given an image-label pair $(x,y)$ from an unknown data distribution $p$ and a classifier $f$, an adversarial example is a perturbed image $\tilde{x} = x + \delta$ which is mis-classified by the model, \ie, $f(\tilde{x}) \neq y$. While our results can be confirmed using other attacks and norms (see the supplementary material for \cite{CarliniSP2017} and transfer attacks), for clarity, we concentrate on the $L_{\infty}$ white-box attack by Madry \etal \cite{MadryICLR2018} that directly maximizes the training loss,
\vskip -14px
\begin{align}
	\max_\delta \cL(f(x + \delta), y)\quad\text{s.t.}\quad\|\delta\|_{\infty} \leq \epsilon, \tilde{x}_i \in [0,1],\label{eq:main-off-manifold-attack}
\end{align}
\vskip -2px
\noindent using projected gradient descent; where $\cL$ is the cross-entropy loss and $\tilde{x}=x+\delta$. The $\epsilon$-constraint is meant to ensure perceptual similarity. We run $40$ iterations of ADAM \cite{KingmaICLR2015} with learning rate $0.005$ and consider $5$ restarts, (distance and direction) uniformly sampled in the $\epsilon$-ball for $\epsilon = 0.3$. Optimization is stopped as soon as the predicted label changes, \ie, $f(\tilde{x}) \neq y$. We attack $1000$ test images.

\begin{figure}
    \centering
    \vskip -0.3cm
    \hskip -0.1cm
    \begin{subfigure}[t]{0.5\textwidth}
        \begin{subfigure}[t]{0.49\textwidth}
            \includegraphics[width=0.9\textwidth]{experiments_hypo1_b.pdf}
        \end{subfigure}
        \begin{subfigure}[t]{0.49\textwidth}
            \includegraphics[width=0.9\textwidth]{experiments_hypo1_c.pdf}
        \end{subfigure}
    \end{subfigure}
    \vskip -6px
    \caption{Distance of adversarial examples to the true, on \Fonts (left), or approximated, on \MNIST (right), manifold. We show normalized histograms of the $L_2$ distance of adversarial examples to their projections onto the manifold ($4377$/$3837$ regular adversarial examples on \Fonts/\MNIST; $667$ on-manifold adversarial examples on \MNIST). Regular adversarial examples exhibit a significant distance to the manifold; on \MNIST, clearly distinguishable from on-manifold adversarial examples.}
    \label{fig:main-hypo1}
    \vskip 0px
\end{figure}

\myparagraph{Adversarial Training:} An established defense is adversarial training, \ie, training on adversarial examples crafted during training \cite{ZantedschiAISEC2017,MiyatoICLR2016,HuangARXIV2015,ShahamNEUROCOMPUTING2018,SinhaICLR2018,LeeARXIV2017b,MadryICLR2018}. Madry \etal \cite{MadryICLR2018} consider the min-max problem
\vskip -14px
\begin{align}
	\hskip -8px \min_w \sum_{n = 1}^N{\max_{\|\delta\|_{\infty} \leq \epsilon,x_{n,i}{+}\delta_i\in[0,1]}}{\cL}(f(x_n{+}\delta; w), y_n)
	\label{eq:main-off-manifold-adversarial-training}
\end{align}
\vskip -2px
\noindent where $w$ are the classifier's weights and $x_n$ the training images. As shown in the supplementary material, we considered different variants \cite{SzegedyARXIV2013,GoodfellowARXIV2014,MadryICLR2018}; in the paper, however, we follow common practice and train on $50\%$ clean images and $50\%$ adversarial examples \cite{SzegedyARXIV2013}. For $\epsilon = 0.3$, the attack (for the inner optimization problem) is run for full $40$ iterations, \ie, is not stopped at the first adversarial example found. Robustness of the obtained network is measured by computing the attack \textbf{success rate}, \ie, the fraction of successful attacks on correctly classified test images, as, \eg, in \cite{CarliniSP2017}, for a fixed~$\epsilon$; lower success rate indicates higher robustness of the network.

\vskip 0px
\subsection{Adversarial Examples Leave the Manifold}

The idea of adversarial examples leaving the manifold is intuitive on \MNIST where particular background pixels are known to be constant, see \figref{fig:main-examples}. If an adversarial example $\tilde{x}$ manipulates these pixels, it has zero probability under the data distribution and its distance to the manifold, \ie, the distance to its projection $\pi(\tilde{x})$ onto the manifold, should be non-zero. On \Fonts, with known generative process in the form of a decoder $\dec$ mapping latent variables $z$ to images $x$, the projection is obtained iteratively: $\pi(\tilde{x}) = \dec(\tilde{z})$ with $\tilde{z} = \argmin_{z} \|\dec(z) - \tilde{x})\|_2$ and $z$ constrained to valid transformations (font and class, known from the test image $x$, stay constant). On \MNIST, as illustrated in \red{\figref{fig:main-illustration-2} (right)}, the manifold is approximated using $50$ nearest neighbors; the projection $\pi(\tilde{x})$ onto the sub-space spanned by the $x$-centered nearest neighbors is computed through least squares. On both \Fonts and \MNIST, the distance $\|\tilde{x} - \pi(\tilde{x})\|_2$ is considered to asses whether the adversarial example $\tilde{x}$ actually left the manifold.

On \Fonts, \figref{fig:main-hypo1} (left) shows that regular adversarial examples clearly exhibit non-zero distance to the manifold. In fact, the projections of these adversarial examples to the manifold are almost always the original test images; as a result, the distance to the manifold is essentially the norm of the corresponding perturbation: $\|\tilde{x} - \pi(\tilde{x})\|_2 \approx \|\tilde{x} - x\|_2 = \|\delta\|_2$. This suggests that the adversarial examples leave the manifold in an almost orthogonal direction. On \MNIST, in \figref{fig:main-hypo1} (right), these results can be confirmed in spite of the crude local approximation of the manifold. Again, regular adversarial examples seem to leave the manifold almost orthogonally, \ie, their distance to the manifold coincides with the norm of the corresponding perturbations. These results show that regular adversarial examples essentially \emph{are} off-manifold adversarial examples; this finding is intuitive as for well-trained classifiers, leaving the manifold should be the ``easiest'' way to fool it; \red{results on \Fashion as well as a more formal statement of this intuition can be found in the supplementary material.}

\vskip 0px
\subsection{On-Manifold Adversarial Examples}

Given that regular adversarial examples leave the manifold, we intend to explicitly compute on-manifold adversarial examples. To this end, we assume our data distribution $p(x,y)$ to be conditional on the latent variables $z$, \ie, $p(x,y|z)$, corresponding to the underlying, low-dimensional manifold. \red{On this manifold, however, there is no notion of ``perceptual similarity'' in order to ensure label invariance, \ie, distinguish valid on-manifold adversarial examples, \figref{fig:introduction} (b), from invalid ones that change the actual, true label, \figref{fig:introduction} (c):}

\begin{definition}[On-Manifold Adversarial Example]
    Given the data distribution $p$, an on-manifold adversarial example for $x$ with label $y$ is a perturbed version $\tilde{x}$ such that $f(\tilde{x}) \neq y$ but $p(y | \tilde{x}) > p(y' | \tilde{x}) \forall y' \neq y$.\label{def:main-on-manifold-adversarial-example}
\end{definition}
\vskip 2px

\noindent Note that the posteriors $p(y|\tilde{x})$ correspond to the true, unknown data distribution; any on-manifold adversarial example $\tilde{x}$ violating \defref{def:main-on-manifold-adversarial-example} changed its actual, true label.

In practice, we assume access to an encoder and decoder modeling the (class-conditional) distributions $p(z|x,y)$ and $p(x|z,y)$ -- in our case, achieved using \VAEGANs \cite{LarsenICML2016,RoscaARXIV2017}. Then, given the encoder \red{$\enc$ and decoder $\dec$} and as illustrated in \figref{fig:main-illustration-2} (left), we obtain the latent code $z = \enc(x)$ and compute the perturbation $\zeta$ by maximizing:
\vskip -14px
\begin{align}
\max_\zeta \cL(f(\dec(z + \zeta)), y)\quad\text{s.t.}\quad\|\zeta\|_{\infty}\leq \eta.\label{eq:main-on-manifold-attack}
\end{align}
\vskip -2px
\noindent The image-constraint, \ie, $\dec(z + \zeta) \in[0,1]$, is enforced by the decoder; the $\eta$-constraint can, again, be enforced by projection; and we can additionally enforce a constraint on $z + \zeta$, \eg, corresponding to a prior on $z$. Label invariance, as in \defref{def:main-on-manifold-adversarial-example}, is ensured by considering only class-specific encoders and decoders, \ie, the data distribution is approximated per class. We use $\eta = 0.3$ and the same optimization procedure as for \eqnref{eq:main-off-manifold-attack}; on approximated manifolds, the perturbation $z + \zeta$ is additionally constrained to $[-2,2]^{10}$, corresponding to a truncated normal prior from the class-specific \VAEGANs; we attack $2500$ test images.

On-manifold adversarial examples obtained through \eqnref{eq:main-on-manifold-attack} are similar to those crafted in \cite{GilmerICLRWORK2018}, \cite{SchottARXIV2018}, \cite{AthalyeARXIV2018} or \cite{ZhaoICLR2018}. However, in contrast to \cite{GilmerICLRWORK2018,SchottARXIV2018,AthalyeARXIV2018}, we directly compute the perturbation $\zeta$ on the manifold instead of computing the perturbation $\delta$ in the image space and subsequently projecting $x + \delta$ to the manifold. Also note that enforcing any similarity constraint through a norm on the manifold is significantly more meaningful compared to using a norm on the image space, as becomes apparent when comparing the obtained on-manifold adversarial examples in \figref{fig:main-examples} to their regular counterparts. Compared to \cite{ZhaoICLR2018}, we find on-manifold adversarial examples using a gradient-based approach instead of randomly sampling the latent space.

\figref{fig:main-examples} shows on-manifold adversarial examples for all datasets, which we found significantly harder to obtain compared to their regular counterparts. On \Fonts, using the true, known class manifolds, on-manifold adversarial examples clearly correspond to transformations of the original test image -- reflecting the true latent space. For the learned class manifolds, the perturbations are less pronounced, often manipulating boldness or details of the characters. Due to the approximate nature of the learned \VAEGANs, these adversarial examples are strictly speaking not always part of the true manifold -- as can be seen for the irregular ``A'' (\figref{fig:main-examples}, 6th column). On \MNIST and \Fashion, on-manifold adversarial examples represent meaningful manipulations, such as removing the tail of a hand-drawn ``8'' (\figref{fig:main-examples}, 10th column) or removing the collar of a pullover (\figref{fig:main-examples}, 11th column), in contrast to the random noise patterns of regular adversarial examples. However, these usually incur a smaller change in the images space; which also explains why regular, unconstrained adversarial examples almost always leave the manifold. Still, on-manifold adversarial examples are perceptually close to the original images. On \Celeb, the quality of on-manifold adversarial examples is clearly limited by the approximation quality of our \VAEGANs. Finally, \figref{fig:main-hypo1} (right) shows that on-manifold adversarial examples are closer to the manifold than regular adversarial examples -- in spite of the crude approximation of the manifold on \MNIST.

\begin{figure}[t]
    \centering
    \vskip -0.3cm
    \begin{subfigure}{0.28\textwidth}
        \includegraphics[width=1\textwidth]{main_illustration_c.pdf}
    \end{subfigure}
    \begin{subfigure}{0.18\textwidth}
        \includegraphics[width=1\textwidth,trim={0 0 1.9cm 0},clip]{main_illustration_a.pdf}
    \end{subfigure}
    \vskip -8px
    \caption{\red{Left: On-manifold adversarial examples can be computed using learned, class-specific \VAEGANs \cite{LarsenICML2016,RoscaARXIV2017}. The perturbation $\zeta$ is obtained via \eqnref{eq:main-on-manifold-attack} and added to the latent code $z = \enc(x)$ yielding the adversarial example $\tilde{x} = \dec(z + \zeta)$ with difference $\delta = \tilde{x} - x$ in image space. Right: The distance of a regular adversarial example $\tilde{x}$ to the manifold, approximated using nearest neighbors, is computed as the distance to its orthogonal projection $\pi(\tilde{x})$: $\|\tilde{x} - \pi(\tilde{x})\|_2$. Large distances indicate that the adversarial example likely left the manifold.}}
    \label{fig:main-illustration-2}
    \vskip 0px
\end{figure}

\vskip 0px
\subsection{On-Manifold Robustness is Essentially\\[2px]\hspace*{-5px}Generalization}

\begin{figure*}
    \centering
    \vskip -0.3cm
    \hskip -0.25cm
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_fonts_error.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_fonts_on_true.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_emnist_error.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_fashion_error.pdf}
    \end{subfigure}
    \\
    \hskip -0.25cm
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_fonts_error_on_true_alt.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_fonts_error_on_learned_alt.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_emnist_error_on_learned_alt.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo3_fashion_error_on_learned_alt.pdf}
    \end{subfigure}
    \\
    \fcolorbox{black!50}{white}{
    \begin{subfigure}[t]{0.975\textwidth}
        \centering
        \includegraphics[width=0.9\textwidth]{experiments_hypo3_legend.pdf}
    \end{subfigure}
    }
    \vskip -6px
    \caption{On-manifold robustness is strongly related to generalization, as shown on \Fonts, \MNIST and \Fashion considering on-manifold success rate and test error. Top: test error and on-manifold success rate in relation to the number of training images. As test error reduces, so does on-manifold success rate. Bottom: on-manifold success rate plotted against test error reveals the strong relationship between on-manifold robustness and generalization.}
    \label{fig:main-hypo3}
    \vskip 0px
\end{figure*}

We argue that on-manifold robustness is nothing different than generalization: as on-manifold adversarial examples have non-zero probability under the data distribution, they are merely generalization errors. This is shown in \figref{fig:main-hypo3} (top left) where test error and on-manifold success rate on \Fonts are shown. As expected, better generalization, \ie, using more training images $N$, also reduces on-manifold success rate. In order to make this relationship explicit, \figref{fig:main-hypo3} (bottom) plots on-manifold success rate against test error. Then, especially for \Fonts and \MNIST, the relationship of on-manifold robustness and generalization becomes apparent. On \Fashion, the relationship is less pronounced because on-manifold adversarial examples, computed using our \VAEGANs, are not close enough to real generalization errors. However, even on \Fashion, the experiments show a clear relationship between on-manifold robustness and generalization.

\vskip 0px
\subsubsection{On-Manifold Adversarial Training\\[2px]Boosts Generalization}

Given that generalization positively influences on-manifold robustness, we propose to adapt adversarial training to the on-manifold case in order to boost generalization:
\vskip -14px
\begin{align}
    \min_w \sum_{n=1}^N \max_{\|\zeta\|_{\infty} \leq \eta} \cL(f(\dec(z_n + \zeta); w), y_n).
    \label{eq:main-on-manifold-adversarial-training}
\end{align}
\vskip -2px
\noindent with $z_n = \dec(x_n)$ being the latent codes corresponding to training images $x_n$. Then, on-manifold adversarial training corresponds to robust optimization \wrt the true, or approximated, data distribution. For example, with \matthias{the} perfect decoder on \Fonts, the inner optimization problem finds ``hard'' images irrespective of their likelihood under the data distribution. For approximate $\dec$, the benefit of on-manifold adversarial training depends on how well the true data distribution is matched, \ie, how realistic the obtained on-manifold adversarial examples are; in our case, this depends on the quality of the learned \VAEGANs.

\begin{figure*}
    \centering
    \vskip -0.3cm
    \hskip -0.25cm
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo4_fonts_off.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo4_fonts_error_off_alt.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo4_emnist_error_off_alt.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.24\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_hypo4_fashion_error_off_alt.pdf}
    \end{subfigure}
    \\
    \fcolorbox{black!50}{white}{
    \begin{subfigure}[t]{0.975\textwidth}
        \centering
        \includegraphics[width=0.9\textwidth]{experiments_hypo4_legend.pdf}
    \end{subfigure}
    }
    \vskip -6px
    \caption{Regular robustness is not related to generalization, as demonstrated on \Fonts, \MNIST and \Fashion considering test error and (regular) success rate. On \Fonts (left), success rate is not influenced by test error, except for adversarial training. Plotting success rate against test error highlights the independence of robustness and generalization; however, different training strategies exhibit different robustness-generalization characteristics.}
    \label{fig:main-hypo4}
    \vskip 0px
\end{figure*}

Instead of approximating the manifold using generative models, we can exploit known invariances of the data. Then, adversarial training can be applied to these invariances, assuming that they are part of the true manifold. In practice, this can, for example, be accomplished using adversarial deformations \cite{AlaifariARXIV2018,XiaoICLR2018,EngstromARXIV2017}, \ie, adversarially crafted transformations of the image. For example, as on \Fonts, we consider $6$-degrees-of-freedom transformations corresponding to translation, shear, scaling and rotation:
\vskip -14px
\begin{align}
    \min_w \sum_{n = 1}^N \max_{\|t\|_{\infty} \leq \eta, t \in \mR^6} \cL(f(T(x_n; t); w), y_n).
    \label{eq:main-stn-adversarial-training}
\end{align}
\vskip -2px
\noindent where $T(x; t)$ denotes the transformation of image $x$ with parameters $t$ and the $\eta$-constraint ensures similarity and label invariance. Again, the transformations can be applied using spatial transformer networks \cite{JaderbergNIPS2015} such that $T$ is differentiable; $t$ can additionally be constrained to a reasonable space of transformations. We note that a similar approach has been used by Fawzi \etal \cite{FawziICIP2016} to boost generalization on, \eg, MNIST \cite{LecunIEEE1998}. However, the approach was considered as an adversarial variant of data augmentation and not motivated through the lens of on-manifold robustness. We refer to \eqnref{eq:main-stn-adversarial-training} as adversarial transformation training and note that, on \Fonts, this approach is equivalent to on-manifold adversarial training as the transformations coincide with the actual, true manifold by construction. We also include a data augmentation baseline, where the transformations $t$ are applied randomly.

We demonstrate the effectiveness of on-manifold adversarial training in \figref{fig:main-hypo3} (top). On \Fonts, with access to the true manifold, on-manifold adversarial training is able to boost generalization significantly, especially for low $N$, \ie, few training images. Our \VAEGAN approximation on \Fonts seems to be good enough to preserve the benefit of on-manifold adversarial training. On \MNIST and \Fashion, the benefit reduces with the difficulty of approximating the manifold; this is the ``cost'' of imperfect approximation. While the benefit is still significant on \MNIST, it diminishes on \Fashion. However, both on \MNIST and \Fashion, identifying invariances and utilizing adversarial transformation training recovers the boost in generalization; especially in contrast to the random data augmentation baseline. Overall, on-manifold adversarial training is a promising tool for improving generalization and we expect its benefit to increase with better generative models.

\vskip 0px
\subsection{Regular Robustness is Independent of\\[2px]\hspace*{-5px}Generalization}

\red{We argue that generalization, as measured \emph{on} the manifold \wrt the data distribution, is mostly independent of robustness against regular, possibly off-manifold, adversarial examples when varying the amount of training data}. Specifically, in \figref{fig:main-hypo4} (left) for \Fonts, it can be observed that -- except for adversarial training -- the success rate is invariant to the test error. \red{This can best be seen when plotting the success rate against test error for different numbers of training examples, \cf \figref{fig:main-hypo4} (middle left): only for adversarial training there exists a clear relationship; for the remaining training schemes success rate is barely influenced by the test error. In particular, better generalization does not worsen robustness.} Similar behavior can be observed on \MNIST and \Fashion, see \figref{fig:main-hypo4} (right). Here, it can also be seen that different training strategies exhibit different characteristics \wrt robustness and generalization. \red{Overall, regular robustness and generalization are not necessarily contradicting goals.}

As mentioned in \secref{sec:introduction}, these findings are in contrast to related work \cite{TsiprasARXIV2018,SuARXV2018} claiming that an inherent trade-off between robustness and generalization exists. For example, Tsipras \etal \cite{TsiprasARXIV2018} use a synthetic toy dataset to theoretically show that no model can be both robust and accurate (on this dataset). However, they allow the adversary to produce perturbations that change the actual, true label \wrt the data distribution, \ie, the considered adversarial examples are not adversarial examples according to \defref{def:main-on-manifold-adversarial-example}. Thus, it is unclear whether the suggested trade-off actually exists \red{for real datasets}; our experiments, \red{at least, as well as further analysis in the supplementary material} seem to indicate the contrary. Similarly, Su \etal \cite{SuARXV2018} experimentally show a trade-off between adversarial robustness and generalization by studying different models on ImageNet \cite{RussakovskyIJCV2015}. However, Su \etal compare the robustness and generalization characteristics of different models (\ie, different architectures, training strategies \etc), while we found that the generalization performance does not influence robustness for any \emph{arbitrary, but fixed} model.

\vskip 0px
\subsection{Discussion}
\label{subsec:main-discussion}

Our results imply that robustness and generalization are not \red{necessarily} conflicting goals, as believed in related work \cite{TsiprasARXIV2018,SuARXV2018}. This means, in practice, for any arbitrary but fixed model, better generalization will not worsen regular robustness. Different models (architectures, training strategies \etc) might, however, exhibit different robustness and generalization characteristics, as also shown in \cite{SuARXV2018,RozsaICMLA2016}. For adversarial training, on regular adversarial examples, the commonly observed trade-off between robustness and generalization is explained by the tendency of adversarial examples to leave the manifold. As result, the network has to learn (seemingly) random, but adversarial, noise patterns \emph{in addition} to the actual task at hand; rendering the learning problem harder. On simple datasets, such as \MNIST, these adversarial directions might avoid overfitting; on harder tasks, \eg, \Fonts or \Fashion, the discrepancy in test error between normal and adversarial training increases. \red{Our results also support the hypothesis that regular adversarial training has higher sample complexity \cite{SchmidtARXIV2018,KhouryARXIV2018}. In fact, on \Fonts, adversarial training can reach the same accuracy as normal training with roughly twice the amount of training data, as demonstrated in \figref{fig:main-disc-2} (top). Furthermore, as illustrated in \figref{fig:main-disc-2} (bottom), the trade-off between regular robustness and generalization can be controlled by combining regular and on-manifold adversarial training, \ie boost generalization while reducing robustness.}

\begin{figure}
    \centering
    \vskip -0.3cm
    \hskip -0.25cm
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_fonts_error_resnet.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_fonts_error_off_resnet.pdf}
    \end{subfigure}
    \\
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_fonts_error_off_accuracy.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_emnist_error_off_accuracy.pdf}
    \end{subfigure}
    \\[-2px]
    \fcolorbox{black!50}{white}{
    \begin{subfigure}[t]{0.45\textwidth}
        \centering
        \includegraphics[width=1.025\textwidth]{experiments_disc_2_legend.pdf}
    \end{subfigure}
    }
    \vskip -6px
    \caption{\red{Adversarial training on regular adversarial examples, potentially leaving the manifold, renders the learning problem more difficult. Top: With roughly $1.5$ to $2$ times the training data, adversarial training can still reach the same accuracy as normal training; results for ResNet-13~\cite{HeCVPR2016}. Bottom: Additionally, the trade-off can be controlled by combining regular and on-manifold adversarial training; results averaged over $3$ models.}}
    \label{fig:main-disc-2}
    \vskip 0px
\end{figure}

The presented results can also be confirmed on more complex datasets, such as \Celeb, and using different threat models, \ie, attacks. On \Celeb, where \VAEGANs have difficulties approximating the manifold, \figref{fig:main-disc-1} (top left) shows that on-manifold robustness still improves with generalization although most on-manifold adversarial examples are not very realistic, see \figref{fig:main-examples}. Similarly, regular robustness, see \figref{fig:main-disc-1} (top right), is not influenced by generalization; here, we also show that the average distance of the perturbation, \ie, average $\|\delta\|_{\infty}$, when used to asses robustness leads to the same conclusions. Similarly, as shown in \figref{fig:main-disc-1} (bottom), our findings are confirmed using Carlini and Wagner's attack \cite{CarliniSP2017} with $L_2$-norm -- to show that the results generalize across norms. However, overall, we observed lower success rates using \cite{CarliniSP2017} and the $L_2$ norm. \red{Finally, our results can also be reproduced using transfer attacks (\ie, black-box attacks, which are generally assumed to be subsumed by white-box attacks \cite{AthalyeARXIV2018}) as well as and different architectures such as multi-layer perceptrons, ResNets~\cite{HeCVPR2016} and VGG~\cite{SimonyanARXIV2014}, as detailed in the supplementary material.}

\begin{figure}
    \centering
    \vskip -0.3cm
    \hskip -0.25cm
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_celeba_on_learned.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_celeba_off_distance.pdf}
    \end{subfigure}
    \\
    \hskip -0.25cm
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_emnist_error_on_learned_cw_alt.pdf}
    \end{subfigure}
    \begin{subfigure}[t]{0.235\textwidth}
        \vskip 0px
        \centering
        \includegraphics[width=1\textwidth]{experiments_disc_emnist_error_off_cw_alt.pdf}
    \end{subfigure}
    \\[-2px]
    \fcolorbox{black!50}{white}{
    \begin{subfigure}[t]{0.45\textwidth}
        \centering
        \includegraphics[width=1.01\textwidth]{experiments_disc_1_legend.pdf}
    \end{subfigure}
    }
    \vskip -6px
    \caption{Results on \Celeb and using the $L_2$ Carlini and Wagner \cite{CarliniSP2017} attack. On \Celeb, as the class manifolds are significantly harder to approximate, the benefit of on-manifold adversarial training diminishes. For \cite{CarliniSP2017}, we used $120$ iterations; our hypotheses are confirmed, although \cite{CarliniSP2017} does not use the training loss as attack objective and the $L_2$ norm changes the similarity-constraint for regular and on-manifold adversarial examples.}
    \label{fig:main-disc-1}
    \vskip 0px
\end{figure}


\section{Logarithms}


\section{Serializable types}\label{sec:dsdl_serializable_types}

Values of the serializable type category can be exchanged between nodes over the UAVCAN network.
This is opposed to the expression types (section~\ref{sec:dsdl_expression_types}),
instances of which can only exist while DSDL definitions are being evaluated.
The data serialization rules are defined in section~\ref{sec:dsdl_data_serialization}.

\subsection{Void types}

Void types are used for padding purposes.
As will be explained in later sections, it is desirable to align field attributes at byte boundaries;
void types can be used to facilitate that.

Void-typed field attributes are set to zero when an object is serialized and ignored when it is deserialized.
Void types can be used to reserve space in data type definitions for possible use in later versions of the data type.

The DSDL name pattern for void types is as follows: ``\verb|void[1-9]\d*|'',
where the trailing integer represents its width, in bits,
ranging from 1 to 64, inclusive.

Void types can be referred to directly by their name from any namespace.

\subsection{Primitive types}\label{sec:dsdl_primitive_types}

Primitive types are assumed to be known to DSDL processing tools a priori,
and as such, they need not be defined by the user.
Primitive types can be referred to directly by their name from any namespace.

\subsubsection{Hierarchy}

The hierarchy of primitive types is documented below.

\begin{itemize}
    \item \textbf{Boolean types.} A boolean-typed value represents a variable of the Boolean algebra.
    A Boolean-typed value can have two values: true and false.
    The corresponding DSDL data type name is ``\verb|bool|''.

    \item \textbf{Algebraic types.} Those are types for which conventional algebraic operators are defined.
    \begin{itemize}
        \item \textbf{Integer types} are used to represent signed and unsigned integer values.
        See table~\ref{table:dsd_integer_properties}.
        \begin{itemize}
            \item \textbf{Signed integer types.} These are used to represent values which can be negative.
            The corresponding DSDL data type name pattern is ``\verb|int[1-9]\d*|'',
            where the trailing integer represents the length of the
            serialized representation of the value, in bits, ranging from 2 to 64, inclusive.

            \item \textbf{Unsigned integer types.} These are used to represent non-negative values.
            The corresponding DSDL data type name pattern is ``\verb|uint[1-9]\d*|'',
            where the trailing integer represents the length of the
            serialized representation of the value, in bits, ranging from 1 to 64, inclusive.
        \end{itemize}

        \item \textbf{Floating point types} are used to approximately represent real values.
        The underlying serialized representation follows the IEEE 754 standard.
        The corresponding DSDL data type name pattern is ``\verb~float(16|32|64)~'', where the trailing
        integer represents the type of the IEEE 754 representation.
        See table~\ref{table:dsd_floating_point_properties}.
    \end{itemize}
\end{itemize}

\begin{UAVCANSimpleTable}{Properties of integer types}{|l X l|}%
    \label{table:dsd_integer_properties}%
    Category &
    DSDL names &
    Range, $X$ is bit length \\

    Signed integers &
    \texttt{int2}, \texttt{int3}, \texttt{int4} \ldots{} \texttt{int62}, \texttt{int63}, \texttt{int64} &
    $\left[-\frac{2^{X}}{2},\frac{2^{X}}{2}-1\right]$ \\

    Unsigned integers &
    \texttt{uint1}, \texttt{uint2}, \texttt{uint3} \ldots{} \texttt{uint62}, \texttt{uint63}, \texttt{uint64} &
    $\left[0,2^{X}-1\right]$ \\
\end{UAVCANSimpleTable}

\begin{UAVCANSimpleTable}{Properties of floating point types}{|X X X X|}
    DSDL name        & Representation    & Approximate epsilon   & Approximate range
    \label{table:dsd_floating_point_properties} \\

    \texttt{float16} & IEEE 754 binary16 & $0.001$               & $\pm{}65504$      \\
    \texttt{float32} & IEEE 754 binary32 & $10^{-7}$             & $\pm{}10^{39}$    \\
    \texttt{float64} & IEEE 754 binary64 & $2 \times{} 10^{-16}$ & $\pm{}10^{308}$   \\
\end{UAVCANSimpleTable}

\subsubsection{Cast mode}

The concept of \emph{cast mode} is defined for all primitive types.
The cast mode defines the behavior when a primitive-typed entity is assigned a value that exceeds its range.
Such assignment requires some of the information to be discarded;
due to the loss of information involved, it is called a \emph{lossy assignment}.

The following cast modes are defined:
\begin{description}
    \item[Truncated mode] --- denoted with the keyword ``\verb|truncated|'' placed before the primitive type name.
    \item[Saturated mode] --- denoted with the optional keyword ``\verb|saturated|''
    placed before the primitive type name. If neither cast mode is specified, saturated mode is assumed by default.
    This essentially makes the ``\verb|saturated|'' keyword redundant; it is provided only for consistency.
\end{description}

When a primitive-typed entity is assigned a value that exceeds its range,
the resulting value is chosen according to the lossy assignment rules
specified in table~\ref{table:dsdl_cast_mode}.
Cases that are marked as illegal are not permitted in DSDL definitions.

\begin{UAVCANSimpleTable}{Lossy assignment rules per cast mode}{|l X X|}
    Type category & Truncated mode & Saturated mode (default)
    \label{table:dsdl_cast_mode} \\

    Boolean &
    Illegal: boolean type with truncated cast mode is not allowed. &
    Falsity if the value is zero or false, truth otherwise. \\

    Signed integer &
    Illegal: signed integer types with truncated cast mode are not allowed. &
    Nearest reachable value. \\

    Unsigned integer &
    Most significant bits are discarded. &
    Nearest reachable value. \\

    Floating point &
    Infinity with the same sign, unless the original value is not-a-number, in which case it will be preserved. &
    If the original value is finite, the nearest finite value will be used.
    Otherwise, in the case of infinity or not-a-number, the original value will be preserved. \\
\end{UAVCANSimpleTable}

Rules of conversion between values of different type categories do not affect compatibility at the protocol level,
and as such, they are to be implementation-defined.

\subsubsection{Expressions}

At the time of DSDL definition processing,
values of primitive types are represented as instances of the \verb|rational| type (section~\ref{sec:dsdl_rational}),
with the exception of the type \verb|bool|,
instances of which are usable in DSDL expressions as-is.

\begin{UAVCANSimpleTable}{Operators defined on instances of type boolean}{|l X X|}
    Op & Type & Description \\

    \texttt{\textbf{!}}     & $(\texttt{bool}) \rightarrow \texttt{bool}$ & Logical not. \\

    \texttt{\textbf{||}}    & $(\texttt{bool}, \texttt{bool}) \rightarrow \texttt{bool}$ & Logical or. \\
    \texttt{\textbf{\&\&}}  & $(\texttt{bool}, \texttt{bool}) \rightarrow \texttt{bool}$ & Logical and. \\

    \texttt{\textbf{==}}    & $(\texttt{bool}, \texttt{bool}) \rightarrow \texttt{bool}$ & Equality. \\
    \texttt{\textbf{!=}}    & $(\texttt{bool}, \texttt{bool}) \rightarrow \texttt{bool}$ & Inequality. \\

\end{UAVCANSimpleTable}

\subsubsection{Reference list}

An exhaustive list of all void and primitive types
ordered by bit length is provided below for reference.
Note that the cast mode specifier is omitted intentionally.

\immediate\write18{rm -f ../latex.tmp}
\immediate\write18{../render_list_of_void_and_primitive_types.py > ../latex.tmp}
\immediate\input{../latex.tmp}

\subsection{Array types}

An array type represents an ordered collection of values.
All values in the collection share the same type, which is referred to as \emph{array element type}.
The array element type can be any type except:
\begin{itemize}
    \item void type;
    \item array type\footnote{%
              Meaning that nested arrays are not allowed;
              however, the array element type can be a composite type which in turn may contain arrays.
              In other words, indirect nesting of arrays is permitted.
          }.
\end{itemize}

The number of elements in the array can be specified as a constant expression at the data type definition time,
in which case the array is said to be a \emph{fixed-length array}.
Alternatively, the number of elements can vary between zero and some static limit specified
at the data type definition time,
in which case the array is said to be a \emph{variable-length array}.
Variable-length arrays with unbounded maximum number of elements are not allowed.

Arrays are defined by adding a pair of square brackets after the array element type specification,
where the brackets contain the \emph{array capacity expression}.
The array capacity expression shall yield a positive integer of type ``\verb|rational|'' upon its evaluation;
any other value or type renders the current DSDL definition invalid.

The array capacity expression can be prefixed with the following character sequences in order to define
a variable-length array:
\begin{itemize}
    \item ``\verb|<|'' (ASCII code 60) --- indicates that the integer value yielded by the array capacity expression
    specifies the non-inclusive upper boundary of the number of elements.
    In this case, the integer value yielded by the array capacity expression shall be greater than one.

    \item ``\verb|<=|'' (ASCII code 60 followed by 61) --- same as above, but the upper boundary is inclusive.
\end{itemize}
If neither of the above prefixes are provided, the resultant definition is that of a fixed-length array.

\subsection{Composite types}\label{sec:dsdl_composite_types}

\subsubsection{Kinds}

There are two kinds of composite data type definitions: message types and service types.
All versions of a data type shall be of the same kind\footnote{%
    For example, if a data type version 0.1 is of a message kind, all later versions of it shall be messages, too.
}.

A message data type defines one data schema.

A service data type contains two data schema definitions:
one for service request object, and one for service response object, in that order.
The two schemas are separated by the service response marker (``\verb|---|'') on a separate line.

The presence of the service response marker indicates that the data type definition at hand is of the service kind.
At most one service response marker shall appear in a given definition.

\subsubsection{Dependencies}

In order to refer to a composite type from another composite type definition
(e.g., for nesting or for referring to an external constant),
one has to specify the full data type name of the referred data type followed by its
major and minor version numbers separated by the namespace separator character,
as demonstrated on figure~\ref{fig:dsdl_nested_reference}.

To facilitate look-up of external dependencies,
implementations are expected to obtain from external sources\footnote{%
    For example, from user-provided configuration options.
} the list of directories that are the roots of namespaces containing the referred dependencies.

\begin{figure}[H]
    $$
    \underbrace{\texttt{\huge{uavcan.node.Heartbeat}}}_{\text{full data type name}}%
    \texttt{\huge{.}}%
    \underbrace{\texttt{\huge{1.0}}}_{\substack{\text{version} \\ \text{numbers}}}%
    $$
    \caption{Reference to an external composite data type definition\label{fig:dsdl_nested_reference}}
\end{figure}

If the referred data type and the referring data type share the same full namespace name,
it is allowed to omit the namespace from the referred data type specification
leaving only the short data type name, as demonstrated on figure~\ref{fig:dsdl_nested_reference_short}.
In this case, the referred data type will be looked for in the namespace of the referrer.
Partial omission of namespace components is not permitted.

\begin{figure}[H]
    $$
    \underbrace{\texttt{\huge{Heartbeat}}}_{\text{short data type name}}%
    \texttt{\huge{.}}%
    \underbrace{\texttt{\huge{1.0}}}_{\substack{\text{version} \\ \text{numbers}}}%
    $$
    \caption{Reference to an external composite data type definition located in the same namespace
             \label{fig:dsdl_nested_reference_short}}
\end{figure}

Circular dependencies are not permitted.
A circular dependency renders all of the definitions involved in the dependency loop invalid.

If any of the referred definitions are marked as deprecated,
the referring definition shall be marked deprecated as well\footnote{%
    Deprecation is indicated with the help of a special directive, as explained in section~\ref{sec:dsdl_directives}.
}.
If a non-deprecated definition refers to a deprecated definition,
the referring definition is malformed\footnote{%
    This tainting behavior is designed to prevent unexpected breakage of
    type hierarchies when one of the deprecated dependencies reaches its end of life.
}.

When a data type is referred to from within an expression context,
it constitutes a literal of type ``\verb|metaserializable|'' (section~\ref{sec:dsdl_metaserializable}).
If the referred data type is of the message kind,
its attributes are accessible in the referring expression through application of the
attribute reference operator ``\verb|.|''.
The available attributes and their semantics are documented in the section~\ref{sec:dsdl_local_attributes}.

\begin{remark}
    \begin{minted}{python}
        uint64 MY_CONSTANT = vendor.MessageType.1.0.OTHER_CONSTANT
        uint64 MY_CONSTANT = MessageType.1.0.OTHER_CONSTANT
        # The above is valid if the referring definition and the referred definition
        # are located inside the root namespace "vendor".
        @print MessageType.1.0
    \end{minted}
\end{remark}

\subsubsection{Unions}\label{sec:dsdl_composite_tagged_unions}

Any data schema definition can be supplied with a special directive (section~\ref{sec:dsdl_directives})
indicating that it forms a tagged union.

A tagged union schema shall contain at least two field attributes.
A tagged union shall not contain padding field attributes.

The value of a tagged union object is a function of the field attribute which value it is currently holding
and the value of the field attribute itself.

\subsubsection{Type polymorphism and equivalency}

Type polymorphism is supported in DSDL through structural subtyping.
The following definition relies on the concept of \emph{field attribute},
which is introduced in section~\ref{sec:dsdl_attributes}.

Let $B$ and $D$ be DSDL message types whose data schemas define $b$ and $d$ field attributes, respectively.
Let each field attribute be assigned a sequential index according to its position in the DSDL definition
(see section~\ref{sec:dsdl_grammar} on statement ordering).

$D$ is a \emph{structural subtype} of $B$ if all conditions are satisfied:
\begin{itemize}
    \item neither $B$ nor $D$ define a tagged union;
    \item $B$ is not $D$;
    \item $b \leq d$;
    \item for each field attribute of $B$ at index $i$ there is an equal\footnote{%
              Field attribute equality is defined in section~\ref{sec:dsdl_attributes}.
          } field attribute in $D$ at index $i$.
\end{itemize}

$D$ is a structural subtype of $B$ if all conditions are satisfied:
\begin{itemize}
    \item both $B$ and $D$ define tagged unions;
    \item $B$ is not $D$;
    \item $b \leq d$;
    \item $\lceil{}\log_2 b\rceil{} = \lceil{}\log_2 d\rceil{}$;
    \item for $i \in \left[0, b\right)$, the type of the field attribute of $D$ at index $i$
          is the same or is a subtype of the type of the field attribute of $B$ at index $i$.
    \item for $i \in \left[0, b\right)$, the name of the field attribute of $D$ at index $i$
          is the same as the name of the field attribute of $B$ at index $i$.
\end{itemize}

$D$ is a structural subtype of $B$ if $b = 0$.

If $D$ is a structural subtype of $B$, then $B$ is a \emph{structural supertype} of $D$.

$D$ and $B$ are \emph{structurally equivalent} if $D$ is a structural subtype and a structural supertype of $B$.

A \emph{type hierarchy} is an ordered set of data types such that for each pair of its members
one type is a subtype of another, and for any member its supertypes are located on the left.

\begin{remark}
    Subtyping example for structure (non-union) types. First type:

    \begin{minted}{python}
        float64 a       # Index 0
        int16[<=9] b    # Index 1
    \end{minted}

    The second type is a structural subtype of the first type:

    \begin{minted}{python}
        float64 a       # Index 0
        int16[<=9] b    # Index 1
        uint8 foo       # Index 2
    \end{minted}

    Subtyping example for union types. First type:

    \begin{minted}{python}
        @union                            # The implicit union tag field is 2 bits wide
        uavcan.primitive.Empty.1.0 foo
        float16 bar
        uint8 zoo
    \end{minted}

    The second type is a structural subtype of the first type:

    \begin{minted}{python}
        @union                            # The implicit union tag field is 2 bits wide
        uavcan.diagnostic.Record.1.0 foo  # Subtype
        float16 bar                       # Same
        uint8 zoo                         # Same
        int64[<=64] baz                   # New field
    \end{minted}

    A message type that defines zero fields is a structural supertype of any other message type, regardless of either
    or both being a union.
    A message type may have an arbitrary number of supertypes as long as the field equality constraint is satisfied.

    Polymorphic relations are not defined on service types;
    however, this may change in a future revision of the standard.
\end{remark}

\begin{remark}[breakable]
    The following example in C demonstrates the concept of polymorphic compatibility detached from DSDL.

    \begin{samepage}
        \begin{minted}{c}
            struct base
            {
                int a;
                float b;
            };

            struct derived_first
            {
                int a;
                float b;
                double c;
            };

            struct derived_second
            {
                int a;
                float b;
                short d;
            };

            float compute(struct base* value)
            {
                return (float)value->a + value->b;
            }

            int main()
            {
                struct derived_first foo  = { .a = 123, .b = -456.789F, .c = 123.456 };
                struct derived_second bar = { .a = -123, .b = 456.789F, .d = -123 };
                // Both derived_first and derived_second are structural subtypes of base. The program returns zero.
                return compute(&foo) + compute(&bar);
            }
        \end{minted}
    \end{samepage}
\end{remark}


\subsubsection{\stid{3.15} Sake Sub-project: Kokkos Kernels}
\paragraph{Overview}
The Kokkos Kernels~\footnote{https://github.com/kokkos/kokkos-kernels} subproject primarily focuses on performance portable kernels
for sparse/dense linear algebra, graphs, and machine learning, with emphasis on
kernels that are key to the performance of
several ECP applications. We work closely with ECP applications to identify the
performance-critical kernels and develop portable algorithms for these kernels.
The primary focus of this subproject is to support ECP application needs, develop new
kernels needed with an emphasis towards software releases, tutorials, boot camps
and user support. The Kokkos Kernels project also works closely with several vendors
(AMD, ARM, Cray, Intel, and NVIDIA) as part of both the ECP collaborations and
NNSA's Center for Excellence efforts. These collaborations will  enable vendor solutions
that are targeted towards ECP application needs.

\paragraph{Key  Challenges}
There are several challenges in allowing ECP applications move to the hardware architectures
announced in the next few years. We highlight the four primary challenges here:
\begin{enumerate}
\item
The next three supercomputers that will be deployed will have
three different accelerators from AMD, Intel and NVIDIA. While we have been expecting diversity of architectures, three
different architectures in such a short timeframe adds pressure on the portability
solutions such as Kokkos Kernels to optimize and support the kernels on all the platforms.
\item
The design of several ECP applications and a software stack that rely on a component-based
approach results in an extremely high number of kernel launches on the accelerators, which
results in the latency costs becoming the primary bottleneck in scaling the applications.
\item
The change in the needs of applications from device-level kernels to smaller team-level kernels. Vendor
libraries are not ready for such a drastic change in software design.
\item
The reliance of ECP applications on certain kernels that do not port well to the  accelerator architectures.
\end{enumerate}

These challenges require a collaborative effort to explore new algorithmic choices,
working with the vendors to incorporate ECP needs into their library plans, to develop
portable kernels from scratch, and to deploy them in a robust software ecosystem. The Kokkos
Kernels project will pursue all of these choices in an effort to address these challenges.

\paragraph{Solution Strategy}

Our primary solution strategy to address these challenges are:
\begin{enumerate}
    \item \textbf{Codesign portable kernels with vendors and applications:}
    We rely on codesign of Kokkos Kernels implementations for
    specializations that are key to the performance of ECP applications. This
    requires tuning kernels even up to the problem sizes that are of interest
    to our users. Once we have developed a version, we provide these
    to all the vendors so their teams can optimize these kernels even
    further in vendor-provided math libraries.
   \item \textbf{Emphasis on software support and usability:}
	The Kokkos Kernels project devotes a considerable amount of time working with
	ECP applications, integrating the kernels into application codes, tuning
	for application needs, and providing tutorials and user support. We invest
	in delivering a robust software ecosystem that serves the
	needs of diverse ECP applications on all platforms of interest.
   \item \textbf{Invest in algorithmic research to reduce latency costs and new accelerator focused approaches:}
   To resolve latency cost issues, the Kokkos Kernels team is considering several solutions from computer science
   perspectives and also from algorithmic applied mathematics perspectives.
   For example, from a computer science perspective, we are focusing on the
   use of streams or other latency reducing techniques such as cuda graphs. From
   the applied mathematics perspective we are developing new algorithms such
   as cluster-based approaches for preconditioners, such as Gauss-Seidel precondtioners,
   to reduce the number of kernel launches.
\end{enumerate}

\paragraph{Recent Progress}
\begin{enumerate}
\item Kokkos Kernels has implemented and released its HIP backend to support deployment on early
access platform Spock. While all of the kernels are available for this backend further work on
performance improvement and vendor TPL support are forth coming.
\item New graph algorithm for distance 2 maximum independent set and coarsening have been developed.
They provide support for the coarsening set of multigrid algorithms on GPUs, and of the applications
dependent on it such as ExaWind and EMPIRE.
\item Recently the SYCL backend has been enabled in Kokkos Kernels, it is not currently supporting all
algorithms (sparse tests are disabled) but provides initial capabilities and will be released in upcoming
Kokkos Kernels version 3.5.0.
\item Kokkos Kernels team has developed and integrated tutorial materials to the Kokkos tutorials. The
tutorials are maintained as a common resource for the entire Kokkos ecosystem.
\end{enumerate}

\paragraph{Next Steps}

Kokkos Kernels team is focused on:
\begin{enumerate}
	\item \textbf{A major software release}: Kokkos ecosystem 4.0 release
	will be available to the ECP applications in FY 2022. This includes several new
	kernels that are requested by ECP applications, performance improvements of
	kernels that are already being used by ECP applications and further support for
        new SYCL and OpenMP Target to enable applications to be tested on Intel architectures.
	\item \textbf{Developing new sparse batched kernels}: Kokkos Kernels team is working on
        sparse batched kernels that allow application to solve small linear system that might
        exist at integration point embeded in larger systems of equations. Such algorithms will
        support ECP applications Pele and XGC.
	\item \textbf{Collaboration with vendors}: Kokkos Kernels team is working with vendor
	libraries team to incorporate ECP application needs in the vendor library roadmap.
	Conversation regarding batched algorithms is on going and collaboration are being
        actively leveraged to further prepare the library for ExaScale platforms readiness.
\end{enumerate}


\section{Path Following}
\label{chap:path_following}
% Explain what should a path follower do and how, then describe how it has been implemented
The first task our controller should accomplish is to follow a reference path.
How this reference path is given to the controller is described in Section \ref{chap:SignalAcquisition} - Signal Acquisition, where we have assumed that we have a high level device which is able to provide the controller with a sampled map composed by a sequence of way-points reporting $x$, $y$, $\theta$, and $v$ for each time-step of the control. We pass to the controller the next \textit{p} steps in the reference map, where \textit{p} is the number of steps of the prediction horizon.\\
Before the final testing process, we have performed several simulations in order to obtain a fine controller tuning regarding its weights (relative to output variables and manipulated variables) and other influential quantities. \\
The best results for path following have been achieved with the following configuration:
\begin{table}[H]
\centering
\begin{tabular}{|l|r|}\hline
Sample Time                        & 0.01 $s$           \\ \hline
Prediction Horizon                 & 15 steps         \\ \hline
Control Horizon                    & 5 steps          \\ \hline
Output Variables Weights           & {[}30 30 8 30{]} \\ \hline
Manipulated Variables Weights      & {[}0 0{]}        \\ \hline
Manipulated Variables Rate Weights & {[}0 0{]}       \\ \hline
\end{tabular}
\caption{Configuration parameters adopted for Path Following}
\label{tab:configuration}
\end{table}
For the sake of the Path Following task, only the constraints on the inputs have been considered while neglecting any constraint on the states.\\
Moreover, we have set the initial value for the \textit{covariance matrix} of the internal state estimator to $\left[ \begin{smallmatrix} 1000 & 0 & 0 & 0 & 0 & 0 \\0 & 1000 & 0 & 0 & 0 & 0 \\0 & 0 & 1000 & 0 & 0 & 0 \\ 0 & 0 & 0 & 1000 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1000 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1000 \end{smallmatrix} \right] $ in order to avoid oscillations in the starting phase of the control, due to the discrepancies between the plant model given to the controller and the dynamic plant model used for simulation.
The covariance is a $6\times 6$ matrix because it is related to 4 states and 2 inputs.\\
This matrix is automatically updated by the MPC algorithm at runtime.
\subsection{Path Following - Testing} \label{subsection:path_following_testing}
% Describe the tests to validate the path following algorithm
The path following tests we have set up, aim to evaluate the performance of a path follower in different scenarios with speed ranging from 10 to 100 $km/h$ as stated in Section \ref{System_Requirements} - System Requirements.
In order to satisfy requirements 1 and 5 regarding  the \textit{maximum lateral error} and the \textit{maximum lateral acceleration} respectively, we have set up a test-bench to assert these quantities. In detail, to verify the lateral error, we have built a function to approximate the distance between the actual point in which the vehicle is and the reference trajectory. This function is based on the assumption that the vehicle is close enough to the reference point considered and that the reference, in the close proximity of the point itself, can be approximated as a straight line. These considerations are summarized by the following equation:
\begin{equation}
    Lateral\_dev = \frac{\|y_{vehicle} - (mx_{vehicle} + q)\|}{\sqrt{1 + m^2}} \label{eq:lateral_dev}
\end{equation}
where:\\
\begin{center}
 $   m = \tan(\theta_{ref}) $\\
 $  q = y_{ref}-mx_{ref}$
\end{center}
\vspace{10pt}
The tests are considered passed if the following conditions are verified:
\begin{itemize}
    \item lateral deviation always lower than 1.0 $m$;
    \item lateral deviation does not exceed 0.75 $m$ for more than 1 $s$.
\end{itemize}
\vspace{10pt}
For what concerns the lateral acceleration we get it directly from the state of the dynamic model as reported in Section \ref{chap:Vehicle_model}, equation \ref{dynamic_equation}: $\ddot{y} = -\dot{\psi}\dot{x} + \frac{2}{m}\left(F_fcos(\delta) + F_r\right)$.\\
The test is considered passed if the lateral acceleration does not exceed 2.0 $m/s^2$ for more than 0.5 $s$.\\ \\
Each path following test has been performed with all the vehicles stored in our database, as reported in section \ref{chap:Vehicle_model} Table \ref{tab:vehicle_data}.

\pagebreak
The scenarios we have selected are the following:
\begin{enumerate}
    \item \textbf{Straight line}: this is the simplest possible scenario, consisting of a straight line starting from $(0,0)$ up to $(1000,0)$ in the X-Y reference frame, with 0° orientation. We repeated this scenario twice with speeds respectively of 10 $km/h$ and 100 $km/h$;
    \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/StraightMap.png}
    \caption{Straight line Map}
      \label{fig:StraightMap}
\end{figure}
\vspace{1cm}
    \item \textbf{Tilted straight line - 45°}: this is a simple scenario consisting of a straight line starting from $(0,0)$ up to $(1000,1000)$, with an orientation of 45°. We selected a speed of 10 $km/h$ for this scenario;
    \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/Tilted45Map.png}
    \caption{Tilted straight 45° map}
      \label{fig:Tilted45Map}
\end{figure}
\pagebreak
    \item \textbf{Tilted straight line - 135°}: this is a simple scenario consisting of a straight line starting from $(0,0)$ up to $(-1000,1000)$, with an orientation of 135°. We selected a speed of 100 $km/h$ for this scenario;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/Tilted135Map.png}
    \caption{Tilted straight 135° map}
      \label{fig:Tilted135Map}
\end{figure}
\vspace{1cm}
    \item \textbf{100m radius Curve}: this scenario consist of a road with constant curvature with 100 $m$ radius. We repeated this scenario twice with speeds respectively of 10 $km/h$ and 100 $km/h$;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/ShortCurveMap.png}
    \caption{100 $m$ radius Curve Map}
      \label{fig:100mCurveMap}
\end{figure}
\pagebreak
    \item \textbf{1000m radius Curve}: this scenario consist of a road with constant curvature with 1000 $m$ radius. We repeated this scenario twice with speeds respectively of 20 $km/h$ and 100 $km/h$;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/LongCurveMap.png}
    \caption{1000 $m$ radius Curve Map}
      \label{fig:1000mCurveMap}
\end{figure}
\vspace{1cm}
    \item \textbf{Puglia}: this scenario is taken from a city road and it is straight for the most part with some smooth corners. In this test we try to follow the path with a speed of 40 $km/h$;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/PugliaMap.png}
    \caption{Puglia Map}
      \label{fig:PugliaMap}
\end{figure}
\pagebreak
    \item \textbf{Switzerland}: this is the slowest scenario considered, with lots of corners one after another. We try to follow this scenario with 15 $km/h$ speed;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/SwitzerlandMap.png}
    \caption{Switzerland Map}
      \label{fig:SwitzerlandMap}
\end{figure}
\vspace{1cm}
    \item \textbf{Campania}: this scenario is made up by a sequence of smooth corners. We try to follow this path with 30 $km/h$ speed;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/CampaniaMap.png}
    \caption{Campania Map}
      \label{fig:Campania}
\end{figure}
\pagebreak
    \item \textbf{Adriatic Highway - A14}: This scenario is taken from the A14 highway which is a straight road for the most of it, with some high-speed corners. We simulate this scenario at 100 $km/h$;
     \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/A14Map.png}
    \caption{Adriatic Highway - A14 Map}
      \label{fig:A14Map}
\end{figure}
\vspace{1cm}
    \item \textbf{Indianapolis Speedway}: This scenario is taken from the Indianapolis Speedway. We simulate it at 100 $km/h$.
    \begin{figure}[H]
    \centering
    \includegraphics[width=0.7\textwidth]{Figures/IndianapolisMap.png}
    \caption{Indianapolis Speedway Map}
      \label{fig:Indianapolis}
\end{figure}
\end{enumerate}

\pagebreak
Details about the path-following tests performed can be found in the documents included in the repository\footnote{The ``Path\_Following-Test\_Report", the ``Path\_Following\_Test-Annex\_A" and the ``Path\_Following-Test\_Report - Failed Tests" files generated by Simulink Test are included in the /Documentation/Test Reports/ file path.}.\\
Performing the tests we have found that in simple scenarios, such as straight lines and the Highway scenario, the performances of our controller are very good, with little deviations from the reference path, while in winding scenarios, such as the \textit{Switzerland} one, we have found out that the vehicle is not able to keep its position inside the road boundaries in all the corners.
Figures \ref{subfig:A14} and \ref{subfig:Switzerland} show the lateral deviation found in the test performed on the Adriatic Highway A14 and the one found in Switzerland respectively. As shown below, the lateral deviation in the former never exceeds the limits imposed (described at the beginning of this section), while in the latter these limits are exceeded few times resulting in the failure of the test.

\begin{figure}[H]
\centering

    \begin{subfigure}{.5\textwidth}
    \centering
   \includegraphics[width=\textwidth,keepaspectratio]{Figures/A14_lateralDev.png}
    \caption{Lateral Deviation Plot (Adriatic Highway A14)}
    \label{subfig:A14}
    \end{subfigure}%
    \begin{subfigure}{.5\textwidth}
    \centering
    \includegraphics[width=0.95\textwidth,keepaspectratio]{Figures/Switzerland_lateralDev.png}
    \caption{Lateral Deviation Plot (Switzerland)}
    \label{subfig:Switzerland}
    \end{subfigure}
    \caption{Lateral Deviation Comparison}
    \label{fig:lateral_dev_comparison}
\end{figure}

\subsubsection{Failed Tests Analysis}
In the ``Path\_Following-Test\_Report - Failed Tests" we have included the results of the failed tests. In this section we will describe in more detail the issues we have faced during these tests.
What we want to point out is that in the failed tests, the lateral deviation condition is violated only in some specific points, where the curvature of the reference road is, maybe, too high to be performed at constant speed by the vehicle. We encountered this problem in the \textit{Switzerland} scenario with all vehicles and in the \textit{Campania} scenario only with the vehicle Ford E150, which is a heavy van, and it suffers, as expected, of a higher lateral slip compared with other vehicles.
In the following figures, we report some details about the test performed with the Hyundai Azera in the \textit{Switzerland} Scenario. Figure \ref{fig:SwitzPass} shows how the vehicle behaves in paths with smooth corners, satisfying the lateral deviation requirement, while Figure \ref{fig:SwitzFail} refers to one of the most critical sections of the map, where it can be appreciated how the vehicle goes outside the lane in the apex of the hairpin turn.

    \begin{figure}[H]
    \centering
    \includegraphics[width=1\textwidth]{Figures/Switz_pass.png}
    \caption{Trajectory of the vehicle in some corners, where all requirements are satisfied}
      \label{fig:SwitzPass}
\end{figure}

    \begin{figure}[H]
    \centering
    \includegraphics[width=1\textwidth]{Figures/Switz_fail.png}
    \caption{Particular where trajectory of the vehicle is out of the lane boundaries, causing the ``fail" of the test}
      \label{fig:SwitzFail}
\end{figure}

Although these failed tests, we have decided to keep the same controller setup for the obstacle avoidance tests as well, excluding the \textit{Switzerland} Scenario from further simulations.


\documentclass[]{AVSSimReportMemo}
\usepackage{AVS}
\usepackage{colortbl}

\newcommand{\ModuleName}{velocityPoint}
\newcommand{\subject}{Guidance Module for Velocity Axis Pointing}
\newcommand{\status}{Initial Version}
\newcommand{\preparer}{M. Cols}
\newcommand{\summary}{Generate the attitude reference to perform a constant pointing towards a Velocity orbit axis}

\begin{document}

\makeCover

%
% enter the revision documentation here
% to add more lines, copy the table entry and the \hline, and paste after the current entry.
%
\pagestyle{empty}
{\renewcommand{\arraystretch}{2}
\noindent
\begin{longtable}{|p{0.5in}|p{4.5in}|p{1.14in}|}
\hline
{\bfseries Rev}: & {\bfseries Change Description} & {\bfseries By} \\
\hline
Draft & initial copy & M. Cols \\
\hline

\end{longtable}
}

\newpage
\setcounter{page}{1}
\pagestyle{fancy}

\tableofcontents
~\\ \hrule ~\\

\section{Module Input and Ouptut}

Table \ref{tab:inputNavTable} shows the input message from the navigation system.
\begin{table}[h!]
	\centering
	\caption{Input Navigation Message}
	\begin{tabular}{|l|l|l|p{3in}|}
		\hline
		\rowcolor{BrickRed}
		\textcolor{white}{Name} & \textcolor{white}{Type} &
		\textcolor{white}{Length} &
		\textcolor{white}{Description}  \\ \hline
		$\bm{R}_S$ & double [] & 3 &
		Position vector of the spacecraft body-point with respect to the inertial frame in inertial frame components
		($\leftexp{N}{\bm{r}}_{B/N}$) . \\ \hline
		$\bm{v}_S$ & double [] & 3 &
		Velocity vector of the spacecraft point with respect to the inertial frame in inertial frame components
		($\leftexp{N}{\bm{v}}_{B/N}$). \\ \hline
	\end{tabular}
	\label{tab:inputNavTable}
\end{table}

Table \ref{tab:inputCelTable} shows the input message from Spice about the main celestial body.
\begin{table}[h!]
	\centering
	\caption{Input Spice Planet Message}
	\begin{tabular}{|l|l|l|p{3in}|}
		\hline
		\rowcolor{BrickRed}
		\textcolor{white}{Name} & \textcolor{white}{Type} &
		\textcolor{white}{Length} &
		\textcolor{white}{Description}  \\ \hline
		$\bm{R}_P$  & double [] & 3 &
		Position vector of the main celestial object with respect to the inertial frame in inertial frame components . \\ \hline
		$\bm{v}_P$  & double [] & 3 &
		Velocity vector of the main celestial object with respect to the inertial frame in inertial frame components . \\ \hline
	\end{tabular}
	\label{tab:inputCelTable}
\end{table}

Table \ref{tab:outputTable} shows the Attitude Reference output message of the module Velocity Point.
\begin{table}[h!]
	\centering
	\caption{Output Attitude Reference Message}
	\begin{tabular}{|l|l|l|p{3in}|}
		\hline
		\rowcolor{BrickRed}
		\textcolor{white}{Name} & \textcolor{white}{Type} &
		\textcolor{white}{Length} &
		\textcolor{white}{Description}  \\ \hline
		$\bm{\sigma}_{R/N}$ & double [] & 3 &
		MRP attitude set of the reference frame with respect to the reference. \\ \hline
		$\leftexp{N} {\bm{\omega}_{R/N}}$ & double [] & 3 &
		Angular rate vector of the reference frame with respect to the inertial expressed in inertial frame components. \\ \hline
		$\leftexp{N} {\bm{\dot{\omega}}_{R/N}}$ & double [] & 3 &
		Angular acceleration vector of the reference frame with respect to the inertial expressed in inertial frame components. \\ \hline
	\end{tabular}
	\label{tab:outputTable}
\end{table}
\newpage

\section{Velocity Frame Definition}
\begin{figure}[htb]
  \centerline{ \includegraphics{Figures/Fig2} }
  \caption{Illustration of the planet orbit frames, Hill $\mathcal{H}$ and Velocity $\mathcal{V}$.}
  \label{fig:Fig1}
\end{figure}
The Velocity reference frame takes the spacecraft's orbital plane as the principal one and has origin in the center of the spacecraft. It is defined by the right-handed set of axes $\mathcal{V}:\{ \hat{\bm\imath}_{n}, \hat{\bm\imath}_{v}, \hat{\bm\imath}_{h} \}$, where\par
$\hat {\bm\imath}_{v}$  is tangent to the orbit and parallel to the velocity. \par
$\hat {\bm\imath}_{h}$ is defined normal to the orbital plane in the direction of the angular momentum. \par
$\hat {\bm\imath}_{n}$ completes the right-handed triode.\par

Figure~\ref{fig:Fig1} illustrates the relation between the Hill orbit frame $\mathcal{H}$ and the Velocity orbit frame.  $\mathcal{V}$. These two frames differ by a 3-axis rotation with the angle $-\beta$.  In terms of $\beta$, the DCM to map from $\mathcal{H}$ to $\mathcal{V}$ is given by
\begin{equation}
  \label{eq:VH1}
  [VH] = [M_{3}(-\beta)] = \begin{bmatrix}
    \cos\beta & -\sin\beta & 0 \\
    \sin\beta & \cos\beta &0 \\
    0 & 0 & 1
  \end{bmatrix}
\end{equation}
In terms of the classical set of orbital elements, this DCM can also be expressed as
\begin{equation}
  \label{eq:VH2}
  [VH] = \begin{bmatrix}
    \dfrac{1 + e \cos f}{\sqrt{1+e^{2}+2e \cos f}} &
    -\dfrac{e \sin f}{\sqrt{1+e^{2}+2e \cos f} }
    & 0 \\
    \dfrac{e \sin f}{\sqrt{1+e^{2}+2e \cos f}}  & \dfrac{1 + e \cos f}{\sqrt{1+e^{2}+2e \cos f}} & 0\\
    0 & 0 & 1
  \end{bmatrix}
\end{equation}

\section{Reference Frame Generation}
\begin{figure}[htb]
  \centerline{ \includegraphics{Figures/Fig1}}
  \caption{Illustration of the inertial frame $\mathcal{N}:\{ \hat{\bm n}_{1}, \hat{\bm n}_{2}, \hat{\bm n}_{3} \}$ and the orbit frames Hill $\mathcal{H}:\{ \hat{\bm\imath}_{r}, \hat{\bm\imath}_{\theta}, \hat{\bm\imath}_{h} \}$ and Velocity $\mathcal{V}:\{ \hat{\bm\imath}_{n}, \hat{\bm\imath}_{v}, \hat{\bm\imath}_{h} \}$ .}
  \label{fig:Fig2}
\end{figure}

In this module, the output reference frame $\mathcal{R}$ is to be aligned with the Velocity reference frame $\mathcal{V}$. Note that the presented technique does not require the planet-fixed  frame to coincide with the inertial frame $\mathcal{N}:\{ \hat{\bm n}_{1}, \hat{\bm n}_{2}, \hat{\bm n}_{3} \}$. Figure 1 illustrates the general situation in which $\bm{R}_{s}$ is the position vector of the spacecraft with respect to the inertial frame and $\bm{R_{p}}$ is the position vector of the  celestial body with respect to the inertial frame as well.

The relative position of the spacecraft with respect to the planet is obtained by simple subtraction:
\begin{equation}
  \label{eq:r}
  \bm r = \bm R_{s} -  \bm R_{p}
\end{equation}
The same methodology is applied to compute the relative velocity vector:
\begin{equation}
  \label{eq:v}
  \bm v = \bm v_{s} -  \bm v_{p}
\end{equation}
Note that the position and velocity vectors of the spacecraft and the celestial body, $\bm{R}_S$,  $\bm{R}_P$,  $\bm{v}_S$ and  $\bm{v}_P$ are the only inputs that this module requires. Having $\bm r$ and $\bm v$, the Velocity frame orientation is completely defined:
\begin{subequations}
  \begin{equation}
  \hat{\bm\imath}_{v} = \frac{\bm v}{v}
  \end{equation}
  \begin{equation}
  \hat{\bm\imath}_{h} = \frac{\bm{r}\times{\bm{v}}}{rv}
  \end{equation}
  \begin{equation}
  \hat{\bm\imath}_{n} = \hat{\bm\imath}_{v} \times \hat{\bm\imath}_{h}
  \end{equation}
\end{subequations}
And the Direction Cosine Matrix to map from the reference frame to the inertial is obtained:
\begin{equation}
	[RN] =  \begin{bmatrix}
       		\hat{\bm\imath}_{n} \\
		\hat{\bm\imath}_{v} \\
		\hat{\bm\imath}_{h}
      \end{bmatrix}
\end{equation}
The corresponding MRP attitude set is computed using the following function from the Rigid Body Kinematics library of Reference~\citenum{schaub}:
$$ [RN] = \textrm{C2MRP}(\bm\sigma_{R/N})$$
\section{Velocity Frame Rate Development}
Next the Velocity frame rate $\dot\beta$ relative to the orbit frame is determined.
Note the following identities from ~\eqref{eq:VH1} and ~\eqref{eq:VH2}:
\begin{equation}
  \label{eq:tanbeta}
  \tan\beta = \frac{e \sin f}{1 + e \cos f}
\end{equation}
\begin{equation}
  \label{eq:cos2beta}
  \cos^{2}\beta^{} = \frac{(1+e \cos f)^{2}}{1+e^{2}+2 e \cos f}
\end{equation}
Taking the derivative of Eq.~\eqref{eq:tanbeta} yields
\begin{equation}
  \label{eq:dbeta}
  \dot\beta =   \frac{ e (e+\cos f)}{1+e^{2}+2 e \cos f} \dot f
\end{equation}
To find the relative angular acceleration $\ddot\beta$, Eq.~\eqref{eq:dbeta} is differentiated again.
\begin{equation}
  \label{eq:ddbeta}
  \ddot\beta =
   \frac{e(e+\cos f)}{1+e^{2}+2 e \cos f} \ddot f
   +\frac{
  e  (e^{2}-1)\sin f
  }{
  (1+e^{2}+2 e \cos f)^{2}
  } \dot f^{2}
\end{equation}
The true anomaly rate and acceleration are determined through the standard astrodynamics relations:
\begin{align}
  \dot f &= \frac{h}{r^{2}}
  \\
  \ddot f &= - 2 \frac{\bm v \cdot \hat{\bm\imath}_{r}}{r} \dot f
\end{align}
Following, let us evaluate the Velocity frame angular vectors.  As both the $\mathcal{V}$ and $\mathcal{H}$ frame rotate about the common $\hat{\bm\imath}_{h}$ axis, note that
\begin{equation}
  \label{eq:omegaVN}
  \bm\omega_{V/N} = \bm\omega_{V/H} + \bm\omega_{H/N} = (-\dot\beta + \dot f) \hat{\bm\imath}_{h}
\end{equation}
\begin{equation}
  \label{eq:domegaVN}
  \dot{\bm\omega}_{V/N} = \dot{\bm\omega}_{V/H} + \dot{\bm\omega}_{H/N} = (-\ddot\beta + \ddot f) \hat{\bm\imath}_{h}
\end{equation}

\section{Angular Velocity Descriptions}
Let $\mathcal{R}_{0}$ reference the Velocity orbit frame. Thus, $\bm\omega_{R_{0}/N}$ and $\dot{\bm\omega}_{R_{0}/N}$ correspond to the expressions in ~\eqref{eq:omegaVN} and ~\eqref{eq:domegaVN} respectively.
The angular rate $\bm\omega_{R/N}$ and acceleration $\dot{\bm\omega}_{R/N}$ of the output reference frame $\mathcal{R}$  still need to be computed.
Since the desired attitude is a fixed-pointing one, $\mathcal{R}$ does not move relative to $\mathcal{R}_{0}$. Thus, the angular velocity of the reference frame happens to be:
\begin{equation}
  \label{eq:omega_R}
  \bm\omega_{R/N} = \bm\omega_{R/R_{0}} + \bm\omega_{R_{0}/N} = \bm\omega_{R_{0}/N} = \frac{1+e \cos f}{1+e^{2} + 2 e \cos f} \dot f \hat{\bm\imath}_{h}
\end{equation}
where $\dot\beta$ and $\dot f$ have been substituted by their corresponding expressions.\newline
Similarly, we find
\begin{equation}
  \label{eq:domega_R}
  \dot{\bm\omega}_{R/N} = \dot{\bm\omega}_{R/R_{0}} + \dot{\bm\omega}_{R_{0}/N} = \dot{\bm\omega}_{R_{0}/N} =
  \left( \frac{1+e\cos f}{1+e^{2}+2 e \cos f} \ddot f
  - \frac{
  e  (e^{2}-1)\sin f
  }{
  (1+e^{2}+2 e \cos f)^{2}
  } \dot f^{2} \right) \hat{\bm\imath}_{h}
\end{equation}
where $\ddot\beta$ and $\ddot f$ have been substituted by their corresponding expressions.

Both $\bm\omega_{R/N}$ and $\bm{\dot\omega}_{R/N} $ need to be expressed in the inertial frame $N$. \newline
Given
\begin{subequations}
\begin{equation}
\leftexp{R}{\bm \omega_{R/N} } =
      \begin{bmatrix}
       0\\ 0 \\ \dot{f}-\dot\beta
      \end{bmatrix}
\end{equation}
\begin{equation}
 \leftexp{R}{\bm{\dot{\omega}}_{R/N}  } =
      \begin{bmatrix}
       0\\ 0 \\ \ddot{f}-\ddot\beta
      \end{bmatrix}
\end{equation}
\end{subequations}
Then,
\begin{subequations}
\begin{equation}
	\leftexp{N} {\bm{\omega}_{R/N}} =  [NR] \textrm{ }\leftexp{R} {\bm{\omega}_{R/N} }
\end{equation}
\begin{equation}
	\leftexp{N} {\bm{\dot{\omega}}_{R/N} }=[NR]\textrm{ }\leftexp{R} {\bm{\dot{\omega}}_{R/N}}
\end{equation}
\end{subequations}
Where $ [NR] = [RN]^T$.

\bibliographystyle{unsrt}
\bibliography{references}

\end{document}