\section{Background}
\label{sec:background}
This section generally introduces blockchain and consensus protocols, gives an overview of Algorand, and provides the pertinent Coq background.

\subsection{Blockchains and Consensus Protocols}

A blockchain is an ordered sequence of cryptographically linked blocks of records, in much the same way as links in a chain are firmly latched in sequence. A blockchain provides a persistent, tamper-proof and globally accessible ledger of transactions. It is built using well-established and publicly known cryptographic tools, including most importantly one-way hash functions. Anyone can verify the validity of its transactions, and no one can tamper with a transaction or claim a transaction exists that does not appear in the chain.

This makes blockchains particularly well suited for building self-governing and autonomous distributed systems. Indeed, a blockchain is a key component for allowing a collection of nodes in a communication network (who do not necessarily trust each other) to work collectively and make decentralized decisions. Nevertheless, the nodes will need more than just a blockchain to achieve proper decentralized operation. Specifically, they need a mechanism for identifying and agreeing on the next block of transactions to be appended to the chain, a mechanism known as a consensus protocol.

A consensus protocol typically proceeds in rounds, where the objective of a round is to produce the next block and record it in the chain. As such, a consensus protocol includes both:
(1) a mechanism for decentralized selection of one or more block proposers or producers of a round in the protocol, and
(2) a mechanism for achieving decentralized consensus on a single block to be appended to the chain.

In an idealistic (and rather unrealistic) setup, in which communication networks are perfectly reliable with zero-delay message delivery, and in which all participating nodes run error-free code and behave honestly, achieving consensus on a single block in each round is a trivial task. All nodes would instantaneously see all proposed blocks and their trusted opinions about them. In reality, however, the situation is very different. Distributed systems utilize global Internet (IP-based) communication networks, which are inherently unreliable and where significant message transmission delays are not uncommon. Furthermore, nodes may deviate from the protocol, either intentionally (when compromised by malicious users) or unintentionally (due to internal errors). These complications can hinder the consensus process, resulting potentially in the system being unusable or in losing large amounts of assets maintained by the system.

Therefore, consensus protocols need to maintain a consistent global view of the system while relying only on the local knowledge at the level of its individual (honest) nodes. Moreover, these protocols must ensure continuous and fair operation of the system in the presence of both benign node or network failures and maliciously behaving nodes. Consequently, consensus protocols are inherently complex, consisting of multiple asynchronous steps and utilizing a wealth of cryptographic and randomization schemes, and possibly some economic incentive structures, whose goal is to reward compliant behaviors and penalize deviations from the protocol.

A class of consensus protocols, pioneered by Bitcoin~\cite{Nakamoto2008} and referred to as Proof-of-Work (PoW), achieves consensus primarily through a process called mining, in which all participating nodes in the network compete on solving complex cryptographic puzzles to produce and record blocks. Although it provided workable solutions to the decentralized consensus challenge, PoW is now widely known to suffer from efficiency, scalability, and security problems. The mining process is inherently computationally expensive, slow, and wasteful, resulting in wasting significant amounts of energy while imposing high transaction fees. Furthermore, PoW tends in practice to result in centralized systems, in which the bulk of the mining power (and hence the power of deciding the fate of the blockchain) falls in the hands of a small subset of users: those who can afford to invest in very powerful computing resources, or who can simply join forces in mining pools. Centralization defeats the purpose of the protocol and poses a major security concern: those who control the majority of the mining power need to be trusted, and even if they are, they make the system a much easier target for attacks.

An alternative mechanism that promises to alleviate these problems is Proof-of-Stake (PoS), in which the burden of producing blocks and selecting the next block to augment the blockchain is placed on a suitably selected subset of nodes (typically much smaller than the entire set of participating nodes), called a committee. In some variations of PoS, called delegated PoS, the committee is trusted and is usually fixed. In other variations, generally referred to as bonded PoS, participation in a committee requires staking some of the blockchain's underlying cryptographic assets (such as cryptocurrency or tokens), or locking up these assets for an extended period of time so that they cannot be expended or moved. The stake of a committee member (relative to the total stake in the system) determines the member's voting power when deciding the next block for a given round in the protocol. Furthermore, a node's stake can also be used as collateral to be (fully or partially) reclaimed by the system if a node is found to misbehave.

However, many important questions arise when designing a PoS protocol. How is the committee selected? One would want to have a selection process that is fair and representative, and hard to manipulate by a malicious user. Moreover, a committee that is fixed for a prolonged period of time can be easily attacked and presents a potential single point-of-failure for the system. But how often should the committee be changed? What is a suitable size of the committee in relation to the total population of nodes in the network? How should the voting process be designed? These and other design choices can significantly affect the efficiency, scalability and security of a PoS blockchain system.

\subsection{Overview of Algorand}

We highlight below some of Algorand's unique features. More detailed descriptions of these features and others are available elsewhere~\cite{Chen2018,Chen2019}.

Algorand almost never forks. Forking happens when consensus on a single block for a round is not reached and multiple different blocks are added for that round. This is a notorious problem in PoW, but is also possible in PoS systems. Having different subsets of nodes decide on appending different blocks to the chain means that transactions appearing in these conflicting blocks are not finalized since only one of these blocks will eventually belong to the canonical chain, the chain that is deemed most accepted by the nodes (there are different methods for deciding the canonical chain, e.g. the longest chain in Bitcoin). Algorand avoids this problem by design: at most one block can receive the majority of votes in a round.

Algorand is very efficient. In Bitcoin and other PoW protocols, the rate at which blocks are produced (and hence the rate at which transactions are processed in the blockchain) is determined by the complexity of the cryptographic puzzle to be solved by the miners: the more complex the problem is, the longer it takes to produce a block. Although simplifying these puzzles would mean faster block production, the effective transaction processing rate would likely be adversely impacted as the forking rate will also increase. More miners will now compete to have their blocks in the chain and many more transactions will now belong to chains that end up being abandoned (they will have to be re-processed and attempted again). Being a PoS system, Algorand does not have this problem, and is in fact designed to produce a block every second. This high block production rate when combined with the fact that the chain forks only with negligible probability means that transactions appear very quickly in the chain, and once they do, they can immediately be considered final. This has the potential of allowing extremely high levels of scalability.

Algorand provides security against two types of attacks.
First, attacks on the protocol, which involve corrupting participating nodes so that they no longer follow the protocol, e.g., by producing blocks with fake transactions or by casting votes for the wrong block or casting multiple conflicting votes.
Second, attacks on the underlying network on which the system is deployed, for instance by intercepting, manipulating and delaying messages. When an attacker gains full control of message delivery in a network, the network is said to be partitioned. Most existing consensus protocols do not consider what happens when a network is partitioned or after the network recovers from a partition.
An attacker's goal is generally to manipulate the chain to reverse transactions or rewrite history (e.g. double-spend attacks), or to attempt to break consensus, leaving the system in an inconsistent state, or even cause the system to halt, where the chain can no longer grow.

The key to Algorand's security in its pure PoS protocol is maintaining decentralization at every step of the protocol. This is generally achieved through several techniques relating to how a committee, which is a group of nodes selected to make a decision on behalf of all nodes in the network, is managed. We highlight below key aspects of committee management.

\paragraph{Committee membership.} Membership in a committee is decided through a unique process called \textit{cryptographic self-selection}, which is run individually by each node. Essentially, a node locally runs a cryptographically fair and irrefutable lottery whose outcome (which can be verified by everyone else) decides membership in the committee.

\paragraph{Committee formation.} Committees are not fixed and change very frequently throughout execution of the protocol. In fact, every step of execution may have its own committee. Furthermore, committees can be of different sizes depending on the type of task the committee is entrusted to perform. For instance, the committee responsible for producing a block is typically much smaller than the committee that votes on blocks.

\paragraph{Attackers and committees.} Cryptographic self-selection is performed secretively by each node in isolation of all other nodes. Only the node knows whether it is part of the current-step committee, until it announces its membership with its vote or block to the network. An attacker has no way of knowing beforehand whether a node belongs to a committee. Once the node announces its membership, it will already be too late for an attacker to corrupt the node and send a different message. The node's original messages are already out and being propagated through the network (note also these messages are digitally signed, so they cannot be tampered with without the receiver knowing).

\paragraph{Ephemeral keys.} Another distinguishing feature of Algorand is the use of ephemeral keys, which are temporary, single-use encryption-decryption keys, for signing vote and block messages. Once a node uses an ephemeral key to sign a message, the node immediately destroys it, so that if the node gets compromised later on by an attacker, the attacker won't be able to claim that the node sent a different message for a previous step in the protocol. This effectively prevents an attacker from going back in history and forging different messages from what the nodes originally sent.\\

\noindent By performing cryptographic self-selection at every step of the protocol and using ephemeral keys for signing block and vote messages, Algorand decouples the execution of a protocol step from the identities of nodes participating in that step, a property referred to as \emph{player replaceability}. Indeed, the execution of a step does not depend on the state of a node, and a node is never obliged to participate in a minimum number of steps in the protocol.

The Algorand consensus protocol proceeds in rounds and any node can participate in the protocol (i.e. Algorand is permissionless). A new block is generated at each round. A round is divided into periods, and a period is further divided into steps. A potential block is proposed in each period of a round, and the round ultimately ends when the block is finalized.

The four kinds of steps in each period are the proposing step, filtering step, certifying step, and finally the finishing step(s). Present in each of these steps is Algorand's cryptographic self-selection process highlighted above. That is, a verifiable random function is used to privately and securely select committee members at each of these steps. The random, verifiable, self-selecting nature of this process is key to the correctness of the Algorand protocol.

\paragraph{Proposing step.} Committee members in this first step (chosen by cryptographic self-selection) are known as potential leaders. These members propose a block and propagate it to other members of the protocol.

\paragraph{Filtering step.} Here, committee members identify their leader, also using cryptographic self-selection. Committee members evaluate their current state, and potentially soft-vote for the proposed block.

\paragraph{Certifying step.} In the certifying step, committee members simply evaluate if there are enough soft-votes for the potential block. If so, each committee member submits a cert-vote for that block. If there are enough certvotes for a block, the block is ultimately approved or certified, and users move to the next round.

\paragraph{Finishing steps.} The last stage of a period, the finishing steps allow users to move to a new period without certifying a block. Here, committee members evaluate the proposed block, as well as votes received from other members, including votes in the last period. This allows users to potentially propagate a next-vote message. Sufficient next-votes result in the period increasing without a block being certified.

A node may proceed to the next period of a round when it observes a quorum of next-votes for a block (or for a special bottom value indicating that no potential block has enough votes yet). Once a node observes that a block gets a certificate, which is a large-enough quorum of cert-votes, the node certifies the block and moves on to the next round.

%For a detailed and more precise description of these steps, the reader is referred to Algorand's technical reports[ADD LINKS].

\subsection{Deductive Verification and Proof Assistants}

One emerging approach to the challenge of designing and implementing correct and robust consensus protocols is to apply formal methods during design and development. More specifically, in this approach, engineers formally specify both a protocol's design and the properties that the design is required to meet. Then, they mathematically verify that the design meets the requirements, or, if it does not, determine how the design may violate the requirements. In the latter case, they can use their findings to revise the design. When a design is successfully verified, engineers also precisely pin down the assumptions under which the design satisfies its requirements, which are important to consider when deploying an implementation of a design and for systems built on top of an implementation.

The benefits of applying formal methods early in the design process of complex systems are well documented~\cite{Yang2011,Fonseca2017}. Most directly, formal methods can uncover fundamental errors that would otherwise go undetected, and which are costly to correct in a later phase. Moreover, obtaining formal guarantees about a protocol's design significantly increases the confidence in not just the implementation of the protocol, but also in the systems built on top of it, and ultimately increases trust in whole platforms among developers and users, facilitating wider adoption and support.

Researchers have developed a large spectrum of techniques and tools for applying formal methods to model, specify, and verify general software systems and distributed protocols. The techniques vary across dimensions such as expressive power, automation, and tool support. Perhaps most importantly, they differ in the mathematical guarantees they provide and therefore in trustworthiness. Generally, to obtain stronger guarantees, more effort is required by formal verification engineers during both modeling and verification, and less extensive automation is available.

%(see the chart in Figure~\ref{fig:verification}).
%\begin{figure}
%  \includegraphics[width=\linewidth]{assets/verification.png}
%  \caption{Different verification methods and how they generally compare with respect to the confidence-effort trade off.}
%  \label{fig:verification}
%\end{figure}

The strongest guarantees, and thus the highest degree of trustworthiness, are obtained through deductive verification and theorem proving. In deductive verification, systems are modeled and specified inside expressive formal logical systems, and verified in a similar way to how mathematicians prove theorems - in principle by elaborating proofs of statements step-by-step. Among tools for deductive verification, \emph{proof assistants}~\cite{Geuvers2009,Harrison2014} offer the smallest trusted computing base necessary to trust verified statements, realized in modestly-sized trusted checkers that only assume a small set of axioms accepted by almost all mathematicians. Although they provide such strong guarantees, proof assistants can offer extensive automation and user support. Nevertheless, the process of figuring out what properties are relevant, how these properties are specified and building up their proofs normally requires a great deal of human ingenuity and experience. However, once everything is specified, proofs developed in proof assistants can be machine-checked, and persisted to serve as independently verifiable evidence that the properties hold for the given system~\cite{Pollack1998}.

To model and formally verify the Algorand consensus protocol, we use deductive verification and theorem proving, as facilitated by the Coq proof assistant~\cite{CoqArt}.

\subsection{Coq and the Mathematical Components Library}

Coq is a proof assistant based on higher-order type theory, and can be viewed as consisting of, on the one hand, a small and powerful purely functional programming language, and on the other hand, a system for specifying and proving properties. A Coq user writes functions and data and then interactively constructs the proof of a theorem by trying different proof tactics that transform the state of the in-progress proof. Coq only accepts the theorem after its checker has been run on the purported proof. The Mathematical Components (MathComp) project~\cite{MathComp} provides a set of Coq libraries with standard mathematical data structures such as sequences, finite sets, and multisets~\cite{Cohen2019}, packaged to facilitate extensions and formal proofs~\cite{Garillot2009}. Proofs in MathComp are written in the SSReflect proof language~\cite{Gonthier2010}, which extends Coq's proof tactic language.

%Coq has previously been used to develop a verified C compiler, formally prove mathematical results such as the four-color theorem, and verify distributed protocols (see, for example, our previous Casper formal verification effort for the Ethereum Foundation).

%We employ several existing Coq libraries which already formalized
%the majority of the mathematics we need to define and reason about Casper.
%Mathematical Components~\cite{MathComp} is a Coq library based on packaging
%mathematical structures and results in the form of Coq \emph{canonical
%  structures}, which can be reused and specialized when
%required~\cite{Garillot2009}.
%The library was used by Gonthier~et~al.\ to capture finite group theory and
%prove fundamental results in abstract algebra~\cite{Gonthier2013}.
%In addition to structures from abstract algebra, the library also contains
%encodings of and results about many standard data structures, such as numbers,
%lists, and finite sets.


From our trained random forest regression model, we get a glimpse of both what
useful, contributing redditors look like, and what bad, non-contributing
redditors look like.

Our test data set was about 3,200 redditors, all of their post data (around
1GB), and all of their comment data (around 3GB). From this test data set, we
used our our regressive model to get a reliability score between $-1 \leq s_r
\leq 1$. Then, we re-correlate this score with input features to intuitively see
what features are important or not, and what features indicated useful and
not-useful redditors.

\subsection{Distribution of Users} % (fold)
\label{sub:distribution_of_users}

In general, we around 24\% of users to have a negative score, and the remaining
76\% to have a positive score. This falls in line with a YouGov study that
revealed about a quarter of Americans admit to acting as `trolls', or other
sorts of argumentative, non-constructive behavior online \cite{trollstats}. A
more detailed distribution of scores is in \Cref{fig:data_20}. Most of the
scores assigned were closer to $1$, with a relatively uniform distribution of
other scores.

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/data_20.png}
    \caption{The distribution of the reliability score $s_r$ of the sampled redditors, binned into twenty bins.}
    \label{fig:data_20}
\end{figure}

% subsection distribution_of_users (end)

\subsection{Feature Efficacy} % (fold)
\label{sub:feature_efficacy}

Though we gathered several different features, not all of them ended up being
`useful' in terms of creating a reliability score. The full table of the
features that we used, as well as the percentage importance our model placed on
them, appears in \Cref{tab:features}. It's worth noting that some featured are
highly correlated, like karma from top 100 subreddits and total karma, so even
though these features might be correlated with reliability, the model did not
place importance on them.

\subsubsection{What Did Work} % (fold)
\label{ssub:what_did_work}

The most important features unsurprisingly turned out to be the amount of post
and comment karma a user had. As this is the built in voting system of
\reddit{}, we expected to see this. Plots of how post and comment karma vary
with reliability appear in
\Cref{fig:reliability_post_karma,fig:reliability_comment_karma}, respectively.
Both of these plots suggest that the voting system works to root out unreliable
users; users with $s_r < 1$ don't get much karma. Of users with $s_r \geq 1$,
the distribution is bimodal. We hypothesize that our model manages to
differentiate between users who are casual, and post well received but non-
useful content, and those who post useful, reliable content which is also well
received by \reddit{} on the whole. Similarly, the voting system seems to
discourage participation from unreliable users.
\Cref{fig:reliability_post_count,fig:reliability_comment_count} suggest that
users who are unreliable do not end up posting much, but those who are more
reliable are more likely to post far more content.

A more surprising feature was the importance of a user verifying their email. A
split histogram of $s_r$ of users, split on if they have verified their email or
not (\Cref{fig:data_20_email}), suggests that though the model only placed
0.65\% importance on this feature, having an unverified email means a user is
more likely to not be reliable. This stands to reason; people who act badly on
the Internet often do so only when their anonymity is guaranteed
\cite{cho1999privacy}, as it is on \reddit{}, so they'd be less likely to link a
non-anonymous account like email to their \reddit{} account.

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/reliability_post_karma.png}
    \caption{The reliability score $s_r$ plotted against the post Karma the redditor has.}
    \label{fig:reliability_post_karma}
\end{figure}

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/reliability_comment_karma.png}
    \caption{The reliability score $s_r$ plotted against the comment Karma the redditor has.}
    \label{fig:reliability_comment_karma}
\end{figure}

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/reliability_post_count.png}
    \caption{The reliability score $s_r$ plotted against the number of posts the redditor has made.}
    \label{fig:reliability_post_count}
\end{figure}

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/reliability_comment_count.png}
    \caption{The reliability score $s_r$ plotted against the number of comments the redditor has made.}
    \label{fig:reliability_comment_count}
\end{figure}

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/data_20_email.png}
    \caption{The distribution of the reliability score $s_r$ of the sampled redditors, binned into twenty bins, separated by if they have a verified email address or not.}
    \label{fig:data_20_email}
\end{figure}

% subsubsection what_did_work (end)

\ \newline % Layout based newline.
\subsubsection{What Didn't Work} % (fold)
\label{ssub:what_didn_t_work}

Though we had hoped that features of the words that redditors used would
indicate how reliable they were, we didn't find this to be the case. An
examination of \Cref{fig:reliability_readability} suggest that there is not
relationship between reliability and Flesch--Kincaid readability of comments.
Our model placed similarly low importance on the frequency of using bad words
and readability of comments. This suggests to us that many kinds of speech
patterns are accepted on \reddit{}, and has little bearing on reliability. Our
model also placed low importance on the amount of karma earned in subsets of
subreddits.

We also thought that gilding would be an important feature, but it turned out
not to be, as seen in \Cref{fig:reliability_gilded}. Not enough users get gilded
on \reddit{}, which makes sense, as putting real money into the service is a
much higher friction action than simply using it for free.

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/reliability_readability.png}
    \caption{The reliability score $s_r$ plotted against the Flesch--Kincaid readability of their comments.}
    \label{fig:reliability_readability}
\end{figure}

\begin{figure}[tb]
    \centering
    \includegraphics[width=\linewidth]{../src/do_regression/figs/reliability_gilded.png}
    \caption{The reliability score $s_r$ plotted against the percentage of gilded posts.}
    \label{fig:reliability_gilded}
\end{figure}

% subsubsection what_didn_t_work (end)

% subsection feature_efficacy (end)


\section{Results} % (fold)
\label{sec:results}
% - It would be nice to compare the performance of your reinforcement learning (RL) or Markov decision process (MDP) model to some naive baseline policies based on heuristic rules. These rules are the one you would adopt when playing yourself the game (or controlling the system).
% Then the question is "Does RL or MDP perform better than the baseline strategy ?" Experimental simulations/comparisons should be carried out in order to answer this question.

In this section, we wish to validate our model through its application on a real case problem and answer our research question detailed in section~\ref{sec:objective}.

Before jumping into the experiments themselves, let us describe the dataset we used.
We use data from a real distribution network with real generation and load profiles.

Our dataset is a slightly simplified version of the data used in Niels \textsc{Leemput} PhD thesis~\cite{Niels} which is a real urban feeder topology that was provided for the EIT-KIC InnoEnergy EVCity project~\cite{evcity} to which the second author has access for his master thesis~\cite{Quentin}.

The system is 5 nodes distribution network made of 3 households, 1 small factory considered as a flexible load and 1 transformer node of maximum power rating of $250\si{\kilo\volt}$ linking the distribution network to the medium voltage network.
All households are assumed to be connected to the same phase, with a rated neutral-to-phase voltage of $230\si{\volt}$.
In what concerns the distances, the distance between the substation node and the first connection node is $350\si{\meter}$.
Then, the distance between two successive nodes is $7.2\si{\meter}$.
A schema of the distribution network can be found in figure~\ref{fig:schema_line}.
\input{line_tikz}
The characteristics of the lines are summarized in table~\ref{tab:char_lines}.
\begin{table}[H]
  \caption{Characteristics of the line.}
  \label{tab:char_lines}
  \centering
  \begin{tabular}{c|c}
  \hline
   Impedance [$\si{\ohm\per\kilo\meter}$] & Current limit [$\si{\ampere}$] \\
   \hline
   $0.31 + \mathbf{i}0.0713$ & $210$ \\
   \hline
  \end{tabular}
\end{table}

We use real single-phase household electric power consumption profiles
that were sampled in 2008, with a 1 hour time resolution.
We also have real PV power generation profiles for all these households that are based upon measurements at an installation of the KU Leuven, also with a 1 hour resolution.
The horizon used is one week and the profiles were acquired in May.

Before comparing the optimization and the RL algorithm, we present a typical curtailment profile of the RL method on figure \ref{fig:power_cut}.

\begin{figure}
  \begin{center}
    \includegraphics[scale=0.5]{./img/power_cut.png}
  \end{center}
  \caption{Power cut of house 4 following power forecast.}
  \label{fig:power_cut}
\end{figure}

This figure shows that at some key time steps, curtailment is indeed needed to ensure the reliability of the network.

Let us now present our results for the comparison between reinforcement learning based on the Markov chain model and multi-time-steps deterministic optimization.

\subsection{Assumptions}

The main assumptions of the RL decision framework are the following :
\begin{itemize}
\item We suppose that the probability distribution for load and production curtailment is discrete.
\item We suppose that the curtailment actions are discrete.
\item The flexible load is either on or off, but we can't activate the flexibility of part of the load.
\item We have to pay a large cost when voltage and current bounds are violated.
\end{itemize}

The assumptions of the optimization part are
\begin{itemize}
\item The problem is deterministic.
\item We don't have flexible loads.
\item Curtailment is allowed to be continuous.
\item Voltage and current bounds are added as constraints so they can't be violated.
\item We minimize the total cost of curtailment.
\end{itemize}

Note that it would have been possible to relax the deterministic assumption of the optimization problem by using stochastic programming.
However, stochastic programming is much less computationally efficient.
We could also have considered flexible loads, but the problem would have become mixed-intger.
This again, is cumbersome and would have decreased greatly the computational efficiency of the problem.

Therefore, we can say that the scope of the RL framework is larger because it takes naturally uncertainty into account but the assumption that curtailment is discrete is quite strict for real case applications.

\subsection{Computational complexity}
Figure~\ref{fig:time} represents the time taken by the two methods.

\begin{figure}
\centering
\includegraphics[scale=0.5]{img/time}
\caption{Comparion of the total time of the two methods.}
\label{fig:time}
\end{figure}

As we can see on the figure, the RL method takes more than 10 times the time of the optimization method.
However, note that the assumptions of the multi-time-steps optimization program are quite strict.
If we would have added the flexible load and the uncertainty, the modelisation as an optimization program would have been more complicated and the time would have increased.

\subsection{Optimal decision}
The comparison of the two methods in terms of the total curtailment on the network is shown in figure \ref{fig:curt_comp}.

\begin{figure}
\centering
\includegraphics[scale=0.5]{img/curt_comp}
\caption{Comparion of the total curtailment in the two methods.}
\label{fig:curt_comp}
\end{figure}

Interestingly enough, both curtailment are very similar.
This suggests that the discretization of the curtailment actions for RL was actually a reasonable assumption, which is a good argument in favor of this method for the Active Network Management of Distribution Network.

% section results (end)


\newcommand{\GCNetname}{Guided Calibration Network\xspace}
\newcommand{\gcnetname}{guided calibration network\xspace}
\newcommand{\gcnetacronymnospace}{GCNet}
\newcommand{\gcnetacronym}{\gcnetacronymnospace\xspace}
\newcommand{\gbr}{GBR\xspace}
\newcommand{\LCNetreg}{LCNet$_\text{reg}$\xspace}

\chapter{Learning Lighting Calibration for Photometric Stereo}
\label{ch:lcnet}

\section{Introduction}
Photometric stereo aims at recovering the surface normal of a static object from a set of images captured under different light directions~\cite{woodham1980ps,silver1980determining}. \emph{Calibrated} photometric stereo methods assume known light directions, and promising results have been reported~\cite{shi2019benchmark} at the cost of tedious light source calibration. The problem of \emph{uncalibrated} photometric stereo, where light directions are unknown, still remains an open challenge, and its stable solution is wanted because of the ease of setting. In this chapter, we study the problem of uncalibrated photometric stereo for surfaces with general and unknown isotropic reflectance.

Most of the existing methods for uncalibrated photometric stereo~\cite{alldrin2007r,shi2010self,papad14closed} assume a simplified reflectance model, such as the Lambertian model, and focus on resolving the shape-light ambiguity, such as the Generalized Bas-Relief (GBR) ambiguity~\cite{belhumeur1999bas}. Although methods of~\cite{lu2013uncalibrated,lu2015uncalibrated} can handle surfaces with general bidirectional reflectance distribution functions (BRDFs), they rely on a uniform distribution of light directions for deriving a solution.

Recently, with the great success of deep learning in various computer vision tasks, deep learning based methods have been introduced to calibrated photometric stereo~\cite{santo2017deep,Taniai18,ikehata2018cnn,chen2018ps}. Instead of explicitly modeling complex surface reflectances, they directly learn the mapping from reflectance observations to surface normals given light directions. Although they have produced promising results in a calibrated setting, they cannot handle the more challenging problem of \emph{uncalibrated} photometric stereo, where light directions and intensities are unknown. One simple strategy to handle uncalibrated photometric stereo with deep learning is to directly learn the mapping from images to surface normals without taking the light directions as input.
However, as reported in~\Sref{sub:upsfcn}, the performance of such a model lags far behind those which take both images and light directions as input.

Instead of directly predicting surface normals from images, we propose to first estimate light directions and intensities from images. The problem of uncalibrated photometric stereo can then be reduced to a calibrated one, which can be effectively solved by existing calibrated methods~\cite{shi2014bi,chen2018ps,ikehata2018cnn}.
The rationales behind this two-stage approach are as follows. First, lighting information is very important for normal estimation since lighting is the source of various cues, such as shading and reflectance, and estimating the light directions ($3$-vectors) and intensities (scalars) is in principle much easier than directly estimating the normal map (a $3$-vector at each pixel location) together with the lighting conditions.
Second, by explicitly learning to estimate light directions and intensities, the model can take advantage of the intermediate supervision by the ground-truth lighting, resulting in a more interpretable behavior.

This work focuses on learning lighting calibration for uncalibrated photometric stereo. The contributions of this work can be summarized as follows:
\begin{itemize}
    \item We introduce a lighting calibration network, named LCNet, for estimating light directions and intensities from images.
    \item We discuss the differences between LCNet and traditional uncalibrated methods, and analyze the features learned by LCNet to resolve the \gbr ambiguity.
    \item We find that attached shadows, shadings, and specular highlights are key elements for lighting estimation, and that LCNet extracts features independently from each input image without exploiting any inter-image information (``inter-image'' means information shared by all images).
    \item Based on our findings, we propose a \gcnetname (\gcnetacronym) that explicitly utilizes object shape and shading information as guidances for better lighting estimation.
\end{itemize}

Preliminary results of this research have been published in~\cite{chen2019self,chen2020chen_gcnet}.
Our code and models can be found at \url{https://guanyingc.github.io/SDPS-Net}.

\section{Related Work}
\label{sec:lcnet_relatedwork}
In this section, we review uncalibrated photometric stereo methods and the loosely related work on learning based lighting estimation. Review for recent deep calibrated photometric stereo methods can be found in~\Sref{sec:psfcn_related_work}.

\paragraph{Uncalibrated photometric stereo}
When lighting is unknown, the surface normals of a Lambertian object can only be estimated up to a $3\times 3$ linear ambiguity~\cite{hayakawa1994photometric}, which can be reduced to a $3$-parameter GBR ambiguity~\cite{belhumeur1999bas,yuille1999determining} using the surface integrability constraint. Previous work used additional clues like albedo priors~\cite{alldrin2007r,shi2010self}, inter-reflections~\cite{chandraker2005reflections}, specular spikes~\cite{drbohlav2005can}, Torrance and Sparrow reflectance model~\cite{georghiades2003incorporating}, reflectance symmetry~\cite{tan2007isotropy,wu2013calib}, multi-view images~\cite{esteban2008multiview}, and local diffuse maxima~\cite{papad14closed}, to resolve the GBR ambiguity. Cho~\etal~\cite{cho2016photometric} considered a semi-calibrated case where the light directions are known but not their intensities. There are few works that can handle non-Lambertian surfaces under unknown lighting. Hertzmann and Seitz~\cite{hertzmann2005example} proposed an exemplar based method by inserting an additional reference object to the scene. Methods based on cues like similarity in radiance changes~\cite{sato2007shape,lu2013uncalibrated} and attached shadow~\cite{okabe2009attached} were also introduced, but they require the light sources to be uniformly distributed on the entire viewing sphere. Recently, Lu~\etal~\cite{lu2018symps} introduced a method based on the ``constrained half-vector symmetry'' to work with non-uniform lightings. Different from these traditional methods, our method can deal with surfaces with general and unknown isotropic reflectance without the need of explicitly utilizing any additional clues or reference objects, solving a complex optimization problem at test time, or making assumptions on the light source distribution.

Other methods related to uncalibrated photometric stereo include exemplar-based methods~\cite{hertzmann2005example}, regression-based methods~\cite{midorikawa2016uncalibrated}, semi-calibrated photometric stereo~\cite{cho2018semi}, inaccurate lighting refinement~\cite{queau2017non}, and photometric stereo under general lighting~\cite{basri2007photometric,mo2018uncalibrated,haefner2019variational}.

\paragraph{Learning based lighting estimation}
Recently, learning based single-image lighting estimation methods have attracted considerable attention. Gardner~\etal~\cite{gardner2017learning} introduced a CNN for estimating HDR environment lighting from an indoor scene image. Hold-Goeffroy~\etal~\cite{hold2017deep} learned outdoor lighting using a physically-based sky model. Weber~\etal~\cite{weber2018learning} estimated indoor environment lighting from an image of an object with known shape. Zhou~\etal~\cite{Zhou_2018_CVPR} estimated lighting, in the form of Spherical Harmonics, from a human face image by assuming a Lambertian reflectance model. Different from the above methods, our method can estimate accurate directional lightings from multiple images of a static object with general shape and non-Lambertian surface.

\section{Lighting Calibration Network (LCNet)}
\label{sec:lcnet}
In the rest of this chapter, we refer to light direction and intensity as ``lighting''.
To estimate lighting from the images, an intuitive approach would be directly regressing the light direction vectors and intensity values.
However, we propose that formulating the lighting estimation as a classification problem is a superior choice, as will be verified by our experiments. Our arguments are as follows. First, classifying a light direction into a certain range is easier than regressing the exact value(s), and this will reduce the learning difficulty.
Second, when training a normal estimation network, taking discretized light directions as input will allow it to better tolerate small errors in the estimated light directions.

\begin{figure}[htbp] \centering
    \input{ch-lcnet/figures/spherical_coord2}
    \raisebox{0.05\height}{\includegraphics[width=0.28\textwidth]{ch-lcnet/images/Method/hemisphere.pdf}} \\ \vspace{-0.9em}
    \makebox[0.28\textwidth]{\small (a)}
    \makebox[0.28\textwidth]{\small (b)} \\
    \caption[Illustration of an example discretization of the lighting space]{(a) Illustration of the coordinate system ($z$ axis is the viewing direction). $\phi \in [0\degree, 180\degree]$ and $\theta \in [-90\degree, 90\degree]$ are the azimuth and elevation of the light direction, respectively. (b) Example discretization of the light direction space when $K_d=18$.} \label{fig:coord}
\end{figure}

\subsection{Discretization of Lighting Space} Since we cast our lighting estimation as a classification problem, we need to discretize the continuous lighting space. Note that a light direction in the upper-hemisphere can be described by its azimuth $\phi \in [0\degree, 180\degree]$ and elevation $\theta \in [-90\degree, 90\degree]$ (see \fref{fig:coord}~(a)). We can discretize the light direction space by evenly dividing both the azimuth and elevation into $K_d$ bins, resulting in $K_d^2$ classes (see \fref{fig:coord}~(b)). Solving a $K_d^2$-class classification problem is not computationally efficient, as the softmax probability vector will have a very high dimension even when $K_d$ is not large (\eg, $K_d^2=1,296$ when $K_d=36$).
Instead, we estimate the azimuth and elevation of a light direction separately, leading to two $K_d$-class classification problems. Similarly, we evenly divide the range of possible light intensities into $K_e$ classes (\eg, $K_e=20$ for a possible light intensity range of $[0.2, 2.0]$).

\begin{figure}[htbp] \centering
	\includegraphics[width=1\textwidth]{ch-gcnet/images/Method/LCNet.pdf}
    \caption[Network architecture of LCNet]{Network architecture of LCNet. Each layer's value indicates its output channel number.} \label{fig:LCNet}
\end{figure}

\subsection{Local-global Feature Fusion}
A straightforward approach to estimate the lighting for each image is simply taking a single image as input, encoding it into a feature map using a CNN, and feeding the feature map to a lighting prediction layer. It is not surprising that the result of such a simple solution is far from satisfactory. Note that the appearance of an object is determined by its surface geometry, reflectance model and the lighting. The feature map extracted from a single observation obviously does not provide sufficient information for resolving the shape-light ambiguity. Thanks to the nature of photometric stereo where multiple observations of an object are considered, we propose a local-global feature fusion strategy to extract more comprehensive information from multiple observations.

Specifically, we separately feed each image into a shared-weight feature extractor to extract a feature map, which we call \emph{local feature} as it only provides information from a single observation. All local features of the input images are then aggregated into a \emph{global feature} through a max-pooling operation, which has been proven to be efficient and robust on aggregating salient features from a varying number of unordered inputs~\cite{wiles2017silnet,chen2018ps}. Such a global feature is expected to convey implicit surface geometry and reflectance information of the object which help resolve the ambiguity in lighting estimation. Each local feature is concatenated with the global feature, and fed to a shared-weight lighting estimation sub-network to predict the lighting for each individual image. By taking both local and global features into account, our model can produce much more reliable results than using the local features alone. We empirically found that additionally including the object mask as input can effectively improve the performance of lighting estimation, as will be seen in the experiment section.

\subsection{Network Architecture}
LCNet is a multi-input-multi-output (MIMO) network that consists of a shared-weight \emph{feature extractor}, an \emph{aggregation layer} (\ie, max-pooling layer), and a shared-weight \emph{lighting estimation sub-network} (see \fref{fig:LCNet}).
It takes the observations of the object together with the object mask as input, and outputs the light directions and intensities in the form of softmax probability vectors of dimension $K_d$ (azimuth), $K_d$ (elevation) and $K_e$ (intensity), respectively.
We convert the output of LCNet to $3$-vector light directions and scalar intensity values by simply taking the middle value of the range with the highest probability. We have experimentally verified that alternative ways like taking the expectation of the probability vector or performing quadratic interpolation in the neighborhood of the peak value do not improve the result.

\paragraph{Loss function}
Multi-class cross entropy loss is adopted for both light direction and intensity estimation, and the overall loss function is
\begin{align}
    \label{eq:cls_loss}
    \mathcal{L}_{\text{Light}} & = \lambda_{l_a} \mathcal{L}_{l_a} + \lambda_{l_e} \mathcal{L}_{l_e} + \lambda_e \mathcal{L}_e,
\end{align}
where $\mathcal{L}_{l_a}$ and $\mathcal{L}_{l_e}$ are the loss terms for azimuth and elevation of the light direction, and $\mathcal{L}_e$ is the loss term for light intensity.
For example, given $F$ input images,
\begin{align}
    \label{eq:cls_detail_loss}
    \mathcal{L}_{l_a}=-\frac{1}{F} \sum_{f=1}^{F} \sum_{i=1}^{K_d} \{y_{i}^f=1\} \log(p_i^f),
\end{align}
where $\{\cdot\}$ is a binary indicator (0 or 1) function, $y_i^f$ is the ground-truth label ($0$ or $1$) and $p_i^f$ is the predicted probability for bin $i$ ($K_d$ bins in our case) for the $f$\textsuperscript{th} image. Detailed definition of other loss terms are similar to \eref{eq:cls_detail_loss}.
During training, weights $\lambda_{l_a}$, $\lambda_{l_e}$, and $\lambda_e$ for the loss terms are set to $1$.

\subsection{Training Data}
We adopted the synthetic Blobby and Sculpture datasets introduced in~\Sref{sec:psfcn_dataset} for training.
Blobby and Sculpture datasets provide surfaces with complex normal distributions and diverse materials from MERL dataset~\cite{matusik2003merl}. Effects of cast shadow and inter-reflection were considered during rendering using the physically based raytracer Mitsuba~\cite{jakob2010mitsuba}. There are $85,212$ samples in total. Each sample was rendered under $64$ distinct light directions sampled from the upper-hemisphere with uniform light intensity, resulting in $5,453,568$ images ($85,212 \times 64$). The rendered images have a dimension of $128\times 128$.

To simulate images under different light intensities, we randomly generated light intensities in the range of $[0.2, 2.0]$ to scale the magnitude of the images (\ie, the ratio of the highest light intensity to the lowest one is $10$)\footnote{Note that the ratio (other than the exact value) matters, since light intensity can only be estimated up to a scale factor.}. Note that this selected range contains a wider range of intensity value than the public photometric stereo datasets like \diligent~\cite{shi2019benchmark} and \gourd~\cite{alldrin2008p}. The color intensities of the input images were normalized to the range of $[0, 1]$.
During training, we applied noise perturbation in the range of $[-0.025, 0.025]$ for data augmentation, and the input image size for LCNet was $128\times 128$. At test time, the input for LCNet is rescaled to $128\times 128$ as it contains fully-connected layers and requires the input to have a fixed spatial dimension. Trained only on the synthetic dataset, we will show that our model can generalize well on real datasets.

\paragraph{Implementation details}
Our method was implemented in PyTorch~\cite{paszke2017pytorch} and Adam optimizer~\cite{kingma2014adam} was used with default parameters. LCNet contains $4.4$ million parameters.
We trained LCNet using a batch size of $32$ for $20$ epochs, and the learning rate was initially set to $0.0005$ and halved every $5$ epochs.
It took about $22$ hours to train LCNet on a single Titan X Pascal GPU with a fixed input image number of $32$.

\subsection{Evaluation of LCNet with Synthetic Data}
\label{sub:eval_lcnet}
We evaluate LCNet on the synthetic test dataset \syntestMERL introduced in \Sref{subsec:synth_test_data}.
To measure the accuracy of the predicted light directions, the widely used mean angular error (MAE) in degree is adopted.
Since the light intensities among the testing images can only be estimated up to a scale factor $s$, we introduce the scale-invariant relative error (RE)
\begin{equation}
    RE_{scale} = \frac{1}{q} \sum_i^q \left(\frac{|s e_i -\tilde{e}_i|}{\tilde{e}_i} \right),
\end{equation}
where $q$ is the number of images, $e_i$ and $\tilde{e}_i$ are the estimated and ground-truth light intensities, respectively, for image $i$. The scale factor $s$ is computed by solving \hbox{$\argmin_s \sum_i^n (s e_i -\tilde{e}_i)^2$} with least squares.
As the calibrated intensity in the real dataset is in the form of a $3$-vector, we repeat the estimated intensity to form a $3$-vector and calculate the average result.

For all experiments on synthetic dataset involving input with unknown light intensities, we randomly generated light intensities in the range of $[0.2, 2.0]$. Each experiment was repeated five times and the average results were reported.

\paragraph{Discretization of lighting space}
For a given number of bins $K_d$, the maximum deviation angle for azimuth and elevation of a light direction is $\delta = 180\degree/(K_d\times 2)$ after discretization (\eg, $\delta=2.5\degree$ when $K_d=36$).
Note that discretizing azimuth and elevation angles independently indicates that lighting space is more densely discretized around the poles and less around the equator.
This suggests that the link between the quantization of the lighting space and surface normal estimation error correlates with the lighting distribution.
To investigate how the light direction discretization affects the surface normal estimation accuracy, we tested PS-FCN on \emphobject{Sphere} and \emphobject{Bunny} rendered under three different lighting distributions, namely, \textit{Near Uniform}, \textit{Around Equator}, and \textit{Around Poles} (see \fref{fig:discretization}~(a)).

We divided the azimuth and elevation angles of light directions into different numbers of bins ranging from $4$ to $180$.
For a specific bin number, we perturbed the azimuth and elevation of each ground-truth light direction by the maximum deviation angle, leading to four light directions that have the maximum possible angular deviations after discretization (see \fref{fig:discretization}~(b)).
We then used these light directions as input for PS-FCN to infer surface normals. The normal estimation error reported in \fref{fig:discretization} (c) is the upper-bound error for PS-FCN caused by discretization.
We can see that the error increase caused by discretization is marginal for all three lighting distributions when $K_d\ge30$.
We chose a relatively sparse discretization of lighting space in this work as it allows PS-FCN to learn to better tolerate small errors in the estimated lighting at test time.

\begin{figure}[tbp] \centering
    \input{ch-lcnet/figures/res_discretization_v2}
    \caption[Results of LCNet under different light direction space discretization levels]{(a) Three different lighting distributions. (b) Light directions $A$, $B$, $C$, and $D$ have the maximum deviation angles with the actual light direction $P$ after discretization. (c) Normal estimation error of PS-FCN on \emphobject{Sphere} (solid lines) and \emphobject{Bunny} (dashed lines) under different light direction space discretization levels ($\infty$ indicates no discretization).} \label{fig:discretization}
\end{figure}

\paragraph{Effectiveness of LCNet}
We first investigated the effect of object mask input and local-global feature fusion.
\Tref{tab:quant_light_synth} shows that taking the object mask as input and adopting the proposed local-global feature fusion strategy can effectively improve the lighting estimation results.
This might be explained by that object mask helps the network distinguish the shadow region from the non-object region, while the proposed local-global feature fusion strategy can effectively make use of information from multiple observations.

\begin{table}[tbp] \centering
    \caption[Lighting estimation results of LCNet on \syntestMERL dataset]{Lighting estimation results (MAE in degree for light direction and relative error for intensity) on \syntestMERL dataset. The results are averaged over samples rendered with $100$ BRDFs. (Value the lower the better)}
    \input{ch-lcnet/tables/res_quant_light_synth_v2} \label{tab:quant_light_synth}
\end{table}

\begin{table}[tbp] \centering
    \caption[Results on \emphobject{Sphere} and \emphobject{Bunny} under different lighting distributions]{Results of LCNet and \LCNetreg on \emphobject{Sphere} and \emphobject{Bunny} rendered under different lighting distributions.} \label{tab:quant_light_synth_regression}
    \input{ch-lcnet/tables/res_quant_light_synth_regression}
\end{table}

We then compared LCNet with a regression based baseline, denoted as \LCNetreg, to validate the effectiveness of the classification based model.
\LCNetreg shares the same architecture with LCNet, except that LCNet$_{\text{reg}}$ estimates a 3-vector for light direction and a scalar value for light intensity, rather than the softmax probability vectors.
 Given $q$ images, the loss function for the lighting regression is
 \begin{align}
     \label{eq:loss_reg}
     \mathcal{L}_{\text{Reg}} & = \lambda_{l} \frac{1}{q} \sum_{i}^q (1 - \boldsymbol{l}_{i}^\top \tilde{\boldsymbol{l}}_{i} )+ \lambda_{e} \frac{1}{q} \sum_{i}^q (e_{i} - \tilde{e}_{i} )^2,
 \end{align}
 where $\lambda_{l}$ and $\lambda_{e}$ are the weighting factors for the loss terms, $\boldsymbol{l}_{i}$ ($e_{i}$) and $\tilde{\boldsymbol{l}}_{i}$ ($\tilde{e}_{i}$) denote the predicted light direction (intensity) and the ground truth, respectively, for image $i$. During training, $\lambda_{l}$ and $\lambda_{e}$ are set to 1 (we found that using other weighting factors produce similar results).

Specifically, we tested LCNet and \LCNetreg on three different lighting distributions illustrated in \fref{fig:discretization}~(a). The results are shown in \Tref{tab:quant_light_synth_regression}. The proposed classification based LCNet consistently outperforms \LCNetreg on both light direction and intensity estimation. This echoes our hypothesis that classifying a light direction to a certain range is easier than regressing an exact value. Thus, solving the classification problem reduces the learning difficulty and improves the performance.
It can also be seen that both methods perform better on \textit{Around Equator} and worse on \textit{Around Poles}.
This suggests that lightings around the poles are more difficult to estimate due to their extremely directions, independent of the lighting space discretization.

\begin{figure}[htbp] \centering
    \includegraphics[width=0.46\textwidth]{ch-lcnet/images/Results/synth/SDPS-Net_img_sense_direction_img_num_sensitivity.pdf}
    \includegraphics[width=0.48\textwidth]{ch-lcnet/images/Results/synth/SDPS-Net_img_sense_intensity_img_num_sensitivity.pdf}
    \\ \vspace{-0.5em}
    \makebox[0.46\textwidth]{\footnotesize (a) Light direction estimation}
    \makebox[0.48\textwidth]{\footnotesize (b) Light intensity estimation}\\
    \caption[Results of LCNet on \syntestMERL dataset with varying image numbers]{Lighting estimation results of LCNet on \syntestMERL dataset with varying input image numbers.} \label{fig:img_num_syn}
\end{figure}

\Fref{fig:img_num_syn} shows that the performance of LCNet increases with the number of input images. This is expected, since more useful information can be used to infer lightings with more input images.

\newcommand{\PSFCNdag}{PS-FCN$^\dag$\xspace}

\paragraph{Integration with PS-FCN for normal estimation}
For uncalibrated photometric stereo, we train a variant of PS-FCN, denoted as \PSFCNdag, using the lighting estimated by LCNet.
Note that the weights of LCNet was fixed during the training of \PSFCNdag, as we found that end-to-end fine-tuning did not improve the performance.
Experiments with IDs C1 \& C2 in \Tref{tab:quant_normal_syn} show that after training with the discretized lighting estimated by LCNet, \PSFCNdag performs better than PS-FCN given possibly noisy lightings at test time. Besides, experiments with IDs C2 \& C3 show that \PSFCNdag coupled with the classification based LCNet consistently outperforms that with the regression based \LCNetreg.

\begin{table}[tbp] \centering
    \caption[Normal estimation results on \syntestMERL dataset]{Normal estimation results on \syntestMERL dataset. \PSFCNdag was trained given lightings estimated by LCNet or \LCNetreg.} %
    \input{ch-lcnet/tables/res_quant_normal_synth_shrink} \label{tab:quant_normal_syn}
\end{table}

\paragraph{Comparison with single-stage deep uncalibrated models}
To validate the effectiveness of the proposed two-stage approach, we compared our method with two different single-stage baseline models.
We first train a variant of PS-FCN, denoted as UPS-FCN, without taking the light direction as input during training and testing.
We then increased the model capacity of UPS-FCN by training a deeper network, denoted as UPS-FCN$_{\text{deep+mask}}$, that takes both the images and object mask as input (see \fref{fig:upsfcn_model_variants}).

\begin{figure}[tbp] \centering
    \includegraphics[width=\textwidth]{ch-lcnet/images/Method/UPS-FCN_Variants}
    \caption[Network architectures of UPS-FCN and UPS-FCN$_{\text{deep+mask}}$.]{Network architectures of UPS-FCN (top) and UPS-FCN$_{\text{deep+mask}}$ (bottom).} \label{fig:upsfcn_model_variants}
\end{figure}

Experiments with IDs C4 \& C5 in \Tref{tab:quant_normal_syn} show that utilizing a deeper network and taking the object mask as input can improve the performance of single-stage model.
However, experiments with IDs C2 \& C4 show that the proposed method significantly outperforms the single-stage model, when the input as well as the number of parameters are comparable.
This result indicates that simply increasing the depth of the network cannot produce optimal results.

\section{Analyzing What is Learned in LCNet}
\label{sec:LCNet_analysis}
\Sref{sub:eval_lcnet} shows that the proposed LCNet can predict accurate lightings for different testing objects. Also, given the lightings estimated by LCNet, the calibrated method PS-FCN significantly outperforms the single-stage uncalibrated method UPS-FCN.
However, what specifically inside the LCNet contributes to its success remains a mystery.

In this section, we discuss the inherent ambiguity in uncalibrated photometric stereo of Lambertian surfaces, the fact that it can be resolved for non-Lambertian surfaces, and the features learned by LCNet to resolve such ambiguity.

\newcommand{\numpixels}{\ensuremath{P}}
\newcommand{\pixelindex}{\ensuremath{p}}
\newcommand{\numlightings}{\ensuremath{F}}
\newcommand{\lightingindex}{\ensuremath{f}}
\subsection{Lambertian Surfaces and the \gbr Ambiguity}
When ignoring shadows (\ie, attached and cast shadows) and inter-reflections, the image formation of a Lambertian surface with $\numpixels$ pixels captured under $\numlightings$ lightings can be written as
\begin{equation}
    \V{M} = \V{N}^\top \V{L},
\end{equation}
where \hbox{$\V{M} \in \mathbb{R}^{\numpixels \times \numlightings}$} is the measurement matrix.
\hbox{$\V{N} \in \mathbb{R}^{3\times \numpixels}$} is the surface normal matrix whose columns are albedo scaled normals \hbox{$\V{N}_{:,\pixelindex}=\rho_\pixelindex\vn_\pixelindex$}, where $\rho_\pixelindex$ and $\vn_\pixelindex$ are the albedo and unit-length surface normal of pixel $\pixelindex$.
\hbox{$\V{L} \in \mathbb{R}^{3\times \numlightings}$} is the lighting matrix whose columns are intensity scaled light directions $\V{L}_{:,f} = e_f\vl_f$, where $e_f$ and $\vl_f$ are the light intensity and unit-length light direction of image~$f$.

\begin{figure}[htbp] \centering
	\input{ch-gcnet/figures/sphere_gbr}
    \caption[Results of PF14 and LCNet on shapes under different GBR transformation]{Row~1 is the true shape of a \emphobject{Sphere}, while rows~2 and~3 are shapes under two different GBR transformations. In column~(c), the points' positions and colors indicate light direction and relative intensity, respectively. Columns (e) and (f) show the lightings estimated by PF14~\cite{papad14closed} and LCNet.}\label{fig:gbr}
\end{figure}

By matrix factorization and applying the surface integrability constraint, $\V{N}$ and $\V{L}$ can be recovered up to an unknown $3$-parameter GBR transformation $\V{G}$~\cite{belhumeur1999bas} such that \hbox{$\V{M} = (\V{G}^{-\top}\V{N})^\top (\V{G}\V{L})$}. This GBR ambiguity indicates that there are infinitely many combinations of albedo $\rho$, normal $\vn$, light direction $\vl$, and light intensity~$e$ that produce the same appearance $\V{M}$ (see~\fref{fig:gbr}~(a)-(d)):
\begin{align}
    \hat{\rho} \!=\! \rho|\V{G}^{-\top}\vn|,\,\,
    \hat{\vn} \!=\! \frac{\V{G}^{-\top}\vn}{|\V{G}^{-\top}\vn|},\,\,
    \hat{\vl} \!=\! \frac{\V{G}\vl}{|\V{G}\vl|},\,\,
    \hat{e} \!=\! e |\V{G} \vl|.
\end{align}

Although the surface's appearance remains the same after GBR transformation (\ie, \hbox{$\hat{\rho} \hat{\vn}^{\!\top} \hat{\vl}  \hat{e} \!=\! \rho \vn^\top \vl e$}, see \fref{fig:gbr}~(d)), a surface point's albedo will be scaled by \hbox{$|\V{G}^{-\top} \vn|$}. As a result, the albedo of an object will change gradually and become spatially-varying.
Because this
kind of spatially-varying albedo distribution resulting %
from \gbr transformations rarely occurs on real world objects, some previous methods make explicit assumptions on the albedo distribution (\eg, constant albedo~\cite{belhumeur1999bas,papad14closed} or low entropy~\cite{alldrin2007r}) to resolve the ambiguity.

PF14~\cite{papad14closed}, a state-of-the-art non-learning uncalibrated method~\cite{shi2019benchmark}, detects Lambertian diffuse reflectance maxima (\ie, image points satisfying $\vn^\top\vl=1$) to estimate $\V{G}$'s $3$ parameters. %
We will later use it as a non-learning benchmark in our comparative experiments.

\subsection{LCNet and the GBR Ambiguity}
\Fref{fig:gbr}~(e)-(f) compare the results of LCNet and PF14 on surfaces that differ by GBR transformations.
Since the input images are the same in all cases, LCNet estimates the same lightings in all cases, namely the most likely lightings for the input images. The same also applies to PF14.
Although LCNet's result does not exactly equal the lightings for uniform albedo, we note that it learned from the training data that GBR-transformed surfaces are unlikely.

\begin{table}[t] \centering
    \caption[Results of PF14 and LCNet on a \emphobject{Sphere} rendered with different BRDFs]{Light direction estimation results of PF14~\cite{papad14closed} and LCNet on a \emphobject{Sphere} rendered with different BRDF types. Non-Lambertian BRDFs are taken from the MERL dataset~\cite{matusik2003merl}.} \label{tab:res_sphere_brdfs}
    \input{ch-gcnet/tables/compare_PF14_MERL_BRDFs_Sphere}
\end{table}

Although uncalibrated photometric stereo has an intrinsic GBR ambiguity for Lambertian surfaces, it was shown that GBR transformations do not preserve specularities~\cite{belhumeur1999bas,georghiades2003incorporating,drbohlav2005can}. Hence, specularities are helpful for ambiguity-free lighting estimation.
However, traditional methods often treat non-Lambertian observations as outliers, and thus fail to %
make full use of specularities for disambiguation~\cite{papad14closed}.
In contrast, learning-based methods can learn the relation between specular highlights and light directions through end-to-end learning.
As shown in \Tref{tab:res_sphere_brdfs}, LCNet achieves good results for non-Lambertian surfaces while PF14 completely fails when non-Lambertian observations dominate.

\begin{figure}[htbp]\centering
	\input{ch-gcnet/figures/feature_visual_v5}
    \caption[Feature visualization of \hbox{LCNet} on a non-Lambertian \emphobject{Sphere}]{Feature visualization of \hbox{LCNet} on a non-Lambertian \emphobject{Sphere}. \emph{Column~1:}~$5$ of the $96$ input images; \emph{Columns~2--4:}~Specular high\-light centers, attached shadows, and shading rendered from ground truth; \emph{Columns~5--7:}~$3$ of LCNet's $256$ features maps. The last row shows the global features produced by fusing local features with max-pooling. All features are normalized to $[0, 1]$ and color coded.} \label{fig:visualization}
\end{figure}

\subsection{Feature Analysis for LCNet}
To analyze the features learned by LCNet, we first visualize the learned local and global features. \Fref{fig:visualization} shows $3$ representative features selected from $256$ feature maps extracted by LCNet from images of a non-Lambertian \emphobject{Sphere} dataset.
Comparing \fref{fig:visualization}'s Column 2 with Column 5, Column 3 with Column 6, and Column 4 with Column 7, we can see that some feature maps are highly correlated with attached shadows (regions where the angle $\measuredangle(\vn,\vl)\!\ge\! 90\degree$), shadings ($\vn^{\!\top}\vl$), and specular highlights (regions where $\vn$ is close to the half angle $\vh\!=\!\frac{\vl+\vv}{\abs{\vl+\vv}}$ of $\vl$ and viewing direction $\vv$).
As discussed earlier in the related work (\Sref{sec:lcnet_relatedwork}), these provide strong clues for resolving the ambiguity. %

To further verify our observations, we did the following. We computed (a)~attached shadows, (b)~the ``specular components'' (with a bit of concept abuse, we denote $\vh^\top\vn$ as specular component), and (c)~shadings for the synthetic Blobby and Sculpture datasets from their ground-truth light directions and normals.
We then trained $4$ variants of the LCNet, taking (a), (b), (c), and (a) + (b) + (c), respectively, as input instead of regular images.
We compared these $4$ variant networks with LCNet (\ie, the network trained with Blobby and Sculpture images) on a synthetic test dataset introduced in \Sref{subsec:synthetic_data}. Similar to LCNet, the variant networks also took the object mask as input.
\Tref{tab:proxy} shows that the variant models achieve results comparable to or even better than the model trained on regular images.

\begin{table}[t] \centering
    \caption[Results of LCNet trained with different inputs]{Light direction estimation results of LCNet trained with different inputs. Values indicate mean angular error in degree.} \label{tab:proxy}
    \input{ch-gcnet/tables/res_synth_proxy}
\end{table}

We can see that shadings contribute more than attached shadows and specular components for lighting estimation.
This is because shading information actually includes attached shadows (\ie, pixels with a zero value in the shading for synthetic data), and can be considered as an image of an object with uniform albedo (albedo equals to $1$). The uniform albedo constraint is a well-known clue for resolving the GBR ambiguity~\cite{belhumeur1999bas,papad14closed}.
In practice, attached shadows, shadings, and the specular components are not directly available as input, but this confirms our assumption that they provide strong clues for accurate lighting estimation.

As discussed before, LCNet learns to resolve ambiguity by assuming that a surface with a gradually changing albedo corresponding to \gbr transformations rarely exists.
However, we have not observed features apparently related to albedo distribution. We hypothesize that the albedo distribution prior is implicitly employed to restrict the mapping space, since \mbox{LCNet} learns the mapping from extracted features to lightings.

\section{\GCNetname (\gcnetacronym)}
We have analyzed the features learned by LCNet and discussed how it resolves the ambiguity. In this section, we present the motivations for our \gcnetname (\gcnetacronym) and detail its structure.

\subsection{Guided Feature Extraction}
\label{sec:guided_feature_extraction}
As we have seen, features like attached shadows, shadings, and specularities are important for lighting estimation, and a lighting estimation network may benefit greatly from being able to estimate them accurately.
We further know that these features are completely determined by the light direction for each image as well as the inter-image shape information derived from the surface normal map.
However, LCNet extracts features independently from each input image and thus cannot exploit any inter-image information during feature extraction. This observation also indicates that simply increasing the layer number of LCNet's shared-weight feature extractor cannot %
produce significant improvement.

\paragraph{Surface normal as inter-image guidance}
Intuitively, if we can provide such inter-image shape information as input to the network to guide the feature extraction process, it should be able to perform better.
This, however, constitutes a chicken-and-egg problem where we require normals and lightings for accurate feature extraction but at the same time we require these features for estimating accurate lightings. We therefore suggest a cyclic network structure in which
we first estimate initial lightings, and then use them to estimate normals as inter-image information to guide the extraction of local (\ie, per-image) features to ultimately estimate final lightings.
An alternative idea might be directly estimating surface normals from images. %
However, we have shown in \Tref{fig:upsfcn_model_variants} that surface normals estimated directly from images are not accurate.

\paragraph{Shading as intra-image guidance}
Another advantage of first estimating initial lighting and surface normals is that we can easily compute coarse attached shadows, shadings, or specular components as intra-image guidance for the feature extraction process (intra-image means the information is different %
for each image).
As shading information already includes attached shadows, and not all materials exhibit specular highlights, we only compute the shading for each image as the dot-product of the estimated lighting with the surface normals, and use it as intra-image guidance.
We experimentally verified that additionally including the specular component as network input does not improve results.
The computed shading, on the other hand, does improve results and can assist the network to extract better features.

\begin{figure}[tbp] \centering
	\includegraphics[width=\textwidth]{ch-gcnet/images/Method/PG-LENet_v2}
    \caption[Network architecture of \gcnetacronym]{(a) Structure of the lighting estimation sub-network L-Net. (b) Structure of the normal estimation sub-network N-Net. (c)~The entire \mbox{\gcnetacronym}. Values in layers indicate the output channel number.} \label{fig:netname}
\end{figure}

\subsection{Network Architecture}
As shown in~\fref{fig:netname}, the proposed \gcnetacronym consists of two lighting estimation sub-networks (L-Net) and a normal estimation sub-network (N-Net).
The first L-Net, ``L-Net$_1$'', estimates initial lightings given the input images and object masks.
The N-Net then estimates surface normals from the lightings estimated by L-Net$_1$ and the input images.
Finally, the second L-Net, ``L-Net$_2$'', estimates more accurate lightings based on the input images, object masks, the estimated normals, and the computed shadings.

\paragraph{L-Net} The L-Net is designed based on LCNet but has less channels in the convolutional layers to reduce the model size (see \fref{fig:netname}~(a)).
Compared to LCNet's $4.4$ million parameters, each L-Net has only \num{1.78} million parameters.

Following LCNet, we discretize the lighting space and treat lighting estimation as a classification problem.
Specifically, L-Net's output light direction and intensity are in the form of softmax probability vectors (a $32$-vector for elevation, a $32$-vector for azimuth, and a $20$-vector for intensity).
Given $\numlightings$ images, the loss function for L-Net is
\begin{align}
    \label{eq:light_loss}
    \mathcal{L}_{\text{light}} & =  \frac{1}{\numlightings} \sum_\lightingindex(\mathcal{L}^\lightingindex_{l_a} + \mathcal{L}^\lightingindex_{l_e} + \mathcal{L}^\lightingindex_e),
\end{align}
where $\mathcal{L}^\lightingindex_{l_a}, \mathcal{L}^\lightingindex_{l_e}$, and $\mathcal{L}^\lightingindex_{e}$ are the cross-entropy loss for light azimuth, elevation, and intensity classifications for the $\lightingindex$\textsuperscript{th} input image, respectively.
The output probability vectors can be converted to a $3$-vector light direction $\vl_f$ and a scalar light intensity $e_f$ by taking the probability vector's expectation, which is differentiable for later end-to-end fine-tuning.

L-Net$_1$ and L-Net$_2$ differ in that L-Net$_1$ has $4$ input channels ($3$ for the image, $1$ for the object mask) while L-Net$_2$ has $8$ ($3$ additional channels for normals, $1$ for shading).

\paragraph{N-Net} The N-Net is designed based on PS-FCN~\cite{chen2018ps} but with less channels, resulting in \num{1.1} million parameters compared to PS-FCN's \num{2.2} million parameters (see \fref{fig:netname}~(b) for details). Following PS-FCN, the \mbox{N-Net's} loss function is
\begin{align}
    \label{eq:normal_loss}
    \mathcal{L}_{\text{normal}} = \frac{1}{\numpixels} \sum_{\pixelindex} \left(1 - \vn_\pixelindex^\top \tilde{\vn}_{\pixelindex} \right),
\end{align}
where $\numpixels$ is the number of pixels per image, and $\vn_\pixelindex$ and $\tilde{\vn}_\pixelindex$ are the predicted and the ground-truth normal at pixel $\pixelindex$, respectively.

\paragraph{End-to-end fine-tuning}
We train L-Net$_1$, N-Net, and L-Net$_2$ one after another until convergence and then fine-tune the entire network end-to-end using the following loss
\begin{align}
    \label{eq:finetune}
    \mathcal{L}_{\text{fine-tune}} & = \mathcal{L}_{\text{light}_1} + \mathcal{L}_{\text{normal}} + \mathcal{L}_{\text{shading}} + \mathcal{L}_{\text{light}_2}, \\
    \mathcal{L}_{\text{shading}} & = \frac{1}{\numlightings\numpixels} \sum_{\lightingindex} \sum_{\pixelindex}(\vn_\pixelindex^\top \vl_\lightingindex - \tilde{\vn}_\pixelindex^\top \tilde{\vl}_\lightingindex)^2,
\end{align}
where \hbox{$\mathcal{L}_{\text{light}_1}$} and \hbox{$\mathcal{L}_{\text{light}_2}$} denote the lighting estimation loss for L-Net$_1$ and L-Net$_2$.
The shading loss term \hbox{$\mathcal{L}_{\text{shading}}$} is included to encourage better shading estimation,
and $\vl_\lightingindex$ and $\tilde{\vl}_\lightingindex$ denote the light direction predicted by L-Net$_1$ and ground-truth light direction for the $\lightingindex$\textsuperscript{th} image, respectively.

\paragraph{Training details}
Following LCNet, we trained the networks on the synthetic Blobby and Sculpture Dataset which contains $85,212$ surfaces, each rendered under $64$ random light directions.

First, we train L-Net$_1$ from scratch for $20$ epochs, halving the learning rate every $5$ epochs.
Second, we train \mbox{N-Net} using ground-truth lightings and input images following PS-FCN's training procedure (see \Sref{sec:psfcn_exp}), and then retrain N-Net given the lightings estimated by L-Net$_1$ for $5$ epochs, halving the learning rate every $2$ epochs.
Third, we train \mbox{L-Net$_2$} given the input images, object masks, estimated normals, and computed shadings for $20$ epochs, halving the learning rate every $5$ epochs.
The initial learning rate is $0.0005$ for L-Net$_1$ and L-Net$_2$, and $0.0002$ for retraining N-Net.
End-to-end training is done for $20$ epochs with an initial learning rate of $0.0001$, halving it every $5$ epochs.

We implemented our framework in PyTorch~\cite{paszke2017pytorch} and used the Adam optimizer~\cite{kingma2014adam} with default parameters.
The full network has a total of \num{4.66} million parameters which is comparable to LCNet (\num{4.4} million).
The batch size and the input image number for each object are fixed to $32$ during training.
The input images for all sub-networks are resized to $128\!\times\! 128$ at both training and test time.

\section{Experimental Results}
We first compare LCNet and \gcnetacronym on the synthetic test dataset \syntestMERL introduced in \Sref{subsec:synth_test_data}, and then compared our methods with existing methods on the publicly available real datasets. Similarly, the widely used mean angular error (MAE) in degree is adopted to measure the accuracy of the predicted light directions and surface normals. The scale-invariant relative error (RE) is used to measure the accuracy of the predicted light intensities (see \Sref{sub:eval_lcnet} for definition).

For all experiments on synthetic dataset involving input with unknown light intensities, we randomly generated light intensities in the range of $[0.2, 2.0]$. Each experiment was repeated five times and the average results were reported.

\subsection{Evaluation on Synthetic Data}
\label{subsec:synthetic_data}
\paragraph{Ablation study}
To validate the design of the proposed \gcnetacronym, we performed an ablation study and summarized the results in \Tref{tab:synth_test_light}.
The comparison between experiments with IDs 2-4 verifies that taking both the estimated normals and shading as input is beneficial for lighting estimation.
The comparison between experiments with IDs 1 \& 2 demonstrates that end-to-end fine-tuning further improves the performance.
We can also see that \hbox{L-Net$_1$} achieves results comparable to LCNet despite using only half of the network parameters, which indicates that simply increasing the channel number of the convolutional layers cannot guarantee better feature extraction. In the rest of this chapter, we denote the results of ``L-Net$_1$ + N-Net + L-Net$_2$ + Finetune'' as ``\gcnetacronym''.

\Tref{tab:synth_test_normal} shows that, as expected, the calibrated photometric stereo method PS-FCN (trained with ground-truth lightings) can estimate more accurate normals given better estimated lighting.

\begin{table}[tbp]
    \caption[Ablation study for network architecture of \gcnetacronym]{Ablation study for network architecture of \gcnetacronym. Lighting estimation results of \gcnetacronym on \syntestMERL dataset. The results are averaged over $100$ MERL BRDFs (bold fonts indicates the best).}
    \input{ch-gcnet/tables/res_synth_test_light_v2} \label{tab:synth_test_light}
\end{table}

\begin{table}[tbp] \centering
    \caption[Normal estimation results on \syntestMERL dataset]{Normal estimation results on \syntestMERL dataset. The estimated normals are predicted by PS-FCN~\cite{chen2018ps} given the lightings estimated LCNet and \gcnetacronym.}
    \input{ch-gcnet/tables/res_synth_test_normal}
    \label{tab:synth_test_normal}
\end{table}

\paragraph{Comparison of different cascaded structures}
Cascaded structure is a popular strategy to improve performance. We compared the proposed structure for \gcnetacronym with three different structures (see \fref{fig:cascaded}) to verify our method's effectiveness.

\begin{figure}[tbp] \centering%
	\includegraphics[width=\textwidth]{ch-gcnet/images/Method/different_structures_simple.pdf}
    \caption[Three different cascaded structures]{Three different cascaded structures. (a) L-Net$_1$ + L-Net$_2$. (b) L-Net$_1$ + N-Net + L-Net$_2^\text{(w/o shading)}$. (c) L-Net$_1$ + N-Net + L-Net$_2^\text{(w/o shading; w/ light)}$. We skip the input of L-Net$_1$ and the output of L-Net$_2$ for all models to simplify the illustration.} \label{fig:cascaded}
\end{figure}

\Fref{fig:cascaded}~(a) is a common structure to refine a network's estimation using another similar network. As discussed in \Sref{sec:guided_feature_extraction}, L-Net's bottleneck is the lack of inter-image information (\eg, normals) during feature extraction, making this structure sub-optimal (compare the experiments with IDs 5 \& 8 in \Tref{tab:synth_test_light}).
\Fref{fig:cascaded}~(b) is a sequential structure where L-Net$_2$ additionally takes estimated normals as input. However, the experiments with the IDs~5~\&~7 show that taking the estimated shading (intra-image information) as input is beneficial.
In \fref{fig:cascaded}~(c), L-Net$_2$ takes estimated normals and lightings as input.
Our experiment shows that taking lightings as input can lead to faster convergence, but the final performance is worse than the proposed method, as show in the experiments with the IDs 5 \& 6.
We suspect that L-Net$_2$ becomes more dependent on input lightings if directly taking them as input during training.
When \mbox{L-Net$_1$'s} estimated lightings are not accurate, the refined estimation may not be good.

\begin{table}[tbp] \centering
    \caption[Results on \emphobject{Armadillo} under three different lighting distributions]{Results on \emphobject{Armadillo} under three different lighting distributions.}
    \input{ch-gcnet/tables/res_synth_biased_lighting}
    \label{tab:synth_biased_light}
\end{table}

\paragraph{Results on different lighting distributions}
To analyze the effect of biased lighting distributions on the proposed method, we evaluated \gcnetacronym on the \emphobject{Armadillo} illuminated under three different lighting distributions: a near uniform, a narrow, and an upward-biased distribution.
\Tref{tab:synth_biased_light} shows that both \gcnetacronym and LCNet have decreased performance under biased lighting distributions (\eg, the upward-biased distribution), but \gcnetacronym consistently outperforms LCNet.

\begin{table}[htbp]
    \caption[Lighting estimation results on \emphobject{Bunny} rendered with SVBRDFs]{Lighting estimation results on \emphobject{Bunny} rendered with SVBRDFs. (a) \emphobject{Bunny} with uniform BRDF. (b) and (c) show the ``Ramp'' and ``Irregular'' material maps and two sample images of \emphobject{Bunny} with the corresponding SVBRDFs.}
    \input{ch-gcnet/tables/res_synth_SVBRDFs_v2}
    \label{tab:synth_SVBRDFs}
\end{table}

\paragraph{Results on surfaces with SVBRDFs}
To analyze the effect of SVBRDFs, we used two different material maps to generate a synthetic dataset of surfaces with SVBRDFs following Goldman \etal~\cite{goldman2010shape}. Specifically, we rendered $100$ test objects by randomly sampling two MERL BRDFs and blended the BRDFs for \emphobject{Bunny} using ``Ramp''  and ``Irregular'' material maps shown in \Tref{tab:synth_SVBRDFs}~(b) and~(c).
\Tref{tab:synth_SVBRDFs} shows that although both methods perform worse on surfaces with SVBRDFs compared to uniform BRDFs, our method is still reasonably good even though it was trained on surfaces with uniform BRDFs.
This might be explained by that although SVBRDFs may affect the feature extraction of some important clues such as shading, others such as attached shadows and specular highlights are less affected and can still be extracted to estimate reliable lightings.

\begin{table}[htbp] \centering
    \caption[Lighting estimation results on surface regions cropped from \emphobject{Bunny}]{Lighting estimation results on surface regions cropped from \emphobject{Bunny}.}
    \input{ch-gcnet/tables/res_silhouette_v2} \label{tab:silhouette}
\end{table}

\paragraph{Effect of the object silhouette}
Object silhouette can provide useful information for lighting calibration (\eg, normals at the occluding contour are perpendicular to the viewing direction).
To investigate the effect of the silhouette, we first rendered the \emphobject{Bunny} using two different types of BRDFs (\textit{alumina-oxide} and \textit{beige-fabric}) under $100$ lightings sampled randomly from the upper hemisphere, and then cropped surface regions with different sizes for testing.
\Tref{tab:silhouette} shows that both LCNet and our method perform robustly for surface regions with or without silhouette, while our method consistently outperforms LCNet. This is because the training data for both methods was generated by randomly cropping image patches from the Blobby and Sculpture datasets, which contains surface regions without silhouette. %

\subsection{Evaluation on Real Data}
\begin{table}[htbp] \centering
    \caption[Lighting estimation results on \diligent]{Lighting estimation results on \diligent. Bold font indicates the best result.} %
    \input{ch-gcnet/tables/res_diligent_light_2col} \label{tab:quant_light_diligent}
\end{table}

To demonstrate the proposed method's capability to handle real-world non-Lambertian objects, we evaluated our method on the challenging \emph{\diligent}~\cite{shi2019benchmark} and the \emph{\lightstage}~\cite{einarsson2006relighting}.

\paragraph{Results on lighting estimation}
We first compared our GCNet with the LCNet and non-learning method PF14~\cite{papad14closed}.
PF14 is the state-of-the-art traditional method for uncalibrated photometric stereo, and we used its publicly available code for testing.
\Tref{tab:quant_light_diligent} shows that \gcnetacronym achieves the best average results on the \diligent with an MAE of $3.32$ for light directions and a relative error of $0.052$ for light intensities.
Although our \gcnetacronym does not achieve the best results for all objects, it exhibits the most robust performance with a maximum MAE of $5.77$ and a maximum relative error of $0.101$ compared with LCNet (MAE:~$10.36$, relative error:~$0.105$) and PF14 (MAE:~$33.22$, relative error:~$0.223$).
\Frefs{fig:qual_lighting}~(a)-(b) visualize the lighting estimation results for the \emphobject{Pot1} and the \emphobject{Goblet}. The non-learning method PF14 works well for near-diffuse surfaces (\eg, \emphobject{Pot1}), but quickly degenerates on highly specular surfaces (\eg, \emphobject{Goblet}). Compared with LCNet, \gcnetacronym is more robust to surfaces with different reflectances and shapes.

\begin{table}[tbp] \centering%
    \caption[Lighting estimation results on \lightstage]{Lighting estimation results on \lightstage.}%
    \input{ch-gcnet/tables/res_lightstage_light_wide} \label{tab:quant_light_lightstage}
\end{table}

\Tref{tab:quant_light_lightstage} shows lighting estimation results on the \lightstage. Our \gcnetacronym significantly outperforms LCNet and PF14, and achieves an average MAE of $9.20$ for light directions and a relative error of $0.163$ for light intensities, improving the results of LCNet by $32.4\%$ and $26.4\%$ for light directions and light intensities respectively.
\Frefs{fig:qual_lighting}~(c)-(d) visualize lighting estimation results for the \lightstage's \emphobject{Standing Knight} and \emphobject{Plant}.

\begin{figure}[tbp] \centering%
    \input{ch-gcnet/figures/qual_lighting}
    \caption[Visualization of the estimated lighting distributions]{Visualization of the ground-truth and estimated lighting distributions for the \diligent and \lightstage. } \label{fig:qual_lighting} %
\end{figure}

\paragraph{Results on surface normal estimation}
We then verified that the proposed \gcnetacronym can be seamlessly integrated with existing calibrated methods to handle uncalibrated photometric stereo.
Specifically, we integrated the \gcnetacronym with a state-of-the-art non-learning calibrated method ST14~\cite{shi2014bi} and two learning-based methods PS-FCN and IS18~\cite{ikehata2018cnn}.
\Tref{tab:quant_diligent_normal} shows that these integrations can outperform existing state-of-the-art uncalibrated methods~\cite{alldrin2007r,shi2010self,wu2013calib,lu2013uncalibrated,papad14closed,lu2018symps} by a large margin on the \diligent.
We can further see that ST14, PS-FCN, as well as IS18 perform better with \gcnetacronym's instead of LCNet's predicted lightings: $10.8$ vs.\ $12.1$ for PS14, $8.7$ vs.\ $9.7$ for PS-FCN, and $8.6$ vs.\ $16.5$ for IS18.
\Fref{fig:qual_diligent_normal} presents visual comparisons on the \emphobject{Pot1} and \emphobject{Goblet} from the \diligent.

\begin{table}[tbp] \centering
    \caption[Normal estimation results on \diligent]{Normal estimation results on \diligent.}
    \input{ch-gcnet/tables/res_diligent_normal} \label{tab:quant_diligent_normal}
\end{table}

\begin{figure}[htbp] \centering
    \input{ch-gcnet/figures/qual_diligent_normal_larger}
    \caption[Visual comparisons of normal estimation for \emphobject{Pot1} and \emphobject{Goblet}]{Visual comparisons of normal estimation for \emphobject{Goblet} in the \diligent. We compared the normal estimation results of a calibrated method IS18~\cite{ikehata2018cnn} given lightings estimated by \gcnetacronym and LCNet.} \label{fig:qual_diligent_normal}
\end{figure}

\Fref{fig:qual_lightstage} shows the surface normals of the \emphobject{Standing Knight} predicted by PS-FCN given lighting estimated by \gcnetacronym and LCNet. We can see that coupled with \gcnetacronym's more accurate lightings, PS-FCN can produce more reliable normal estimation for thin regions.

\begin{figure}[htbp] \centering
    \input{ch-gcnet/figures/qual_lightstage}
    \caption{Visual comparison of normal estimation for the \lightstage's \emphobject{Standing Knight}.} \label{fig:qual_lightstage}
\end{figure}

\subsection{Failure Cases}
\begin{figure}[htbp] \centering
    \caption[Failure cases]{Failure cases. (a)~Results on a piecewise planar surface with sparse normal distribution. (b)~Results on a highly-concave bowl. The estimated normals are predicted by PS-FCN given GCNet's estimated lightings.} %
	\label{fig:failure}
	\input{ch-gcnet/figures/failure_cases_v2}
\end{figure}

As discussed in \Sref{sec:LCNet_analysis}, LCNet relies on features like attached shadows, shading, and specular highlights, which is also true for \gcnetacronym.
For piecewise planar surfaces with a sparse normal distribution such as the one in \fref{fig:failure}~(a), few useful features can be extracted and as a result \gcnetacronym cannot predict reliable lightings for such surfaces.
For highly-concave shapes under directional lightings, strong cast shadows largely affect the extraction of useful features.
\Fref{fig:failure}~(b) shows that \gcnetacronym erroneously estimates a highly-concave bowl to be convex.
Note that LCNet and PF14~\cite{papad14closed} also have similar problems.

\section{Conclusion}
In this chapter, we have first introduced a lighting calibration network, named LCNet, to estimated directional lightings for uncalibrated photometric stereo.
To understand what have been learned by LCNet for lighting estimation, we analyze the features learned by the network, and find that attached shadows, shadings, and specular highlights are key elements for lighting estimation.
Based on our findings, we then introduced the \gcnetname, named \gcnetacronym, that explicitly leverages inter-image information of object shape and intra-image information of shading to estimate more reliable lightings.
Experiments on both synthetic and real datasets showed that \gcnetacronym achieves significantly better results than LCNet, and demonstrated that our method can be integrated with existing calibrated photometric stereo methods to handle uncalibrated setups.


\section{Parametric Molecular Electrostatic Potential (PMEP)}\index{PMEP|ff}
\index{Ford@{\bf Ford, George}}
\index{Wang@{\bf Wang, Bingze}}
\index{Electrostatic Potential!PMEP model}
The PMEP procedure~\cite{pmep1,pmep2} is a technique for rapidly calculating
the electrostatic properties of a molecule.  Written by Prof.\ George Ford and
Dr.\ Bingze  Wang\footnote{Current address: IRBM, Via Pontina Km.30.600, 00040
Pomezia (Roma), Italy} at Southern Methodist University, Dallas, Texas, the
procedure is ideally suited for large systems.

The PMEP procedure has two main functionalities: first, to generate a 2-D grid
of points giving the Electrostatic Potential (ESP) in a cross-section through a
system, and second, to generate atomic charges based on the calculated ESP. At
present, the method is limited to AM1 systems containing H, C, N, O, F, Cl,
only.

\subsection{2-D Electrostatic Potential Plots}
ESP plots are generated in two steps.  First, a MOPAC calculation generates a
2-D grid of points.  This grid is then converted into a picture by the utility
program \comp{ESPLOT}.  \comp{ESPLOT} is very simple to use: the command is
\comp{esplot $<$filename$>$}, where \comp{$<$filename$>$} is the name of the
data-set.  \comp{ESPLOT} generates an on-line picture of the PMEP, and a HPGL
file suitable for use in generating hard-copy.  Because \comp{ESPLOT} is so
simple, it will not be discussed further.  Instead, the rest of this discussion
applies to the MOPAC calculation.

The grid generated by MOPAC consists of a 2-D array of points representing a
cross-section through the system.  The distance between points is a constant
0.1 \AA ngstroms.  The size represented by the grid is roughly 4 \AA ngstroms
plus twice the size of the system.  For example, N$_2$ has a N--N distance of
1.1 \AA , and the default associated grid represents a rectangular area of 5.8
by 4.6 \AA .    Each grid point represents the potential in kcal/mol which a
unit positive charge would experience due to the electrostatic field of the
system.

ESP grids are generated by specifying
\hyperref[pageref]{\comp{PMEP}}{(see p.~}{)}{pmep} and \comp{PRTMEP}.
An example of a data-set for the PMEP procedure is shown in
Figure~\ref{pmepdata}. The PMEP plot for this data set is shown in
Figure~\ref{pmepplot}. This plot can be compared with the
\hyperref[pageref]{\comp{MEP} plot}{ on page~}{}{mep1plot}.

\begin{figure}
\begin{makeimage}
\end{makeimage}
\begin{verbatim}
 1scf  AM1 PMEP MINMEP PRTMEP
 Formaldehyde (Cross-section in plane of molecule)
 Generate a 2-D grid of PMEP potentials for `esplot' to use
  O    0.00000000 0     0.0000000 0    0.0000000 0    0 0 0     -0.2759
  C    1.22732374 1     0.0000000 0    0.0000000 0    1 0 0      0.1384
  H    1.11047287 1   122.2253516 1    0.0000000 0    2 1 0      0.0688
  H    1.11048351 1   122.2158646 1  179.9998136 1    2 1 3      0.0687
\end{verbatim}
\caption{\label{pmepdata}Data Set for PMEP Calculation of Formaldehyde}
\end{figure}

\begin{figure}
\begin{makeimage}
\end{makeimage}
\begin{center}
\includegraphics{ch2o-ford}
\end{center}
\caption{\label{pmepplot}Parametric Molecular Electrostatic Potential around
Formaldehyde}
\end{figure}

\subsubsection{Choice of Plane to be Calculated}
By default, the grid is centered on the center of the molecule, and the X-Y
plane at Z=0 is selected.  Other grids can be chosen using \comp{PMEPR}.
\comp{PMEPR} uses three atoms and an optional offset to define the plane to be
used.  It has enough options to allow any plane to be easily specified.

\subsection{Atomic Charges}
By use of \comp{QPMEP}, a set of atomic charges can be calculated.  This set of
charges is the best least squares fit to the charges which reproduce the ESP of
the Connolly or Williams surfaces.


\section{Discussion}
The aim of this paper were to uncover the information in the spike-train of head direction neurons in the brain.
We found that the neurons contains useful information about the head direction of mice. By creating tuning curves and calculating the mutual information between head direction and neural activity, we have postulated how the neurons can be used to update the set of possible head direction, much the same way a detective use evidence to limit the search for a suspect. By only knowing the spike-train of multiple HD cells, we can make edjucated guesses about the true head direction.

Further, by applying a change of basis using PCA and by utlizing the first few principal components we have found a circular relationship between the head direction and the first two principal components.  This could be used to develop good models for for the head direction, using simple statistical learning methods.

While we are able to create edjuacted guesses, we can only approach the problem of predicting head direction using a probabilistic approach. Each bit of information we gain help us shape the probability distribution over different head directions, but we never gain a definitive answer.

\cite{projectdata} have used the same dataset, and created online estimators using bayesian methods. By using the activity of different cells, they were able to use the data to update a prior belief (the likelihood of different head directions) and create good posterior distributions. Their experiments also showed similar activity during sleep. This appears to be a working proof-of-concept, indicating it is in fact possible to predict head direction from neural activity. Their experiments also provide experimental evidence of the postulated ring-attractor hypotheis, where theoretical models assume the HD cells lies on a virtual ring. This hypoethesis fits well with our findings on the circular dependencies between the first two principal components and the head direction.

Further research into the field may uncover better models for predicting head direction, which may be used to gain more insight into how our brain process information. It may also lead to better human-machine interfaces in the future, by allowing machines to predict and understand our senses.