\section{Resiliency and Survivability}
\label{sec:survivability}

Being a platform for contractual money, Ergo should also support long-term contracts for a period of at least an average person's lifetime.
However, even young existing smart contract platforms are experiencing issues with performance degradation and adaptability to external conditions.
This leads to a situation where the cryptocurrency depends on a small group of developers to provide a fixing hard-fork, or the cryptocurrency won't survive.
For example, the Ethereum network was started with a Proof-of-Work consensus algorithm with a plan to switch to Proof-of-Stake in the near future.
However, delays in the Proof-of-Stake development have led to several fixing hard-forks~\cite{ethDifficultyBomb} and the community is still forced to rely on core developers promising to implement the next hard-fork.

The first common survivability issue is that in pursuit of popularity, developers tend to implement ad-hoc solutions without proper preliminary research and testing.
Such solutions inevitably lead to bugs, which then lead to hasty bug fixes, then to fixes of those bug fixes, and so on, making the network unreliable and even less secure.
A notable example is the IOTA cryptocurrency, which implemented various scaling solutions, including its own hash function and DAG structure, that allowed it to achieve high popularity and market capitalization.
However, a detailed analysis of these solutions revealed multiple serious problems, including practical attacks that enabled token theft~\cite{heilmancryptanalysis, de2018break}.
A subsequent hard-fork~\cite{IOTAReport} then fixed these problems by switching to the well-known SHA3 hash function, thereby confirming the uselessness of such kind of innovations.
Ergo's approach here is to use stable well-tested solutions, even if they lead to slower short-term innovations.
Most of the solutions used in Ergo are formalized in papers presented at peer-reviewed
conferences~\cite{reyzin2017improving,meshkov2017short,chepurnoy2018systematic,chepurnoy2018self,chepurnoy2018checking,duong2018multi}
and have been widely discussed in the community.

A second problem that decentralization (and thus survivability) faces is the lack of secure trustless light clients. Ergo tries to fix this problem of blockchain technology without creating new ones. Since Ergo is a PoW blockchain, it easily allows extraction of a small header from the block content.
This header alone permits validation of the work done in the block and a headers-only chain is enough for best chain selection and synchronization with the network.
A headers-only chain, although much smaller than the full blockchain, still grows linearly with time.
Recent research on light clients provides a way for light clients to synchronize with the network by downloading an even smaller amount of data, thereby unlocking the ability to join the network using untrusted low-end hardware such as mobile phones~\cite{kiayias2017non,luuflyclient}.
Ergo uses an authenticated state~\ref{sec:utxo} and for transactions included in a block, a client may download a proof of their correctness.
Thus, regardless of the blockchain size, a regular user with
a mobile phone can join the network and start using Ergo with the same security
guarantees as a full node.

Readers may notice a third potential problem in that although support for light clients solves the problem for Ergo users, it does not solve the problem for Ergo miners, who still need to keep the whole state for efficient transaction validation.
In existing blockchain systems, users can put arbitrary data into this state. This data, which lasts forever, creates a lot of dust and its size increases endlessly over time~\cite{perez2019another}.
A large state size leads to serious security issues because when the state does not fit in random-access memory, an adversary can generate transactions whose validation become very slow due to required random access to the miner's storage. This can lead to DoS attacks such as the one on Ethereum in 2016~\cite{ethDos2016}.
Moreover, the community's fear of such attacks along with the problem of ``state bloat'' without any sort of compensation to miners or users holding the state may have prevented scaling solutions that otherwise
could have been implemented (such as larger block sizes, for example).
To prevent this, Ergo has a storage rent component: if an
output remains in the state for 4 years without being consumed, a miner may charge a small fee for every
byte kept in the state.

This idea, which is similar to regular cloud storage services, was only proposed quite recently for cryptocurrencies~\cite{chepurnoy2017space} and has several important consequences.
Firstly, it ensures that Ergo mining will always be stable, unlike Bitcoin and other PoW currencies, where mining may become unstable after emission is done~\cite{carlsten2016instability}.
Secondly, growth of the state's size becomes controllable and predictable, thereby helping Ergo miners to manage their hardware requirements.
Thirdly, by collecting storage fees from outdated boxes, miners can return coins to circulation, and thus, prevent the steady decrease of circulating supply due to lost keys~\cite{wsj2018}.
All these effects should support Ergo's long-term survivability, both technically and economically.

A fourth vital challenge to survivability is that of changes in the external environment and demands placed on the protocol.
A protocol should adapt to the ever changing hardware infrastructure, new ideas to improve security or
scalability that emerge over time, the evolution of use-cases, and so on.
If all the rules are fixed without any ability to change them in a decentralized manner, even
a simple constant change can lead to heated debates and community splits. For instance,  discussion of the block-size limit in Bitcoin led to its splitting into several independent coins.
In contrast, Ergo protocol is self-amendable and is able to adapt to the changing environment.
In Ergo, parameters like block size can be changed on-the-fly via voting of miners.
At the beginning of each 1024-block voting epoch, a miner proposes changes of up to 2 parameters (such as an increase of block size and a decrease of storage fee factor). During the rest of the epoch, miners vote to approve or reject the changes.
If a majority of votes within the epoch support the change, the new values are written into the extension section of the first block of the next epoch, and
the network starts using the updated values for block mining and validation.

To absorb more fundamental changes, Ergo follows the approach of {\em soft-forkability} that
allows changing the protocol significantly while keeping old nodes operational.
At the beginning of an epoch, a miner can also propose to vote for a more fundamental change~(e.g., adding a new instruction to ErgoScript) describing affected validation rules.
Voting for such breaking changes continues for 32,768 blocks and requires at least $90\%$ of
``Yes'' votes to be accepted. Once being accepted, a 32,768-blocks long activation period starts to give outdated nodes time to update their software version.
If a node software is still not updated after the activation period, then it skips the specified checks but continues to validate all the known rules.
List of previous soft-fork changes is recorded into the extension to allow light nodes of
any software version to join the network and catch up to the current validation rules.
A combination of soft-forkability with the voting protocol allows changing of almost all the parameters of the network except the PoW rules that are responsible for the voting itself.


\documentclass{warpdoc}
\newlength\lengthfigure                  % declare a figure width unit
\setlength\lengthfigure{0.158\textwidth} % make the figure width unit scale with the textwidth
\usepackage{psfrag}         % use it to substitute a string in a eps figure
\usepackage{subfigure}
\usepackage{rotating}
\usepackage{pstricks}
\usepackage[innercaption]{sidecap} % the cute space-saving side captions
\usepackage{scalefnt}

%%%%%%%%%%%%%=--NEW COMMANDS BEGINS--=%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\alb}{\vspace{0.2cm}\\} % array line break
\newcommand{\mfd}{\displaystyle}
\renewcommand{\fontsizetable}{\footnotesize\scalefont{0.7}}
\setcounter{tocdepth}{3}
\let\citen\cite

%%%%%%%%%%%%%=--NEW COMMANDS ENDS--=%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%=--NEW COMMANDS BEGINS--=%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\newcommand{\ns}{{{n}_{\rm s}}}
\newcommand{\efficiency}{\eta}
\newcommand{\ordi}{{\rm d}}
\newcommand{\unitvecdiff}[2]{\overline{\vec{#1} - \vec{#2}}}
%\let\vec\bf
%\renewcommand{\vec}[1]{\pmb{#1}}
\newcommand{\betae}{\beta_{\rm e}}
\newcommand{\betai}{\beta_{\rm i}}
\newcommand{\sigmatilde}{\widetilde{\sigma}}
\newcommand{\mdot}{\dot{m}}
\newcommand{\efp}{\psi}
\newcommand{\vion}{\vec{v^{\rm i}}}
\newcommand{\vneutral}{\vec{v^{\rm n}}}
\newcommand{\velectron}{\vec{v^{\rm e}}}
\newcommand{\vflow}{\vec{v}}
\newcommand{\betaiprime}{\betai^\prime}
\newcommand{\betaeprime}{\betae^\prime}
\newcommand{\Eprime}{\vec{E}^\prime}
\newcommand{\tauei}{\tau_{\rm ei}}
\newcommand{\tauen}{\tau_{\rm en}}
\newcommand{\tauin}{\tau_{\rm in}}
\newcommand{\momin}{{k_{\rm in}}}
\newcommand{\momen}{{k_{\rm en}}}
\newcommand{\momei}{{k_{\rm ei}}}
\newcommand{\ie}{{\it i.e.}}
\newcommand{\rhoeDveDt}{\rho_{\rm e} \frac{{\rm D}v_{\rm e}}{{\rm D} t}}
\newcommand{\rhoiDviDt}{\rho_{\rm i} \frac{{\rm D}v_{\rm i}}{{\rm D} t}}
\newcommand{\gradPe}{\nabla P_{\rm e}}
\newcommand{\gradPi}{\nabla P_{\rm i}}
\newcommand{\zetae}{\zeta_{\rm e}}
\newcommand{\zetai}{\zeta_{\rm i}}
\newcommand{\zetan}{\zeta_{\rm n}}
\newcommand{\ev}{{e_{\rm v}}}
\newcommand{\ee}{{e_{\rm e}}}
\newcommand{\evzero}{{e_{\rm v}^0}}
\newcommand{\eezero}{{e_{\rm e}^0}}

\author{
  Bernard Parent
}

\email{
  bparent@princeton.edu
}

\department{
  Dept.\ of Mechanical and Aerospace Engineering
}

\institution{
  Princeton University
}

\title{
 Electron Beam Models
}

\date{
  January -- February 2005
}

%\setlength\nomenclaturelabelwidth{0.13\hsize}  % optional, default is 0.03\hsize
%\setlength\nomenclaturecolumnsep{0.09\hsize}  % optional, default is 0.06\hsize

\nomenclature{

  \begin{nomenclaturelist}{Roman symbols}
   \item[$a$] speed of sound
  \end{nomenclaturelist}
}

\abstract{
abstract
}

\begin{document}
  \pagestyle{headings}
  \pagenumbering{arabic}
  \setcounter{page}{1}
%%  \maketitle
  \makewarpdoctitle
%  \makeabstract
%  \tableofcontents
%  \makenomenclature
%%  \listoftables
%%  \listoffigures

\section{Fixed E-Beam Model}

For the ``fixed'' electron beam model, the power deposited per unit volume, $Q_{\rm b}$ is fixed to a constant within the control file as a function of the node position and is not altered as the solution converges.

\section{Albebraic E-Beam Model}

 The power deposited by the e-beam can be taken as:
%
\begin{equation}
  Q_{\rm b} = b_a + \frac{b_b}{b_w} \exp \left[ \frac{-2 \left( s- b_m \right)^2}{b_w^2 }  \right]
\end{equation}
%
where $b_m=b_{L}/3.21$ and $b_w=1.64 b_m$. The constants $b_a$ and $b_b$ can be determined from two conditions.
The first condition is to set $Q_{\rm b}$ to zero when $s=b_{L}$, yielding the following expression for $b_a$
%
\begin{equation}
  b_a =  - \frac{b_b}{b_w} \exp \left[ \frac{-2 \left( b_{L}- b_m \right)^2}{b_w^2 }  \right]
      =  - \frac{b_b}{b_w} \exp \left[ \frac{-2 \times 2.21^2}{1.64^2 }  \right]
  \label{eqn:ba}
\end{equation}
%
and the second condition consists of finding the total heat deposited to the flow from $s=0$ to $s=b_{L}$ and setting it equal to $J_{\rm b} \epsilon_{\rm b}$:
%
\begin{equation}
  \epsilon_{\rm b} J_{\rm b}= \int_{s=0}^{s=b_{L}} Q_{\rm b} {\rm d} s
\end{equation}
%
or
%
\begin{equation}
  \epsilon_{\rm b} J_{\rm b}=
   b_{a} b_{L}  +    \frac{b_b}{b_w} \int_{s=0}^{s=b_{L}}
\exp \left[ \frac{-2 \left( s- b_m \right)^2}{b_w^2 }  \right]
   {\rm d} s
\end{equation}
%
Then, substituting $b_a$ from Eq.\ (\ref{eqn:ba}) and isolating $b_b$:
%
\begin{equation}
  {b_b}=
\frac{
\epsilon_{\rm b} J_{\rm b} b_w
}{\mfd      \int_{s=0}^{s=b_L}
\exp \left[ \frac{-2 \left( s- b_m \right)^2}{b_w^2 }  \right]
   {\rm d} s
   - b_{L} \exp \left[ \frac{-2 \times 2.21^2}{1.64^2 }  \right]
}
\end{equation}
%
where the RHS is all known since $J_{\rm b}$ is user-specified and $\epsilon_{\rm b}$ is set to:
%
\begin{equation}
  \epsilon_{\rm b} = 10^3 \left(\frac{b_{L} \overline{N}}{1.1 \times 10^{21}}  \right)^{1/1.7}
\end{equation}
%
where $b_{L}$ is user-defined and corresponds to the total length of the path along which air is ionized by the e-beam. The average number density is computed from the local number density as follows:
%
\begin{equation}
  \overline{N} = \frac{1}{b_{L}} \int_{s=0}^{s=b_{L}} N {\rm d} s
\end{equation}
%

In order to determine the power deposited by the e-beam, the following constants must be user-specified: the number density $N$ in 1/m$^3$, the average cost of ionization for an air particule $W_{\rm i}$ is in eV, the length of ionization $b_L$ is in meters, and the e-beam current density at the surface $J_{\rm b}$ in A/m$^2$, and the power deposited $Q_{\rm b}$ is in W/m$^3$. Since $Q_{\rm b} \sim \epsilon_{\rm b} J_{\rm b}/b_L$, and since $J_{\rm b}$ is in A/m$^2$ and $b_L$ is in m, it follows that $\epsilon_{\rm b}$ is the electron beam electromotive force in Volts.

Once $Q_{\rm b}$ is found through the above, it can be used to find the local ionization rate $Q_{\rm b}/(34~{\rm eV} \cdot e)$  where $e$ is the electron charge in Coulomb and 34 eV is the average cost of ionization for an air particule  \citen{misc:1968:peterson}.

\section{Kinetic Model}

Following \cite[Ch. 5]{book:1991:raizer}, let us first consider an electron distribution function $f(t,x,y,z,v_x,v_y,v_z)$ which is such that $f\, dx \, dy \, dz \, dv_x \, dv_y \, dv_z$ gives the number of electrons at a time $t$ within the volume $dx \, dy \, dz$ that have an $x$-velocity component located between $v_x$ and $v_x+dv_x$, a $y$-velocity component located between $v_y$ and $v_y+dv_y$, etc. The integral of $f$ over all velocities would yield the electron number density $N_{\rm e}$.

The change in time of $f$ (in a reference frame following the particules) can be written as:
%
\begin{equation}
  \frac{d f}{d t} = \left(\frac{\partial f}{\partial t}  \right)_{\rm coll}
\end{equation}
%
where the RHS refers to the change in time of the distribution function due to collisions. We can expand the total derivative on the LHS as follows:
%
\begin{equation}
  \frac{d f}{d t} = \frac{\partial f}{\partial t}
                  + \frac{\partial f}{\partial x} \frac{dx }{dt}
                  + \frac{\partial f}{\partial y} \frac{dy }{dt}
                  + \frac{\partial f}{\partial z} \frac{dz }{dt}
                  + \frac{\partial f}{\partial v_x} \frac{d v_x }{dt}
                  + \frac{\partial f}{\partial v_y} \frac{d v_y }{dt}
                  + \frac{\partial f}{\partial v_z} \frac{d v_z }{dt}
\end{equation}
%
or,
%
\begin{equation}
  \frac{d f}{d t} = \frac{\partial f}{\partial t}
                  + v_x \frac{\partial f}{\partial x}
                  + v_y \frac{\partial f}{\partial y}
                  + v_z \frac{\partial f}{\partial z}
                  + F_x \frac{\partial f}{\partial v_x}
                  + F_y \frac{\partial f}{\partial v_y}
                  + F_z \frac{\partial f}{\partial v_z}
\end{equation}
%
where $F$ is the specific force (the force per unit mass) acting on the electrons. Substituting the latter in the former, we get the Boltzman equation:
%
\begin{equation}
  \frac{\partial f}{\partial t}
                  + v_x \frac{\partial f}{\partial x}
                  + v_y \frac{\partial f}{\partial y}
                  + v_z \frac{\partial f}{\partial z}
                  + F_x \frac{\partial f}{\partial v_x}
                  + F_y \frac{\partial f}{\partial v_y}
                  + F_z \frac{\partial f}{\partial v_z}
 = \left(\frac{\partial f}{\partial t}  \right)_{\rm coll}
\label{eqn:boltzman}
\end{equation}
%
The main difficulty is solving the Boltzman equation (other than the fact that it is 6-dimensional) is the determination of the collision terms on the RHS. See Chapter 5 in Ref.\ \cite{book:1991:raizer} for more information on this point. However, the collision terms become simplified should we neglect the electron flux in the ``backward'' direction (the electrons that bounce off the neutrals in the direction opposite of the one in which the e-beam points). I.e., this is equivalent to assuming that the elastic collisions are negligible compared to the non-elastic collisions. An example of a modified 1D Boltzmann equation in which the elastic collisions are neglected can be found in Eq.\ (8) in Ref.\ \cite{tvt:1989:raizer}.

\bibliographystyle{warpdoc}
\bibliography{local}

\end{document}


\chapter{Alloy analysis}

\begin{lstlisting}
module Travlendar

open util/integer as Integer
open util/boolean as boolean
// impose ordering among relations
open util/ordering[Date] as DO
open util/ordering[Time] as TO
open util/ordering[ScheduledAppointment] as SAO

-----------------------------------------------
sig Date{}
-----------------------------------------------
sig Time{}

pred TimeIsBetween[t:Time, t1:Time, t2:Time] {
	t in TO/nexts[t1] and t in TO/prevs[t2]
}
-----------------------------------------------
sig TimeSlot{
	start: one Time,
	end: one Time
}

-----------------------------------------------
sig OptimizingCriteria{}{
	this in Schedule.optimizingCriteria
}

-----------------------------------------------
sig User{
	hasCar: one Bool,
	hasBike: one Bool,
	hasPass: one Bool
}{
	this in Appointment.user
}

-----------------------------------------------
sig Schedule{
	date: one Date,
	wakeUpTime: one Time,
	optimizingCriteria: one OptimizingCriteria,
	constraints : set ConstraintOnSchedule,
	initialNumberOfPeopleInvolved : one Int
}{
	initialNumberOfPeopleInvolved >= 1
}

// there aren't schedules that doesn't have at least one ScheduledAppointment
fact NoScheduleUnlinked{
	all s:Schedule | s in ScheduledAppointment.schedule
}

// we can't have two different schedule for the same day with the same optimizing criteria
fact DifferentOptimizingCriteriaForScheduleOfTheSameDay{
	all s,s1 : Schedule| s != s1 and s.date = s1.date  => s.optimizingCriteria != s1.optimizingCriteria
}

// no schedule of scheduled appointment belonging to different users
fact belongingCoeherence{
	no sa1,sa2 :ScheduledAppointment | sa1.schedule = sa2.schedule
	and sa1.appointment.user != sa2.appointment.user
}

-----------------------------------------------
sig Appointment{
	user: one User,
	date: one Date,
	startingTime: lone Time,
	timeSlot: lone TimeSlot,
	constraints: set ConstraintOnAppointment,
	variationNumberInvolvedPeople: one Int
}
{
	// startingTime and timeSlot are mutually exclusive
	startingTime = none <=> timeSlot != none
	timeSlot = none <=> startingTime != none
}

// all appointments must be scheduled into some ScheduledAppointment
fact AppointmentAssociationCoherence {
	all a : Appointment | a in ScheduledAppointment.appointment
}

-----------------------------------------------
sig ScheduledAppointment{
	schedule: one Schedule,
	date: one Date,
	appointment: one Appointment,
	startingTravelTime: one Time,
	ETA: one Time,
	endingTime: one Time,
	numberOfInvolvedPeople: one Int,
	weather: one Weather
}
{	startingTravelTime in TO/prevs[endingTime]
//	ETA in TO/nexts[startingTravelTime] and ETA in TO/prevs[endingTime]		// ETA must be between starting and ending times
	TimeIsBetween[ETA, startingTravelTime, endingTime]
	date = schedule.date
	date = appointment.date
	numberOfInvolvedPeople >= 0
	startingTravelTime in TO/nexts[schedule.wakeUpTime]
}

// appointments starting time ordering coherence
fact ScheduledAppointmentsOrderingConsistence {
	all a1,a2 : ScheduledAppointment | a1.schedule = a2.schedule and a1 in SAO/prevs[a2]
	=> a1.startingTravelTime in TO/prevs[a2.startingTravelTime]
}

// if two scheduled appointments are relative to the same appointment then they must
//belong to different schedules
fact AppointmentOnMultipleSchedules {
	all s1,s2 : ScheduledAppointment | s1 != s2 and s1.appointment = s2.appointment
	=> s1.schedule != s2.schedule
}

// Starting time of a scheduled appointment must coincide with the starting time of the
//non-scheduled appointment if timeSlot is not specified
fact StartingTimeCoherence{
	all s : ScheduledAppointment | s.appointment.timeSlot = none =>
	s.startingTravelTime = s.appointment.startingTime
}

// The number of involved people in each travel is coherent with the number of seats of each
//travel mean used for it
fact numberOfPeopleInvolvedCoherentWithSeats{
	no p : Path | p.travelMean.seats < p.source.numberOfInvolvedPeople
}

// The selected travel mean for a path can provide enough seats for the people involved in the appointment
fact coherenceOnNumberOfInvolvedPeople{
	all sa:ScheduledAppointment, a:Appointment, s:Schedule |
	sa in schedule.s and a in sa.appointment =>
	sa.numberOfInvolvedPeople = add[s.initialNumberOfPeopleInvolved,
		sum e : SAO/prevs[sa] | e.appointment.variationNumberInvolvedPeople]
}

----------------------------------------------------
sig Path{
	lenght: one Int,
	source: one ScheduledAppointment,
	dest: one ScheduledAppointment,
	travelMean: one TravelMean
}
{
	lenght >= 0
	source != dest
	source.schedule = dest.schedule
}

// No path linking to scheduled appointment belonging to different user
fact SchedulePathBelongingConsistency{
	all p:Path | p.source.appointment.user = p.dest.appointment.user
}

// No scheduled appointment unreachable
fact AllScheduledAppointmentCanBeReachedByPaths{
	all s : ScheduledAppointment | s in (Path.source + Path.dest)
}

// No path belonging to a user with a travel mean that he/she doesn't own
fact TravelMeanConsistency{
	no p:Path| (p.travelMean = Bike and p.source.appointment.user.hasBike in False) or
	(p.travelMean = Car and p.source.appointment.user.hasCar in False)
}

// No path linking two scheduled appointments that are not sequential
fact PathOrderConsistency{
	all p:Path | p.source = SAO/prev[p.dest]
}

-----------------------------------------------------
abstract sig Constraint{
	travelMean: one TravelMean,
	maxTravelDistance: one Int // with travel mean above
}
{maxTravelDistance >= 0} //if 0 implies that the travel mean is deactivated

sig ConstraintOnAppointment extends Constraint{
}
{maxTravelDistance = 0}

sig ConstraintOnSchedule extends Constraint{
	weather: set Weather,
	timeSlot: lone TimeSlot
	// in which travel mean can't be used
}{
	weather != none => maxTravelDistance = 0
	weather != none => (timeSlot .start = timeSlot.end)
}

fact NoConstraintUnlinked{
	all c : Constraint | c in Appointment.constraints or c in Schedule.constraints
}
-----------------------------------------------
sig StrikeDay {
	strikeDate: one Date,
	strikingTimeSlot: one TimeSlot,
	strikingTravelMeans: some PublicTravelMean
}
-----------------------------------------------
abstract sig Weather{}

one sig  Sunny extends Weather{}
one sig Snowy extends Weather{}
one sig Rainy extends Weather{}
one sig Foggy extends Weather{}
one sig Cloudy extends Weather{}

fact NoWeatherUnlinked {
	all w : Weather | w in ConstraintOnSchedule.weather
}

------------------------------------------------
abstract sig TravelMean{
	seats: one Int
}
//100 seats encode an unbounded number of seats

abstract sig PublicTravelMean extends TravelMean{}
abstract sig SharedTravelMean extends TravelMean{}
abstract sig PrivateTravelMean extends TravelMean{}

sig Train extends PublicTravelMean{}{
	seats=100
}
sig Bus extends PublicTravelMean{}{
	seats=100
}
sig Tram extends PublicTravelMean{}{
	seats=100
}
sig Taxi extends PublicTravelMean{}{
	seats=3
}
sig Underground extends PublicTravelMean{}{
	seats=100
}
sig CarSharing extends SharedTravelMean{}{
	seats=5
}
sig BikeSharing extends SharedTravelMean{}{
	seats=1
}
sig Bike extends PrivateTravelMean{}{
	seats=1
}
sig Car extends PrivateTravelMean{}{
	seats=5
}
sig Walking extends PrivateTravelMean{}{
	seats=100
}

-----------------------------------------------
// if a schedule appointment is on a schedule, it must be in the list of scheduled appointments of that schedule
ScheduledAppointmentAndScheduleBiunivocity : check{
	all a : ScheduledAppointment | all s : Schedule | a.schedule = s => a in schedule.s
} for 8
-----------------------------------------------------
// if a scheduled appointment is in a schedule, they must have the same date
EveryAppoinmentOfScheduleIsInItsDay : check {
	all s : Schedule, a : ScheduledAppointment | a.schedule = s => s.date = a.date
}
for 8
-----------------------------------------------------
ScheduleIsOwnedByOnlyOneUser : check {
	all s1,s2:ScheduledAppointment | s1.schedule = s2.schedule implies s1.appointment.user = s2.appointment.user
} for 8
-----------------------------------------------------
// all scheduled appointment date must be equal to its original appointment date
AppointmentDateIsEqualToItsScheduledAppt : check {
	all a : ScheduledAppointment | a.date = a.appointment.date
} for 8
-----------------------------------------------------
//source and dest of each path must refer to the same schedule
sourceDestOfPathCoherence : check {
	all p:Path | p.dest.schedule = p.source.schedule
} for 8
-----------------------------------------------------
// scheduled appointments are ordered according to their path precedence
PathOrderConsistency : check {
	all p:Path | p.dest = SAO/next[p.source]
} for 8
-----------------------------------------------------
/* if two appointments belong to different user then they should belong to different
    schedule since a schedule belongs to only one user */
TwoAppointmentsWithSameUserNoInSameSchedule : check {
	all a,a1:Appointment | a.user != a1.user => appointment.a.schedule != appointment.a1.schedule
} for 8
-----------------------------------------------------

// return all the paths of the specified schedule
fun pathOfSchedule [s : Schedule] : set Path {
	(source+dest).(schedule.s)
}

// return the total distance travelled in the specified schedule with a specific travel mean
fun distanceTravelledWithMeanInSchedule [s : Schedule, t: TravelMean] : Int {
	sum e : (pathOfSchedule[s]) & (travelMean.t) | e.lenght
}

// checks if the specified path links two scheduled appointment in the given schedule
pred doesPathBelongToSchedule [s:Schedule, p:Path] {
	some sa:ScheduledAppointment | sa in (Path.source + Path.dest) and sa.schedule=s
}

// checks if a schedule satisfying the specified constraint exists
pred doesConstraintSatisfySchedule (c : ConstraintOnSchedule){
	some s : Schedule | all p:Path, st:StrikeDay |
	doesPathBelongToSchedule[s, p] and
	p.travelMean  != c.travelMean or (p.travelMean = c.travelMean
					and (schedule.s).weather not in c.weather
					and ((c.timeSlot.start = c.timeSlot.end)
						or ((schedule.s).ETA in TO/prevs[c.timeSlot.start]
						or (schedule.s).startingTravelTime in TO/nexts[c.timeSlot.end] ))
					and (s.date = st.strikeDate and (
								TimeIsBetween[p.source.startingTravelTime, st.strikingTimeSlot.start, st.strikingTimeSlot.end] or
								TimeIsBetween[p.source.ETA, st.strikingTimeSlot.start, st.strikingTimeSlot.end]
							)
								implies p.travelMean not in st.strikingTravelMeans)
					and (distanceTravelledWithMeanInSchedule[s,c.travelMean] < c.maxTravelDistance )
				)
}

// checks if a scheduled appointment satisfying the specified constraint exists
pred doesConstraintSatisfiesAppointment(c : ConstraintOnAppointment){
	some s: Schedule | all sa:ScheduledAppointment, a: Appointment, p: Path |
		sa in schedule.s and sa.appointment = a and p.dest=sa
		=> p.travelMean != c.travelMean
}

pred NoOverlappingScheduledAppointmentInSchedule (s : Schedule) {
	all s1,s2 : ScheduledAppointment |  s1.schedule = s and  s2.schedule = s and s1 != s2
	=> s1.endingTime in TO/prevs[s2.startingTravelTime] - s2.startingTravelTime
	or s2.endingTime in TO/prevs[s1.startingTravelTime] - s1.startingTravelTime
}

// checks if a valid schedule exists (see def. on RASD document)
pred validSchedule {
	some s: Schedule | all cs:ConstraintOnSchedule, ca:ConstraintOnAppointment  |
	cs in s.constraints and ca in (schedule.s).appointment.constraints
	and doesConstraintSatisfySchedule[cs] and doesConstraintSatisfiesAppointment[ca]
	and NoOverlappingScheduledAppointmentInSchedule[s]
}
-----------------------------------------------------

// new appointment insertion action
pred InsertNewAppointment[u,u1 : User, a1 : Appointment]{
	//precondition
	all a : Appointment | a.user = u and a1.user != a.user
	//postcondition
	user.u1 = user.u + a1
}

// checks if the insertion of a new appointment is correct
InsertNewAppointmentIsCorrect : check {
	all u,u1 : User, a,a1 : Appointment | a.user = u and a1.user != a.user and InsertNewAppointment[u,u1,a1]
	=> a1 not in user.u and a1 in user.u1
} for 8
-----------------------------------------------------

// editing appointment action
pred ModifyAppointment[a,a1 : Appointment, u : User]{
	//precondition
	a in user.u and a1 not in user.u
	//postcondition
	a1 in user.u and a not in user.u
}

// checks if the modification of a new appointment is correct
ModifyAppointmentIsCorrect : check {
	all u : User, a,a1 : Appointment | a in user.u and a1 not in user.u and ModifyAppointment[a,a1,u]
	=> a1 in user.u and a not in user.u
} for 8

pred show(){}

run { show and validSchedule} for 6

\end{lstlisting}

\subsection{Generated World}
\begin{figure}[H]
\begin{center}
\includegraphics[height=\textheight, keepaspectratio]{"images/generatedWorld".png}
\caption{Generated world}
\end{center}
\end{figure}

\subsection{Execution Results}
\begin{figure}[H]
\begin{center}
\includegraphics[width=\textwidth, keepaspectratio]{"images/alloyResult".jpg}
\caption{Execution results}
\end{center}
\end{figure}


\chapter{Conclusion and Outlook}\label{ch:summary}
%
The PhotonStream offers a great number of new opportunities for future IACT
analyses, only a few of which have been investigated in this analysis.
In this work a very first analysis of the new IACT data representation
PhotonStream is performed. It exclusively uses openly accessible data from the
FACT open data sample~\cite{fact-data} and only open source software tools. The
necessary data handling and feature generation tools have been implemented in
the openly available \texttt{FeatureStream}~\cite{FeatureStream}. The extensive
investigations on the different characteristics of the new representation and
the implications for the analysis of PhotonStream data resulted in several
remarks and yield the basis for a new, dedicated PhotonStream analysis.

The PhotonStream opens new ways to represent IACT data and to parametrize air-
shower events for analysis. The single photon extraction results in a larger
number of reconstructed photons and enables a more efficient and exact way to
reconstruct air-shower events. The resulting images still suit the classical LP
representation very well. The new per photon timing information impacts every
analysis step. The density-based cleaning DBSCAN is a completely new way to
clean air-shower events of the background. The resulting data set is
characterized by brighter images with larger distributions along the spatial
coordinates, that require new thresholds for cleanings and special treatment of mismatches between simulations and observations. Generally, the PhotonStream data contains the
information neccessary to generate the classical Hillas features and additional
fetaures used in FACT analyses. Additionally, the new three-dimensional
representation offers a great number of new features to be engineered. The
classical features have been investigated in this work and show significant
mismatches to the respective simulations. The mismatches have a strong impact
on the analysis results and need to be accounted for by quality cuts or
improving the simulation quality.

The timing information within the PhotonStream shows even bigger mismatches and
therefore needs better simulation quality and optimized analysis methods to
boost the analysis performance. The different topology of cleaned air-shower
events strongly affects the used machine learning techniques for the origin
reconstruction. The orientation of air-showers shows significant mismatches,
making the reconstrcution error very large and the reconstructed signal source
blurred. The disp-method used for the origin reconstruction does not perform
very well, even on the MC simulations it is trained on. The separation of gamma
air-showers and hadronic air-showers is working very well on MC simulations.
The performance measures yield better results than the comparative FACT-Tools
analysis. The energy estimation of the gamma air-showers yields a significantly
better performance than the FACT-Tools analysis. Both these results indicate
that the cleaned air-shower clusters show a better agreement with the true air-
showers and therefore can better be reconstructed. The used unsupervised
clustering algorithm DBSCAN shows much better results on PhotonStream data than
the classical threshold based cleaning. An improvement of the mismatches
between observations and MC simulations via this algorithm's hyperparameters
could not be achieved. It seems that the data needs to be cleaned of noises and
the different topologies of observed data and MC simulations need to be
understood better to improve. To achieve this dedicated studies for quality
cuts and the noises in the data samples are needed. However, since the two-dimensional cleaning on the shower pixels has a poor performance the three-dimensional space is the most promising space for event cleaning.

The PhotonStream data representation shows very good tendencies for energy
reconstruction and gamma hadron separation and therefore promising
opportunities for the field of imaging air Cherenkov telescopes. It is not only
a very intuitive data representation but also one suited very well for
analysing air-shower images, as is done by FACT, representing a great new development in IACT data.


\documentclass[main.tex]{subfiles}
\begin{document}

\marginpar{Thursday \\ 2020-4-02, \\ compiled \\ \today}

On the Right Hand Side, instead, since the stress energy tensor is given by
%
\begin{align}
T^{i}_{j} = \sum _{\text{all species \(a\)}} g_{a}
\int  \frac{ \dd[3]{p}}{(2 \pi )^3}
\frac{p^{i}p_{j}}{E_{a}(p)} f_a (\vec{x}, \vec{p}, t)
\,
\end{align}
%
we obtain
%
\begin{align}
\qty(\hat{k}_{i} \hat{k}^{j} - \frac{1}{3} \delta^{i}_{j})
\widetilde{T}^{i}_{j}
= \sum _{a} g_{a} \int  \frac{\dd[3]{p}}{(2\pi )^3} \frac{p^2\mu^2 - p^2/3}{E_a (p)} f_a (\vec{p})
\,,
\end{align}
%
where \(\mu \) is the cosine of the angle between \(\hat{k}\) and \(\hat{p}\).

But we know that
%
\begin{align}
\mu^2 - \frac{1}{3} = \frac{2}{3} P_2 (\mu )
\,,
\end{align}
%
where \(P_2\) is the second Legendre polynomial; recall that these Legendre polynomials are precisely those used in the definition of the multipole moments \(\theta_{k}\) [definition ref].
Using this, we get
%
\begin{subequations}
\begin{align}
\qty(\hat{k}_{i} \hat{k}^{j} - \frac{1}{3} \delta^{i}_{j})
\widetilde{T}^{i}_{j} &=
-2 \int_{0}^{ \infty } \frac{ \dd{p} p^2}{2\pi^2} \pdv{f^{(0)}}{p}p^2
\int_{-1}^{1} \frac{ \dd{\mu }}{2} \frac{2}{3} P_2 (\mu ) \theta (\mu )  \\
&= 2 \frac{2}{3} \widetilde{\theta}_{2} \int_{0}^{ \infty } \frac{ \dd{p}p^2}{2 \pi^2} p^2 \pdv{f^{(0)}}{p} = - \frac{8}{3} p_{\gamma } \widetilde{\theta}_{2}
\,,
\end{align}
\end{subequations}
%
we have integrated by parts, and the quadrupole contribution has vanished by the orthogonality property of the Legendre polynomials.

[why is it \(\rho_{\gamma, 0}\)? we are not computing it now, right?]
We are writing \(\rho_{\gamma }\) and the such we really mean \(\rho_{\gamma }^{(0)}\), an average at monopole order.

In a universe without neutrinos, we would have no difference between \(\Phi \) and \(\Psi \).

\todo[inline]{What would be the geometric meaning of the difference between \(\Phi \) and \(\Psi \)?}

So, if we collect terms we find that the whole Einstein equation reads:
%
\begin{align} \label{eq:radiation-temperature-perturbation-term}
k^2 \qty(\Phi - \Psi ) &= - 32 \pi G a^2
\underbrace{\qty[\rho_{\gamma } \widetilde{\theta}_{2} + \rho_{\nu } \widetilde{w}_{2}] }_{\rho_{r} \theta_{r, 2}}
\,.
\end{align}

Since we have added the neutrino contribution, we are accounting for all of radiation.
See Dodelson \cite[page 99]{dodelsonModernCosmology2003}.

The Einstein 00 equation \eqref{eq:einstein-00-perturbed-equation} can be written in Fourier space as
%
\begin{align}
k^2 \Phi + 3 \frac{\dot{a}}{a} \qty(\dot{\Psi} + \frac{\dot{a}}{a} \Phi )
= -4 \pi G a^2 \qty(\rho_{m } \delta_{m} + 4 \rho_{r} \widetilde{\theta}_{r, 0})
\,,
\end{align}
%
where
%
\begin{align}
\rho_{m} \delta_{m} = \rho_{m} \delta + \rho_{b} \delta_{b}
\qquad \text{and} \qquad
\rho_{r} \theta_{r, 0} = \rho_{\gamma } \theta + \rho_{\nu } w
\,.
\end{align}

Dark Matter and baryons do not contribute to the quadrupole (at first order).

Recall that the EFE are not independent: we could also have chosen the \(^0_i\) one: those components of the stress-energy tensor are given by
%
\begin{align}
T^{0}_{i} = \sum _{\alpha } a \rho_{\alpha }
\int \frac{ \dd[3]{p}}{(2\pi )^3} p_{i} f_\alpha  (\vec{x}, \vec{p}, t)
\,,
\end{align}
%
so with reasoning similar to what was done before we find that the \(^0_{i}\) equation reads
%
\begin{align}
\dot{\Psi} + a H \Phi = - \frac{4 \pi G a^2}{ik}
\qty[\rho_{m} v_{m} - 4 i \rho_{r} \theta_{r, 1}]
\,,
\end{align}
%
where
%
\begin{align}
\rho_{m} v_{m} = \rho _{\text{dm}} v + \rho_{b} v_{b}
\qquad \text{and} \qquad
\rho_{r} \theta_{r, 1} = \rho_{g} \theta_1 + \rho_{\nu } w_{1}
\,.
\end{align}

In this case then we would find
%
\begin{align}
k^2 \Psi = -4\pi G a^2
\qty[\rho_{m} \delta_{m} + 4 \rho_{r} \theta_{r, 0}
+ \frac{32H}{k} \qty(i \rho_{m} v_m + 4 \rho_{r} \theta_{r, 1})]
\,.
\end{align}

\subsubsection{Tensor models}

Now that we have found the equation of motion of the scalar field, let us discuss tensor modes: we can find the equation of motion for these starting from the transverse traceless part of the \(^i_{j}\) equation. We set \(\chi_{ij} = 2 h_{ij}\), and with this we can write the traceless Christoffel symbols:
%
\begin{subequations}
\begin{align}
\Gamma^{0}_{ij} &= \frac{1}{2} \dot{h}_{ij} + \frac{\dot{a}}{a} h_{ij}  \\
\Gamma^{i}_{0j} &= \frac{1}{2} \dot{h}  \\
\Gamma^{i}_{jk} &= h^{i}_{(j,k)} - \frac{1}{2} \tensor{h}{_{jk}^{,i}}  \\
\,,
\end{align}
\end{subequations}
%
which yield the traceless Einstein tensor
%
\begin{align}
G^{i}_{j} &= \frac{\dot{a}}{a} \dot{h}^{i}_{j}
- \frac{1}{2} \nabla^2 h^{i}_{j} + \frac{1}{2} \ddot{h}^{i}_{j}
\,.
\end{align}

In order to get the correct contribution on the RHS we need to project it: we apply the projection operator
%
\begin{align}
\tensor{\mathcal{P}}{^{k}_{i}^{j}_{l}} =
\mathcal{P}^{k}_{i} \mathcal{P}^{j}_{l}
- \frac{1}{2} \mathcal{P}^{k}_{l} \mathcal{P}^{j}_{i}
\,,
\end{align}
%
where
%
\begin{align}
\mathcal{P}^{i}_{j} = \delta^{i}_{j} - \hat{k}^{i}\hat{k}_{j}
\,.
\end{align}

\todo[inline]{The work \cite[]{carboneUnifiedTreatmentCosmological2005} is mentioned, but the work only the two-index \(\mathcal{P}\) is defined (in position space and not in momentum space, but it's the same) (eq. 13); but the four-index combination is not explicitly written\dots}

What is usually done is to completely neglect the right hand side of the EFE (since it is second order in \(v/c\)?), to get the homogeneous expression
%
\begin{align}
\ddot{h}_{ij} + 2 \frac{\dot{a}}{a} h_{ij} - \nabla^2 h_{ij} = 0
\,.
\end{align}

We can express the tensor perturbation in Fourier space as
%
\begin{align}
h_{ij} = \frac{1}{(2\pi )^3} \int \dd[3]{k} e^{- \vec{k} \cdot \vec{x}}
h_{ij} (\vec{k}, t)
\,,
\end{align}
%
where \(h_{ij} (\vec{k}, t)\) can be decomposed into the two basis polarizations \(+\) and \(\times \)
so that our equation will read
%
\begin{align}
\ddot{h}_{\epsilon } + 2 \frac{\dot{a}}{a} \dot{h}_{\epsilon } + k^2 h_{\epsilon } = 16 \pi G \pi_{\epsilon }
\,,
\end{align}
%
where \(\epsilon = +, \times \) (see Pritchard \& Kamionkowski \cite[]{pritchardCosmicMicrowaveBackground2005}).
Here \(\pi_{\epsilon } \) is the traceless tensor part of the stress-energy tensor, which contains a negligible contribution from photons and a contribution from neutrinos which has a non-negligible effect on gravity waves; the latter leads to
%
\begin{align}
\ddot{h}_{\epsilon } + 2 \frac{\dot{a}}{a} \dot{h}_{\epsilon }
+ k^2 h_{\epsilon } =
-24 f_{\nu }(\eta ) \qty(\frac{\dot{a}}{a})^2
\int_{0}^{\eta } \dd{\widetilde{\eta} } K (k (\eta - \widetilde{\eta})) \dot{h}_{\epsilon }(\widetilde{\eta})
\,,
\end{align}
%
where \(f_{\nu } = p_{\nu }^{(0)} / p^{0}\) and
%
\begin{align}
K(s) = - \frac{\sin s}{s^3} - \frac{3 \cos s}{s^{4}}
+ \frac{3 \sin s}{s^{5}}
\,.
\end{align}

This is a damping, not a source term

\todo[inline]{Is it? as long as \(s < 5\), sure, but is that guaranteed? around \(s = 5.5\) the function changes sign\dots}

Solutions to the tensor perturbation equations are given by
%
\begin{subequations}
\begin{align}
h _{\text{rad}} (\eta ) &= h(0) \dots  \\
h _{\text{mat}} (\eta ) &= 3 h(0) \dots
\,.
\end{align}
\end{subequations}

\subsubsection{Initial conditions}

Plot: comoving scale versus log of scale factor. From Lucchin-Coles? \cite[]{colespCosmology2002}.

We can say that the universe starts at the end of inflation.

Let us put the curvature \(k\) back into the equations we found.
At early times, but after inflation, (???) are outside the horizon
so we have \(k \eta \ll 1\) and \(\dot{\tau} \rightarrow \infty \), so the initial conditions look like
%
\begin{subequations}
\begin{align}
\dot{\theta}_{0} &= \dot{\Psi}
\qquad \qquad
\dot{w}_{0} = \dot{\psi} \\
\dot{\delta} &= 3 \dot{\psi}
\qquad  \qquad
\dot{\delta}_{b} = 3 \dot{\psi}
\,;
\end{align}
\end{subequations}
%
these equations assume that all the multipoles from the dipole onward are suppressed.

This is called the ``separate universe'' approximation. Locally, the universe looks like a separate FRLW curved universe.
\todo[inline]{Clarify}

Also, the dipole must be much smaller than the monopole, and we must have the velocity of the baryons \(v_b\) be equal to \(-3 i \theta \) because of tight coupling.

In this limit, photons behave like a fluid, and this fluid has the same behavior as the \(e^{-}\) fluid.

We assume that the neutrino quadrupole is negligible.
We are considering a radiation-dominated epoch, so we neglect matter.

We have then \(\Psi = \Phi \), and
%
\begin{align}
3 \frac{\dot{a}}{a} \qty(\dot{\Psi} + \frac{\dot{a}}{a} \Phi ) = - 16 \pi G a^2 \rho_{r} \Theta_{r, 0}
\,,
\end{align}
%
which becomes
%
\begin{align}
\frac{\dot{\Phi}}{\eta } + \frac{1}{\eta^2} \Phi
= - \frac{16 \pi G}{3} \rho_{r} a^2 \qty(\frac{\rho_{\gamma }}{\rho_{r} \theta_{0} + \frac{\rho_{\nu}}{\rho_{r}} w_0 })
\,.
\end{align}

Then the phase space distribution looks like
%
\begin{align}
f_{\nu } = \frac{\rho_{\nu }}{\rho_{\gamma } + \rho_{\nu }}
\,,
\end{align}
%
since \(a \propto \eta \) and \(\rho_{r} \propto a^{-4} \) and \(\rho_{r} = \rho_{\gamma } + \rho_{\nu }\).

If we assume \(\dot{\theta}_{0} = \dot{w}_{0} = \dot{\Phi}\). This finally yields
%
\begin{align}
\ddot{\Phi} \eta + 4 \dot{\Phi} = 0
\,,
\end{align}
%
which has a vanishing solution, and a constant solution. We choose the latter.

We require the perturbation to be isentropic (see \cite[]{colespCosmology2002}). This yields
%
\begin{align}
\delta_{m} = 3 \theta_0 = \frac{3}{4} \delta_{r}
\,.
\end{align}

Why do we assume this? If the perturbation is sourced by a single scalar field, then we can only consider a single mode: the isentropic mode.

With these, the 00 EFE becomes
%
\begin{align}
\Phi = -2 \qty[(1- f_{\nu }) \theta_0 + f_{\nu }w_0 ]
\,,
\end{align}
%
where we must have \(f_{\nu } = \const\).

Adiabaticity implies
%
\begin{align}
\phi = - 2 \theta_0 \implies \delta = - \frac{3}{2} \Phi
\,.
\end{align}

If we had kept the neutrino quadrupole, we would have gotten
%
\begin{align}
\Phi = \Psi \qty(1 + \frac{2 f_{\nu }}{5})
\,.
\end{align}

\(f_{\nu }\) is a constant at any order: it is a number fixed by the number of neutrino species.

\end{document}


\documentclass[10pt]{article}

\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{xspace}
\usepackage[binary-units]{siunitx}
\sisetup{round-mode=places,round-precision=2}
\usepackage[USenglish]{babel}
\usepackage{graphicx}
\usepackage{longtable}

\usepackage[font=footnotesize]{caption}

\usepackage{lua-utils}

\newcommand*{\eg}{e.\,g.\@\xspace}
\newcommand*{\ie}{i.\,e.\@\xspace}

%% Lua magic

\directlua{dofile("variables_for_tex.lua")}

\title{IPFS Crawler -- Report}

\begin{document}
\maketitle

\section{Basic Statistics}
\label{sec:basic}

This is an automatically-generated report on the IPFS-crawl from \lraw{crawlStartDate} until \lraw{crawlEndDate}.
In the \SI{\lraw{crawlPeriod}}{\day} we performed a total of \lraw{numCrawls} crawls in total, each of which took \SI{\lraw{avgCrawlDuration}}{\minute} to complete, on average.
We found an average number of \lraw{avgNumberOfNodes} nodes per crawl; the crawler was able to connect to \SI{\lraw{avgOnline}}{\percent} of them.
In total, we found \lraw{totalDistinctNodes} distinct nodes during the whole crawl period.
%
\begin{figure}[ht]
\centering
  \includegraphics[width=.7\textwidth]{figures/num_nodes.png}
  \caption{Number of nodes over time, distinguished by all and reachable (=answered to our query) nodes. Times are in UTC.}
  \label{fig:num_nodes}
\end{figure}
%
The number of nodes over time is depicted in Figure~\ref{fig:num_nodes}, where time is on the x-axis and the node count on the y-axis.
The figure distinguishes between all nodes and nodes that were reachable, \ie, the crawler was able to establish a connection to these nodes.

Using the crawl data, we interpolate the uptime of nodes, the results of which are gathered in Table~\ref{tab:sessionLengths}.
To this end, we use a simple assumption: if we see a node in consecutive crawls, we assume that it was online in the meantime.
Therefore, the crawl duration (\SI{\lraw{avgCrawlDuration}}{\minute}) is the granularity.
The table depicts the inverse cumulative distribution of uptimes: each row gives the number of sessions (and total percentage) that were \emph{longer} than the given duration.
%
\begin{table}
  \center
  \input{tables/session_lengths}
  \caption{Inverse cumulative session lengths: each row gives the number of sessions (and total percentage) that were \emph{longer} than the given duration.}
  \label{tab:sessionLengths}
\end{table}
%

We also gather agent versions, the results of which are depicted in Figure~\ref{figs:agent_version_distribution}.
It shows the ten agent versions with the highest popularity, according to the data.
In particular, the average number of peers with that agent version per crawl is shown (similar to the country distribution in Tab.~\ref{tab:geoip}).
The complete table for all agent versions and their counts can be found in the Appendix.
%
\begin{figure}[ht]
  \centering
  \includegraphics[width=.7\textwidth]{figures/agent_version_distribution.png}
  % \input{figures/agent_version_distribution}
  \caption{Agent version distribution, the top ten versions include \SI{\lraw{AgentVersionIncludeTruncatedPercentage}}{\percent}} of all seen agent versions.
  \label{figs:agent_version_distribution}
\end{figure}

\section{Node Distribution over Countries and Protocol Usage}
\label{sec:eval_country_distribution}
%
\begin{table}[htb]
\input{tables/geoIP_statistics}
\caption{The top \lraw{topNNodes} countries per crawl, differentiated by all discovered nodes and nodes that were reachable. Depicted is the average count per country per crawl as well as confidence intervals.}
\label{tab:geoip}
\end{table}
%

Table~\ref{tab:geoip} depicts the top \lraw{topNNodes} countries, both for all discovered nodes and for nodes that were reachable by the crawler.
These \lraw{topNNodes} countries contain \SI{\lraw{geoIPtopNAllPercentage}}{\percent} \SI{\lraw{geoIPtopNReachablePercentage}}{\percent} in the case of reachable nodes) of the whole network.
The complete table can be found in the Appendix.

The table shows the mean count per country per crawl as well as \SI{95}{\percent} confidence intervals (assuming a student t distribution of observations).
Furthermore, the statistics are distinguished between all and only reachable peers.
This means a row should be read as follows: for country $c$ we have seen ``Count'' peers from that country per crawl, on average.
The ``Conf. Int.'' column shows how large the spread around this average value is.

Of all seen nodes \SI{\lraw{PercLocalIPs}}{\percent} of all nodes \emph{only} provide local or private IP addresses, thus making it impossible to connect to them.
This is in line with the default behavior of IPFS when operated behind a NAT.
When a node first comes online, it does not know its external IP address and therefore advertises the internal IP addresses to peers it connects to.
These entries enter the DHT, since IPFS aims to provide support for building private IPFS networks.
Over time, a node learns about its external multi-addresses (multi-addresses contain address and port from its peers.
An IPFS considers these observed multi-addresses reachable, if at least four peers have reported the same multi-address in the last 40 minutes and the multi-address has been seen in the last ten minutes.
This is never the case for symmetric NATs, which assign a unique external address and port for every connection, yielding a different multi-address for every connected peer.
%
\begin{table}[htb]
  \center
  \input{tables/protocol_stats}
  \caption{Network statistics: protocol capabilities among IPFS peers.}
  \label{tab:protocols}
\end{table}
%

IPFS supports connections through multiple network layer protocols; Table~\ref{tab:protocols} shows the prevalence of encountered protocols during our crawls.
If a node was reachable through multiple, say IPv4 addresses, we only count it as one occurrence of IPv4 to not distort the count.
This is aggregated over all crawls: if a peer supported, \eg, IPv6 at some time, then it will be counted as supporting IPv6 in general.
The protocols ``ipfs'' and ``p2p-circuit'' are connections through IPFS' relay nodes, ``dnsaddr'', ``dns4/6'' are DNS-resolvable addresses and ``onion3'' signals TOR capabilities.

\section{Overlay Graph Properties}
\label{sec:graph_properties}

Figures~\ref{fig:log_all_nodes_degree_distribution} and~\ref{fig:log_online_nodes_degree_distribution} depict the log-log in-degree distribution from the first crawl.
You can change the number in the \texttt{Makefile} to check different crawls; in the best case the distributions should not differ too much between individuals crawls.
We differentiate between all found nodes (Figure~\ref{fig:log_all_nodes_degree_distribution}) and only reachable nodes (Figure~\ref{fig:log_online_nodes_degree_distribution}).

If the the plotted points form (roughly) a straight line, then the in-degree distribution can be approximated by a power-law and the graph could be classified as scale-free.
This would be in line with prior measurements on other Kademlia systems.

\begin{figure}[!htb]
\centering
        \includegraphics[width=.8\textwidth]{figures/log_all_nodes_degree_distribution.png}
        % \input{figures/log_all_nodes_degree_distribution}
        \caption{In-Degree distribution from the first crawl including \emph{all} found nodes. Other crawls yielded similar shapes.}
        \label{fig:log_all_nodes_degree_distribution}
\end{figure}
\begin{figure}[!htb]
        \includegraphics[width=.8\textwidth]{figures/log_online_nodes_degree_distribution.png}
        % \input{figures/log_online_nodes_degree_distribution}
        \caption{The same distribution as Figure~\ref{fig:log_all_nodes_degree_distribution}, but only including \emph{reachable} (\ie, online) nodes.}
        \label{fig:log_online_nodes_degree_distribution}
\end{figure}

Table~\ref{tab:degreeStats} depicts the aggregated statistics on the degree of each crawls' graph.
Depicted is the mean, median, minimum and maximum degree, differentiated by in- and out-degree.
\input{tables/degree_stats}
%

\section*{Appendix}
\label{sec:appendix}

Full table of agent versions, complete version of Fig.~\ref{figs:agent_version_distribution}.
\input{tables/full_agent_version_tab}

Full table of countries, complete version of Tab.~\ref{tab:geoip}.
\input{tables/all_countries_geoIP_statistics}
\end{document}


\chapter{CCD and Detector Parameters}
In this chapter, I will deal with some miscellaneous but important topics about CCD or other types of detectors (e.g., there is no available CCD in infrared wavelength, so technically they're not CCD).

\section{Calibration Frames}
There are mostly three types of calibration frames in observational astronomy: bias, dark, and flat. Others inlcude comparison arc lamp images, various types of flat (dome, sky, etc). In this section, I referred HowellSB (2006) ``Handbook of CCD Astronomy'' 2e, chapter 4.

\subsection{Detector Readout}
To understand why we need the bias, which is a technical than scientific reason, we need to know how the detector is being read out, so I separated this part from the \cref{ss: calib-bias}. When a CCD is being read out\footnote{It is assumed you know how the CCD or photodetectors transfer electrons; if not, search for, e.g., CCD clock pattern.}, the electronics need to convert the number of (photo-)electrons into a digital unit (so-called ADU or DN), where the number of ADU is, by the definition, number of electrons divided by the electron gain.

But how can it count the number of electrons? Consider we have a voltmeter, attached to the `measuring position' (the output gate). By the clock pattern of CCD, the \textbf{correlated double sampling} (CDS) process is done as follows:
\begin{enumerate}
  \item Reset the output gate's voltage: $ V_\mathrm{out, 1} = V_\mathrm{reset} $.
  \item Measure $ V_\mathrm{out} $.
  \item Dump the electrons to the output gate, so the voltage will change by $ V_\mathrm{e} $.
  \item Measure the output gate's voltage again $ V_\mathrm{out, 2} $ (which is now $ V_\mathrm{reset} + V_\mathrm{e} $)
  \item Get $ \Delta V = V_\mathrm{out, 2} - V_\mathrm{out, 1} = V_\mathrm{e}$ (trivially the number of electrons is linearly related with this)
  \item Iterate this again and again until all the pixels are read out.
\end{enumerate}
The $ V_\mathrm{e} $ is then converted to ADU or DN by the Analog-to-Digital Converter (ADC).

What I want to emphasize here is that the processes number 1 (resetting) and 4 (dumping) are not perfect, so the voltage over time ($ V(t) $) graph is never a combination of linear lines as in \cref{fig:cds}. The resetting and dumping involves movement of electrons, which results in a damped oscillation in the $ V(t) $ graph. If we had plenty of time, we can just wait until it stablized; however, we need to read out pixels as soon as possible (consider reading out 1000 by 1000 pixel CCD: to read out it in 1 sec, we need to read each pixel in $ \SI{1}{\mu s} $!). If it takes long time, you may lose observation time, and the dark current at the first-read pixel and the last-read pixel may differ, etc. Therefore, it is impossible to avoid the \emph{readout noise}. For some CCDs, there are options you can choose to compromise these readout time and readout noise.

\begin{figure}
\centering
\includegraphics[width=0.5\linewidth]{figs/CDS}
\caption{CCD output and correlated doubling sampling. The vertical axis is the voltage and horizontal axis is time. You can see the voltage is not a combination of linear line segments, but noisy curves. (Image from Texas Instruments, 2018: \href{ti.com/lit/an/snaa322/snaa322.pdf}{ti.com/lit/an/snaa322/snaa322.pdf})}
\label{fig:cds}
\end{figure}

\subsection{Signed and Unsigned \texttt{int}}
In computer science, there are two ways to express integers: signed and unsigned. For instance, an 8-bit integer of number 3 can be expressed as \texttt{0000 0011}. How about negative 3? \texttt{1000 0011}. That means, the first part of the 8-bit is used as \emph{sign}. From this sense, an 8-bit integer can express from $ (-2^7) = -128 $ to $ (+2^7-1) = +127 $. But in astronomical science, we know there should be no negative pixel value, and therefore there is absolutely no need to use the first bit as sign bit. Then we can store the numbers from $ 0 $ to $2^8 - 1 = 255 $, called the \emph{unsigned} integer format. Thus we could express almost double amount of data (twice the resolution) without losing any storage.

Although astronomers use \texttt{BZERO} and \texttt{BSCALE} technique in the FITS file format in reality, the idea is similar to unsigned integer. The output images are frequently \textbf{assumed to have no negative pixel values}. This is one of the reasons why we need the bias frame, explained a bit more in \cref{ss: calib-bias}.

\subsection{Bias} \label{ss: calib-bias}
\begin{defn}[Bias]
A \textbf{bias frame} is ideally a frame with neither source signal nor thermal electrons. In practice, it is obtained with 0-sec (or the shortest possible) exposure time with shutter closed.
\end{defn}

Now you know we have to go with readout noise; we cannot remove it. As technology improves, the readout noise $ R $ decreased down to only few electrons per readout. Early days when CCD was first used in astronomy, it was at least tens of electrons (e.g., $ R \sim \SI{50}{e} $, and it's still true in infra-red detectors), or up to hundreds of electrons. Since gain is $ \sim 1\mathrm{-}10 \,\mathrm{e/ADU} $, that means each pixel might have at least $ \pm $ tens of ADU uncertainty. Thus, even if you take a bias frame, which must have nearly a constant value $ b $, you may have a Gaussian distribution of, e.g., $ \mathcal{N}(b, 10^2) $.

Consider a 1K by 1K CCD with bias follows Gaussian of $ \mathcal{N}(b, 10^2) $. You have $ 10^6 $ pixels, and if you observe for 1 year, you may have taken $ 10^5 $ to $ 10^6 $ frames, which translates into the number of pixels read out per year of $ 10^{10} $ to $ 10^{11} $ pixels. If the readout was a perfect Gaussian, about one pixel of pixel value $ v = -7 \sigma = b - 70 $ should have been detected at least once. Probabilistically speaking, even $ b - 100 $ may happen once in our lives. Thus, for all pixels to have non-negative value, we need to have $ b $ value of at least around 100 or larger. Therefore, most detectors have bias level of around 100 to 1000 or even more, depending on the readout noise.

You may ask that, given the full capacity of the usual detectors (16-bit \texttt{int}) is $ 2^{16}-1 = 65535 $ ADU, so we need to take as small bias value as possible to achieve as large dynamic range (DR) as possible. You may have seen a detector such as $ R \sim 3 \,\mathrm{ADU} $ while $ b \sim 1000 \,\mathrm{ADU} $. It seems $ b \sim 100\,\mathrm{ADU} $ is more than enough and about 900 ADU capacity (DR) is wasted. In practice, however, we are not losing the DR at all. The CCD anyway loses linearity or saturate at around 40k (40,000 ADU), i.e., anyway you should carefully use any CCD pixels which have pixel values higher than around 40k after bias subtraction. In principle, therefore, it's even possible to have a bias of 10,000 ADU without losing any scientific performance.

A short note: bias is also temperature dependent due to the electronics performance, as well as dark.

\subsection{Dark} \label{ss: calib-dark}

\begin{defn}[Dark]
A \textbf{dark frame} is ideally a frame with no source signal but with thermal electrons. In practice, it is obtained with preset exposure time with shutter closed.
\end{defn}

In the potential well made by the detector electronics, some electrons are not \emph{photo}electrons, but \texttt{thermal}electrons. The former is the electrons pumped up from the valence to conduction band due to the signal photons, while the latter is those which obtained enough kinetic energy from random thermal energy distribution of electrons according to quantum mechanics. Simply put, probabilistically, few electrons may have obtained enough thermal energy to be detached from any parent atom/molecule of the electronics, and captured in the potential well after the detachment. The thermal energy is a sensitive (exponential) function of temperature, so this probability is dependent of the electronics temperature.

In reality, there are at least 5 differnt sources of dark current which have different functional forms\footnote{See JanesickJR (2001) ``Scientific Charge-Coupled Devices'' chapter 7 and PankoveJ (1971) ``Optical Processes in Semiconductors'' p. 27.}. Among them, the most widely cited source of the dark is ther \emph{backside dark current}, which is the dominant one for backside-illuminated CCDs:
\begin{equation}\label{eq: dark current}
  d(T) \propto
    P_S T^{1.5}
    \exp \qty[- \frac{E_g(T)}{2k_B}
      \qty(\frac{1}{T} - \frac{1}{300 \,\mathrm{K}}) ]
    \quad [\mathrm{electrons/sec/pixel}]
\end{equation}
where $ P_S $ is the pixel size, $ T $ is the tempareture, and
\begin{equation}
  E_g(T) = a - \frac{b T^2}{c + T}
\end{equation}
for $ a = 1.1557 $, $ b = 7.021 \times 10^{-4} $, and $ c = 1108 $ are the experimentally determined constants. The dark current is very sensitive to temperature, and gets enormously large as temperature increases.

In real observations, therefore, it is highly recommended to (1) reduce the temperature at least much lower than the room temperature and (2) try to maintain the temperature constant throughout the observation (otherwise, dark current may significantly vary).

\subsection{Flat} \label{ss: calib-flat}
\begin{defn}[Flat]
A \textbf{flat frame} is ideally a frame with both the flat signal with thermal electrons. The flat signal means a spatially (and spectrally/polarimetrically for spectroscopic/polarimetric observation) homogeneous, high signal-to-noise ratio signal. In practice, it is obtained with flat lamp and dome flat, twilight/dawn flat, and/or dark night flat.
\end{defn}

Flat is not an easy topic: ``To CCD experts, the term ``flat field'' can cause shivers to run up and down their spine\footnote{HowellSB (2006) ``Handbook of CCD Astronomy'' 2e, p.67.}''. The pixel-to-pixel non-uniformity of quantum efficiency at the given wavelength is not necessarily identical, so flat frame is obtained to remove this variation.

The flat field \emph{in photometry} is composed of mainly the following two components:
\begin{enumerate}
\item \textbf{large-scale variation}: Even though the telescope is uniformly illuminated, vignetting can occur because of the unavoidable pixel-scale variation, i.e., distortion, throughout the field of view.
\item \textbf{small-scale variation}: The vignetting effect within, e.g., 10 by 10 pixels may be negligible. But each pixel can have different sensitivity, so local variation may appear.
\end{enumerate}

The large scale variation could be extracted even from the object frame by using large median or boxcar kernel (it has been used since IRAF: see \texttt{ILLUMCOR}). This is sometimes needed because the variation pattern may differ from flat to object, caused by a slight movement of loosely fixed optics during the motion of the telescope.

Especially for the small-scale variation, we need to think the difference in the spectral response of each pixel. Consider two nearby pixels, where vignetting is ignored. Say the filter profile is $ f(\lambda) $, $ i $-th pixel has the spectral response $ r_i(\lambda)$, flat and object have the spectral energy distribution $ s_f(\lambda) $ and $ s_o(\lambda) $, respectively. Then what we have from flat and the object frame at the $ i $-th pixel are
\begin{equation}
  F_i = \int_{0}^{\infty} s_f(\lambda) r_i(\lambda) f(\lambda) d\lambda
  \sep
  I_i = \int_{0}^{\infty} s_o(\lambda) r_i(\lambda) f(\lambda) d\lambda ~.
\end{equation}
By the flat field correction, $ I_i/F_i $, what we are \emph{hoping} is that the integrand function $ \tilde{f}(\lambda) $ is identical for all the pixels:
\begin{equation}
  I_i/F_i = \int_{0}^{\infty} s_o(\lambda) \tilde{f}(\lambda) d\lambda
  \sep
  \tilde{f}(\lambda)
    = \frac{r_i(\lambda) f(\lambda)}{F_i}
    = \frac{r_i(\lambda) f(\lambda)}
      {\int_{0}^{\infty} s_f(\lambda) r_i(\lambda) f(\lambda) d\lambda}
\end{equation}
This is not always true (because the $ r_i(\lambda) $ do not simply cancel out), and it is a reason from the technological point of view why the instrument-dependent correction term, the transformation coefficient ($ k_f $; see \cref{eq: std}) appears, and making the spectroscopic flat very difficult to achieve.

One lesson from this is that, it \emph{might be} risky to use a monochromatic flat (e.g., an LED), but even a spectrally flat lamp may not be a perfect flat, unless our flat has the spectrum identical to the (unknown) target object. In spectroscopy, however, a spectrally flat lamp is important to obtain good signal-to-noise ratio for all wavelength pixels.

\section{Gain and Readout Noise}
Now that you are familiar with preprocessing and data reduction. In the error-analysis, you may have used the gain and readout noise to estimate the pixel noise. But how can we determine the gain and readout noise? Frequently both of them are \textit{provided from the CCD manufacturer}, but sometimes the user has to determine them. There are few ways to determine those two.

\subsection{Gain and Readout Noise in FITS Header}
The two parameters usually appear in the FITS header. Gain appears as the keyword \texttt{GAIN}, but many times people use the keyword \texttt{EGAIN}, which is not preferred. The readout noise is also called the read noise in short, and appear as \texttt{RDNOISE}. Sometimes \texttt{RONOISE} is used, but not preferred.

\subsection{Janesick's Method}
Janesick's method is the most classical way of deriving the gain and readout noise value. Although it's the most widely used in many textbooks, they mostly don't provide even simple ideas of proof, I here provide the full proof as well as the formulae.

\begin{thm}[Janesick's Method] \label{thm: janesick method}
If the two flat images have pixel values of $ F_1 $ and $ F_2 $ and two biases have $ B_1 $ and $ B_2 $ (all in ADU), the gain and readout noise are
\begin{equation}\label{eq: janesick method}
  g = \frac{ (\bar{F}_1 + \bar{F}_2) - (\bar{B}_1 + \bar{B}_2)}{\sigma^2_{F_1 - F_2} - \sigma^2_{B_1 - B_2}} ~\mathrm{[e/ADU]}
  \quad;\quad
  R = g\frac{\sigma_{B_1 - B_2}}{\sqrt{2}} ~\mathrm{[e]}
\end{equation}
Here $ \bar{X} $ means the average of all the pixels in the frame $ X $, and $ \sigma_X $ is the true standard deviation of the frame $ X $, estimated from the sample standard deviation $ \sigma_X \approx \sqrt{(\sum_i (X_i - \bar{X})^2) / (N - 1)} $.
\end{thm}

\begin{proof}[of Janesick's Method]
Note that in this theorem, by saying \textit{flat}, we are implicitly assuming that those frames should share the identical expected values. Also the pixel-wise sensitivity variation is ignored, as well as cosmic-ray events, bad pixels, etc.

The bias frame is nothing but the offset voltage added with readout noise. Therefore, if the true bias level is $ b $ in ADU, any bias image will follow a normal distribution:
\begin{equation*}
  B \sim \mathcal{N} \qty( b, \qty(\frac{R}{g})^2 ) ~\mathrm{[ADU]}
  \quad \rightarrow \quad
  B_1 - B_2 \sim \mathcal{N} \qty( 0, 2\qty(\frac{R}{g})^2 ) ~\mathrm{[ADU]}
\end{equation*}
where $ g $ is introduced in the denominator to convert $ R $, in [e], to [ADU]. Hence,
\begin{equation*}
  \sigma^2_{B_1 - B_2}
    = 2\frac{R^2}{g^2} ~,
\end{equation*}
so the second equation is proven. Since the LHS is approximated by the sample standard deviation, you can write $ \sigma^2_{B_1 - B_2} \approx $ \pyth{np.std(B1-B2, ddof=1)**2} in python.

The (raw) flat image consist of photons with dark plus bias level. Therefore, if $ f $ is the true flat level \emph{plus dark} in ADU, any flat will roughly follow a normal distribution, similar to the bias case:
\begin{equation*}
  F \simdot \mathcal{N} \qty( f + b, \frac{f}{g} + \qty(\frac{R}{g})^2 ) ~\mathrm{[ADU]}
  \quad \rightarrow \quad
  \left \{
  \begin{aligned}
    F_1 - F_2 &\simdot
      \mathcal{N} \qty( 0, 2\frac{f}{g} + 2 \qty(\frac{R}{g})^2 ) &~\mathrm{[ADU]}\\
    (F_1 + F_2) - (B_1 + B_2) & \simdot
      \mathcal{N} \qty(2f, 2 \frac{f}{g} + 4 \qty( \frac{R}{g} )^2) &~\mathrm{[ADU]}
  \end{aligned}
  \right .
\end{equation*}
The first term in the variance is the Poisson noise term\footnote{Reminder: the Poisson noise term is also called photon noise or shot noise in astronomy. Both photoelectron from true flat and dark current follow Poisson distribution. The Poisson distribution is usually assumed to be Gaussian, because $ f/g \gg 1 $ (Poisson distribution asymptotically approaches to Gaussian when the mean value gets larger, as we saw in Thm \ref{thm: Pois gauss}).} and the second term is the readoise term, respectively. From these, you can extract
\begin{align*}
  \sigma^2_{F_1 - F_2} - \sigma^2_{B_1 - B_2}
    &\approx 2\frac{f}{g}\\
  (\bar{F}_1 + \bar{F}_2) - (\bar{B}_1 + \bar{B}_2)
   &\approx 2f
\end{align*}
This proves the first equation. Since the $ \sigma $ values are approximated by the sample standard deviation, $ \sigma^2_{F_1 - F_2} - \sigma^2_{B_1 - B_2} \approx $ \pyth{np.std(F1-F2, ddof=1)**2 - np.std(B1-B2, ddof=1)**2} in python, and from simple mathematics, $ (\bar{F}_1 + \bar{F}_2) - (\bar{B}_1 + \bar{B}_2) = $ \pyth{np.mean((F1+F2) - (B1+B2))} in python.
\end{proof}

Although we can use any one flat and one bias out of two of each to obtain $ F_i - B_i \simdot \mathcal{N} (f, f/g + 2(R/g)^2) $, Janesick's method uses all the information from all the four frames at once. In real application, we may take a lot of bias and flat frames, say $ N_b $ and $ N_f $ frames, respectively. Then select 2 from each, making $ \binom{N_b}{2} \times \binom{N_f}{2} $ possible gain and readout noise estimations. We can use the mean of those results to estimate the gain and readout noise and their uncertainties by sample standard deviation of the estimates. Because there always are vignetting in flat frames, you \textbf{must extract only the smooth part of flat} (and thus the corresponding region in bias) for this analysis to meet the assumptions given in the beginning of the proof, and should not na\"{i}vely use all the pixels in the image.

Also note that, in real physical unit system, both $ \mathrm{e/ADU} $ and $ \mathrm{e} $ are unitless, so don't be confused if you see something like $ \sqrt{R^2/g} $ has the unit of ADU, not $ \sqrt{\mathrm{e \cdot ADU}} $.

\subsection{Graphical Method (Line Fitting) }
There is another method, which has no name as far as I know. It fits a linear line to some values. \textbf{This method is preferred over Janesick's method}, because (1) it's a linear regression so you can simply estimate the uncertainty of $ g $ and $ R $ from simple statistics, (2) using the identical data obtained from this method, you can check the linearity of the detector.

Similar to Janesick's method, consider you have a master bias $ B $, and assume the flat frame has no sensitivity variation in the pixels we select. Take $ N^{(i)}_f (\ge 2) $ flat images at each exposure time $ t_i $, $ t_i \in \{ t_1, t_2, \cdots, t_N \} $ ($ t_1 < t_2 < \cdots < t_N $), so that the expected value of each exposure time will be $ I_0 t_i $.

\begin{thm} [Gain and Readnoise Determination - Graphical (not recommended)]
For each exposure, select 2 flats among $  N^{(i)}_f $ flat frames, and calculate the variance of the difference between two frames, $ \sigma^2 \qty({F^{(i)}_j - F^{(i)}_k}) $ $ (j \neq k) $, so that you have $ \binom{N_f^{(i)}}{2} $ values. Then
\begin{equation}
  \sigma^2 \qty({F^{(i)}_j - F^{(i)}_k})
    = 2\qty [ \frac{1}{g} f^{(i)} + \qty(\frac{R}{g})^2 ]
  \quad\rightarrow\quad
  \left \{
  \begin{aligned}
    g &= \frac{2}{\mathrm{slope}} &&~\mathrm{[e/ADU]} \\
    R &= g \sqrt{\mathrm{intercept}}/2 &&~\mathrm{[e]}
  \end{aligned}
  \right .
\end{equation}
where $ f^{(i)} \approx \mathrm{mean} \qty( \sum_j (F^{(i)}_j - B) ) $ is the true flat \emph{plus dark} in ADU. There will be $  N^{(i)}_f $ points for the fixed $ f^{(i)} $ value on the $ x $-axis, and the number of $ x $ values is $ N $.
\end{thm}

There is actually no need to prove this: from Thm \ref{thm: janesick method}, we already derived $ F_1 - F_2 \simdot \mathcal{N} (f,\, 2f/g + 2(R/g)^2) $, thus the above equation is proven. The assumptions used are that all flats share same bias (true bias $ b $ is estimated from the master bias $ B $), all flats share identical gain and readout noise, and that $ \mathrm{mean} \qty( \sum_j (F^{(i)}_j - B) ) $ can be used as the estimator for the true signal in ADU. One may think some extended version of assumptions such as the variation of the detector's temperature is ignorable, etc. All these assumptions are too trivial for well-established instruments, so manytimes we just do not explicitly state these.

Usually the error-bar of $ f^{(i)} $ will be very small because you will use tens of thousands of pixels to estimate this, and the CLT (Thm \ref{thm: clt}) decreases the error-bar. There is no need to spend long time to accurately derive the error-bar in the $ y $-direction either, because this kind of \textit{performance evaluation} is a one-time task per year or so, and hence you can just plot a huge number of $ y $ values rather than thinking about error-bars of each point. Let's just ignore error-bars and proceed the line fitting.

As mentioned before, the data used in this analysis can be \textbf{recycled for the linearity check}. We can simply plot $ f^{(i)} $ as a function of $ t_i $, and you should see a linear line. Depending on the dynamic range of the detector, the linearity will break at some exposure $ t_i > t_{i_\mathrm{crit}} $. The $ F^{(i_\mathrm{crit})} = f^{(i_\mathrm{crit})} + b $ will then be the maximum ADU value you should trust. Any pixel value higher than that must be cautiously dealt when you do the data analysis.

A tip is that, it is better to take a fake flat, i.e., a flat with illumination gradient. That is, for each exposure, certain range of pixel values are obtained across the image. Therefore, in both the linear regression and linearity plot, the $ x $-axis (mean ADU) will be filled more densely.

To increase the number of data points in linear regression, you need to take too many flats. Another way is to chop each frame into mesh (e.g., 10 by 10 pixel meshes), and use each such region as one data point. If you have 1 flat at 5 exposures ($ N_f^{(i)} = 1 $ for $ i = 1, .., 5 $) and flat is 1,000 by 1,000 pixels, you can make 10,000 meshes per each exposure! Therefore, a simpler method is :

\begin{thm} [Gain and Readnoise Determination - Graphical (recommended)]
For each flat, subtract bias and chop it into many meshes. For each mesh, calculate pixel mean and variances. If the incident flux is assumed to be homogeneous in this small mesh area, $ \mathrm{variance} = \mathrm{mean}/g + (R/g)^2 $.

Fit a line for $ (x, y) $ where $ x $: $ \mathrm{mean} (= f = \mathrm{flat} - \mathrm{bias}) $ and $ y $: $ \mathrm{variance} $ for all meshes and exposures. Then
\begin{equation}
  \left \{
  \begin{aligned}
    g &= \frac{1}{\mathrm{slope}} &&~\mathrm{[e/ADU]} \\
    R &= g \sqrt{\mathrm{intercept}} &&~\mathrm{[e]}
  \end{aligned}
  \right .
\end{equation}
Note the factor 2 is not appearing in this method.
\end{thm}

\subsection{Note}
A good thing for the above two methods are that they are independent of the spectral energy distribution (SED) of the flat. This is because both the gain and readout noise are device's output-gate-related, not related to the pixel's sensitivity or whatever. Thus, important things are the pixel count and mathematics (statistics), which means you can calculate gain and readout noise even with a monochromatic laser, for example. In real observation, it's better not to use monochromatic light source for the flat (see \cref{ss: calib-flat}).


\section{Product Functions}

\subsection{User Functions}

	\begin{aims}

		\item[F11] The user can access the implemented features as a part of MNE Scan (see section \ref{3_1_scope}).
		\item[F12] Either MEG or EEG data can be used as input.
		\item[F13] Any desired surface mesh and set of sensor data, selected by the user, can be used as input data for 							further calculations.
		\item[F14] A preferred subset of vertices can be selected, to perform distance calculations on.
		\item[F15] The user is able to select a threshold for identifying all relevant vertices while interpolating.
		\item[F16] As part of MNE Scan/Disp3D all graphical options are available to the user, meaning e.g. different 								coloring and rotation of the object.

	\end{aims}

\subsection{Offline Functions}

	These functions are performed one time and are executed prior to the interpolation process.

	\begin{aims}

		\item[F21]	The distance between two vertices on a given mesh, is calculated following the structure of the surface. 						Therefore the euclidian distance is not used.
		\item[F22] All floating sensor points are projected onto the nearest vertices on the input mesh.
		\item[F23] A weight matrix, containing values for all vertices, is generated.

	\end{aims}

\subsection{Real-Time Functions}

	These functions are performed continuously and in real-time.

	\begin{aims}

		\item[F31]	Based off prior calculations, the interpolation assigns every vertex a value.
		%\item[F32] A visual output is generated. %TODO bleibt das so?

	\end{aims}