Bayesian data analysis

Bayesian data analysis is based on Bayesian inference. Briefly, this approach is based on the following straightforward property of probability distributions. Let p(x,y) be the joint probability of observing x and y simultaneously. Let p(x|y) be the conditional probability of observing x, given y. Then, by definition

$p(x|y)p(y)=p(x,y)=p(y|x)p(x)\,$

from which follows Bayes' theorem:

$p(x|y)={\frac {p(y|x)p(x)}{p(y)}}$

Interpretation of Bayes' Theorem

The interpretation of this expression in the framework of data interpretation is as follows. Given the initial knowledge of a system, quantified by the prior probability p(x) of system states x, one makes an observation y with probability (or degree of confidence) p(y), adding to the prior knowledge. The posterior probability p(x|y) quantifies this enhanced knowledge of the system, given the observation y. Thus, the Bayesian inference rule allows one to (a) gradually improve the knowledge of a system by adding observations, and (b) easily combining information from diverse sources by formulating the degree of knowledge of the system and the observations in terms of probability distributions. The quantity p(y|x) is a fundamental quantity linking the prior and posterior distributions, called the likelihood, and expresses the probability of observing y, given the prior knowledge x.

Forward modelling

The likelihood is evaluated using a forward model of the experiment, returning the value of simulated measurements while assuming a given physical state x of the experimental system. Mathematically, this forward model (mapping system parameters to measurements) is often much easier to evaluate than the reverse mapping (from measurements to system parameters), as the latter is often the inverse of a projection, which is therefore typically ill-determined. On the other hand, evaluating the forward model requires detailed knowledge of the physical system and the complete measurement process.

The Likelihood

The forward model is used to predict the measurements y, based on the physical state x of the system. The Likelihood p(y|x) specifies the most probable measurement outcome, which corresponds to the maximum of the distribution p(y|x), as well as its uncertainty, given by the width of the distribution.

In a typical case, assume that the model is such that the measurement outcomes are distributed like a Gaussian around a most probable value y₀, with an error Δ y. Then the likelihood will be

$p(y|x)={\frac {1}{\sqrt {2\pi \Delta y^{2}}}}\exp \left(-{\frac {(y-y_{0})^{2}}{2\Delta y^{2}}}\right)$

Note that the negative logarithm of the likelihood is proportional to the χ² of the fit of the data y to the model y₀, and maximizing the likelihood will minimize χ², thus establishing the link between the Bayesian approach and the common least squares fit. However, the Bayesian approach is more general than the standard least squares fit, as it can handle any type of probability distribution.

Parametric formulation

The description of the system state is usually done by defining a parametric representation, e.g., by defining density or temperature profiles as a function of space via a vector of N parameters, α: n(r,α) or T(r,α). The parameters obey a prior distribution p(α), expressing physical or other constraints. Applying Bayes theorem one obtains

$p(\alpha |D)={\frac {p(D|\alpha )p(\alpha )}{p(D)}}$

where D represents the available data. The likelihood p(D|α) specifies the probability of a specific measurement outcome D for a given choice of parameters α. The advantage of the parametric representation is that the abstract 'system state' is reduced to a finite set of parameters, greatly facilitating numerical analysis. This parametrization may involve, e.g., smooth (orthogonal) expansion functions such as Fourier-Bessel functions, or discretely defined functionals on a grid.

Maximization

The maximization of the posterior probability as a function of the parameters α yields the most likely value of the parameters, given the data D, which is the basic answer to the data interpretation problem.

Marginalization

The width of the posterior distribution yields the error in the parameters. To obtain the error in a given parameter α_i, the posterior distribution is marginalized by integrating over the remaining N-1 parameters:

$p(\alpha _{i}|D)=\int {p(\alpha |D)d\alpha _{1}\cdots d\alpha _{i-1}d\alpha _{i+1}\cdots d\alpha _{N}}$

The width of this one-dimensional distribution is found using standard procedures.

Comparison with Function Parametrization

Function parametrization (FP) is another statistical technique for recovering system parameters from diverse measurements. Both FP and Bayesian data analysis require having a forward model to predict the measurement readings for any given state of the physical system, and the state of the physical system and the measurement process is parametrized. However

• instead of computing an estimate of the inverse of the forward model (as with FP), Bayesian analysis finds the best model state corresponding to a specific measurement by a maximization procedure (maximization of the likelihood);
• the handling of error propagation is more sophisticated within Bayesian analysis, allowing non-Gaussian error distributions and absolutely general and complex parameter interdependencies; and
• additionally, it provides a systematic way to include prior knowledge into the analysis.

Typically, the maximization process is CPU intensive, so that Bayesian analysis is not usually suited for real-time data analysis (unlike FP).

Integrated Data Analysis

The goal of Integrated Data Analysis (IDA) is to combine the information from a set of diagnostics providing complementary information in order to recover the best possible reconstruction of the actual state of the system subjected to measurement. This goal overlaps with the goal of Bayesian data analysis, but IDA applies Bayesian inference in a relatively loose manner to allow incorporating information obtained with traditional or non-Bayesian methods.

See also

• Markov chain Monte Carlo