4,378
edits
Error propagation: Difference between revisions
no edit summary
No edit summary |
|||
| (9 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
Proper reporting of experimental measurements requires the calculation of error bars or "confidence intervals". The appropriate and satisfactory calibration of data and analysis of errors is essential to be able to judge the relevance of observed trends. Below, a brief definition of the main concepts and a discussion of generic ways to obtain error estimates is provided. | Proper reporting of experimental measurements requires the calculation of error bars or "confidence intervals". The appropriate and satisfactory calibration of data and analysis of errors is essential to be able to judge the relevance of observed trends. Below, a brief definition of the main concepts and a discussion of generic ways to obtain error estimates is provided. | ||
<ref>[http://www.nrbook.com/ W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in FORTRAN (Cambridge University Press, 1992), 2nd ed.]</ref> | <ref>[http://www.nrbook.com/ W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in FORTRAN (Cambridge University Press, 1992), 2nd ed.]</ref> | ||
<ref>P. Bevington and D. Robinson, Data Reduction and Error Analysis for the Physical Sciences (McGraw-Hill, UK, 2003), 3rd ed. ISBN 978-0072472271</ref> | <ref>P. Bevington and D. Robinson, Data Reduction and Error Analysis for the Physical Sciences (McGraw-Hill, UK, 2003), 3rd ed. {{ISBN|978-0072472271}}</ref> | ||
Of course, any particular measuring device generally requires specific techniques. | Of course, any particular measuring device generally requires specific techniques. | ||
| Line 33: | Line 33: | ||
:<math>z = f(x, y, ...)\,</math> | :<math>z = f(x, y, ...)\,</math> | ||
:<math>(\Delta z)^2 = \left ( \frac{\partial f}{\partial x}\right )^2 \Delta x^2 + \left ( \frac{\partial f}{\partial y}\right )^2 \Delta y^2 + ... </math> | :<math>(\Delta z)^2 = \left ( \frac{\partial f}{\partial x}\right )^2 \Delta x^2 + \left ( \frac{\partial f}{\partial y}\right )^2 \Delta y^2 + ... </math> | ||
This formula holds exclusively for a Gaussian (normal) distribution of errors (assuming the errors are small and that the independent variables ''x'', ''y'', ... are indeed independent). | This formula holds exclusively for a Gaussian (normal) distribution of errors (assuming the errors are small and that the independent variables ''x'', ''y'', ... are indeed independent). | ||
<ref>[http://mathworld.wolfram.com/ErrorPropagation.html Error Propagation (MathWorld)]</ref> | |||
One should be aware that many situations exist where error distributions are not normal (see below). | One should be aware that many situations exist where error distributions are not normal (see below). | ||
One can easily check whether the error distribution is normal by doing repeated experiments under the same conditions and observing the resulting distribution of ''s''. | One can easily check whether the error distribution is normal by doing repeated experiments under the same conditions and observing the resulting distribution of ''s''. | ||
| Line 46: | Line 47: | ||
This technique also provides a quick method to check for possible problems such as ill-conditioning, cited above. | This technique also provides a quick method to check for possible problems such as ill-conditioning, cited above. | ||
When the model relating ''s'' and ''p'' is known, as well as the error distributions (and the latter may either be Gaussian or not), a more systematic approach to error propagation is provided by a technique known as the [[:Wikipedia:Maximum likelihood|maximum likelihood method]]. | When the model relating ''s'' and ''p'' is known, as well as the error distributions (and the latter may either be Gaussian or not), a more systematic approach to error propagation is provided by a technique known as the [[:Wikipedia:Maximum likelihood|maximum likelihood method]]. | ||
<ref>[ | <ref>Particle Data Group, [[doi:10.1007/s10052-998-0104-x|Eur. Phys. J. C 3, 1 (1998)]]</ref> | ||
This technique is simply the generalisation of standard error propagation to general error distributions (i.e. not limited to Gaussians). | This technique is simply the generalisation of standard error propagation to general error distributions (i.e. not limited to Gaussians). | ||
| Line 81: | Line 82: | ||
The simplest case is when the physically interesting phenomenon is slowly varying in time. | The simplest case is when the physically interesting phenomenon is slowly varying in time. | ||
Random noise is usually characterised by a high frequency, so that a filter in frequency space can then separate signal and noise neatly. | Random noise is usually characterised by a high frequency, so that a filter in frequency space can then separate signal and noise neatly. | ||
<ref>D. Newland, An Introduction to Random Vibrations, Spectral and Wavelet Analysis (Dover, New York, 1993) ISBN 0486442748</ref> | <ref>D. Newland, An Introduction to Random Vibrations, Spectral and Wavelet Analysis (Dover, New York, 1993) {{ISBN|0486442748}}</ref> | ||
However, when the physically interesting information is fluctuating, this signal-noise separation by frequency is not feasible, and much care is needed when analysing data. | However, when the physically interesting information is fluctuating, this signal-noise separation by frequency is not feasible, and much care is needed when analysing data. | ||
The application of a set of techniques is required to understand such signals (cross correlation, conditional averaging, spectral analysis, bi-spectral analysis, | The application of a set of techniques is required to understand such signals (cross correlation, conditional averaging, spectral analysis, [[Bicoherence|bi-spectral analysis]], | ||
<ref>J. van den Berg, ed., Wavelets in Physics (Cambridge University Press, 1999) ISBN 978-0521593113</ref>, [[Biorthogonal decomposition]], | <ref>J. van den Berg, ed., Wavelets in Physics (Cambridge University Press, 1999) {{ISBN|978-0521593113}}</ref>, [[Biorthogonal decomposition]], | ||
determination of fractal dimension, mutual information, reconstruction of chaotic attractor, | determination of fractal dimension, mutual information, reconstruction of chaotic attractor, | ||
<ref> | <ref>H. Abarbanel, R. Brown, J. Sidorowich, and L. S. Tsimring, [[doi:10.1103/RevModPhys.65.1331|Rev. Mod. Phys. 65, 1331 (1993)]]</ref> ...). | ||
== Non-Gaussian statistics == | == Non-Gaussian statistics == | ||
| Line 92: | Line 93: | ||
The distribution of random variations of a signal ''s'' around its mean value need not be Gaussian. | The distribution of random variations of a signal ''s'' around its mean value need not be Gaussian. | ||
E.g., photon statistics are typically of the [[:Wikipedia:Poisson distribution|Poisson]] type, which is especially important for low signal levels. | E.g., photon statistics are typically of the [[:Wikipedia:Poisson distribution|Poisson]] type, which is especially important for low signal levels. | ||
<ref> | <ref>B. van Milligen, I. Classen, and C. Barth, [[doi:10.1063/1.1597951|Rev. Sci. Instrum. 74, 3998 (2003)]]</ref> | ||
In other cases, the random component of the signal ''s'' is simply a non-linear function of a (Gaussian) noise source, causing the distribution to be skewed or distorted. | In other cases, the random component of the signal ''s'' is simply a non-linear function of a (Gaussian) noise source, causing the distribution to be skewed or distorted. | ||
Or the random component of the measured signal could correspond to the maximum or minimum value of a (Gaussian) random number, leading to extremal ([[:Wikipedia:Gumbel distribution|Gumbel]]) distributions. | Or the random component of the measured signal could correspond to the maximum or minimum value of a (Gaussian) random number, leading to extremal ([[:Wikipedia:Gumbel distribution|Gumbel]]) distributions. | ||
<ref> | <ref>B. van Milligen, R. Sánchez, B. Carreras, V. Lynch, B. LaBombard, M. Pedrosa, C. Hidalgo, B. Gonçalves, and R. Balbín, [[doi:10.1063/1.1884615|Phys. Plasmas 12, 052507 (2005)]]</ref> | ||
The [[:Wikipedia:Log-normal distribution|log-normal distribution]] is also quite common (e.g. in potential fluctuations). | The [[:Wikipedia:Log-normal distribution|log-normal distribution]] is also quite common (e.g. in potential fluctuations). | ||
<ref> | <ref>F. Sattin, N. Vianello, and M. Valisa, [[doi:10.1063/1.1797671|Phys. Plasmas 11, 5032 (2004)]]</ref> | ||
However, all the previous distributions can be obtained by suitable manipulations of Gaussian random variables. | However, all the previous distributions can be obtained by suitable manipulations of Gaussian random variables. | ||
A totally different class of statistics is known as [[:Wikipedia:Lévy distribution|Lévy distributions]] (of which the Gaussian distribution is only a special case), which is the class of distributions satisfying the requirement that the sum of independent random variables with a distribution ''P'' again has a distribution ''P'' (generalisation of the Central Limit Theorem). | A totally different class of statistics is known as [[:Wikipedia:Lévy distribution|Lévy distributions]] (of which the Gaussian distribution is only a special case), which is the class of distributions satisfying the requirement that the sum of independent random variables with a distribution ''P'' again has a distribution ''P'' (generalisation of the Central Limit Theorem). | ||
Such distributions are expected to appear in [[Self-Organised Criticality|self-organised systems]] (such as plasmas). | Such distributions are expected to appear in [[Self-Organised Criticality|self-organised systems]] (such as plasmas). | ||
In general, the detection of this type of non-Gaussian statistics is difficult. Some techniques are however available, such as renormalisation, [[:Wikipedia:Rescaled range|rescaled-range]] analysis, | In general, the detection of this type of non-Gaussian statistics is difficult. Some techniques are however available, such as renormalisation, [[:Wikipedia:Rescaled range|rescaled-range]] analysis, | ||
<ref> | <ref>B. Carreras, B. van Milligen, M. Pedrosa, R. Balbín, C. Hidalgo, D. Newman, E. Sánchez, R. Bravenec, G. McKee, I. García-Cortés, et al., [[doi:10.1063/1.873490|Phys. Plasmas 6, 1885 (1999)]]</ref> | ||
the detection of long-range time dependence, | the detection of [[Long-range correlation|long-range time dependence]], | ||
<ref> | <ref>B. Carreras, D. Newman, B. van Milligen, and C. Hidalgo, [[doi:10.1063/1.873192|Phys. Plasmas 6, 485 (1999)]]</ref> | ||
finite-size [[:Wikipedia:Lyapunov exponent|Lyapunov exponents]], | finite-size [[:Wikipedia:Lyapunov exponent|Lyapunov exponents]], | ||
<ref> | <ref>B. Carreras, V. Lynch, and G. Zaslavski, [[doi:10.1063/1.1416180|Phys. Plasmas 8, 5096 (2001)]]</ref> etc. | ||
Sometimes it is possible to obtain information on the nature of the errors by averaging experimental data (in space or time) - this is the renormalisation technique referred to above. | Sometimes it is possible to obtain information on the nature of the errors by averaging experimental data (in space or time) - this is the renormalisation technique referred to above. | ||
When averaging over ''N'' samples, the variation of the ''N''-averaged (or smoothed) data is less than that of the original data. | When averaging over ''N'' samples, the variation of the ''N''-averaged (or smoothed) data is less than that of the original data. | ||
| Line 113: | Line 114: | ||
== Integrated data analysis == | == Integrated data analysis == | ||
Often, various different diagnostics provide information on the same physical parameter (e.g., in a typical fusion plasma experiment, the electron temperature ''T<sub>e</sub>'' is possibly measured by Thomson Scattering, ECE | Often, various different diagnostics provide information on the same physical parameter (e.g., in a typical fusion plasma experiment, the electron temperature ''T<sub>e</sub>'' is possibly measured by Thomson Scattering, ECE, and indirectly also by SXR, although mixed with information on the electron density ''n<sub>e</sub>'' and ''Z<sub>eff</sub>''. The electron density is measured directly by Thomson Scattering, the HIBP, reflectometry, and interferometry, and indirectly by SXR). | ||
Part of this information is local, and part is line-integrated. Instead of cross-checking these diagnostics for one or a few discharges, one could decide to make an integrated analysis of data. | Part of this information is local, and part is line-integrated. Instead of cross-checking these diagnostics for one or a few discharges, one could decide to make an integrated analysis of data. | ||
This means using all information available to make the best possible reconstruction of, e.g., the electron density and temperature that is compatible with all diagnostics simultaneously. | This means using all information available to make the best possible reconstruction of, e.g., the electron density and temperature that is compatible with all diagnostics simultaneously. | ||
To do this, the following conditions must apply: 1) The data should not contradict each other mutually. This requires a previous study concerning the mutual compatibility, i.e. data validation. 2) The data should be available with proper calibration and independent error estimates in a routine fashion. This means regular calibrations of the measuring device and crosschecks. 3) A suitably detailed model of the physical system should be available, capable of modelling all experimental conditions and all corresponding measurement data. Techniques based on e.g. [[Bayesian data analysis|Bayesian statistics]] then allow finding the most probable value of all physical parameters in the model, compatible with all measured signals. | To do this, the following conditions must apply: 1) The data should not contradict each other mutually. This requires a previous study concerning the mutual compatibility, i.e. data validation. 2) The data should be available with proper calibration and independent error estimates in a routine fashion. This means regular calibrations of the measuring device and crosschecks. 3) A suitably detailed model of the physical system should be available, capable of modelling all experimental conditions and all corresponding measurement data. Techniques based on e.g. [[Bayesian data analysis|Bayesian statistics]] then allow finding the most probable value of all physical parameters in the model, compatible with all measured signals. | ||
== Summary == | == Summary == | ||
| Line 122: | Line 123: | ||
A proper analysis of error propagation requires having a reasonable model ''M<sub>p</sub>'' that relates the measured signals ''s'' to the corresponding physical parameters ''p''. | A proper analysis of error propagation requires having a reasonable model ''M<sub>p</sub>'' that relates the measured signals ''s'' to the corresponding physical parameters ''p''. | ||
Such a model can initially be rudimentary, implying the probable existence of large systematic errors. | Such a model can initially be rudimentary, implying the probable existence of large systematic errors. | ||
The systematic observation and analysis of the results ''p'' and their properly propagated random errors ''p'', and their comparison with similar results from other diagnostics should allow improvement of the model, thus reducing the systematic error and improving the agreement between independent measurements (and/or models). | The systematic observation and analysis of the results ''p'' and their properly propagated random errors ''Δp'', and their comparison with similar results from other diagnostics should allow improvement of the model, thus reducing the systematic error and improving the agreement between independent measurements (and/or models). | ||
This gradual improvement of the physics model is the basis of scientific progress. | This gradual improvement of the physics model is the basis of scientific progress. | ||
== References == | == References == | ||
<references /> | <references /> | ||