4,377
edits
Model validation: Difference between revisions
m
→The logical trap
| (2 intermediate revisions by the same user not shown) | |||
| Line 13: | Line 13: | ||
Anyone would agree that the logical inference 'if A is true, then B must be true' combined with the observation that 'B is true' does not imply that 'A is true'. | Anyone would agree that the logical inference 'if A is true, then B must be true' combined with the observation that 'B is true' does not imply that 'A is true'. | ||
And yet this mistake appears to be rather common: if a given plasma model (A) describes a given | And yet this mistake appears to be rather common: if a given plasma model (A) describes a given experimental result (B), it is inferred that the model must be 'OK' - erroneously, because the agreement may be fortuitous or due to constraints that are hard to identify, or the data interpretation problem may be ''badly posed'' (see below). | ||
There are several ways of avoiding this trap: | There are several ways of avoiding this trap: | ||
| Line 22: | Line 22: | ||
Second, the number of experiments for which 'A implies B' holds can be increased. E.g., if a model (A) is found to describe the obervations (B) for a large number of significantly different cases (experimental situations), without 'tweaking' parameters, then the validity of the model is enhanced; although it can never be ''proven'' that the model will always work in this way. Its validity will always remain subject to further testing. | Second, the number of experiments for which 'A implies B' holds can be increased. E.g., if a model (A) is found to describe the obervations (B) for a large number of significantly different cases (experimental situations), without 'tweaking' parameters, then the validity of the model is enhanced; although it can never be ''proven'' that the model will always work in this way. Its validity will always remain subject to further testing. | ||
== Hidden assumptions == | |||
The equations describing the behaviour of plasmas are mostly known (Maxwell's equations, etc.) but are untractable due to the large number of particles involved. | |||
Hence, simplifying assumptions are always made, usually of the type 'assume ''X >> Y'' '. It is quite common that these assumptions are not made fully explicit, which entrains the risk that the assumptions are violated in some specific case without this circumstance being detected. Therefore, it is important to clarify as precisely as possible under what conditions the model is valid, and check that these conditions are met for all relevant applications of the model. | |||
Of course, initial and boundary conditions are just as important to specify. | |||
== Circular reasoning == | |||
Ideally, one would like the model only to take input from boundary and/or initial conditions, and predict the experimental outcome. | |||
However, often experimental measurements are taken as input (e.g., a density profile) to predict (using a model, e.g., NC theory) another experimental profile (e.g., the radial electric field). In this case, coincidence with radial electric field measurements only show ''consistency'' with the theory used, but does not prove the model is correct in and of itself, as the coincidence may be due to constraints also obeyed by other (transport) theories, while the input profile may be (partly) a result of turbulent transport (not contained in NC theory). | |||
== Badly posed problems == | == Badly posed problems == | ||
| Line 31: | Line 43: | ||
* [[Bayesian data analysis]] | * [[Bayesian data analysis]] | ||
* [[Error propagation]] | * [[Error propagation]] | ||
Models can of course not be validated with respect to any parameters that are badly determined in an experimental situation. | |||
== References == | == References == | ||
<references /> | <references /> | ||