Uncertainty Quantification — Polynomial Chaos and Kriging Surrogate
- Introduced by
- Norbert Wiener (polynomial chaos, 1938); extended to Wiener–Askey scheme by Xiu & Karniadakis (2002)
- Year
- Seminal modern form: 2002
- Type
- Computational uncertainty analysis framework
What does it do?
Uncertainty Quantification (UQ) is a computational framework for systematically measuring how uncertainty in the inputs of a model propagates into uncertainty in its outputs.
Building on Wiener's polynomial chaos theory (1938) and formalised for general stochastic problems by Xiu and Karniadakis (2002), UQ uses two primary strategies: Polynomial Chaos Expansion (PCE), which represents the model output as a series of orthogonal polynomials matched to the input distributions, and Kriging (Gaussian process) surrogates, which replace an expensive simulation with a fast statistical approximation fitted to a small set of carefully chosen runs.
The intuition behind the method
Imagine you are running a costly computer simulation — a finite-element structural analysis or a climate model — and you know that several of its inputs are not precise numbers but uncertain quantities drawn from distributions. Running the simulation thousands of times with random inputs (plain Monte Carlo) would be accurate but prohibitively slow. UQ offers a smarter route: either express the output mathematically as a polynomial function of the uncertain inputs (PCE), so that mean, variance, and sensitivity indices follow analytically, or train a cheap surrogate model (Kriging) on a small number of strategically placed simulation runs and then sample from the surrogate at negligible cost. Both strategies turn expensive, black-box simulations into tractable problems, letting you quantify output variability and identify which inputs drive most of that variability.
History
The mathematical roots of polynomial chaos lie in Wiener's homogeneous chaos theory (1938), which represented Brownian-motion-driven processes using Hermite polynomial expansions. The method remained largely a theoretical tool until Ghanem and Spanos (1991) applied it to stochastic finite-element analysis. The decisive generalisation came from Xiu and Karniadakis (2002), who showed that the optimal polynomial basis depends on the input distribution — the Wiener–Askey scheme — enabling efficient PCE for a wide range of engineering problems. Concurrently, Kriging surrogates, originating in geostatistics (Krige, 1951; Matheron, 1963) and formalised as Gaussian process regression in machine learning, became a standard tool in computer experiment design. Smith (2013) provided the comprehensive synthesis of UQ theory, implementation, and application that now serves as the canonical graduate-level reference. Sparse PCE, adaptive strategies such as EGO (Efficient Global Optimisation), and deep-learning surrogates are active areas of current research.
When to use it
UQ is appropriate whenever a computational model has uncertain inputs — parameters measured with error, boundary conditions that are only approximately known, or material properties with natural variability — and the analyst needs to characterise how that uncertainty affects model outputs. It is especially valuable when the simulator is computationally expensive (minutes to hours per run), making direct Monte Carlo infeasible. Input uncertainties must be expressible as probability distributions; if only worst-case bounds are available without probabilistic information, interval or fuzzy methods may be more suitable. PCE is well-suited to moderate numbers of inputs (roughly up to 10–15 dimensions before the curse of dimensionality requires sparse variants); Kriging surrogates can handle somewhat higher dimensions but require careful design-of-experiments to keep the number of training runs manageable. Both approaches assume that the model is sufficiently smooth for a polynomial or Gaussian process approximation to be accurate.
Limitations and considerations
- PCE suffers from the curse of dimensionality: the number of polynomial terms grows rapidly with the number of uncertain inputs and polynomial degree, requiring sparse PCE or dimensionality-reduction strategies for high-dimensional problems.
- The accuracy of a Kriging surrogate depends critically on the design of experiments; too few or poorly placed training points yield unreliable predictions between them.
- Both methods assume the simulator response is smooth enough to be approximated well by polynomials or a Gaussian process; discontinuities or sharp gradients in the response surface degrade accuracy.
- The quality of the UQ result is only as good as the input distributions chosen; poorly specified input distributions propagate incorrect uncertainty into the output.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Provides a full characterisation of the output distribution — not just a mean or worst case — enabling rigorous probabilistic statements about model predictions.
- Sobol sensitivity indices identify which uncertain inputs drive output variability, guiding experimental design and data collection priorities.
- Once the surrogate is built, propagation and sensitivity analysis are computationally negligible, making UQ practical even for expensive simulators.
- PCE delivers analytical expressions for mean, variance, and sensitivity indices, avoiding Monte Carlo sampling noise in those quantities.
Applications
- Quantifying the effect of material property variability on structural reliability in aerospace and civil engineering simulations.
- Propagating parameter uncertainty through climate and earth-system models to characterise the range of projected temperature or sea-level outcomes.
- Assessing the impact of measurement uncertainty on medical device performance predictions in computational biomechanics.
- Supporting robust design optimisation by identifying which design parameters most reduce output variability, prioritising tighter manufacturing tolerances.
- Evaluating confidence intervals on epidemic model forecasts when transmission parameters are imprecisely known.
How it works
How it works
The analyst identifies all uncertain model inputs and assigns each a probability distribution — for example, a uniform, normal, or beta distribution — that represents the state of knowledge about that quantity. This step is foundational: the choice of input distributions directly determines the shape of the output uncertainty and must be grounded in physical knowledge or measured data rather than assumed arbitrarily.
For PCE, the output is expanded in a basis of orthogonal polynomials whose family matches each input distribution (Hermite polynomials for normal inputs, Legendre for uniform, and so on following the Wiener–Askey scheme). The polynomial coefficients are estimated from a set of model evaluations using regression or projection. For a Kriging surrogate, a design-of-experiments strategy — such as Latin hypercube sampling or optimal space-filling designs — selects a small number of representative input points, the simulation is run at those points, and a Gaussian process model is fitted to interpolate between them. The polynomial degree (for PCE) or covariance kernel (for Kriging) must be chosen with care to balance accuracy against computational cost.
Once the surrogate is built, uncertainty propagation is cheap. For PCE the output mean and variance emerge directly from the polynomial coefficients without additional simulations. For a Kriging surrogate, millions of Monte Carlo samples can be drawn from the input distributions and pushed through the fast surrogate rather than the expensive simulator, yielding the full output distribution at a fraction of the cost. The resulting distribution characterises the range and likelihood of all possible outputs given the input uncertainty.
Variance-based global sensitivity indices — known as Sobol indices — decompose the total output variance into contributions attributable to each input and to interactions among inputs. For a PCE surrogate these indices are obtained analytically from the polynomial coefficients; for a Kriging surrogate they are estimated from the propagated Monte Carlo sample. First-order Sobol indices report the fraction of variance explained by each input alone; total-order indices include all higher-order interactions involving that input. This step identifies which inputs matter most and which can be fixed to reduce uncertainty.
Common pitfalls
- Choosing input distributions without physical or empirical justification — assigning a normal distribution simply because it is convenient can produce output distributions that misrepresent the true uncertainty.
- Underestimating the polynomial degree or using a space-filling design that is too sparse relative to the response surface complexity leads to a surrogate that underfits and underestimates output variance.
- Ignoring interactions among inputs by reporting only first-order Sobol indices can be misleading when input interactions are the dominant source of variance; always inspect total-order indices.
- Applying PCE in high dimensions (above roughly 15 inputs) without sparse or adaptive truncation causes the number of terms to explode, making coefficient estimation unreliable.
- Treating the surrogate as perfect and not validating it against held-out simulator evaluations before using it for decision-making is a common and costly error.
Frequently asked questions
What is the difference between PCE and a Kriging surrogate?
PCE represents the model output as an explicit polynomial series in the uncertain inputs, chosen to be orthogonal with respect to the input distributions. Sobol sensitivity indices and moments follow analytically from the coefficients. Kriging treats the model output as a realisation of a Gaussian process and interpolates between training points, providing both a prediction and a prediction variance. PCE tends to be more efficient when the input dimension is low and the response is smooth and globally polynomial; Kriging is more flexible for irregular or locally varying responses and can be extended with active learning strategies such as EGO.
How many simulator runs are needed to build a reliable surrogate?
For PCE the minimum is roughly two to three times the number of polynomial basis terms, but more runs improve accuracy. For a Kriging surrogate a common rule of thumb is ten times the number of uncertain inputs for the initial design, with additional points added adaptively if budget allows. In both cases, validation against held-out runs is essential before the surrogate is used for decision-making.
What are Sobol indices and why do they matter?
Sobol indices are variance-based global sensitivity measures. The first-order index for an input reports the fraction of the total output variance attributable to that input alone. The total-order index includes all interactions between that input and others. Together they tell the analyst which inputs are worth measuring more precisely and which can be treated as fixed without significantly affecting the output uncertainty. Unlike local (derivative-based) sensitivity measures, Sobol indices are valid across the entire input distribution, not just near a nominal point.
Can UQ be applied when input distributions are unknown?
UQ in the probabilistic sense requires that input uncertainties be specified as probability distributions. When only ranges or bounds are available without distributional knowledge, uniform distributions are sometimes used as a conservative choice, but this assumption should be made explicit. If the analyst genuinely cannot assign a distribution — for example in deep uncertainty settings — scenario analysis or robust optimisation approaches may be more appropriate than probabilistic UQ.
Related methods
Sources
Xiu, D. & Karniadakis, G.E. (2002). The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations. SIAM Journal on Scientific Computing, 24(2), 619–644.
Open DOI referenceCopied.Smith, R.C. (2013). Uncertainty Quantification: Theory, Implementation, and Applications. SIAM.
ISBN: 978-1611973211Copied.