Surrogate-Based Optimization — Metamodel-Assisted Design
- Introduced by
- Sacks, Welch, Mitchell & Wynn (computer experiments framework, 1989); Kriging popularised by Matheron (1963)
- Year
- 1989 (computer experiments formulation)
- Type
- Metamodel-assisted black-box optimization
What does it do?
Surrogate-based optimization, formalized in the computer-experiments framework of Sacks et al. (1989) and popularized for engineering by Forrester et al. (2008), replaces a prohibitively expensive simulation or physical experiment with a cheap approximate model — called a surrogate or metamodel — and then optimizes that surrogate instead.
The surrogate is typically a Kriging (Gaussian Process), Radial Basis Function, or polynomial response surface fitted to a small set of carefully chosen design evaluations and periodically updated as the search progresses.
The intuition behind the method
Imagine that every time you want to test a new aircraft wing shape you must run a full computational fluid dynamics (CFD) simulation that takes 48 hours. Running thousands of such evaluations to find the best shape is impossible. Surrogate-based optimization sidesteps this by first running a small batch of simulations at well-spread design points, fitting a fast mathematical stand-in (the surrogate) that predicts performance for any untested design in seconds, optimizing that stand-in, and then strategically picking the most promising new design to actually simulate. The surrogate gets smarter with each new data point added, and eventually the expensive simulator is called only where it truly matters.
History
The statistical foundations were laid by Sacks, Welch, Mitchell, and Wynn in their landmark 1989 paper "Design and Analysis of Computer Experiments" (Statistical Science), which introduced the Kriging framework for emulating deterministic computer codes. Kriging itself traces to geostatistics work by Georges Matheron in the 1960s. The approach was systematised for engineering design optimisation by Forrester, Sobester, and Keane in their 2008 Wiley monograph, which established Latin Hypercube Sampling, cross-validation, and expected-improvement infill as the standard pipeline. Since then the field has expanded to include multi-fidelity surrogates, Bayesian optimization (a closely related probabilistic formulation), and deep-learning surrogates for very high-dimensional problems.
When to use it
Surrogate-based optimization is the right choice when the true objective function is so expensive to evaluate — through CFD, finite-element analysis, lab experiments, or multi-physics simulations — that only tens to a few hundred evaluations are affordable. The method requires that the true response be sufficiently smooth for a surrogate to approximate it; highly discontinuous or stochastic functions may require specialized surrogates or variance-reduction techniques. The design space should be continuous. A space-filling initial design (Latin Hypercube Sampling) and mandatory cross-validation of the surrogate are non-negotiable prerequisites.
Limitations and considerations
- Surrogate accuracy degrades in high-dimensional spaces (the curse of dimensionality); typically practical up to ~20–50 design variables without dimensionality reduction.
- If the true function is highly discontinuous or multimodal, the surrogate may mis-represent it and guide the optimizer to a false optimum.
- The initial training budget must be set carefully; too few points produce a poor surrogate, too many defeat the purpose of avoiding expensive evaluations.
- Kriging fitting and optimization of the acquisition function add computational overhead that grows with training set size.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Dramatically reduces the number of expensive true evaluations needed to find a near-optimal design.
- Kriging surrogates provide uncertainty estimates that enable principled infill strategies such as expected improvement.
- Works on black-box problems where no gradient information is available.
- Applicable across engineering, science, and any domain where simulation cost dominates.
- Reusable surrogate can answer multiple design questions without additional true evaluations.
Applications
- Aerodynamic shape optimization of aircraft wings and turbine blades using CFD simulations.
- Structural optimization in finite-element analyses where each simulation takes hours.
- Automotive crashworthiness and NVH (noise, vibration, harshness) design under tight evaluation budgets.
- Chemical process and reactor design where physical experiments are costly and slow.
- Hyperparameter tuning in machine learning when training a model is itself expensive.
- Pharmaceutical formulation development and clinical trial design.
How it works
How it works
A space-filling experimental design — most commonly Latin Hypercube Sampling — is used to place an initial set of design points across the entire feasible space. The goal is maximal spread so that the surrogate can be trained with minimal bias toward any region.
Each initial design point is passed to the true expensive evaluator: a CFD solver, a finite-element analysis, a physical prototype test, or any other high-fidelity function. This step is the computational bottleneck; its output forms the training dataset for the surrogate.
A surrogate model — Kriging, Radial Basis Function, polynomial response surface, or a neural network — is fitted to the training data. Kriging is preferred in engineering design because it provides not only a prediction but also a quantified uncertainty estimate at every unvisited point.
The surrogate fit is checked by cross-validation or by comparing surrogate predictions at a held-out set of test points against true evaluations. Metrics such as R-squared, RMSE, and maximum absolute error indicate whether the surrogate represents the true function well enough to guide optimization. If validation fails, additional training points are added.
A cheap optimizer — gradient-based or evolutionary — searches the fitted surrogate for the optimum. Because evaluating the surrogate costs almost nothing, many thousands of candidate solutions can be assessed in seconds.
The design predicted to be optimal (or most informative, under expected-improvement criteria) is passed back to the expensive evaluator. Its true response is added to the training set and the surrogate is refitted. This loop continues until a stopping criterion — budget, convergence, or accuracy target — is met.
Common pitfalls
- Skipping cross-validation of the surrogate is the single most common error — a poorly fitting surrogate will systematically mislead the optimizer.
- Using a uniform random initial design instead of a space-filling Latin Hypercube leaves regions of the design space poorly covered, creating surrogate blind spots.
- Applying the method to problems with stochastic responses (noisy simulations or physical experiments) without accounting for noise in the surrogate model leads to over-fitting to noise rather than the true trend.
- Stopping the update loop too early — before the surrogate has converged — can mean the reported optimum is a surrogate artifact rather than a true design improvement.
Frequently asked questions
Which surrogate type should I choose — Kriging, RBF, or polynomial?
Kriging (Gaussian Process) is the default for engineering design because it provides both a prediction and a pointwise uncertainty estimate, enabling principled infill strategies such as expected improvement. Radial Basis Functions are computationally lighter and scale somewhat better. Polynomial response surfaces are simple and interpretable but struggle with nonlinear or multimodal responses. Start with Kriging unless dimensionality or training-set size makes it impractical.
How many initial design points do I need?
A common rule of thumb is 10 times the number of design variables for the initial Latin Hypercube sample, though this depends on the smoothness of the response. Validation metrics (cross-validation R-squared, RMSE) should confirm the surrogate is adequate before optimization proceeds. Additional points are added through the infill loop rather than all at once.
Is surrogate-based optimization the same as Bayesian optimization?
They are closely related but not identical. Bayesian optimization is a specific form of surrogate-based optimization that uses a Gaussian Process surrogate with a probabilistic acquisition function (such as expected improvement) to select infill points. The broader surrogate-based optimization framework includes non-probabilistic surrogates (RBF, polynomial) and other infill criteria, so Bayesian optimization is a subset.
What if my simulation is noisy rather than deterministic?
Standard Kriging assumes a deterministic response. For stochastic simulators, a noise term must be added to the Kriging covariance model (nugget parameter or stochastic Kriging). Without this, the surrogate will over-fit to noise rather than the true trend and the optimizer will be misled.
Related methods
Sources
Forrester, A., Sobester, A., & Keane, A. (2008). Engineering Design via Surrogate Modelling: A Practical Guide. Wiley.
Open publication referenceCopied.Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989). Design and Analysis of Computer Experiments. Statistical Science, 4(4), 409-423.
Open DOI referenceCopied.