Approximate Bayesian Computation — Likelihood-Free Inference
- Year
- 2002
- Type
- Simulation-based Bayesian inference
What does it do?
Approximate Bayesian Computation (ABC) is a family of simulation-based inference methods that estimate posterior distributions without requiring an analytically tractable likelihood function.
Introduced by Beaumont, Zhang and Balding (2002) in the context of population genetics, ABC replaced the intractable likelihood with repeated model simulation and a comparison of summary statistics between simulated and observed data.
The intuition behind the method
Classic Bayesian inference asks: given the observed data, how probable are the model parameters? Answering that question normally requires computing the likelihood — the probability of the data under each parameter setting. For many realistic models in biology, ecology, and economics, that calculation is either analytically impossible or computationally prohibitive. ABC sidesteps it entirely: instead of computing the likelihood, it simulates data from the model, compresses it into a few summary statistics, and accepts a parameter proposal only when those simulated summaries are close enough to the observed summaries. The accepted proposals accumulate to form an approximate posterior. The approximation is exact in the limit as the tolerance threshold approaches zero, and it improves steadily as that threshold tightens — at the cost of accepting fewer proposals.
History
The foundations of likelihood-free inference trace back to rejection-sampling ideas used informally in population genetics, but the method was formalised and named by Beaumont, Zhang and Balding in their landmark 2002 Genetics paper on inferring demographic history. Marjoram et al. (2003) introduced the ABC-MCMC variant to improve efficiency, and Sisson, Fan and Beaumont (2007) developed the sequential Monte Carlo (SMC-ABC) framework that adaptively reduces the tolerance across populations. The 2018 Handbook of Approximate Bayesian Computation, edited by Sisson, Fan and Beaumont, consolidates two decades of methodological development. More recently, neural density estimators and amortised inference have been proposed as ways to learn the posterior directly from simulations, extending ABC into the machine-learning era.
When to use it
ABC is the right choice when the model can be simulated forward but its likelihood function cannot be evaluated analytically — a situation that arises frequently in population genetics, epidemiology, ecology, and agent-based economic modelling. It handles continuous, binary, and count outcome variables and places no normality requirement on the data. There is no hard minimum sample size, but the summary statistics must carry adequate information about the parameters. When the simulation budget is large and tolerance can be set low, ABC approximates the true posterior closely; when simulations are expensive, SMC-ABC or neural posterior estimation variants are preferable. If the likelihood is tractable, standard MCMC is more efficient.
Limitations and considerations
- Naive rejection ABC requires an enormous number of simulations to achieve a low tolerance threshold and meaningful acceptance rate, making it computationally expensive for high-dimensional parameter spaces.
- The quality of the posterior approximation depends critically on the choice of summary statistics; poorly chosen summaries can produce a biased or high-variance posterior.
- There is no analytic guarantee that the chosen summary statistics are sufficient, so information is generally lost relative to the full likelihood.
- Fewer than roughly 1 000 total accepted proposals typically yields an unreliable posterior approximation; when the simulation budget is limited, standard MCMC should be considered.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Operates without a closed-form likelihood, opening Bayesian inference to complex generative models that are easy to simulate but hard to evaluate analytically.
- Flexible enough to handle continuous, binary, and count outcomes under any structural model the analyst can encode as a simulator.
- Variants such as SMC-ABC and ABC-MCMC substantially reduce the number of simulations needed compared to naive rejection sampling.
- The posterior approximation converges to the true posterior as the tolerance threshold approaches zero.
Applications
- Inferring population size, migration, and divergence history in population genetics from allele frequency data.
- Estimating transmission rates and contact parameters in epidemic models where individual-level likelihood is intractable.
- Calibrating agent-based economic models to observed macro-level statistics without a closed-form equilibrium likelihood.
- Fitting stochastic ecological models — predator-prey dynamics, species spread — to field observation summaries.
- Parameter estimation in systems biology models where mechanistic ODEs are replaced by stochastic simulations.
How it works
How it works
The analyst specifies a prior distribution over the model parameters, encoding what is known or assumed before seeing the data. This prior is the source from which candidate parameter values will be drawn throughout the algorithm, so it should cover the plausible range of the parameters.
A parameter proposal is drawn from the prior (or, in adaptive variants, from an importance distribution). The full stochastic model is then run forward using that proposal to generate a synthetic dataset. This simulation step is the computational core of ABC and may be run thousands or millions of times.
Both the synthetic dataset and the observed dataset are compressed into a vector of summary statistics — quantities chosen to carry as much information about the parameters as possible. Ideally the summary statistics are sufficient statistics, but in practice they are chosen based on domain knowledge and diagnostic checks.
The distance between the simulated summary statistics and the observed summary statistics is computed under a chosen metric. If that distance falls within a pre-specified tolerance threshold epsilon, the parameter proposal is accepted; otherwise it is rejected. Smaller epsilon values yield a more accurate posterior approximation but a lower acceptance rate.
The collection of accepted parameter proposals forms a sample from the approximate posterior distribution. This sample can be used to compute posterior means, credible intervals, and predictive distributions in exactly the same way as a standard Bayesian posterior sample. In SMC-ABC, the tolerance is reduced across a sequence of populations to improve efficiency.
Common pitfalls
- Setting the tolerance epsilon too loosely produces an approximation that can differ substantially from the true posterior — always verify that tightening epsilon stabilises the posterior.
- Summary statistics that do not carry sufficient information about the parameters of interest bias the posterior toward the prior; validate statistics by checking whether they change when the true parameter changes.
- Running fewer than 1 000 simulations yields a posterior sample too thin to be reliable; if simulation cost prevents reaching this threshold, switch to MCMC or SMC-ABC.
- The prior must cover the plausible parameter space adequately; a prior that is too narrow will exclude regions of the posterior, while one that is far too broad wastes simulation budget on implausible regions.
Frequently asked questions
How is ABC different from standard MCMC?
Both methods produce samples from a posterior distribution, but MCMC evaluates the likelihood at each proposed parameter value, whereas ABC replaces that evaluation with a forward simulation and a summary-statistic comparison. Use ABC when the likelihood is intractable; use MCMC when it can be computed.
How do I choose summary statistics?
Summary statistics should capture as much information about the parameters of interest as possible. Common practice is to use domain knowledge (e.g., mean, variance, allele frequencies) and to validate choices by checking whether the statistics change when the true parameter changes. Sufficient statistics exist only for exponential-family models; for other models, some information loss is unavoidable.
How small does the tolerance epsilon need to be?
There is no universal rule. A practical approach is to tighten epsilon incrementally and check whether the posterior shape stabilises. The posterior approximation improves as epsilon decreases, but the acceptance rate falls accordingly and more simulations are needed. SMC-ABC automates this trade-off by adaptively reducing epsilon across populations.
When should I use SMC-ABC instead of rejection ABC?
When simulation is expensive, naive rejection ABC wastes most simulations because the acceptance rate can be extremely low at a tight tolerance. SMC-ABC uses a sequence of intermediate tolerance thresholds and importance-weighted populations to concentrate simulations near the posterior, making it far more efficient for the same final epsilon.
Related methods
Sources
Beaumont, M.A., Zhang, W. & Balding, D.J. (2002). Approximate Bayesian Computation in Population Genetics. Genetics, 162(4), 2025-2035.
Open DOI referenceCopied.Sisson, S.A., Fan, Y. & Beaumont, M.A. (Eds.) (2018). Handbook of Approximate Bayesian Computation. Chapman & Hall/CRC.
Open DOI referenceCopied.