Skip to content
Back to search results

Agent-Based Sensitivity Analysis — Quantifying Parameter Influence in Complex Simulation Models

Introduced by
Adapted from global sensitivity analysis (Saltelli et al.) for agent-based models
Year
2000s–2010s
Type
Simulation-based sensitivity analysis
Family
Simulation / optimization
Font size

What does it do?

Agent-based sensitivity analysis (ABSA) applies sensitivity analysis techniques to agent-based models (ABMs) to determine which input parameters most strongly influence emergent outputs.

Because ABMs are stochastic and nonlinear, standard analytical derivatives are unavailable; ABSA uses designed simulation experiments — screening methods, variance-based indices, or regression-based surrogates — to rank parameter importance and guide model calibration and validation.

The intuition behind the method

An agent-based model can have dozens of parameters governing individual agent behaviors, interaction rules, and environment properties. Modelers rarely know in advance which parameters actually drive emergent outcomes. Agent-based sensitivity analysis systematically varies these inputs — often using efficient designs like Morris screening or Sobol sampling — and measures how much each parameter shifts the model output. The result is a ranked importance map: a few parameters typically dominate, allowing the analyst to focus calibration effort where it matters and to trust model conclusions only for well-understood parameter regions.

History

Sensitivity analysis for simulation models has roots in the variance-based global methods developed by Saltelli, Sobol, and colleagues through the 1990s and 2000s. Systematic application to agent-based models grew during the 2000s as ABMs became widespread in social science, ecology, and epidemiology. ten Broeke, van Voorn, and Ligtenberg (2016) provided a widely cited comparative review specifically tailored to ABMs, consolidating ABSA as a recognized methodological step in the ABM development cycle.

When to use it

Use ABSA when an agent-based model has multiple uncertain parameters and you need to know which matter most before calibration, when reviewers or stakeholders require evidence of model robustness, or when computational resources must be allocated efficiently across a large parameter space. Apply Morris screening for exploratory phases with many parameters; switch to Sobol indices when quantitative variance decomposition is needed for a smaller subset. Do NOT use ABSA as a substitute for model validation against real data; sensitivity alone cannot confirm that a model is correct. Avoid variance-based methods when individual ABM runs are extremely slow and the parameter space is very high-dimensional — surrogate emulator approaches may be needed instead.

Limitations and considerations

  • Variance-based Sobol indices require many model runs (thousands to tens of thousands), making them costly for slow ABMs.
  • Results depend on the assumed parameter ranges and distributions; poor range specification leads to misleading importance rankings.
  • Standard sensitivity indices assume output is a scalar; multivariate or time-series outputs require specialized extensions.
  • Stochastic noise in ABM outputs inflates variance estimates, requiring sufficient replication to separate parameter signal from simulation noise.
  • Sensitivity analysis does not identify the correct parameter values — it only ranks their influence.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Reveals which parameters drive emergent outcomes, focusing calibration and validation efforts efficiently.
  • Compatible with any ABM regardless of internal structure, since only model inputs and outputs are required.
  • Handles stochastic models by incorporating replication into the experimental design.
  • Morris screening scales to dozens or hundreds of parameters with modest computational cost.
  • Sobol indices provide a rigorous variance decomposition that decomposes total output variance into individual and interaction effects.
  • Supports transparent documentation of model assumptions for peer review and policy applications.

Applications

  • Epidemiological ABMs: identifying which transmission rate parameters most influence epidemic peak size.
  • Social simulation: determining which behavioral rules drive emergent segregation or opinion polarization.
  • Ecological modeling: ranking habitat and behavior parameters that govern species population dynamics.
  • Financial market ABMs: assessing which trader-strategy parameters shape price volatility.
  • Supply-chain simulation: finding which agent decision rules most affect system-wide disruption resilience.
  • Urban planning models: evaluating sensitivity of land-use emergence to zoning rule parameters.

How it works

How it works

1

Identify the ABM's input parameters (behavioral rules, interaction strengths, population sizes, initialization seeds) and specify feasible ranges or distributions for each. Choose one or more scalar output metrics that capture the emergent phenomenon of interest (e.g., final population size, segregation index, market price). Clarify whether the goal is screening (ranking importance) or quantification (estimating variance decomposition).

2

Select a sampling strategy matched to the goal and budget. Morris one-at-a-time (OAT) elementary-effects screening is efficient for many parameters (10–100+) with limited runs. Sobol quasi-random sequences or Latin hypercube sampling support variance-based indices but require more runs. Define the number of replications per parameter configuration to average out stochastic noise inherent in ABMs.

3

Execute the ABM for every sampled parameter configuration, repeating each configuration multiple times to estimate the mean and variance of the output. Store results in a structured experiment table. Parallelization is standard because ABMs are often computationally expensive; cloud or HPC resources are commonly used for large parameter sweeps.

4

Compute sensitivity metrics from the simulation results. For Morris screening, calculate the mean absolute elementary effect (mu*) and standard deviation (sigma) for each parameter. For Sobol variance-based analysis, estimate first-order indices S_i (direct effect) and total-order indices S_Ti (including interactions). For regression-based approaches, compute standardized regression coefficients (SRC) if the model is approximately linear.

5

Rank parameters by their sensitivity index. Distinguish influential parameters (high mu* or S_Ti) from non-influential ones (near zero). Investigate parameters with high sigma or high S_Ti - S_i, which signal strong nonlinearity or interaction. Use results to narrow calibration targets, justify parameter fixing in publications, and document model robustness. Cross-validate with domain knowledge and, where possible, against empirical data.

Common pitfalls

  • Using too few replications per configuration: stochastic ABM noise dominates, producing unreliable sensitivity estimates.
  • Conflating sensitivity with calibration: a parameter can be highly influential yet still be set incorrectly if empirical data are not used.
  • Ignoring interaction effects by relying solely on first-order indices when agent interactions create strong nonlinearities.
  • Applying Latin hypercube or Sobol sampling without checking that the parameter space is adequately covered at the chosen sample size.
  • Treating sensitivity results as universal: importance rankings can shift substantially when the studied output metric is changed.

Frequently asked questions

How many simulation runs do I need for reliable sensitivity estimates?

It depends on the method. Morris screening typically needs N × (k+1) runs, where k is the number of parameters and N is 10–50 trajectories — feasible even for slow models. Sobol variance-based indices require N × (k+2) runs with N often 500–2000 or more. Always run a convergence check by computing indices at increasing sample sizes until they stabilize.

Can I use ABSA on a model with time-series outputs rather than a single scalar?

Yes, but additional steps are required. Common approaches include computing sensitivity indices on summary statistics of the time series (mean, peak, final value), applying functional sensitivity analysis using principal components of the output trajectories, or using a surrogate model trained on the time-series data.

What is the difference between Morris screening and Sobol indices for ABMs?

Morris screening is a qualitative or semi-quantitative method that ranks parameters cheaply, making it ideal for initial screening of large parameter spaces. Sobol indices provide a full quantitative variance decomposition (first-order and total-order effects including interactions) but require far more runs. The recommended workflow is Morris first, then Sobol on the influential subset.

Does sensitivity analysis replace calibration?

No. Sensitivity analysis identifies which parameters matter and which can be fixed at nominal values; it does not tell you the correct parameter values. Calibration — matching model output to empirical data — is a separate step that should focus on the influential parameters identified by ABSA.

Is a surrogate emulator needed for computationally expensive ABMs?

For ABMs where a single run takes hours, running thousands of configurations for Sobol indices may be impractical. In such cases, a statistical surrogate (Gaussian process, polynomial chaos expansion, or random forest) trained on a limited ABM run budget can approximate the model and allow sensitivity analysis on the surrogate at negligible additional cost.

Related methods

Sources

  1. Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004). Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. John Wiley & Sons.

    ISBN: 9780470870938
    Copied.
  2. ten Broeke, G., van Voorn, G., & Ligtenberg, A. (2016). Which Sensitivity Analysis Method Should I Use for My Agent-Based Model? Journal of Artificial Societies and Social Simulation, 19(1), 5.

    Open DOI reference
    Copied.