Differential Evolution — Global Stochastic Optimiser
- Introduced by
- Rainer Storn & Kenneth Price
- Year
- 1997
- Type
- Population-based stochastic metaheuristic
What does it do?
Differential Evolution (DE), introduced by Rainer Storn and Kenneth Price in 1997, is a population-based stochastic optimisation algorithm designed for continuous parameter spaces.
It generates candidate solutions by combining vector differences between existing population members, making it a powerful and parameter-lean alternative to Genetic Algorithms and Particle Swarm Optimisation when the search landscape is non-convex, multimodal, or poorly suited to gradient-based methods.
The intuition behind the method
Imagine a group of explorers spread across an unknown landscape looking for the lowest valley. At each step, each explorer watches three randomly chosen colleagues, moves in the direction suggested by the difference between two of them scaled by a step-size factor, and then decides whether the new position or the old one is better terrain. Over many rounds, the whole group gradually clusters around the deepest valley it has collectively found. Differential Evolution works exactly this way: the 'difference' between two population vectors acts as the search direction, and the scale factor F controls how bold each step is.
History
Differential Evolution was introduced by Rainer Storn and Kenneth Price in their landmark 1997 paper in the Journal of Global Optimization. The algorithm grew out of Price's earlier work on genetic annealing and Storn's experiments with floating-point crossover. The original variant, now called DE/best/1/bin, proved competitive with — and often superior to — contemporary evolutionary strategies and genetic algorithms on standard benchmarks. A comprehensive review by Das, Mullick, and Suganthan (2016) catalogued over two decades of variants, adaptive schemes, and applications, establishing DE as one of the foundational methods in the metaheuristics literature.
When to use it
Differential Evolution is appropriate when the objective function is continuous, possibly non-differentiable or multimodal, and the search bounds for all parameters are known in advance. It requires no gradient information and no normality assumption, so it suits engineering design, hyperparameter tuning, calibration of simulation models, and any black-box optimisation task. A population size of roughly 10 times the number of parameters is a common starting point. DE should not be applied to purely discrete or combinatorial problems; it also needs at least 1 000 iterations to converge reliably — if the function evaluation budget is very tight, Bayesian optimisation is a more sample-efficient alternative.
Limitations and considerations
- Convergence can be slow if the number of iterations is insufficient; fewer than 1 000 generations often yields a suboptimal result.
- Performance is sensitive to the choice of F and CR — poorly tuned parameters can lead to premature convergence or excessively slow progress.
- Not designed for purely discrete or combinatorial search spaces; discrete extensions exist but are less standard.
- Each iteration requires evaluating the objective function NP times, which can be costly when each evaluation is computationally expensive.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Requires no gradient information — works with any black-box objective function, including non-differentiable and noisy ones.
- Very few control parameters (F, CR, NP) relative to its search power, making it comparatively easy to configure.
- Naturally handles multimodal landscapes where gradient-based optimisers converge to local optima.
- Population evaluations are independent and can be parallelised straightforwardly, reducing wall-clock time on multi-core systems.
- Competitive with Genetic Algorithms and Particle Swarm Optimisation on many benchmarks while using a simpler update rule.
Applications
- Calibrating simulation and agent-based models whose objective functions are non-analytic.
- Hyperparameter optimisation for machine-learning pipelines when the search space is continuous and bounded.
- Engineering design problems such as antenna design, structural optimisation, and controller tuning.
- Training neural networks and fitting non-linear models in engineering and natural sciences.
- Portfolio weight optimisation and parameter estimation in financial time-series models.
How it works
How it works
A population of NP candidate solution vectors is drawn uniformly at random within the predefined search bounds for each parameter. The search bounds must be specified by the user before the algorithm starts; DE cannot infer them from the problem. A good initial population covers the full feasible space so that the algorithm does not miss promising regions at the outset.
For each target vector in the population, three distinct vectors are selected at random. A mutant vector is formed by adding the scaled difference of two of them to the third: v = x_r1 + F*(x_r2 - x_r3), where F is the scale factor (typically 0.4–1.0). This difference-based step is what distinguishes DE from other evolutionary algorithms and gives it self-adaptive directional information without requiring gradient computation.
A trial vector is built by mixing the mutant vector and the original target vector element by element. Each element is taken from the mutant with probability CR (the crossover rate, typically 0.5–1.0) and from the target otherwise; at least one element is always taken from the mutant. This step controls the balance between exploration and exploitation: a high CR favours the mutant, a low CR preserves more of the current solution.
The trial vector is evaluated on the objective function and compared with the target vector. If the trial achieves an equal or better objective value, it replaces the target in the next generation; otherwise the target is retained unchanged. This greedy one-to-one selection keeps the population from deteriorating and ensures monotone improvement of the best solution found.
Mutation, crossover, and selection repeat for a sufficient number of generations (commonly 1 000 or more) until a convergence criterion is met — for example, the change in the best objective value falls below a tolerance threshold, or the maximum iteration count is reached. The best vector in the final population is the algorithm's solution.
Common pitfalls
- Running DE with too few iterations (below ~1 000) is the most common mistake — the algorithm has not had sufficient generations to converge, and the result should be treated as preliminary; switch to Bayesian optimisation if the evaluation budget is tight.
- Setting the scale factor F too low collapses diversity prematurely; setting it too high prevents convergence. Start with F in [0.4, 1.0] and tune empirically.
- Omitting or mis-specifying the search bounds is a critical error — DE uses bounds to initialise the population and will produce meaningless results if bounds are too narrow or absent.
- Treating DE as a one-shot run is risky on multimodal landscapes — run several independent trials from different random seeds and report the best solution found across runs.
Frequently asked questions
How do I choose the scale factor F and crossover rate CR?
The most widely cited defaults are F = 0.8 and CR = 0.9 for the DE/best/1/bin strategy. In practice, F controls step size — values around 0.4–1.0 are typical — and CR controls how much the mutant replaces the target. If the population converges too quickly, raise F or lower CR. Adaptive DE variants (jDE, JADE) adjust these parameters automatically during the run, which is a reasonable choice when manual tuning is impractical.
How many generations are needed?
As a rule of thumb, at least 1 000 generations are recommended. For high-dimensional problems (many parameters) or complex landscapes, more iterations may be necessary. Watch the objective value of the best solution across generations: if it has not changed for hundreds of iterations, DE has likely converged — or stagnated, in which case re-running with a fresh random seed is advisable.
How does DE compare with Genetic Algorithms and Particle Swarm Optimisation?
All three are population-based metaheuristics for continuous optimisation. DE typically uses fewer control parameters than GAs and has a simpler, deterministic update rule compared to PSO. On many standard benchmarks DE is competitive with or outperforms both, but performance is problem-dependent. The choice often comes down to familiarity and available implementations rather than a universal ranking.
Can DE handle integer or categorical parameters?
DE is designed for continuous spaces. Integer parameters can be accommodated by rounding the continuous solution to the nearest integer after each update, but this is an approximation. Purely categorical or combinatorial problems — such as scheduling or routing — are better served by methods designed for discrete spaces, such as Ant Colony Optimisation or Tabu Search.
Related methods
Sources
Storn, R. & Price, K. (1997). Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces. Journal of Global Optimization, 11(4), 341–359.
Open DOI referenceCopied.Das, S., Mullick, S. S., & Suganthan, P. N. (2016). Recent advances in differential evolution – An updated survey. Swarm and Evolutionary Computation, 27, 1–30.
Open DOI referenceCopied.