Skip to content
Back to search results

Evolutionary Strategy (CMA-ES) — Covariance Matrix Adaptation

Introduced by
Nikolaus Hansen & Andreas Ostermeier
Year
2001
Type
Derivative-free continuous black-box optimizer
Font size

What does it do?

CMA-ES, short for Covariance Matrix Adaptation Evolution Strategy, is a modern derivative-free optimizer for continuous black-box functions introduced by Hansen and Ostermeier in 2001.

It maintains a population of candidate solutions drawn from a multivariate normal distribution and iteratively updates the distribution's mean, step size, and full covariance matrix to steer the search toward better regions of the parameter space. It has become the de-facto standard for continuous black-box optimization and is widely used in neural architecture search and reinforcement-learning policy optimization.

The intuition behind the method

Imagine you are searching for the lowest point in a hilly landscape but you cannot see the terrain — you can only probe individual locations and learn their heights. CMA-ES works like a guided swarm: it drops a cloud of probes around a current best guess, measures how good each probe is, and then reshapes the cloud for the next round. The cloud is not a simple sphere — it learns the elongated, tilted shape of the valley it is descending into by remembering which directions led to improvement. Over successive generations the cloud narrows and tilts to align with the curvature of the loss surface, letting the algorithm navigate long narrow valleys that would trap simpler methods.

History

Evolution strategies were introduced in the 1960s and 1970s as a family of population-based optimizers inspired by biological evolution. The covariance matrix adaptation mechanism was developed by Hansen and Ostermeier and fully described in their 2001 Evolutionary Computation paper, which demonstrated that completely derandomized self-adaptation — learning the full second-order geometry of the search landscape from the history of successful steps — dramatically outperformed earlier evolution strategies on ill-conditioned and non-separable problems. Hansen's 2016 tutorial consolidated the algorithm and its theoretical properties, making CMA-ES the standard reference method for continuous black-box optimization benchmarking.

When to use it

CMA-ES is appropriate when the objective function is continuous, possibly non-convex, multimodal, or noisy, and when gradient information is unavailable or too expensive to compute. The ideal dimensionality range is roughly 10 to 1000 parameters; below 10 dimensions simpler methods are often sufficient, and above 1000 the quadratic memory cost of the covariance matrix can become prohibitive. There is no minimum sample size requirement in the statistical sense — the algorithm itself generates its own evaluation points. CMA-ES is a natural fit for hyperparameter tuning, neural architecture search, reinforcement-learning policy optimization, and engineering design problems where the simulator provides only black-box outputs.

Limitations and considerations

  • Quadratic memory and per-generation computational cost in the number of dimensions limits applicability beyond roughly 1000 parameters without dimensionality-reduction variants.
  • Converges to a local optimum in the absence of restarts; for highly multimodal landscapes, multiple restarts (IPOP-CMA-ES or BIPOP-CMA-ES) are needed.
  • Not suited to discrete or combinatorial search spaces — the method is inherently defined over continuous real-valued vectors.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Requires no gradient information — works on any callable objective function, including noisy or simulation-based ones.
  • Learns the local curvature of the search landscape through covariance adaptation, allowing it to navigate elongated and ill-conditioned search spaces that defeat gradient-free methods with fixed search distributions.
  • Invariant to order-preserving transformations of the objective function and to rotations of the search space, making it robust across a wide variety of problem geometries.
  • Well-suited for moderate-to-high-dimensional continuous problems (10–1000 parameters) with a single tunable hyperparameter — the initial step size.

Applications

  • Hyperparameter tuning for machine-learning models, including neural network architectures and reinforcement-learning agents.
  • Neural architecture search — optimizing discrete-encoded architecture parameters via continuous relaxation.
  • Reinforcement-learning policy optimization when the policy is parameterized by a continuous vector.
  • Engineering design optimization, such as aerodynamic shape optimization and structural parameter fitting, where only simulator outputs are available.
  • Scientific model calibration — fitting simulation models to experimental data when the likelihood is non-differentiable or computationally expensive.

How it works

How it works

1

The algorithm is seeded with an initial solution vector and an initial step size (sigma). The covariance matrix starts as the identity matrix — a perfectly spherical search cloud. Population size is set, typically around 4 + floor(3 * ln(dimension)).

2

At each generation a population of candidate solutions is sampled from the current multivariate normal distribution defined by the mean vector, the step size, and the covariance matrix. Because the distribution is multivariate normal, samples reflect both the current best estimate and the learned shape of the search space.

3

Each candidate solution is passed to the objective function and receives a fitness score. No gradient information is required — only the scalar objective value at each sampled point is needed. This makes CMA-ES suitable for noisy, non-differentiable, and simulation-based objectives.

4

The mean is moved toward the better-ranked solutions. The covariance matrix and step size are updated using two complementary adaptation paths — rank-one and rank-mu updates — so that directions that produced good improvements become more likely in the next generation. This is the core mechanism by which CMA-ES learns the local geometry of the landscape.

5

The algorithm repeats the sample–evaluate–update cycle until a stopping criterion is met: a maximum number of function evaluations, a tolerance on the change in the mean or step size, or a detected degeneration of the covariance matrix. The best solution found across all generations is returned.

Common pitfalls

  • Setting the initial step size too small traps the search in a narrow neighbourhood and prevents exploration; too large and early generations waste evaluations far from the optimum. A common rule of thumb is to set sigma so that the optimum lies within three sigma of the starting point.
  • Applying CMA-ES to very low-dimensional problems (fewer than five parameters) is usually wasteful — simpler grid search or Nelder-Mead suffices, and the population overhead of CMA-ES brings no benefit.
  • Running only a single CMA-ES instance on a multimodal landscape risks missing the global optimum; use multi-start or population-based restart strategies (IPOP/BIPOP) when the landscape is known to be highly multimodal.
  • Treating the returned best solution as the true global optimum without restart verification is a common mistake — always confirm with multiple independent runs when the objective is highly non-convex.

Frequently asked questions

How does CMA-ES differ from a simple genetic algorithm?

A classic genetic algorithm maintains a population and applies crossover and mutation with fixed operators. CMA-ES instead models the population as samples from a multivariate normal distribution and continuously adapts that distribution — its mean, step size, and full covariance matrix — based on which solutions were successful. This allows CMA-ES to learn the shape of the loss landscape and navigate correlated, ill-conditioned spaces far more efficiently than a genetic algorithm with fixed operators.

Does CMA-ES guarantee finding the global optimum?

No. CMA-ES is a local search method in the sense that a single run can converge to a local optimum. For multimodal landscapes, multiple independent restarts — possibly with increasing population sizes (IPOP-CMA-ES) — are recommended to improve coverage of the search space.

What is the most important hyperparameter to set?

The initial step size sigma is the single most influential hyperparameter. A good heuristic is to set it so that the expected optimum lies within roughly three sigma of the starting point. Population size and the maximum number of evaluations are secondary controls; the default population size formula (4 + floor(3 * ln(dimension))) works well in practice.

Can CMA-ES handle noisy objective functions?

Yes. Because CMA-ES uses only rank-order information from the population — not absolute function values — it is relatively robust to noise. For highly noisy evaluations, increasing the population size or averaging multiple evaluations per candidate can further improve robustness.

Related methods

Sources

  1. Hansen, N. & Ostermeier, A. (2001). Completely Derandomized Self-Adaptation in Evolutionary Strategies. Evolutionary Computation, 9(2), 159-195.

    Open DOI reference
    Copied.
  2. Hansen, N. (2016). The CMA Evolution Strategy: A Tutorial. arXiv:1604.00772.

    Open publication reference
    Copied.