Elastic Net
- Introduced by
- Zou, H. & Hastie, T.
- Year
- 2005
- Type
- Regularized linear regression (L1 + L2 penalty)
What does it do?
Elastic Net is a regularized linear regression method introduced by Zou and Hastie in 2005 that blends the LASSO (L1) and Ridge (L2) penalties, so it performs variable selection and coefficient shrinkage at the same time.
It is designed for predictive and explanatory modelling on data with many, possibly correlated, predictors.
The intuition behind the method
Plain linear regression can overfit and behaves badly when predictors are correlated or there are more features than observations. LASSO can zero out coefficients to select variables but struggles when groups of predictors are correlated, while Ridge shrinks everyone smoothly but never drops a variable. Elastic Net mixes the two: it keeps LASSO's ability to set some coefficients exactly to zero while borrowing Ridge's stability so that correlated predictors are handled gracefully instead of being chosen arbitrarily.
History
Elastic Net was introduced by Hui Zou and Trevor Hastie in 2005 to overcome two known weaknesses of the LASSO: its erratic behaviour when predictors are highly correlated and its inability to select more variables than there are observations. By adding a Ridge-style L2 term to the LASSO's L1 penalty, they produced an estimator that both selects variables and stabilizes the solution, and it has become a standard tool for high-dimensional regression.
When to use it
Use Elastic Net for prediction or explanation on continuous, binary, or count outcomes when you have many predictors — especially correlated ones or more features than observations — and want automatic variable selection together with shrinkage. It assumes a roughly linear relationship and that features have been normalized. You need at least about 30 observations so that both hyperparameters can be tuned by cross-validation; below that, plain LASSO or Ridge is safer.
Limitations and considerations
- Assumes an approximately linear relationship between predictors and outcome.
- Features must be normalized first, or the penalty unfairly favours large-scale variables.
- Two hyperparameters (penalty strength and L1/L2 ratio) must be tuned, needing more cross-validation data than single-penalty methods.
- On small samples (n below about 30) the L1/L2 mixing ratio is unstable to optimize.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Combines LASSO variable selection with Ridge shrinkage in a single model.
- Handles correlated predictors gracefully, tending to keep or drop them as groups.
- Works when the number of features exceeds the number of observations.
- Distribution-light: does not require normally distributed outcomes and handles continuous, binary, and count features.
Applications
- High-dimensional regression where predictors outnumber observations.
- Genomic and biomarker studies with many correlated features.
- Predictive modelling that needs both variable selection and robust coefficients.
- Explanatory modelling where shrinking and pruning correlated predictors clarifies the signal.
How it works
How it works
The model fits coefficients by minimizing the usual squared-error loss plus a combined penalty: an L1 term that pushes some coefficients to exactly zero (selection) and an L2 term that shrinks them smoothly (stability). The strength of the total penalty is controlled by λ, and the balance between the two penalties by the mixing parameter (l1_ratio, written α here).
Formula
\min_{\beta}\; \tfrac{1}{2n}\,\lVert y - X\beta \rVert_2^2 + \lambda\left( \alpha \lVert\beta\rVert_1 + \tfrac{1-\alpha}{2}\lVert\beta\rVert_2^2 \right)Two hyperparameters govern the fit — the overall penalty strength and the L1/L2 mixing ratio — and they are chosen by cross-validation. Because there are two knobs to tune rather than one, Elastic Net needs enough data (at least about 30 observations) for the cross-validation grid to be reliable. Predictors should be standardized first so the penalty treats every feature on the same scale.
The fitted model returns a sparse set of coefficients: unimportant predictors are driven to zero and dropped, while correlated predictors tend to be kept or removed together as a group rather than one being picked at random. The surviving coefficients, having been shrunk, give a more stable and generalizable model than unpenalized regression.
Common pitfalls
- Fitting without standardizing the predictors first.
- Trying to tune both hyperparameters on too few observations, making the cross-validation grid unreliable.
- Reaching for Elastic Net when a single penalty suffices — LASSO alone if you only need selection, Ridge alone if you only need shrinkage.
- Interpreting shrunken coefficients as unbiased effect sizes.
Frequently asked questions
How is Elastic Net different from LASSO and Ridge?
LASSO uses only the L1 penalty (selection but unstable with correlated predictors); Ridge uses only the L2 penalty (stable shrinkage but never drops variables). Elastic Net blends both, controlled by a mixing ratio, so it selects variables and stays stable with correlated features.
Do I need to scale my features?
Yes. The penalty acts on the size of the coefficients, so features must be normalized first; otherwise large-scale variables are penalized differently and the selection is biased.
How many observations do I need?
At least about 30. Elastic Net tunes two hyperparameters by cross-validation, so it needs more data than single-penalty methods. Below 30, use plain LASSO if you need selection or Ridge if you need shrinkage.
What do the two hyperparameters control?
The overall penalty strength (λ) controls how much the coefficients are shrunk, and the mixing ratio (l1_ratio) sets the balance between the L1 selection penalty and the L2 shrinkage penalty. Both are chosen by cross-validation.
Related methods
Sources
Zou, H. & Hastie, T. (2005). Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society: Series B, 67(2), 301–320.
Open DOI referenceCopied.