Principal Component Analysis
- Introduced by
- Jolliffe, I.T. (textbook); Pearson & Hotelling (origins)
- Year
- 2002
- Type
- Unsupervised dimensionality reduction
What does it do?
Principal component analysis (PCA) represents centred observations along orthogonal directions ordered by the variance they retain.
Keeping fewer components gives a linear, lower-dimensional representation.
The intuition behind the method
The first component follows the direction of greatest variation. Subsequent components retain remaining variation while staying orthogonal to earlier components.
When to use it
Use PCA when a linear representation and variance preservation suit the research objective. Choose preprocessing and retained dimensions explicitly.
Limitations and considerations
- High retained variance does not establish scientific importance, predictive value or causality.
- Results depend on feature units, outliers and the stability of the observed covariance structure.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Orthogonal components summarize variation and support visualization.
Applications
- Exploratory visualization and feature compression.
How it works
How it works
Centre each feature. Scaling is a separate modelling choice when units or feature variances should receive comparable influence.
Decompose the centred data matrix, or its covariance matrix, and order the resulting directions by explained variance.
Project observations onto the selected directions.
Formula
\Sigma = \frac{1}{n-1} X^\top XFormula
\Sigma v = \lambda v
Formula
Z = X V
Common pitfalls
- Treating standardization or a fixed explained-variance threshold as universally mandatory.
- Interpreting components as latent causes or replacing a study-specific stability assessment with a universal minimum sample size.
Related methods
Sources
Jolliffe, I.T. (2002). Principal Component Analysis (2nd ed.). Springer.
Open DOI referenceCopied.scikit-learn Principal Component Analysis documentation
Open publication referenceCopied.