Skip to content
Back to search results

Principal Component Analysis

Introduced by
Jolliffe, I.T. (textbook); Pearson & Hotelling (origins)
Year
2002
Type
Unsupervised dimensionality reduction
Font size

What does it do?

Principal component analysis (PCA) represents centred observations along orthogonal directions ordered by the variance they retain.

Keeping fewer components gives a linear, lower-dimensional representation.

The intuition behind the method

The first component follows the direction of greatest variation. Subsequent components retain remaining variation while staying orthogonal to earlier components.

When to use it

Use PCA when a linear representation and variance preservation suit the research objective. Choose preprocessing and retained dimensions explicitly.

Limitations and considerations

  • High retained variance does not establish scientific importance, predictive value or causality.
  • Results depend on feature units, outliers and the stability of the observed covariance structure.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Orthogonal components summarize variation and support visualization.

Applications

  • Exploratory visualization and feature compression.

How it works

How it works

Centre each feature. Scaling is a separate modelling choice when units or feature variances should receive comparable influence.

Decompose the centred data matrix, or its covariance matrix, and order the resulting directions by explained variance.

Project observations onto the selected directions.

1
Formula
\Sigma = \frac{1}{n-1} X^\top X
2
Formula
\Sigma v = \lambda v
3
Formula
Z = X V

Common pitfalls

  • Treating standardization or a fixed explained-variance threshold as universally mandatory.
  • Interpreting components as latent causes or replacing a study-specific stability assessment with a universal minimum sample size.

Related methods

Sources

  1. Jolliffe, I.T. (2002). Principal Component Analysis (2nd ed.). Springer.

    Open DOI reference
    Copied.
  2. scikit-learn Principal Component Analysis documentation

    Open publication reference
    Copied.