Skip to content
Back to search results

Factor Analysis

Introduced by
Louis Leon Thurstone
Year
1931
Type
Method
Family
dimension-reduction
Font size

What does it do?

Factor analysis is a statistical technique for identifying latent (unobserved) dimensions underlying observed variables, developed by Louis Leon Thurstone in the 1930s and formalized by Jöreskog (1969).

Exploratory factor analysis (EFA) discovers unknown factor structure from data; confirmatory factor analysis (CFA) tests hypothesized relationships between observed and latent variables. Essential in psychometrics (test development), organizational research (measuring constructs like leadership style), and biomedicine (identifying disease subtypes), factor analysis reduces dimensionality while revealing conceptual organization in multivariate data.

The intuition behind the method

Observed variables (questionnaire items, measurements) are often manifestations of underlying latent constructs (intelligence, depression severity, organizational climate) that cannot be directly measured. Factor analysis assumes correlations among observed variables reflect shared latent factors. The mathematics decomposes a correlation matrix into common variance (explained by factors) and unique variance (error and item-specific effects). Exploratory factor analysis asks: what latent structure best explains these correlations? Confirmatory factor analysis asks: does a theoretically proposed structure fit the data? This enables researchers to measure abstract concepts, validate assessment instruments, and understand dimensional organization in complex datasets.

History

Charles Spearman (1904) introduced factor analysis to explain positive correlations among mental abilities (general intelligence 'g'). Louis Leon Thurstone (1931) expanded to multiple factors and developed rotation methods (centroid, principal axes). Karl Jöreskog's (1969) confirmatory factor analysis and structural equation modeling revolutionized hypothesis testing in latent variable models. The Kaiser criterion (Kaiser, 1960) standardized eigenvalue-based factor extraction. Modern developments include item response theory (Rasch, 1960), robust estimation methods (distribution-free estimation), and multigroup/multilevel factor analysis for comparing structures across populations or nested data.

When to use it

Use EFA when exploring dimensional structure of a new measurement construct, refining scales, or understanding relationships among observed variables with no a priori theory. Psychological assessment: item selection for new depression screening, leadership assessment scale. Biomedical: identifying disease subtypes from symptom correlations. Use CFA when validating previously developed scales, testing theoretical measurement models, or comparing rival factor structures. Common in psychology: confirming unidimensional vs multidimensional model of a construct. Also use CFA as the measurement part of larger structural equation models linking latent variables to outcomes.

Limitations and considerations

  • EFA results are sample-dependent; factor structure may not replicate in new sample. Always cross-validate findings or use confirmatory factor analysis on independent sample.
  • Assumes linear relationships between variables and factors; nonlinear or threshold effects undetected.
  • Interpretation is subjective: similar eigenvalue structure and rotations can yield different interpretations. Multiple solutions may fit data equally well.
  • Requires large samples for stable estimates: general rule n > 5 × variables; small samples (n < 100) produce unstable, unreplicable factors.
  • CFA assumes measurement model is correct a priori; if true structure differs, model fit measures may not detect misspecification (local minima, equivalent models).

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Dimensionality reduction: many observed variables compressed into fewer interpretable latent factors, simplifying analysis and communication.
  • Reveals latent constructs: enables measurement of abstract concepts (intelligence, personality, organizational culture) that cannot be directly observed.
  • Statistical basis for test development: supports validity evidence for psychological and educational assessments; CFA formally validates measurement models.
  • Flexible: accommodates binary, ordinal, and continuous data; handles missing data via full-information maximum likelihood (FIML) in CFA.
  • Extends to complex models: foundation for structural equation modeling linking latent variables to outcomes, enabling mediation and moderation tests.

Applications

  • Psychological test development: constructing and validating clinical scales (depression, anxiety), personality inventories (Big Five), and cognitive ability tests; establishing validity evidence.
  • Organizational research: measuring constructs like job satisfaction, organizational commitment, leadership style; confirming multidimensional nature of complex concepts.
  • Quality of life assessment: reducing many health-related items to interpretable dimensions (physical, mental, social functioning) for outcome measurement.
  • Educational assessment: developing curriculum assessments, establishing validity of college entrance exams, reducing item pools while preserving conceptual coverage.
  • Biomedical research: identifying distinct disease phenotypes from symptom/genetic marker correlations (endotypes), reducing high-dimensional genomic data to interpretable latent dimensions.

How it works

How it works

1

Administer exploratory or confirmatory factor analysis by collecting data on multiple observed variables (typically >5) from a sufficiently large sample (general rule: n > 5 × number of variables, or absolute minimum n ≥ 100; ideally n > 200). For EFA: collect data without prior hypothesis about factor structure; variables should be reasonably intercorrelated (check correlation matrix; all correlations < 0.3 suggest factors unlikely). Assess sampling adequacy: Kaiser-Meyer-Olkin (KMO) test compares magnitude of correlations to partial correlations; KMO > 0.6 acceptable, > 0.7 good. Bartlett's test of sphericity should be significant (p < 0.05), indicating correlations differ from identity matrix. For CFA: specify a hypothesized factor structure a priori based on theory and prior research.

2

For exploratory factor analysis: extract factors using principal axis factoring (PAF) or maximum likelihood (ML). Kaiser criterion: retain factors with eigenvalues > 1. Scree plot: plot eigenvalues in decreasing order; retain factors before the elbow (diminishing returns). Parallel analysis: compare eigenvalues to those from random data; retain factors exceeding random benchmark. Rotate factors (varimax = orthogonal/uncorrelated, oblimin = oblique/correlated) to improve interpretability and achieve simple structure (each variable loads strongly on one factor). Report factor loadings (correlations between variables and factors): typically |loading| > 0.4 considered salient; loadings > 0.7 strong. Calculate communalities: proportion of each variable's variance explained by factors; low communalities (<0.3) suggest poor fit for that variable. For confirmatory factor analysis: fit the hypothesized model using structural equation modeling (SEM) software. Report model fit indices: CFI (Comparative Fit Index) > 0.95, TLI (Tucker-Lewis Index) > 0.95, RMSEA (Root Mean Square Error of Approximation) < 0.06, SRMR (Standardized Root Mean Square Residual) < 0.08 (Hu & Bentler, 1999).

3

In EFA, identify latent factors by examining which variables load highly on each factor; interpret factor conceptually based on variable content. For instance, if items about sadness, hopelessness, and anhedonia all load >0.6 on one factor, label it 'depression.' Report factor variances explained: sum of squared loadings divided by number of variables × 100% gives variance per factor; total across factors indicates overall fit. Reliability within each factor: calculate Cronbach's alpha (≥0.70 acceptable, ≥0.80 good) on variables loading on that factor. In CFA, examine standardized factor loadings (all should be > 0.4, ideally > 0.6); nonsignificant loadings suggest items poorly measure the construct. Model fit indices: meeting all criteria (CFI > 0.95, RMSEA < 0.06) supports the proposed model. Modification indices suggest alternative paths; however, data-driven model tweaking (post-hoc) compromises model validity—use cross-validation samples to test modified models. Report variance explained by latent variable for each observed variable (R²).

Common pitfalls

  • Data-driven EFA with small samples: capitalizes on chance; replicability in new sample is poor. Always conduct EFA on exploratory sample and validate in holdout sample.
  • Extracting factors based on Kaiser criterion (eigenvalue > 1) alone; scree plot and parallel analysis provide additional guidance and often suggest fewer factors.
  • Retaining uninterpretable factors: if a factor cannot be conceptually labeled or contains conceptually unrelated loadings, consider removing it rather than forcing interpretation.
  • CFA model specification errors: incorrectly specifying factor structure, omitting theoretically relevant paths, or allowing correlated errors without justification compromises validity.
  • Misinterpreting factor loading magnitude: loadings are correlations, not regressions; a loading of 0.5 explains only 25% of variable variance (R² = 0.5²).
  • Using inappropriate rotation: varimax (orthogonal) imposes uncorrelated factors; oblimin (oblique) allows correlations. Choose based on theory; orthogonal rotation often unrealistic.

Frequently asked questions

What is the difference between exploratory and confirmatory factor analysis?

Exploratory factor analysis (EFA) discovers latent structure from data without prior theory; you let the data guide how many factors and which variables load on which factors. Use EFA when investigating a new construct or scale. Confirmatory factor analysis (CFA) tests a hypothesized factor structure specified before analysis based on theory and prior research. CFA provides formal fit tests and is more statistically rigorous. Standard practice: conduct EFA on training sample to propose model, then validate with CFA on independent test sample (or use cross-validation). Never conduct EFA on the same sample used for CFA validation.

How many factors should I extract? What does the scree plot tell me?

Multiple criteria guide factor extraction: (1) Kaiser criterion (eigenvalue > 1) is traditional but often suggests too many factors. (2) Scree plot: eigenvalues plotted in decreasing order; retain factors before the elbow (point where diminishing returns begin). The 'scree' is the slope after the elbow. (3) Parallel analysis: compare empirical eigenvalues to those from random permutations; retain only factors exceeding random baseline. Parallel analysis is most statistically defensible. (4) Interpretability: retain only factors you can conceptually label and explain. Different criteria often yield different numbers; consult multiple methods and theory.

What does a factor loading of 0.6 mean, and how is it different from a reliability coefficient?

A factor loading of 0.6 is the correlation between an observed variable and a latent factor; it indicates the variable contributes to measuring that factor. The squared loading (0.6² = 0.36) is the proportion of variable variance explained by the factor. Loadings > 0.4 are minimally acceptable; > 0.6 is strong. A reliability coefficient (Cronbach's alpha, typically ≥0.70) measures internal consistency: the degree to which multiple variables measuring the same construct correlate with each other. Loading shows relation to the latent factor; alpha shows whether variables together consistently measure one construct. Both matter: high loadings and high alpha together support a homogeneous latent dimension.

How do I handle variables with low communalities?

Communality is the proportion of a variable's variance explained by all factors (sum of squared loadings across factors). Low communalities (<0.3) suggest the variable is poorly represented by the factors, either because: (1) it measures a unique construct not captured by the factor model, (2) it is unreliable/noisy, or (3) the factor solution is incorrect. Options: remove the variable (simplify model if theoretically justified), add factors (may overfit), or investigate whether the variable genuinely belongs in the scale. Always examine item content; sometimes low communality reflects a poorly worded item rather than conceptual misfit.

Related methods

Sources

  1. Thurstone, L. L. (1947). Multiple Factor Analysis. University of Chicago Press.

    Open DOI reference
    Copied.
  2. Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34(2), 183–202.

    Open DOI reference
    Copied.
  3. Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20(1), 141–151.

    Open DOI reference
    Copied.