Skip to content
Back to search results

Theil Index for Health Inequality

Introduced by
Henri Theil (entropy index); Sam Harper & John Lynch; Luisa Borrell & Makram Talih (health-disparity applications)
Year
2008
Type
Entropy-based decomposable index of inequality applied to health disparities
Family
Health-inequality summary measures
Font size

What does it do?

The Theil index is an entropy-based measure of inequality, adapted from information theory, that quantifies how disproportionately a quantity is distributed across groups - and in social epidemiology it measures how unequally ill health (or its burden) is shared across population subgroups defined by race-ethnicity, area socioeconomic position, or other categories.

Its defining advantage over simpler disparity measures is additive decomposability: total inequality splits cleanly into a between-group component and a within-group component, so analysts can ask how much overall health disparity is due to differences among, say, racial groups versus differences among areas inside those groups. Harper, Lynch, and colleagues' 2008 American Journal of Epidemiology overview placed the Theil index among the recommended summary measures for monitoring social disparities in health, illustrating it with trends in U.S. lung-cancer incidence. Borrell and Talih's 2011 Statistics in Medicine paper introduced a symmetrized version that removes the standard index's dependence on which group is treated as the reference and derived its variance for complex survey data. Because it is reference-free, decomposable, and population-weighted, the Theil index is well suited to multi-group, nested disparity analysis where ordered-rank measures like the concentration index do not apply.

The intuition behind the method

Inequality measures all try to capture the same idea: are the things we care about - cases of disease, years of life lost - spread across people in proportion to how many people there are, or are they piled up on some groups? The Theil index answers this by comparing each group's share of the health burden with its share of the population. If a group holds exactly its fair share of disease, it contributes nothing; if it holds far more (or far less) than its population share, it contributes a lot, with the contribution weighted by an entropy (logarithmic) term that penalizes large disproportions. The number you get is zero when every group bears disease in proportion to its size and grows as the burden concentrates. The truly useful trick is that this total can be carved into two pieces: how much inequality comes from differences between broad groups, and how much from differences among the smaller units nested inside them. That lets you say, for example, that most racial disparity in an outcome arises between racial groups rather than from where people happen to live - an insight that points policy at the right level.

History

Henri Theil derived his inequality index in the 1960s from Shannon's information-theoretic entropy, giving economists a measure of income inequality that, unlike many alternatives, decomposes additively into between- and within-group parts. Public-health researchers later adopted it to summarize health disparities across population subgroups. Harper, Lynch, and colleagues' 2008 American Journal of Epidemiology methods overview catalogued the Theil index among the recommended absolute and relative measures for monitoring social disparities in cancer, demonstrating it on U.S. lung-cancer incidence trends and embedding it in the National Cancer Institute's disparity-measurement tools. Borrell and Talih's 2011 Statistics in Medicine paper addressed two practical weaknesses - reference dependence and the lack of design-based inference - by introducing a symmetrized Theil index and deriving its sampling variance for complex surveys, broadening the method's use in population health-equity research.

When to use it

Use the Theil index when health inequality is structured across multiple, possibly nested, categorical groups - race-ethnicity, area deprivation strata, regions within countries - and you want a single decomposable summary that separates between-group from within-group inequality. It is the right tool when there is no natural socioeconomic ranking to exploit (so the concentration index or slope index do not apply) and when the key question is at what level disparity is concentrated. It also suits hierarchical surveillance, such as tracking how much of a national disparity arises between regions versus within them. The Theil index is less appropriate when groups are inherently ordered and you want a gradient measure, when zero-burden groups create undefined logarithms (requiring care), or when you need an absolute, unit-bearing measure of the gap. For survey data, pair the symmetrized form with design-based variance estimation to ensure valid inference.

Limitations and considerations

  • Undefined when a group has zero burden or zero population, requiring adjustments or pooling of sparse cells.
  • Being relative and unit-free, it conveys no information about the absolute size of disparities.
  • The standard (non-symmetrized) index depends on the reference and is sensitive to high-frequency groups.
  • Its entropy scale is less intuitive to non-specialists than a simple rate ratio or rate difference.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Additively decomposable into between-group and within-group inequality, pinpointing the level at which disparity operates.
  • Handles many groups at once without requiring an ordinal socioeconomic ranking.
  • The symmetrized version is reference-invariant, avoiding dependence on an arbitrary baseline group.
  • Has design-based variance formulas for complex survey data, enabling confidence intervals and trend tests.

Applications

  • Decomposing racial-ethnic disparities in cancer incidence into between-group and within-area components.
  • Quantifying how much of a national health disparity arises between regions versus within them.
  • Tracking trends in multi-group health inequality where no single socioeconomic ranking applies.
  • Measuring disparities in dental caries or other outcomes across subgroups using complex survey data.
  • Comparing inequality structure across countries or time using the reference-invariant symmetrized index.

How it works

How it works

1

The analysis begins by defining mutually exclusive, exhaustive groups - racial-ethnic categories, area socioeconomic strata, or a nesting of both - and computing two sets of shares: each group's share of the total health burden (its proportion of cases, deaths, or other events) and its share of the total population. These paired shares are the raw material of the index, because Theil's measure is fundamentally a comparison of burden-share against population-share. When the data come from a complex survey, the shares must be computed with the appropriate sampling weights so they represent the target population. Defining sensible groups and computing correct shares is the foundation on which the entropy calculation and its decomposition rest.

Formula
s_j = \frac{h_j}{\sum_k h_k},\quad p_j = \frac{N_j}{\sum_k N_k},\quad j = 1,\dots,J
2

The Theil index is then computed as the population-weighted sum, across groups, of each group's burden share times the logarithm of the ratio of its burden share to its population share. Equivalently it is an average of the rate ratio (burden share over population share) weighted by population share and the log of that ratio. The logarithm gives the index its entropy character: groups carrying far more or far less than their proportional burden dominate the total. The result is zero when every group's burden matches its population share - perfect proportionality - and increases as disease concentrates, providing a single scalar summarizing multi-group disproportion regardless of how many groups there are.

Formula
T = \sum_{j=1}^{J} s_j \ln\!\frac{s_j}{p_j} = \sum_{j} p_j\,r_j \ln r_j,\quad r_j = s_j/p_j
3

The signature step is additive decomposition. When groups are nested - for example areas within race-ethnicity, or counties within states - the total Theil index splits exactly into a between-group term, computed by collapsing each broad group to its aggregate shares, and a within-group term, the population-weighted average of the Theil indices computed inside each broad group. This lets the analyst attribute total health inequality to the level at which it operates: a large between-component says the broad grouping captures most of the disparity, while a large within-component says important inequality persists inside those groups. This decomposability, which simpler measures lack, is the main reason the Theil index is favored for structured, hierarchical disparity analysis.

Formula
T = \underbrace{\sum_g S_g \ln\!\frac{S_g}{P_g}}_{T_{\text{between}}} + \underbrace{\sum_g S_g\,T_g}_{T_{\text{within}}}
4

A known drawback of the standard Theil index is that it is not symmetric in how it treats the reference: results can depend on whether one measures the disproportion of health relative to population or vice versa, and the index can be unduly influenced by groups with high disease frequency. Borrell and Talih's symmetrized version averages the two directions, producing a reference-invariant measure that does not hinge on an arbitrary choice of which distribution is the baseline. Using the symmetrized form makes comparisons across populations and over time more defensible, because the index reflects the disparity itself rather than an artifact of the reference convention. It retains the additive between/within decomposition of the original.

Formula
T^{\text{sym}} = \tfrac{1}{2}\sum_j (s_j - p_j)\ln\!\frac{s_j}{p_j}\ \ \text{(reference-symmetric form)}
5

Because health-disparity data usually come from complex sample surveys, point estimates need design-based standard errors. Borrell and Talih extended Taylor-series linearization to derive the sampling variance of the total Theil index and of its between- and within-group components for grouped complex-survey data, so that confidence intervals and trend tests properly reflect stratification, clustering, and weighting. Quantifying uncertainty this way is essential when judging whether a disparity has genuinely changed over time or differs across settings, rather than reflecting sampling noise. Proper variance estimation turns the Theil index from a descriptive number into a basis for statistical inference about health inequality.

Formula
\widehat{\mathrm{Var}}(\hat{T})\ \text{via Taylor linearization on complex-survey design}

Common pitfalls

  • Computing the index with empty or near-empty groups, producing undefined or unstable logarithmic terms.
  • Using the asymmetric standard index for cross-population comparisons where the reference convention distorts results.
  • Reporting the total index without its between/within decomposition, discarding the method's main advantage.
  • Ignoring complex-survey design in variance estimation, understating uncertainty in disparity trends.

Frequently asked questions

Why use the Theil index instead of the concentration index?

The concentration index requires a meaningful socioeconomic ranking and measures the gradient of health along that rank. The Theil index does not need any ordering: it works with unordered categorical groups such as race-ethnicity and many regions at once, and its defining strength is additive decomposition into between-group and within-group inequality. When the question is at what level disparity is concentrated, or when groups are nominal and nested, the Theil index is the natural choice; when there is a clear SES gradient to summarize, the concentration index fits better.

What does the between/within decomposition tell you?

It partitions total inequality into the part arising from differences between broad groups (for example between racial-ethnic categories) and the part arising from differences among smaller units nested within them (for example across areas within each racial-ethnic group). A large between-component means the broad grouping captures most of the disparity, suggesting interventions targeted at group-level differences; a large within-component means substantial inequality persists inside those groups, pointing to finer-grained drivers. This attribution to a level of organization is what makes the Theil index uniquely informative for structured disparities.

Why is a symmetrized version of the Theil index needed?

The standard Theil index is not symmetric: its value can depend on whether you measure the disproportion of the health burden relative to population or the reverse, and it can be dominated by groups with high disease frequency. That makes comparisons across populations or over time sensitive to an arbitrary reference convention. Borrell and Talih's symmetrized index averages the two directions to produce a reference-invariant measure that reflects the disparity itself, while preserving the additive between/within decomposition and supplying design-based variances for survey data.

Related methods

Sources

  1. Harper, S., Lynch, J., Meersman, S. C., Breen, N., Davis, W. W., & Reichman, M. E. (2008). An Overview of Methods for Monitoring Social Disparities in Cancer with an Example Using Trends in Lung Cancer Incidence by Area-Socioeconomic Position and Race-Ethnicity, 1992-2004. American Journal of Epidemiology, 167(8), 889-899.

    Open DOI reference
    Copied.
  2. Borrell, L. N., & Talih, M. (2011). A symmetrized Theil index measure of health disparities: an example using dental caries in U.S. children and adolescents. Statistics in Medicine, 30(3), 277-290.

    Open DOI reference
    Copied.