Hierarchical Clustering
- Introduced by
- Ward, J. H.
- Year
- 1963
- Type
- Unsupervised clustering (agglomerative)
What does it do?
Hierarchical clustering is an unsupervised method that groups observations into nested clusters and draws the result as a dendrogram, so the number of clusters need not be fixed in advance.
Its agglomerative form rests on the objective-function grouping criterion introduced by Joe Ward in 1963.
The intuition behind the method
Imagine every observation starting out as its own tiny group. At each step the two groups that are most similar are merged, then the next two, and so on, until everything joins into a single tree. Cutting that tree at different heights gives you different numbers of clusters, so you can decide how finely to split the data after seeing the structure rather than committing to a fixed count beforehand.
History
The agglomerative form of hierarchical clustering was placed on a formal footing by Joe H. Ward in 1963, who framed cluster merging as the optimisation of an objective function — minimising the increase in within-group variance at each merge. Ward's linkage remains one of the most widely used criteria, and the dendrogram it produces is a standard exploratory tool across the social, health, and business sciences.
When to use it
Use it for exploratory grouping when you want to discover structure without pre-specifying the number of clusters, on continuous, ordinal, or categorical features with at least about 10 observations. It is assumption-light but hinges on two choices: a suitable distance metric and a suitable linkage method. Below about 10 observations the distance matrix is unreliable and the dendrogram cannot show meaningful structure, so K-means is a safer alternative; on very small samples results become overly sensitive to the linkage method and cluster memberships are unstable.
Limitations and considerations
- Results depend heavily on the chosen distance metric and linkage method, which are user decisions.
- On small samples cluster memberships are unstable and overly sensitive to the linkage method.
- With fewer than about 10 observations the distance matrix is unreliable and the dendrogram shows no meaningful structure.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Does not require the number of clusters to be fixed in advance — you read it off the dendrogram.
- The dendrogram gives an interpretable visual map of how clusters nest at different similarity levels.
- Assumption-light: no normality requirement, and it accommodates continuous, ordinal, and categorical features through the distance choice.
- Works as an exploratory tool to reveal structure before committing to a partition.
Applications
- Exploratory segmentation of observations when the number of groups is unknown.
- Building a dendrogram to visualise how cases nest at different similarity levels.
- Grouping respondents, samples, or units on mixed continuous, ordinal, and categorical features.
- Generating candidate cluster counts to inform a later partitioning method.
How it works
How it works
First a distance is computed between every pair of observations using a chosen distance metric, producing a distance matrix. The choice of metric matters: it defines what counts as 'similar', and continuous, ordinal, and categorical features may call for different distance definitions.
Starting from each observation as its own cluster, the algorithm repeatedly merges the two clusters that are closest according to a chosen linkage method. Ward's criterion, for example, merges the pair that yields the smallest increase in within-cluster sum of squares, keeping the resulting clusters as compact as possible. The linkage choice strongly shapes the final grouping.
Formula
\Delta(A,B) = \sum_{i \in A \cup B} \lVert x_i - m_{A \cup B}\rVert^2 - \sum_{i \in A}\lVert x_i - m_A\rVert^2 - \sum_{i \in B}\lVert x_i - m_B\rVert^2The sequence of merges is recorded as a dendrogram, a tree whose branch heights show the distance at which clusters joined. Cutting the dendrogram at a chosen height yields the cluster assignments, so the number of clusters is read off the tree rather than specified in advance.
Common pitfalls
- Running it on fewer than about 10 observations, where the distance matrix and dendrogram are not trustworthy.
- Picking a distance metric and linkage method without justification and treating the result as definitive.
- Trusting unstable cluster memberships on a small sample instead of switching to K-means or a Gaussian mixture.
- Forcing a fixed number of clusters before inspecting the dendrogram's natural cut points.
Frequently asked questions
Do I need to choose the number of clusters first?
No. Hierarchical clustering builds the full merge tree regardless, and you decide the number of clusters afterward by cutting the dendrogram at a chosen height — that is one of its main advantages over methods like K-means.
Which distance metric and linkage should I use?
Both are deliberate choices that strongly shape the result. The distance metric should match your feature types, and Ward's linkage is a common default because it keeps clusters compact by minimising the increase in within-cluster variance at each merge.
How many observations do I need?
At least about 10. Below that the distance matrix is unreliable and the dendrogram cannot reveal meaningful structure, so K-means is the safer alternative.
Why are my clusters unstable?
On small samples the result is overly sensitive to the linkage method, so cluster memberships shift with small changes. In that situation consider K-means or a Gaussian mixture model instead.
Related methods
Sources
Ward, J. H. (1963). Hierarchical Grouping to Optimize an Objective Function. Journal of the American Statistical Association, 58(301), 236–244.
Open DOI referenceCopied.