t-SNE
- Introduced by
- van der Maaten, L. & Hinton, G.
- Year
- 2008
- Type
- Nonlinear dimensionality reduction (manifold visualization)
What does it do?
t-SNE (t-Distributed Stochastic Neighbor Embedding) is a nonlinear dimensionality-reduction method introduced by Laurens van der Maaten and Geoffrey Hinton in 2008 that maps high-dimensional data into a 2D or 3D space for visualization.
It preserves probabilistic local similarities, so points that are neighbours in the original space stay close together, revealing cluster structure and local neighbourhoods.
The intuition behind the method
When data lives in dozens or hundreds of dimensions, we cannot see it directly. t-SNE asks a simpler question for every pair of points: how likely is one to be picked as a neighbour of the other? It then arranges a 2D or 3D picture so that those neighbour probabilities are reproduced as faithfully as possible. The result is a map where tightly related observations gather into visible clumps, letting the eye spot groups that are invisible in the raw high-dimensional table.
History
t-SNE was introduced by Laurens van der Maaten and Geoffrey Hinton in 2008 in the Journal of Machine Learning Research, building on the earlier Stochastic Neighbor Embedding by replacing its Gaussian low-dimensional kernel with a heavy-tailed Student-t distribution to solve the crowding problem. It rapidly became the default tool for visualizing high-dimensional data across machine learning, genomics, and neuroscience. More recently UMAP has emerged as a faster alternative that often preserves more global structure, but t-SNE remains a standard reference for exploratory visualization.
When to use it
Use t-SNE to visualize and explore high-dimensional continuous data — embeddings, gene-expression profiles, image features — when you want to see local cluster structure in 2D or 3D, with at least about 50 observations. It is an exploratory, visual tool: read its maps for grouping and neighbourhood patterns, not for quantitative inference. Standardize features first, tune perplexity (a rule of thumb is around n/10, within 5–50), and check stability across several perplexity values. On very large datasets it is slow, so UMAP is a faster alternative.
Limitations and considerations
- Visualization only — distances and cluster sizes in the map carry no quantitative meaning, so no statistical inference should be drawn from it.
- Slow on large datasets; UMAP is a faster alternative.
- Results are stochastic and depend strongly on perplexity, so different runs or settings can look different.
- On small samples (n below about 50) the embedding is unstable and not reproducible.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Excellent at revealing local cluster structure and neighbourhoods that linear methods miss.
- Nonlinear: captures curved manifold structure that PCA cannot.
- Produces intuitive 2D/3D maps that make high-dimensional groupings visible at a glance.
- The heavy-tailed Student-t kernel avoids the crowding problem, keeping clusters well separated.
Applications
- Visualizing high-dimensional embeddings and learned feature spaces in 2D or 3D.
- Exploring cluster structure in single-cell or gene-expression data.
- Inspecting image or text feature representations from deep models.
- Exploratory neighbourhood analysis before quantitative clustering or classification.
How it works
How it works
For each pair of points in the original high-dimensional space, t-SNE converts their distance into a conditional probability that one would choose the other as a neighbour, using a Gaussian kernel. The width of that kernel is set per point so that the effective number of neighbours matches the chosen perplexity (typically 5–50), which controls the balance between local and global structure.
Formula
p_{j|i} = \frac{\exp(-\lVert x_i - x_j \rVert^2 / 2\sigma_i^2)}{\sum_{k \neq i}\exp(-\lVert x_i - x_k \rVert^2 / 2\sigma_i^2)}Each point gets a position in the low-dimensional map, and the pairwise similarities there are measured with a heavy-tailed Student-t distribution instead of a Gaussian. The heavy tails give distant points room to spread apart, which prevents everything from being crushed into the centre — the 'crowding problem' that earlier methods suffered from.
Formula
q_{ij} = \frac{(1 + \lVert y_i - y_j \rVert^2)^{-1}}{\sum_{k \neq l}(1 + \lVert y_k - y_l \rVert^2)^{-1}}t-SNE moves the low-dimensional points by gradient descent to minimize the Kullback–Leibler divergence between the high-dimensional neighbour probabilities and the low-dimensional ones. As this cost shrinks, the map's neighbourhood structure comes to match the original data's, producing the final clustered layout.
Formula
C = \mathrm{KL}(P \,\Vert\, Q) = \sum_{i \neq j} p_{ij} \log \frac{p_{ij}}{q_{ij}}Common pitfalls
- Reading cluster sizes or between-cluster distances as if they were meaningful — they are not.
- Skipping feature standardization before running the embedding.
- Picking a single perplexity and never checking stability across other values.
- Drawing quantitative conclusions from a tool meant only for visual exploration.
- Running t-SNE on fewer than ~50 observations, where the map becomes stochastic and misleading.
Frequently asked questions
What does perplexity control?
Perplexity sets the effective number of neighbours each point considers, balancing attention to local versus broader structure. Typical values are 5–50, with a rule of thumb around n/10; it is wise to try several and check that the visible structure is stable.
Can I read cluster sizes and distances from a t-SNE plot?
No. t-SNE preserves local neighbourhoods but not global geometry, so the size of a cluster and the gaps between clusters are not quantitatively meaningful. Use the plot to spot grouping, not to measure it.
How is t-SNE different from PCA?
PCA is a linear projection onto directions of maximum variance and preserves global structure, while t-SNE is a nonlinear method that preserves local neighbourhoods and excels at revealing clusters. They are complementary; PCA is often used as a preprocessing step before t-SNE.
Should I use t-SNE or UMAP?
Both are nonlinear visualization methods. t-SNE is a long-standing standard for local structure, while UMAP is faster on large datasets and tends to preserve more global structure, so it is the better choice when data volume is high.
Related methods
Sources
van der Maaten, L. & Hinton, G. (2008). Visualizing Data using t-SNE. Journal of Machine Learning Research, 9(86), 2579–2605.
Open publication referenceCopied.