Capsule Network
- Introduced by
- Sabour, S., Frosst, N. & Hinton, G. E.
- Year
- 2017
- Type
- Deep learning architecture (vector capsules with dynamic routing)
What does it do?
A Capsule Network (CapsNet) is a deep learning architecture introduced by Sara Sabour, Nicholas Frosst and Geoffrey Hinton in 2017 that organises neurons as vectors (capsules) rather than scalar activations, so that spatial hierarchy and pose (orientation) information are encoded directly.
It was proposed to overcome the fragility of convolutional networks to changes in viewpoint.
The intuition behind the method
A standard convolutional network detects whether a feature is present but discards much of its precise spatial relationship to other features, which makes it brittle when an object is rotated or seen from a new angle. A capsule replaces a single scalar neuron with a small vector: its length signals the probability that an entity is present and its direction encodes that entity's properties, such as pose and orientation. Because capsules carry this richer information, the network can reason about how parts relate to wholes instead of merely whether parts appear.
History
Capsule Networks were introduced by Sara Sabour, Nicholas Frosst and Geoffrey Hinton in the 2017 NeurIPS paper Dynamic Routing Between Capsules, motivated by the limitations of convolutional networks in handling viewpoint variation. In 2018 Hinton, Sabour and Frosst extended the idea at ICLR with Matrix Capsules with EM Routing, replacing the original routing scheme with an expectation-maximisation procedure over matrix-valued capsules.
When to use it
Suitable for classification and prediction on image or otherwise structured continuous-feature data where robustness to viewpoint and pose changes matters, with a reasonably large sample (at least about 500 observations). A GPU is recommended and the dynamic routing algorithm must be tuned. Below about 500 observations the routing cannot learn reliably and the model overfits; below roughly 200 a classical classifier such as random forest or an SVM is the better choice.
Limitations and considerations
- Requires a fairly large sample (about 500+); on smaller data the dynamic routing overfits and learns unreliably.
- Iterative dynamic routing is computationally demanding and a GPU is recommended.
- The routing algorithm introduces hyperparameters that must be tuned, adding training complexity.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Encodes pose and spatial-hierarchy information that convolutional pooling discards, improving robustness to viewpoint changes.
- Models explicit part-to-whole relationships through routing-by-agreement.
- Vector capsules preserve entity properties (orientation, position) rather than collapsing them to a single activation.
Applications
- Image classification that must be robust to changes in viewpoint, rotation, or pose.
- Recognition tasks where modelling part-to-whole spatial relationships improves generalisation.
- Classification and prediction on structured continuous-feature inputs with sufficient training data.
How it works
How it works
Instead of scalar neurons, the network groups units into capsules — short vectors whose length represents the probability that a particular entity exists and whose orientation encodes its instantiation parameters such as pose. Lower-level capsules detect parts, while higher-level capsules represent the wholes those parts compose.
A dynamic routing-by-agreement algorithm connects lower-level capsules to higher-level ones. Each lower capsule makes a prediction for the output of every higher capsule via a transformation matrix; coupling coefficients are then refined iteratively so that predictions which agree reinforce one another. This routing must be tuned and replaces the pooling step of a convolutional network with an explicit part-to-whole consensus.
Formula
\hat{u}_{j|i} = W_{ij}\,u_i,\quad s_j = \sum_i c_{ij}\,\hat{u}_{j|i}A non-linear squashing function shrinks short vectors towards zero and long vectors towards unit length, so the magnitude of an output capsule stays a valid probability while its direction preserves the entity's properties. The final layer's capsule lengths give the class predictions used for classification.
Formula
v_j = \frac{\lVert s_j\rVert^2}{1+\lVert s_j\rVert^2}\,\frac{s_j}{\lVert s_j\rVert}Common pitfalls
- Training on too few observations (under ~500), where routing cannot converge to reliable couplings.
- Leaving the dynamic routing iterations and coupling at defaults instead of tuning them for the task.
- Attempting to train without GPU acceleration and accepting prohibitively slow or incomplete training.
- Applying CapsNet to non-image, unstructured inputs it was not designed for instead of a classical model.
Frequently asked questions
How is a capsule different from a normal neuron?
A normal neuron outputs a single scalar activation. A capsule outputs a vector whose length represents the probability that an entity is present and whose direction encodes that entity's properties, such as pose and orientation, so it carries far richer spatial information.
What is dynamic routing?
Dynamic routing-by-agreement is the algorithm that connects lower-level capsules to higher-level ones. Each lower capsule predicts the output of higher capsules, and coupling coefficients are refined iteratively so that predictions which agree reinforce each other, replacing the pooling step of a convolutional network.
How much data and compute do I need?
CapsNet needs a reasonably large dataset — at least about 500 observations — and a GPU is recommended because iterative routing is computationally demanding. Below roughly 200 observations a classical classifier such as random forest or an SVM is more appropriate.
Why use a capsule network instead of a CNN?
CapsNet was proposed specifically to address a CNN's fragility to viewpoint changes. By preserving pose and part-to-whole relationships rather than pooling them away, it can generalise better when objects appear in new orientations.
Related methods
Sources
Sabour, S., Frosst, N. & Hinton, G. E. (2017). Dynamic Routing Between Capsules. Advances in Neural Information Processing Systems (NeurIPS).
Open publication referenceCopied.Hinton, G. E., Sabour, S. & Frosst, N. (2018). Matrix Capsules with EM Routing. International Conference on Learning Representations (ICLR).
Open publication referenceCopied.