Skip to content
Back to search results

Convolutional Neural Network (Classification)

Introduced by
LeCun, Y. et al.
Year
1998
Type
Deep neural network (convolutional)
Font size

What does it do?

A Convolutional Neural Network (CNN) is a deep learning model, established by LeCun and colleagues in 1998, that learns local patterns directly from images and structured data to classify them.

Stacks of convolutional filters discover increasingly abstract features, so manual feature engineering can be largely reduced.

The intuition behind the method

Instead of hand-crafting features, a CNN slides small learnable filters across the input to detect local patterns such as edges, textures, or motifs, regardless of where they appear. Early layers pick up simple cues; deeper layers combine them into higher-level concepts. Because the same filter is reused across the whole input, the network learns efficiently and recognises a pattern wherever it occurs.

History

Convolutional neural networks were brought together in modern form by Yann LeCun and colleagues in their 1998 work on gradient-based learning applied to document recognition, where the LeNet architecture read handwritten digits. The combination of convolution, pooling, and end-to-end gradient training became the template for image classification and, with larger datasets and faster hardware, drove the deep learning revolution in computer vision.

When to use it

Use a CNN for classifying or predicting from images and structured grid-like data when you have a large labelled dataset — roughly 1000 or more observations — and want the model to learn features automatically rather than engineering them by hand. It does not require normally distributed data and handles continuous and categorical inputs. Below about 500 observations the network overfits and a Random Forest is safer; below about 200 the convolutional filters cannot learn meaningful patterns and SVM is preferable.

Limitations and considerations

  • Needs a large labelled dataset (around 1000+ observations) to train reliably.
  • On small samples (n below about 500) the network overfits and classification performance drops.
  • Below roughly 200 observations the convolutional filters cannot learn meaningful patterns at all.
  • Computationally demanding to train, and the learned representation is hard to interpret.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Learns local patterns automatically, greatly reducing the need for manual feature engineering.
  • Weight sharing recognises a pattern wherever it appears and keeps the parameter count manageable.
  • Builds a hierarchy of features, from simple edges to abstract concepts, well suited to images.
  • Assumption-light: no normality requirement, and it accepts continuous and categorical inputs.
  • State-of-the-art accuracy on large labelled image and grid-structured classification tasks.

Applications

  • Image classification and recognition tasks.
  • Classification of structured, grid-like data where local patterns matter.
  • Document and handwritten character recognition.
  • Feature learning where manual feature engineering is impractical.

How it works

How it works

1

Learnable filters (kernels) are convolved across the input, computing local weighted sums to produce feature maps. Each filter responds to a specific local pattern — an edge, a texture, a shape fragment — and because the same weights are shared across all positions, the network detects that pattern anywhere in the input while keeping the parameter count low.

Formula
(I * K)(i,j) = \sum_{m}\sum_{n} I(i+m, j+n)\, K(m,n)
2

Pooling layers (for example max pooling) downsample each feature map, summarising neighbourhoods into a single value. This shrinks the spatial dimensions, makes the representation more robust to small shifts and distortions, and lets later layers build progressively more abstract features over a wider receptive field.

3

The high-level feature maps are flattened and passed through fully connected layers ending in a softmax that outputs class probabilities. The whole network — filters and weights together — is trained end to end by gradient-based learning, minimising classification loss via backpropagation.

Formula
\hat{y} = \operatorname{softmax}(W h + b)

Common pitfalls

  • Training on too little data and overfitting instead of falling back to Random Forest or SVM.
  • Skipping a held-out validation or test split and trusting training accuracy.
  • Ignoring class imbalance instead of re-weighting or resampling the classes.
  • Reporting only accuracy without precision, recall, F1, or a confusion matrix.
  • Forgetting to normalise inputs or applying no regularisation against overfitting.

Frequently asked questions

How much data does a CNN need?

CNNs are data-hungry; a rough floor is around 1000 labelled observations. Below about 500 they tend to overfit, and below about 200 the filters cannot learn meaningful patterns — in those cases Random Forest or SVM are safer choices.

Why convolution instead of a plain neural network?

Convolution shares the same small filter across all positions of the input, so a pattern is detected wherever it appears and the parameter count stays low. A fully connected network would need vastly more weights and would not exploit the spatial structure of the data.

What does pooling do?

Pooling downsamples feature maps by summarising local neighbourhoods, which reduces spatial size, adds robustness to small shifts, and lets deeper layers build more abstract features over a wider receptive field.

What should I check before relying on a CNN?

Confirm you have a large enough labelled dataset, normalise the inputs, set up a proper train/validation/test split, address any class imbalance, and evaluate with precision, recall, F1, and a confusion matrix rather than accuracy alone.

Related methods

Sources

  1. LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. (1998). Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Open DOI reference
    Copied.