Skip to content
Back to search results

Shannon Channel Capacity Theorem

Introduced by
Claude Shannon
Year
1948
Type
fundamental theoretical bound
Family
Information theory
Font size

What does it do?

Shannon's channel capacity theorem, published in 1948, establishes the maximum rate at which information can be reliably transmitted over a noisy channel.

Expressed as C = B log2(1 + S/N) for additive white Gaussian noise (AWGN), it is a fundamental bound in information theory and communications engineering. Shannon proved that reliable communication is possible at any rate below capacity, and impossible above it. This theorem underpins the design of all modern communication systems and motivates coding theory, modulation, and signal processing techniques.

The intuition behind the method

Intuitively, channel capacity answers the question: how much information can I transmit per unit time reliably despite noise? Shannon showed that the answer depends on three factors: (1) bandwidth B (more bandwidth -> more capacity), (2) signal power S (stronger signal -> more capacity), and (3) noise power N (less noise -> more capacity). The logarithmic trade-off log2(1 + S/N) captures a deep truth: doubling power only increases capacity by 1 bit/s/Hz, a diminishing return. This explains why, despite centuries of communication practice, reliable communication is fundamentally limited until the ratio of signal to noise is sufficiently high. It also motivates sophisticated modulation and coding: they approach capacity asymptotically but never exceed it.

History

Shannon's 1948 paper 'A Mathematical Theory of Communication' founded information theory. He proved the noisy channel coding theorem, establishing that reliable communication below capacity is possible, but above capacity is not. This revolutionary result showed that noise does not prevent reliable communication, only limits the rate. In subsequent decades, coding theorists (Hamming, Golay, Convolutional codes, Reed-Solomon) developed practical codes approaching capacity. Modern codes (turbo, LDPC, polar) achieve rates within 0.5-0.7 dB of capacity. Shannon's work transformed communications from an empirical engineering practice to a rigorous mathematical discipline.

When to use it

Shannon capacity is used to set design targets and benchmark system performance. Before designing a communication system, compute capacity to understand the ultimate theoretical limit. When evaluating a system, compare actual throughput to capacity to quantify efficiency. Capacity also guides resource allocation: power control, bandwidth allocation, coding rate selection. Use capacity analysis to justify investment in advanced technologies (MIMO, beamforming) by quantifying potential gains.

Limitations and considerations

  • AWGN capacity formula assumes infinite codeword length; practical finite-length codes are suboptimal
  • Fading capacity requires full channel state information at transmitter for optimal power allocation (water-pouring)
  • Formula assumes Gaussian input; some channels benefit from non-Gaussian modulation (rare)
  • Capacity does not account for latency, complexity, or energy constraints; practical systems prioritize these
  • Multi-user capacity is vastly more complex; interference channels have no closed-form capacity

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Provides a fundamental limit independent of specific code/modulation design
  • Elegant closed-form formula for AWGN channels
  • Extends to fading, interference, multi-user channels via information-theoretic generalizations
  • Motivates entire field of coding theory (turbo, LDPC, polar codes all approach capacity)
  • Invariant across technologies: capacity formula is valid for optical, RF, acoustic channels
  • Allows trade-off analysis: explores gains from bandwidth expansion, power increase, or diversity

Applications

  • Wireless system design: capacity analysis guides antenna selection, power budgets, and spectral efficiency targets in LTE/5G
  • Satellite communications: capacity limits set realistic expectations for download rates from deep space probes
  • Optical fiber communications: capacity formula informs wavelength division multiplexing and amplification strategies
  • Storage systems: capacity bounds error correction overhead needed for reliable HDD/SSD archival
  • Network design: multi-user capacity analysis guides resource allocation and admission control

How it works

How it works

1

Information source generates symbols at rate R (bits per second) to be transmitted.

2

For AWGN channel: capacity C = B log2(1 + P/N0 B), where B is bandwidth, P is signal power, N0 is noise spectral density.

3

For fading channels: capacity depends on channel state; average capacity is C = integral over channel state distribution of B log2(1 + |h|^2 P / N0 B).

4

The channel model includes: transmitted signal X, fading channel h, additive noise Z. Received signal Y = h*X + Z.

5

Reliable transmission at rate R is possible if and only if R ≤ C. If R > C, arbitrarily small error probability cannot be achieved, no matter the code complexity.

Common pitfalls

  • Assuming capacity is easily achieved; gap between theory and practice is large for short packets
  • Ignoring finite-blocklength effects; asymptotic capacity assumes infinite codeword length
  • Not accounting for feedback; limited feedback may increase capacity significantly
  • Assuming fading capacity with partial CSI; channel estimation errors reduce effective capacity
  • Expecting capacity to be achievable with simple schemes; capacity-achieving codes are complex (turbo, LDPC, polar)

Frequently asked questions

What is the relationship between Shannon capacity and spectral efficiency?

Spectral efficiency (bits per second per Hertz) is constrained by capacity divided by bandwidth: η ≤ C/B. For AWGN, η ≤ log2(1 + S/N). If system bandwidth is B and power is P, spectral efficiency is η = C/B. As SNR increases, spectral efficiency increases logarithmically; there is no hard limit but diminishing returns.

How does feedback affect capacity?

For memoryless AWGN channel, Shannon proved capacity is unchanged by feedback: information-theoretically, feedback cannot increase capacity. However, feedback enables adaptive coding (adjust rate based on CSI), which can improve practical performance. For channels with memory or feedback delay, capacity can increase.

What is water-pouring and why is it optimal for fading channels?

Water-pouring is an optimal power allocation strategy for fading channels: allocate more power when the channel is good (high gain) and less when bad (low gain). Mathematically, power P(h) = max(0, λ - N0 / |h|^2), where λ is a Lagrange multiplier set to satisfy total power constraint. This maximizes ergodic capacity.

Can capacity be exceeded with clever coding?

No. Shannon's noisy channel coding theorem is a fundamental limit: if transmission rate exceeds capacity, the probability of error cannot be made arbitrarily small, regardless of code complexity or decoding algorithm. The theorem states that achieving rate R < C with arbitrary error probability ε > 0 requires blocklength exponential in 1/ε.

Related methods

Sources

  1. Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379-423.

    Open DOI reference
    Copied.
  2. Cover, T. M., & Thomas, J. A. (1991). Elements of Information Theory. John Wiley & Sons.

    Open publication reference
    Copied.