Skip to content
Back to search results

MFCC (Mel-Frequency Cepstral Coefficients)

Introduced by
Steven Davis, Paul Mermelstein
Year
1980
Type
Audio feature extraction algorithm
Family
Audio Signal Processing
Font size

What does it do?

Mel-Frequency Cepstral Coefficients (MFCCs) are a compact representation of audio features that mimic human auditory perception.

Introduced by Davis and Mermelstein in 1980, MFCCs are the de facto feature extraction method for speech recognition and environmental sound analysis. They compress the frequency information of audio signals into a small set of coefficients that capture phonetic content while discarding irrelevant details.

The intuition behind the method

Human hearing is not linear in frequency; we perceive pitch logarithmically, and we are more sensitive to lower frequencies. MFCCs exploit this: instead of using a linear frequency scale (Hz), they use the mel scale, which warps frequency to match human perception. Audio is decomposed by frequency, transformed to the mel scale, compressed via logarithm and cosine transform, yielding ~13 numbers that capture the 'shape' of the spectrum. These numbers are remarkably stable across speakers and robust to noise, making them ideal for speech processing.

History

Davis and Mermelstein published MFCCs in 1980, drawing on earlier mel-scale work and cepstral analysis. MFCCs became the standard in speech recognition during the 1990s-2000s, used in systems like HTK and Sphinx. With the rise of deep learning, MFCCs remain popular as input features to neural networks, though learned representations (e.g., from spectrograms) are increasingly competitive.

When to use it

Use MFCCs for speech recognition, speaker identification, emotion detection, and environmental sound classification. They are industry-standard for automatic speech recognition (ASR) systems and deep learning models. Apply to audio signals with clear speech or environmental structure. Avoid for music analysis without modification (music requires different features like spectral flux or chroma).

Limitations and considerations

  • Discards phase information; cannot reconstruct audio from MFCCs alone
  • Assumes stationarity within frames; short-term features miss longer-term patterns (overcome with delta/acceleration)
  • Not optimal for music; music features (spectral centroid, chroma, etc.) are often better
  • Sensitive to microphone characteristics and background noise (requires normalization)

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Compact representation (~13 coefficients per frame) reduces computational cost and storage
  • Aligned with human auditory perception; captures perceptually important information
  • Robust to speaker variability and moderate noise
  • Widely implemented in speech recognition frameworks (HTK, Kaldi, TensorFlow)

Applications

  • Automatic speech recognition in consumer devices (voice assistants, dictation)
  • Speaker identification and verification for security applications
  • Emotion recognition from speech in customer service and mental health applications
  • Environmental sound classification for surveillance and acoustic monitoring

How it works

How it works

1

Record audio signal at a standard sample rate (typically 16 kHz for speech).

2

Divide audio into short frames (20-40 ms, overlapping); apply a windowing function (Hamming window) to reduce spectral leakage.

Formula
\text{Frame audio into overlapping windows; apply windowing function (Hamming)}
3

Compute the fast Fourier transform (FFT) of each frame; apply a bank of triangular filters spaced on the mel scale.

Formula
\text{Compute short-time Fourier transform (STFT); apply mel-scale filter bank}
4

Take the logarithm of the mel-scale power; apply the discrete cosine transform (DCT) to decorrelate the features.

Formula
\text{Compute DCT (discrete cosine transform) of log mel-power; retain first 12-13 coefficients}
5

Retain the first 12-13 MFCC coefficients (plus delta and acceleration derivatives); use for classification or clustering.

Common pitfalls

  • Using too many or too few MFCC coefficients; typically 12-13 are optimal for speech
  • Neglecting delta and acceleration coefficients; these improve speech recognition significantly
  • Failing to normalize features across speakers or conditions; z-score normalization is essential

Frequently asked questions

Why use the mel scale instead of Hz?

Human hearing perceives pitch logarithmically, not linearly. The mel scale reflects this: 1 mel corresponds to 1 just-noticeable change in pitch. Grouping frequencies by mel scale better captures perceptual differences.

What are delta and acceleration coefficients?

Delta coefficients capture the rate of change of MFCCs over time; acceleration coefficients capture the second derivative. Adding these to the static MFCCs significantly improves speech recognition (feature vector ~39 numbers instead of 13).

How do I choose the number of MFCC coefficients?

For speech, 12-13 static coefficients are standard. Including deltas and accelerations, use 39-40 total (13 × 3). For different domains or sample rates, experiment; more coefficients capture more detail but can overfit.

Related methods

Sources

  1. Davis, S., & Mermelstein, P. (1980). Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech, and Signal Processing, 28(4), 357-366.

    Open DOI reference
    Copied.
  2. Young, S. J., Evermann, G., Gales, M. J., et al. (1996). The HTK Book. Cambridge University Engineering Department.

    Open publication reference
    Copied.
  3. Moustakides, G. V., & Rougui, J. A. (2004). Optimal filtering for polynomial signal models. IEEE Transactions on Signal Processing, 52(8), 2219-2230.

    Open publication reference
    Copied.