Doc2Vec — Document Embeddings
- Introduced by
- Quoc V. Le & Tomas Mikolov
- Year
- 2014
- Type
- Document-embedding representation learning
What does it do?
Doc2Vec, also known as Paragraph Vector, is a representation-learning method introduced by Le and Mikolov (2014) that maps whole documents to fixed-length dense vectors.
These vectors place similar documents close together in space, supporting document comparison and classification.
The intuition behind the method
A document is made of many words of varying length, but most models need each document expressed as one fixed-size set of numbers. Doc2Vec learns a single dense vector for each document by training it, alongside its words, to predict the text it contains. Documents that talk about similar things end up with similar vectors, so the geometric distance between two vectors becomes a usable measure of how related the documents are.
History
Doc2Vec, or Paragraph Vector, was introduced by Quoc Le and Tomas Mikolov in their 2014 ICML paper Distributed Representations of Sentences and Documents. It extended the idea of learning dense word vectors to whole sentences and documents, giving each piece of text its own learned fixed-length representation.
When to use it
Doc2Vec fits when you have a text corpus and want a single dense vector per document for comparison, classification, or relationship analysis. It assumes a sufficient number and length of documents — roughly 100 documents or more — and that text preprocessing has been done. With too few or very short documents the learned vectors are unreliable.
Limitations and considerations
- Needs a sufficient number and length of documents to learn stable vectors; small corpora give unreliable embeddings.
- Requires proper text preprocessing before training.
- Performance depends on training choices such as vector size and the number of epochs.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Produces a single fixed-length dense vector for an entire document, regardless of its original length.
- Embeddings place similar documents near each other, enabling direct comparison and similarity search.
- The resulting vectors feed naturally into classification and relationship analysis.
Applications
- Comparing documents by similarity for search and retrieval.
- Classifying documents using their learned vectors as features.
- Clustering or exploring large text corpora by semantic proximity.
- Studying relationships between documents in the humanities, social, and business domains.
How it works
How it works
The corpus is cleaned and tokenised into the units the model will learn from. Consistent text preprocessing is required so that documents are comparable and the learned vectors are stable.
Each document is paired with a unique tag (identifier). This tag is what the model will associate with a learned vector, letting every document keep its own distinct representation throughout training.
A model is trained over the tagged documents for several passes (epochs), jointly learning word and document vectors so that each document vector helps predict the words it contains. The chosen vector size sets the dimensionality of the resulting embeddings.
After training, every document is represented by its fixed-length dense vector. These embeddings can be compared directly — for example by distance or similarity — and fed into downstream classification or relationship tasks.
Common pitfalls
- Running on too few documents (under ~100) yields noisy, unstable vectors.
- Skipping or inconsistently applying preprocessing makes the learned vectors hard to compare.
- Very short documents carry too little signal for a meaningful embedding.
- Treating vector size and epochs as fixed defaults rather than tuning them to the corpus.
Frequently asked questions
How is Doc2Vec different from word embeddings?
Word-embedding methods learn a vector for each word. Doc2Vec learns an additional vector for each whole document, so you get one fixed-length representation per document rather than having to combine word vectors yourself.
How many documents do I need?
The method assumes a sufficient number and length of documents — roughly 100 or more. With far fewer documents, or very short ones, the learned vectors become unreliable.
Do I need to preprocess the text first?
Yes. The corpus should be cleaned and tokenised consistently before training, otherwise the learned document vectors are hard to compare.
What can I do with the resulting vectors?
Each document becomes a fixed-length dense vector you can compare by distance or similarity, or use directly as input features for classification and relationship analysis.
Related methods
Sources
Le, Q. V. & Mikolov, T. (2014). Distributed Representations of Sentences and Documents. Proceedings of the 31st International Conference on Machine Learning (ICML), 1188-1196.
Open publication referenceCopied.