Skip to content
Back to search results

Text Regression — Predicting Numbers from Text

Type
Supervised regression on text features
Font size

What does it do?

Text-based regression predicts a continuous target variable using features extracted from text — TF-IDF scores, embeddings, or n-grams — as the independent variables.

Building on the text-as-data programme consolidated by Gentzkow, Kelly and Taddy (2019), it lets a numeric outcome such as a price, a rating, or a sentiment score be estimated directly from documents, and is widely used in social-science, economics, and finance applications.

The intuition behind the method

A regression model normally needs numeric predictors, but the raw material here is text. The idea is to turn each document into a vector of numbers that captures its wording, then run a regression of the numeric target on those features. Instead of hand-coding what a document means, the method lets the words themselves — weighted by how informative they are — drive the prediction of a continuous quantity.

History

Text regression sits within the broader text-as-data movement surveyed by Gentzkow, Kelly and Taddy (2019), which framed how unstructured text can serve as input to econometric and statistical models. Taddy (2013) advanced the predictive use of text by introducing multinomial inverse regression to recover sentiment and other quantities from word counts, illustrating how numeric outcomes can be read off text features.

When to use it

Use text regression when you have a text predictor and a continuous numeric outcome you want to predict from it, with at least roughly fifty documents to learn from. The text must first be vectorised, and dimensionality reduction is often helpful given how wide text feature spaces are. It does not assume normally distributed inputs. With far fewer documents, or with no numeric target to regress on, the method is not appropriate.

Limitations and considerations

  • Needs a reasonable sample (around fifty documents or more) to estimate stable coefficients.
  • Text must be vectorised first, and the very high dimensionality usually calls for dimensionality reduction.
  • Predictive quality depends on how well the chosen text features carry signal about the numeric target.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Lets a continuous outcome be predicted directly from text, with no manual coding of document content.
  • Works with several feature types — TF-IDF, embeddings, or n-grams — so the representation can match the corpus.
  • Does not require normally distributed inputs, fitting the sparse, high-dimensional features text produces.

Applications

  • Estimating numeric ratings or scores from review and feedback text.
  • Predicting financial or market quantities from news, filings, and disclosures.
  • Recovering continuous sentiment or attitude scores from social-media text in social-science research.
  • Linking written content to measured outcomes in economics and policy studies.

How it works

How it works

1

Each document is converted into numeric features so that text becomes regression-ready input. This vectorisation step produces TF-IDF scores, word or document embeddings, or n-gram counts — turning unstructured wording into the independent variables the regression will read.

2

Because text vectors are typically very high-dimensional, dimensionality reduction is often applied to compress the feature space before fitting. Trimming redundant or sparse features keeps the model stable and helps it generalise rather than memorise rare wordings.

3

A regression model is trained to map the text features onto the continuous target. A regularised linear regressor is a natural fit for the wide, sparse feature matrices that text produces, learning the coefficients that link wording to the numeric outcome.

4

The fitted model is scored on held-out data with regression metrics, and the largest errors are inspected before the model is trusted. This guards against a model that fits the training text but fails to predict the target on new documents.

Common pitfalls

  • Skipping dimensionality reduction on a very wide feature matrix invites overfitting to rare wordings.
  • Fitting on too few documents yields unstable coefficients that do not generalise.
  • Reusing a vectoriser or model built on one corpus or domain for another degrades predictions; refit on the target text.
  • Judging the model on training fit alone — always evaluate on held-out data and inspect the largest errors.

Frequently asked questions

What kind of target does text regression predict?

A continuous numeric variable — for example a price, a rating, or a sentiment score. If the target is a category rather than a number, text classification is the matching method instead.

Which text features should I use as predictors?

TF-IDF scores, word or document embeddings, and n-gram counts are all standard choices. The best representation depends on the corpus and the signal you expect the wording to carry about the target.

How much data do I need?

The method expects on the order of fifty documents or more so the regression can estimate stable coefficients. Because text features are high-dimensional, dimensionality reduction is often advisable even when the sample is adequate.

Why is dimensionality reduction recommended?

Text vectorisation produces very wide, sparse feature matrices. Reducing the dimensionality compresses redundant features, stabilises the fit, and helps the model generalise instead of memorising rare wordings.

Related methods

Sources

  1. Gentzkow, M., Kelly, B. & Taddy, M. (2019). Text as Data. Journal of Economic Literature, 57(3), 535-574.

    Open DOI reference
    Copied.
  2. Taddy, M. (2013). Measuring Political Sentiment on Twitter: Factor Optimal Design for Multinomial Inverse Regression. Technometrics, 55(4), 415-425.

    Open DOI reference
    Copied.