Skip to content
Back to search results

Matrix Scoring and Ranking

Introduced by
Participatory Rural Appraisal tradition (Robert Chambers and colleagues)
Year
1994
Type
Participatory scoring of options against locally generated criteria in a matrix
Family
Participatory Rural Appraisal (PRA)
Font size

What does it do?

Matrix scoring and ranking is a participatory rural appraisal tool in which community members evaluate a set of options — crop varieties, services, trees, livestock breeds, sources of water — against criteria they themselves generate, arranged as a matrix.

Options run along one axis and criteria along the other, and participants score each cell, typically by placing a number of counters such as seeds or stones to show how well an option performs on that criterion. Summing the scores across criteria produces a ranking of the options that reflects the community's own values and priorities.

The intuition behind the method

When people choose between options they weigh several things at once — a crop's yield, its taste, its drought tolerance, its market value, its labour demand. A simple preference ranking hides this reasoning, but a scoring matrix exposes it: every cell is a judgment, and you can see exactly why one variety beats another and where it falls short. Letting participants invent the criteria, rather than supplying them, captures what actually matters locally, and using piles of counters makes the strength of each judgment visible and debatable. The matrix is thus both a decision aid and a window into the criteria behind local preferences.

History

Matrix scoring and ranking developed as one of the core visual tools of participatory rural appraisal in the late 1980s and early 1990s, part of the family of methods Robert Chambers describes in his 1994 account of PRA's origins and practice, and it became especially prominent in participatory varietal selection where farmers scored crop varieties against their own criteria. It built on simpler preference and pairwise ranking techniques by adding an explicit criteria dimension, making the reasoning behind choices visible. Bernard's Research Methods in Anthropology places it among the systematic ranking and rating techniques used to elicit cultural preferences and judgments.

When to use it

Use matrix scoring and ranking when you want to understand how a community evaluates and chooses among a set of comparable options and on what criteria — for example to learn why farmers prefer certain varieties, to compare services, or to prioritize interventions. It works well with a bounded set of concrete, familiar options and a group able to articulate criteria. It assumes the options are genuinely comparable on shared criteria. It is less suitable for one-off or incomparable choices, for very large option sets that overwhelm the matrix, or when a single overall preference (rather than the reasoning behind it) is all that is needed — a simple ranking then suffices.

Limitations and considerations

  • Scores are relative and group-negotiated, so totals are not calibrated measurements and travel poorly across sites.
  • Simple summation treats all criteria as equally important unless explicit weights are elicited and applied.
  • The matrix grows unwieldy as the number of options or criteria rises, straining attention and reliability.
  • Group settings let dominant voices sway scores, so a single matrix may mask important internal disagreement.

Source conditions do not replace independent scientific review or validation by an analysis engine.

Strengths

  • Reveals the criteria behind preferences, not just the final ranking, exposing why options are valued or rejected.
  • Generates the evaluation criteria from participants, grounding the comparison in local values rather than imposed metrics.
  • Counter-based scoring is low-literacy, visual, and engaging, prompting rich discussion as the matrix is filled.
  • Yields a transparent, auditable ranking whose every cell can be questioned and explained by the group.

Applications

  • Participatory varietal selection, in which farmers score crop varieties against their own agronomic and use criteria.
  • Comparing and prioritizing local services, technologies, or livelihood options for planning.
  • Eliciting the criteria and trade-offs behind preferences for trees, livestock breeds, or water sources.
  • Evaluating and ranking project or development options with the intended beneficiaries.
  • Comparing the priorities of different groups by having women, men, or wealth strata each build a matrix.

How it works

How it works

1

The group agrees on the set of options to compare — for example the sorghum varieties they grow, the water sources they use, or the services available to them — and these become the columns (or rows) of the matrix. Keeping the option set bounded and concrete makes the scoring manageable and the comparison meaningful.

2

Participants are asked what makes a good option of that kind, and their answers — yield, taste, storability, drought resistance, cost, distance, reliability — become the evaluation criteria along the other axis. Eliciting criteria from the group rather than imposing them is what makes the matrix reflect local values; negative criteria are usually reworded as positives so that more counters always mean better.

3

For each cell the group decides how well that option meets that criterion and expresses the judgment by placing a quantity of counters — seeds, stones, beans — so that a better performance gets more counters, often against a fixed maximum per cell. Filling the matrix collectively prompts discussion and lets the strength of each judgment, not just its direction, be recorded.

4

The counters are totalled to rank the options. In the simplest case each option's score is the sum across criteria; where some criteria matter more, a weighted sum is used: R_j = Σ_k w_k s_jk, where s_jk is option j's score on criterion k and w_k is the weight (importance) of that criterion. Options are ordered by R_j, and the pattern of scores shows the strengths and weaknesses behind the ranking.

Formula
R_j = \sum_k w_k\, s_{jk}

Common pitfalls

  • Supplying the criteria instead of eliciting them, which turns a participatory tool into the researcher's own evaluation.
  • Mixing positive and negative criteria so that more counters sometimes mean better and sometimes worse, corrupting the totals.
  • Summing across criteria of obviously unequal importance without eliciting and applying weights.
  • Treating the counter totals as precise quantitative scores and comparing them across groups as if calibrated.

Frequently asked questions

What is the difference between matrix scoring and matrix ranking?

In matrix scoring each cell receives an independent score (for example up to a fixed number of counters), so options can tie and the strength of each judgment is recorded. In matrix ranking the options are rank-ordered on each criterion instead. Scoring is generally richer because it preserves how much better one option is, not merely the order.

How are the scores combined into a ranking?

The counters in each cell are totalled per option. The simplest aggregation sums the scores across all criteria. When some criteria matter more than others, the group weights them and a weighted sum is used, R_j = Σ_k w_k s_jk, where s_jk is option j's score on criterion k and w_k is that criterion's weight; options are then ordered by R_j.

Should all criteria be weighted equally?

Not necessarily. Equal weighting is the default and is fine when criteria are of similar importance, but it can mislead when one criterion dominates real decisions. If importance clearly differs, elicit weights from the group — for instance by having them allocate a pile of counters across the criteria — and apply a weighted sum so the ranking reflects what truly matters.

Related methods

Sources

  1. Chambers, R. (1994). The origins and practice of participatory rural appraisal. World Development, 22(7), 953–969.

    Open DOI reference
    Copied.
  2. Bernard, H. R. (2017). Research Methods in Anthropology: Qualitative and Quantitative Approaches (6th ed.). Lanham, MD: Rowman & Littlefield.

    ISBN: 9780759112421
    Copied.