Behaviorally Anchored Rating Scales
- Introduced by
- Patricia Cain Smith & L. M. Kendall
- Year
- 1963
- Type
- Behaviorally anchored performance-rating scale construction
- Family
- Organizational behavior / performance appraisal
What does it do?
Behaviorally anchored rating scales (BARS) are performance-appraisal instruments whose scale points are defined by concrete examples of job behavior rather than by vague adjectives like 'good' or 'excellent.' Patricia Cain Smith and L. M. Kendall introduced the method in 1963 with their technique of retranslation of expectations, a procedure for constructing unambiguous behavioral anchors.
The core problem they tackled is that ordinary rating scales leave raters to guess what each numerical point means, so that one supervisor's 4 is another's 2, fatally undermining reliability and fairness. BARS solves this by attaching specific behavioral descriptions, drawn from critical incidents and vetted by independent expert judges, to each level of each performance dimension. The construction process is deliberately participatory and quantitative, which both improves measurement and builds rater understanding. BARS became one of the most influential and widely studied formats in performance appraisal.
The intuition behind the method
Imagine two managers rating the same employee's customer service on a scale from 1 to 7. One thinks a 5 means 'mostly fine,' the other thinks it means 'consistently excellent,' so their identical scores mean entirely different things and their differing scores may reflect nothing but differing yardsticks. The trouble is that the scale points are just numbers and labels that everyone interprets privately. Smith and Kendall's idea was to replace the private interpretations with public, concrete examples. Instead of anchoring a high point with the word 'excellent,' you anchor it with a vivid behavioral example such as 'could be expected to calm an irate customer and resolve the complaint on the first call.' Now every rater is looking at the same described behavior, so a high rating means the same thing across raters. To make the anchors trustworthy, they are not invented by one person but distilled from real incidents and then independently sorted and scaled by many judges, keeping only the examples that the judges agree on. The numbers finally acquire a shared, behavioral meaning.
History
Behaviorally anchored rating scales arose from dissatisfaction with the ambiguity of conventional graphic rating scales and from the rise of the critical incident technique in the 1950s. Patricia Cain Smith and L. M. Kendall's 1963 Journal of Applied Psychology paper introduced the key innovation, retranslation of expectations, providing a systematic, consensus-based procedure for building unambiguous behavioral anchors, work developed in the context of nursing performance appraisal. The format, originally called behavioral expectation scales, attracted intense research interest through the 1970s as scholars compared it with alternative formats and probed whether its theoretical advantages translated into measurable improvements in reliability and reduced rating error. Although the empirical superiority of BARS over simpler formats turned out to be smaller than hoped, the method profoundly shaped performance appraisal, spawning variants such as behavioral observation scales and embedding behavioral anchoring as a durable principle in rating-scale design.
When to use it
Use behaviorally anchored rating scales when you need defensible, behavior-based performance appraisal and rater consistency matters, for example in legally sensitive evaluation, promotion decisions, or any setting where vague adjective scales have produced inconsistent or contested ratings. BARS are well suited to jobs with identifiable, observable behaviors that recur enough to be captured as critical incidents, and where the considerable development effort can be justified by repeated use. They are less appropriate for highly idiosyncratic or rapidly changing jobs whose key behaviors cannot be stably described, for one-off evaluations where the construction cost cannot be amortized, or where the relevant performance is better captured by objective output metrics than by behavioral observation.
Limitations and considerations
- Construction is labor-intensive and time-consuming, requiring incident collection, retranslation, and scaling by multiple judges.
- Scales are job-specific and must be rebuilt when jobs change or for each new role, limiting transferability.
- Real behavior may fall between anchors or span several, forcing raters to interpolate and reintroducing some judgment error.
- Empirical gains in reliability and reduced rating errors over simpler formats have proven modest and inconsistent across studies.
Source conditions do not replace independent scientific review or validation by an analysis engine.
Strengths
- Replaces ambiguous numeric anchors with concrete behavioral examples, giving scale points a shared meaning across raters.
- Grounds the scale in real job behavior through critical incidents and the retranslation consensus filter, improving content validity.
- Engages the eventual users in construction, increasing rater understanding and acceptance of the appraisal system.
- Provides behaviorally specific, defensible documentation useful for feedback, development, and legal scrutiny.
Applications
- Constructing defensible, behavior-based performance-appraisal instruments for specific jobs.
- Standardizing supervisor ratings to improve consistency and fairness across raters.
- Providing concrete behavioral feedback to employees for development and goal-setting.
- Supplying legally defensible documentation of performance criteria and judgments.
- Serving as a model for anchoring other behavioral measures, including assessment-center and competency rating forms.
How it works
How it works
Construction begins by gathering critical incidents, specific examples of effective and ineffective behavior, from people who know the job well, exactly as in the critical incident technique. These incidents are the raw material from which both the performance dimensions and the eventual anchors will be built. Smith and Kendall designed the procedure so that the people who will use or be judged by the scales participate in generating the incidents, which both grounds the instrument in real behavior and increases acceptance. A rich pool of concrete incidents is essential, because the entire scale is built from them.
Formula
I = \{i_1,\dots,i_N\} \text{ critical incidents of effective/ineffective performance from SMEs}The incidents are then examined to induce the major performance dimensions, the distinct aspects of the job along which people will be rated, such as customer relations, planning, or job knowledge. Rather than imposing dimensions from theory, the developers let them emerge from clusters of related incidents, so the dimensions reflect how performance actually varies in the role. Defining a clear, non-overlapping set of dimensions is important because each will become its own anchored scale, and ambiguous or redundant dimensions would reintroduce the very confusion the method aims to remove.
Formula
D = \{d_1,\dots,d_p\},\quad \text{performance dimensions induced from incidents}The defining step, retranslation, has a second, independent group of expert judges sort each incident back into the dimension they think it belongs to, without seeing the original developers' assignments. Only incidents that a high proportion of judges, typically a clear majority, independently assign to the same dimension are retained. Smith and Kendall introduced this 'retranslation of expectations' precisely to weed out ambiguous incidents: if independent judges cannot agree what an incident is an example of, it will confuse raters and is discarded. This consensus filter is what gives BARS anchors their characteristic clarity.
Formula
\text{retain } i_k \iff \text{agreement}\bigl(d(i_k)\bigr) \ge \tau,\quad \tau \approx 60\text{--}80\%Surviving incidents are then scaled for effectiveness: judges rate how effective or ineffective each behavior is, and the mean rating locates the incident on the dimension's continuum while the spread of ratings indicates agreement. Incidents with high rater disagreement, a large standard deviation, are dropped because their effectiveness value is unclear, and the rest are placed at their mean scale value. This quantitative scaling ensures that each retained anchor not only belongs to a single dimension but also occupies a well-defined, agreed level on that dimension, from poor to excellent.
Formula
v_k = \bar{x}(i_k),\ \sigma_k;\ \ \text{keep } i_k \iff \sigma_k \le \sigma^{*} \text{ (low rater disagreement)}Finally, the retained, scaled incidents are assembled into the finished scale for each dimension, with behavioral anchors arranged along the continuum at their scaled effectiveness values. Raters then judge an employee by deciding which behavioral anchors best describe what the person could be expected to do, a more concrete and defensible task than assigning a bare number. Smith and Kendall's assembled scales give each point a behavioral meaning shared across raters, which is the payoff of the whole procedure and the reason BARS reduce the ambiguity and idiosyncrasy of conventional rating formats.
Formula
\text{scale}(d_j) = \text{continuum with anchors } i_k \text{ placed at } v_kCommon pitfalls
- Skipping or shortcutting the retranslation step, which lets ambiguous incidents back in and defeats the method's purpose.
- Retaining anchors with high judge disagreement on effectiveness, leaving scale points poorly defined.
- Writing anchors as traits or evaluations rather than concrete, observable behaviors.
- Assuming BARS automatically eliminate rater bias, when leniency and halo can still distort how anchors are applied.
Frequently asked questions
What is the retranslation step and why does it matter?
Retranslation is the heart of the BARS method. After developers tentatively assign critical incidents to performance dimensions, a separate, independent group of expert judges is asked to sort the same incidents into dimensions on their own. Only incidents that a high proportion of these judges independently place in the same dimension are kept. Smith and Kendall called this 'retranslation of expectations' because the incident must survive being translated back into a dimension by fresh judges. It matters because it filters out ambiguous incidents: if independent experts cannot agree what an incident exemplifies, it would confuse raters, so removing it is what gives the anchors their shared, unambiguous meaning.
Do BARS actually produce more accurate ratings than ordinary scales?
The theoretical case is strong, but the empirical evidence is mixed. Decades of research comparing BARS with graphic rating scales and other formats found that the improvements in reliability and reductions in errors such as leniency and halo were generally modest and not always present. The construction process is also costly. That said, BARS offer real advantages beyond raw psychometrics: the behavioral anchors improve the clarity and defensibility of ratings, support better feedback, and engage users in development. So BARS are valued less for dramatically higher accuracy than for the shared behavioral language and procedural legitimacy they bring to appraisal.
How do BARS differ from behavioral observation scales?
Both use behavioral content, but they ask raters different questions. Classic BARS, originally behavioral expectation scales, ask the rater to judge what an employee could be expected to do and to locate them against behavioral anchors arranged from poor to excellent. Behavioral observation scales (BOS), a later variant, instead list specific behaviors and ask how frequently the employee actually performed each one, then sum or average the frequencies. BARS thus emphasize expected level of effectiveness anchored by examples, while BOS emphasize observed frequency of concrete behaviors. Both descend from the critical incident technique and the goal of grounding appraisal in behavior rather than vague trait judgments.
Related methods
Sources
Smith, P. C., & Kendall, L. M. (1963). Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Psychology, 47(2), 149-155.
Open DOI referenceCopied.