Bias in Workplace assessment

Assessment is a common but high stakes practice in organisations used for;

• Hiring the right people
• Shaping organizational culture via core competency assessment
• Assessing and developing technical competency
• Assessing and developing leadership competencies
• Ensuring Compliance – for example health & safety behaviours
• Improving productivity via Performance Appraisal
• Evaluating Staff engagement levels
• Assessing organisational climate and culture

So, getting it right is critical.  Assessment is very prone to bias . When organizations don’t get it right trust, quality and compliance suffer.

When we think of assessments, we think of giving a rating or score that represents the demonstration of a work task, a responsibility, a behavior or other performance standard.

But there is a sequence of events that underlies this judgement that we are not consciously aware of.

1. Observation — noticing/recalling behaviours or evidence.
2. Processing — comparing what was seen with an explicit or implicit standard.
3. Interpretation — integrating signals into a rating/comment.

Bias can enter at each stage, but you can minimise it with design and practice.

The observation process is about how assessors pay attention to and actively select information about the assessee and their activities.   Unless the assessment is an on-job assessment this means drawing upon memory.

Assessors must identify relevant information that they will use as a basis for their judgment. That is they must recognise what is relevant and actually perceive it, then store it in memory.

The observational process is influenced by many conscious, unconscious, situational and personality factors.  Research has shown that assessors shown the same video of an assessee will pay attention to different aspects of their activity or behaviour.

This is the phase in which assessors retrieve the information they have gathered and use contextual information and their own prior knowledge to make sense of it.

They make use of a categorisation mechanism – a comparison with some sort of standard – implicit or explicit.

An explicit standard would be a very specific indicator or statement in the assessment tool.

However, in most cases, the standard is not specific. In these cases, the assessor will compare what they have observed with an implicit standard.  A standard derived from their own beliefs and experience; their own idea of competency/performance and some specific examples they can recall.

Examples used include people assessed in the past, recall of their own level of performance and skills in a similar context, and of staff and colleagues with different levels of experience and expertise.

The assessment process is prone to bias at all stages. Bias is a systematic error, or deviation from the truth, in results or inferences.

Biases can operate in either direction: underestimation or overestimation of the true situation.   Bias stems from our thoughts and our feelings.

Availability bias  – use of information that is easily available rather than looking at the whole range of information that exists.

Recall bias –  we remember better recent events, the first and last of a sequence of events (better than middle ones.  Negative more than positive experiences, dramatic events over the routine.  We tend not to remember things that happened in a different context..

Anchor effects – information on a previous assessment used as a basis for current assessment – not a completely independent evaluation.

Selective perception

• expectations and stereotypes about people and situations affect what is seen and heard.
• tendency to see and hear those things that confirm existing beliefs,  and to filter out things that don’t agree.

Feelings trump facts.  Our feelings influence how we think.

Wanting to be liked – assessments more favorable than true opinion

The halo effect – the overall impression of a person is applied to assessments of specific attributes.  Those who are more attractive tend to be rated more positively on any dimension. Many research studies have found performance appraisals are primarily a measure of the staff/supervisor relationship, not the staff member’s actual performance .

• Train awareness of sources of bias
• Run calibration to align on rating scale anchors and what “meets” looks like.
• Require examples tied to indicators for high/low ratings.
• Train to evaluate behaviours, not people; avoid using self-performance as the benchmark.
• Frame marginal results as development opportunities.

• Use terminology relevant to your organization
• Define specific indicators (one behavior per line) and clearly explained scale anchors.
• Include some reverse scored items to prevent/identify blanket selection of the same rating
• Choose the right scale: Yes/No for strict standards,  Frequency for development scans,  Agreement for values/citizenship.
• Prefer 7-point scales (people avoid extremes; 5-point leaves only three usable points).
• Have a  NULL (not observed – neither agree nor disagree = no assessment).
• Hide prior ratings to reduce anchor effects
• Space out the timing of assessments to reduce contrast effects, (the tendency to make the assessment different than the preceding one(s).
• Use auto-scoring aligned to indicators; discourage free-form overall scores.
• Provide clear instruction on the form.

For values based behaviours and core competencies that are less well defined (e.g., professionalism, integrity), variation can reflect differences in the observation context.  Assessments may be complementary perspectives and equally valid.   Use 360 feedback surveys as developmental, not pay-linked, input for these perspectives.