The Harder the Task, the Higher the Score: Findings of a Difficulty Bias

Abstract

Studies have found that going first or last in a sequential order contest leads to a biased outcome, commonly called order bias (or primacy and recency). Studies have also found that judges have a tendency to reward contestants they recognize with additional points, called reference bias. Controlling for known biases, we test for a new type of bias we refer to as “difficulty bias,” which reveals that athletes attempting more difficult routines receive higher execution scores, even when difficulty and execution are judged separately. Despite some identification challenges, we add to the literature by finding strong evidence of a difficulty bias in gymnastics. We also provide generalizations beyond athletics. (JEL L10, L83, D81, J70, Z1)

Sequential effects in Olympic synchronized diving scores

“…If divers perform harder dives and consequently received lower scores in later rounds, this would also produce a pattern in the opposite direction to that predicted by the difficulty bias (higher execution scores given to more difficult routines) found in gymnastics [23]. To test this idea, for each of the 11 judges, the scores they gave throughout the competition were correlated with the difficulties of the dives (which are predefined in the governing body's rules and regulations; [29]).…”

“…), they often lack additional procedures for addressing various human biases. These may include a nationalistic bias, where judges give higher scores to competitors from their own countries [21,22], a difficulty bias, where athletes attempting more difficult routines receive higher execution scores [23], or a bias to conform after receiving feedback about other judges' scores [24,25].…”

Is home-field advantage driven by the fans? Evidence from across the ocean

“…Biases that occur from social pressures are different from other forms of bias, which are internal to the person making the judgment (see Price andWolfers, 2010 or Morgan andRotthoff, 2014). Although both internal and external influences on bias exist, we extend the literature that focuses on the impact of external factors, known as social pressures, on referee bias.…”

A theory-based intervention to prevent calibration effects in serial sport performance evaluations

“…gymnastics, Morgan & Rotthoff, 2014; figure skating, Emerson & Arnold, 2012). For instance, in gymnastics, the mean grades of a judging panel were changed into a complex scoring system with open range of points combining scores for difficulty and execution (gymnastics) and the use of video-based analyses (figure skating) is considered to help judges form more objective evaluations.…”