A case study comparison of objective and subjective evaluation methods of physical qualities in youth soccer players

Abstract

Subjective and objective assessments may be used congruently when making decisions regarding player recruitment in soccer, yet there have been few attempts to examine the level of agreement between these methods. Therefore, we compare levels of agreement between subjective and objective assessments of physical qualities associated with youth soccer performance. In total, 80 male youth soccer players (13.2 ± 1.9 years), and 12 professional coaches volunteered to participate. Players were objectively assessed using five fitness measures: Yo-Yo Intermittent Recovery Test Level 1; Countermovement vertical jump; Functional Movement Screen™; 5/20m sprint; alongside anthropometric measures.Additionally, coaches subjectively rated each player on the same five physical qualities using 5-point Likert scales. Inter-rater agreement between ratings from lead and assistant coaches were established for each age group. Moreover, Bayesian regression models were fitted to determine how well coach ratings were able to predict fitness test performance. Although inter-rater agreement between lead and assistant coaches was moderate-to-substantial (ω=0.48-0.68), relationships between coaches subjective rating's and corresponding fitness test performance were only highly related for the highest and lowest performing players. We suggest that while ratings derived from objective and subjective assessment methods may be related when attempting to differentiate between distinct populations, concerns exist when evaluating homogeneous samples using these methods. Our data highlight the benefits of using both types of measures in the talent identification process.

Nationwide Subjective and Objective Assessments of Potential Talent Predictors in Elite Youth Soccer: An Investigation of Prognostic Validity in a Prospective Study

“…With regard to the assessment of endurance, it should be noted that this performance factor was evaluated subjectively by coaches. Dugdale et al (2020) indicate that coaches reach their limits in subjective evaluations when players show similar performances regarding endurance. Nevertheless, in the present study, the mean univariate effect size of d = 0.43 is within the range of the significant effect sizes presented in the review by Murr et al (2018b) for objective endurance diagnostics (0.28 ≤ d ≤ 1.56).…”

“…Several prospective studies with multidimensional approaches have provided empirical evidence on the significant, yet also partly limited prognostic relevance of objective diagnostics that assess youth players' characteristics, abilities, and skills in soccer (Murr et al, 2018a,b;Sarmento et al, 2018;Ivarsson et al, 2020). However, little is known about the prognostic validity of subjective evaluations of such performance factors by coaches or scouts, and recent studies highlight the need to integrate both subjective and objective evaluations of potential talent predictors (Dugdale et al, 2020;Ford et al, 2020).…”

“…Results indicated that the use of subjective coach assessments and objective performance data was significantly better at predicting under 19 (U19) player status than objective performance data alone. More recently, Dugdale et al (2020) compared levels of agreement between subjective and objective assessment of youth elite Scottish soccer players' (U11-U17 age group) physical performance. Athletes completed different physical performance assessments (e.g., endurance, acceleration, speed), with coaches providing a subjective evaluation of the physical variables.…”

“…In conclusion, the two studies highlight the importance of combining both subjective evaluations and objective test results within the talent identification process. It should be noted, however, that Sieghartsleitner et al (2019a) did not examine subjective performance factors, but general ingame performance, and Dugdale et al (2020) did not address the prognostic validity of subjective rated performance factors with regards to players' future performance level. Thus, given the lack of data on coaches' and scouts' efficacy regarding selection decisions, there is a need to examine the prognostic validity of subjective and objective evaluations of youth players' performance factors in regard to their validity for talent selections over an extended period.…”

Disruptive Technologies and the Sport Ecosystem: A Few Ethical Questions

“…Whereas the coach is still the person with "an eye" to identify the potential, her evaluation and decision-making starts to be supported by algorithms that permit to use a variety of data and metrics and forecast future successes [14]. Indeed, the research shows that although coaches' personal assessments meet objective physical criteria, that happens only with the highest-and lowest-performing future athletes, whereas when concerned with similar-level athletes, it becomes very divergent [15]. It means that the coaches are good at identifying a super talent, yet not that good in choosing from more or less equally talented children.…”