Performance-based assessment of expertise: How to decide if someone is an expert or not

Shanteau, James; Weiss, David J.; Thomas, Rick P.; Pounds, Julia C

doi:10.1016/s0377-2217(01)00113-8

Cited by 234 publications

(196 citation statements)

References 16 publications

Supporting

Mentioning

184

Contrasting

Unclassified

Order By: Relevance

“…A thorough familiarisation with the domain which focuses on the tasks performed is needed to find out what may characterise expert behaviour. If objective outcome measures do not exist, professional criteria that provide social recognition, such as experience, degrees, licenses, job titles, status, and prizes, might be used to define experts (Evetts, Mieg, & Felt, 2006;Hoffman et al, 1995;Mieg, 2006), as well as peer judgments that ask professionals to identify the best performers in their field, or those whom they would go to for advice (Ericsson, 2006a; Kahneman & Klein, 2002;Shanteau, 2002). Other groups of participants must be included to examine in what way experts differ from those who are less experienced within the domain, or those who have worked as long in the profession but are considered to have less expertise.…”

Section: Examining Expertisementioning

confidence: 99%

Examining Expertise Using Interviews and Verbal Protocols

Wiel

2017

FLR

View full text Add to dashboard Cite

To understand expertise and expertise development, interactions between knowledge, cognitive processing and task characteristics must be examined in people at different levels of training, experience, and performance. Interviewing is widely used in the initial exploration of domain expertise. Work and cognitive task analysis chart the knowledge, skills, and strategies experts employ to perform effectively in representative tasks. Interviewing may also shed light on the learning processes involved in acquiring and maintaining expertise and the way experts deal with critical incidents. Interviews may focus on specific tasks, events, scenarios, and examples, but they do not directly tap the representations involved in task performance. Methods that collect verbal protocols during and immediately after task performance better probe the ongoing processes in representing problems and accomplishing tasks. This article provides practical guidelines and examples to help researchers to prepare, conduct, analyse, and report expertise studies using interviews and verbal protocols that are derived from thinking aloud, dialogues or group discussions, free recall, explanation, and retrospective reports. In a multi-method approach, these methods and other techniques need to be combined to fully grasp the nature of expertise. This article shows how the cognitive processes in data collection constrain data quality and highlights how research questions guide the development of coding schemes that enable meaningful interpretation of the rich data obtained. It focuses on professional expertise and provides examples from medicine including visual tasks. This comprehensive review of qualitative research methods aims to contribute to the advancement of expertise.Keywords: expertise, interviews, verbal protocols, cognitive processing, analysis IntroductionFor this special issue on "Methodologies for studying visual expertise", the present article will discuss the qualitative research methods of interviews and verbal protocols to examine expertise and expertise development. This article aims to guide students, practitioners, and researchers new to the field of expertise research, or these types of qualitative research, when and how to use these methods to answer their research questions. Starting from a theoretical framework of expertise and cognitive processing in task performance, this article provides practical guidelines so that researchers can prepare, conduct, analyse, and report expertise studies using interviews and verbal protocols. The rationale behind the methods, as well as their strengths and weaknesses, are explained to understand how procedures should be designed to maximise the quality of the data. Although these guidelines for research are applicable to all expertise domains, the focus here is on professional expertise. Most examples will be drawn from medical expertise research, as it has a long-standing tradition of using diverse methods of verbal protocols, and includes various areas of visual expertise. Thi...

show abstract

Section: Examining Expertisementioning

confidence: 99%

Examining Expertise Using Interviews and Verbal Protocols

Wiel

2017

FLR

View full text Add to dashboard Cite

show abstract

“…While the criteria for 'expertise' have been debated elsewhere (Shanteau et al, 2002) and will not be rehearsed here, judgements about what constitutes an expert is critical to the practice of eliciting expert knowledge (Caley et al, 2014). For this study, the process of sampling experts to contribute to the symposium comprised two stages.…”

Section: Sampling Strategymentioning

confidence: 99%

Expert views of peer-based interventions for prisoner health

Woodall

South

Dixey

et al. 2015

International Journal of Prisoner Health

View full text Add to dashboard Cite

Expert views of peer-based interventions for prisoner health AbstractPurpose: Formalised support services for prisoners that rely on peer methods of delivery show promising health and social outcomes but there is also conjecture that negative effects, both at an individual and organisational level, can occur.Design/methodology/approach: Individuals with recognised professional expertise from various sectors (including ex-prisoners) were invited to contribute to an expert symposium to share their perceptions of the positive and negative effects of peer interventions in prison.Discussions and debate were audio recorded with the consent of all delegates and verbatim transcripts were analysed using Framework Analysis.Findings: According to the participants, peer interventions in the prison setting created both positive and negative impacts. It was clear from the evidence gathered that peer interventions in prisons can impact positively on health outcomes, but these effects were perceived to be more well-defined for peer deliverers. The notion that peer deliverers can be subjected to 'burnout' suggests that supervisory processes for peer workers need to be considered carefully in order to avoid the intervention from being counter-productive.Organizationally, one of the salient issues was the adverse effects that peer interventions cause to the security of the prison. Originality/value: To our knowledge, this is the first time an expert symposium has been conducted to specifically examine peer interventions in prison and to consider the effects, both positive and negative, of such schemes.

show abstract

“…Knowledge of relevant facts, experience in a given field, perceptiveness and communication skills are some of the features of expertise [18].…”

Section: Expert Patient Programmesmentioning

confidence: 99%