Examining Contextual Effects in a Practice Analysis: An Application of Dual Scaling

Abstract

Practice analyses are routinely used in support of the development of occupational and professional certification and licensure examinations. These analyses usually survey incumbents to obtain importance ratings of (1) specific tasks and (2) knowledge, skill, and ability (KSA) statements deemed by subject matter experts as essential to safe and effective practice. Several researchers have made important criticisms of traditional practice analysis procedures, particularly the lack of attention to contextual constructs and the resulting problematic interpretation of mean importance ratings. The present study provides a framework for assessing the impact of context in practice analysis studies. It focuses on a practice analysis of a health profession that sought to enhance the meaning of incumbents' importance ratings by embedding the statements in the context of patient acuities. Results indicate that incumbents' importance ratings varied as a function of patient acuity. Dual scaling analysis was used to obtain a multidimensional visual representation of the associations between importance ratings and contextual content. The implications of the contextual component of the study design for future practice analysis studies are discussed as well as possible applications of this approach to professions in education.

A Survey Evaluating the Current Role of the Nurse Practitioner in Urology

“…• A certification test determines clinical competency and demonstrates that any certified member has been adequately tested according to current practice standards and role descriptions (De Champlain & LaDuca, 2007;Raymond, 2002) • A practice analysis was necessary to establish the importance and frequency ratings for tasks identified as critical to NP role in managing those with urological conditions • These data would provide insight into safe and effective NP practice within Urology, and to support NP specialty certification in urology…”

On the Superior Statistical Properties of Frequency Scales in Job Analyses

“…Some have argued that criticality rating scales are preferable to importance scales as they provide a more specific frame of reference (Manson, Levine, & Brannick, 2000). Research by Champlain, Cuddy, and LaDuca (2007) suggests that adding contextual descriptors to an importance scale for the task being rated results in more meaningful ratings. These findings all support the idea that less ambiguity and subjectivity in the rating scales and/or anchor points yield more reliable and accurate ratings.…”