Commitment to Practice Change: An Evaluator’s Perspective

Abstract

A commitment to practice change (CTC) approach may be used in educational program evaluation to document practice changes, examine the educational impact relative to the instructional focus, and improve understanding of the learning-to-change continuum. The authors reviewed various components and procedures of this approach and discussed some practical aspects of its application using as an example of a study evaluating a presentation on menopausal care for primary care physicians. The CTC approach is a valuable evaluation tool, but it requires supplementation with other data to have a complete picture of the impact of education on practice. From the evaluation perspective, the self-reported nature of the CTC data is a major limitation of this method. Keywords commitment to practice change; program evaluation; continuing educationThe translation of new knowledge into clinical practice can be a slow process, often taking a decade or more for research findings to find their way into routine patient care (Sussman, Valente, Rohrbach, Skara, & Pentz, 2006). The resulting gaps in the quality of healthcare have led to calls for action in many parts of the world (Committee on Quality of Healthcare in America, 2001;Legido-Quigley, McKee, Nolte, & Glinos, 2008). The traditional role of continuing medical education (CME)-updating physicians on the latest scientific evidenceis not sufficient to respond to this reality (Marinopoulos et al., 2007). Therefore, in the past five years there has been a marked shift toward focusing CME activities on improving practice rather than disseminating information (Davis, Davis, & Bloch, 2008;Regnier, Kopelow, Lane, & Alden, 2005). This shift in the goals of CME programming has resulted in a concomitant shift in how CME activities need to be evaluated. Evaluators must now go beyond measuring learner satisfaction and change in medical knowledge to the level of physician performance and patient outcomes. To date, only a limited number of CME evaluations have assessed impact at this level (Tian, Atkinson, Portnoy, & Gold, 2007). There is a pressing need to equip evaluators with valid, reliable and feasible methods of assessing practice and/or patient outcomes (Davis, Barnes, & Fox, 2003;Marinopoulos et al., 2007).The commitment to change (CTC) approach is one tool available to CME evaluators for assessing the impact of education on clinical practice. The central, distinguishing feature of the CTC approach is that it asks participants in an educational activity to write down descriptions of the changes they propose to make as a result of what they learned during the (Purkis, 1982). Since 1982, when the CTC approach was introduced to medical educators as an evaluation method (Purkis, 1982), it has been increasingly used in the CME field to both facilitate and measure practice change (Wakefield, 2004). This trend is likely due, in part, to the perception that the CTC approach is relatively easy to implement and low-cost (Curry & Purkis, 1986;Jones, 1990). There are, however, significant...

Using a Commitment to Change Tool for Evaluation and Planning of a Global Competency-Based Curriculum in Orthogeriatrics

“…Regarding the additional research question “Does level of commitment influence the likelihood to respond to the follow‐up survey and the degree of implementation of intended changes?” Shershneva et al report that a question asking participants to designate a level of commitment may be included, and this has been done in several studies. Overton and MacVicar suggest that this enables learners to make a conscious decision to commit and reflect on the basis for that decision.…”

“…Would a second follow up augment change?) and by Shershneva et al (integration of new CTC statements at follow‐up, review of changes that may be undesirable, predefined lists of changes or barriers).…”

“…Commitment to change (CTC) is one tool that is widely used for this purpose in continuing education, due in part to its relative ease of administration and evidence that physicians who commit to intended changes are more likely to make actual changes in clinical practice . CTC is a theoretical framework and a practical approach for enabling and measuring behavioral change resulting from attending an educational event . Participants are asked to write down descriptions of the changes they propose to make in their practice as a result of what they learned during the educational event, and a follow‐up survey is administered at a set time point to assess the extent of implementation and to identify barriers to making the intended changes.…”

Evaluation of a Collaborative Program on Smoking Cessation: Translating Outcomes Framework Into Practice

“…We used the CTC approach with and without follow-up 17 as a proven surrogate measure of actual performance change. 18,19 In some activities, we embedded a request to list intended changes resulting from attending the activity into the postactivity evaluation.…”

Number Needed to Benefit From Information (NNBI): Proposal From a Mixed Methods Research Study With Practicing Family Physicians

“…In addition, longer follow up on all searches could yield additional evidence of benefit. 23 Longer follow-up may also allow the detection of other patients for which the same information could be used; observed benefits for other patients would have also led to lowering the NNBI. Third, it is likely the 2008 version of Essential Evidence Plus was somewhat difficult to use for searches on a small hand-held computer.…”