The conceptual definition of systems-based practice (SBP) does not easily translate into directly observable actions or behaviors that can be easily assessed. At the Academic Emergency Medicine consensus conference on education research in emergency medicine (EM), a breakout group presented a review of the literature on existing assessment tools for SBP, discussed the recommendations for research tool development during breakout sessions, and developed a research agenda based on this discussion.ACADEMIC EMERGENCY MEDICINE 19:1366-1371 © 2012 by the Society for Academic Emergency Medicine T he inclusion of systems-based practice (SBP) as one of the Accreditation Council for Graduate Medical Education (ACGME) core competencies highlights the importance of the health care system on the ability of a physician to provide competent and effective patient care.1 Physicians must be able to collaborate with other members of the health care team, consider costs when weighing risks and benefits, improve system performance by identifying system errors and implementing potential solutions, and continue to advocate for quality patient care. This set of skills is also described in the CanMEDS Physician Competency Framework as elements of the physician role as a collaborator, manager, and health advocate. Despite a recent shift in the ACGME assessment system to the Next Accreditation System, the framework for assessing milestones within the competencies still requires specific tools to assess SBP.3 Although the conceptual definition of SBP does not easily translate into observable actions or behaviors, the Council of Emergency Medicine Residency Directors (CORD) was able to develop emergency medicine (EM)-specific evaluation domains for SBP. 4 These included specific actions and behaviors that are directly observable and facilitated real-time and summative feedback. However, this set of observable behaviors did overlook some pertinent areas, such as situational awareness and participation in systems improvement. Similarly, by using the Can-MEDS roles of collaborator, manager, and health advocate, Graham et al. also developed a comprehensive list of observable actions that readily translated into assessments.
5This article summarizes the authors' review of the current assessment tools for the SBP competency used in EM and non-EM residencies, both within and outside the United States. As a result of several small group discussions during the breakout session on the assessment of observable learner performance at the 2012 Academic Emergency Medicine (AEM) consensus conference on education research in EM, we developed a research plan for assessment tool development.
PROCESS AND CONSENSUSWe reviewed the literature on SBP assessment, searching the Medline and Pubmed databases by combining the terms "systems based" with "evaluat* OR assess*" and "competenc*" as well as combining "simulation" or "portfolio" with "systems based practice," yielding 156 and 309 references, respectively.