Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education

Abstract

Assessment of educational effectiveness provides vitally important feedback to Institutions of Higher Education. It also provides important information to external stakeholders, such as prospective students, parents, governmental and local regulatory entities, professional and regional accrediting organizations, and representatives of the workforce. However, selecting appropriate indicators of educational effectiveness of programs and institutions is a difficult task, especially when criteria of effectiveness are not well defined. This article proposes a comprehensive and systematic approach to aligning criteria for educational effectiveness with specific indicators of achievement of these criteria by adapting a popular organizational training evaluation framework, the Kirkpatrick's four level model of training criteria (Kirkpatrick 1959;1976;1996), to assessment in Higher Education. The four level model consists of reaction, learning, behavior and results criteria. Adaptation of this model to Higher Education helps to clarify the criteria and create plans for assessment of educational outcomes in which specific instruments and indicators are linked to corresponding criteria. This provides a rich context for understanding the role of various indicators in the overall mosaic of assessment. It also provides Institutions of Higher Education rich and multilevel feedback regarding the effectiveness of their effort to serve their multiple stakeholders. The importance of such feedback is contextualized both in the reality of stakeholder pressures and in theoretical understanding of colleges and universities as open systems according to the systems theory (Katz and Kahn 1966). Although the focus of this article is on Higher Education, core principles and ideas will be applicable to different types and levels of educational programs.

Flipped learning, flipped satisfaction, getting the balance right

“…Measuring learning outcomes is one method to judge the value-add of education institutions (Douglass, Thomson, & Zhao, 2012), or when extrapolated to the individual class level, the value-add of the course material and pedagogy employed. Although there is evidence that people with less skill in a domain can over-assess their own competence (Kruger & Dunning, 1999), students can develop the capacity to make good judgements about their own performance (Boud, Lawson, & Thompson, 2014), and learning outcomes can be measured objectively and perceived by self-report (Praslova, 2010). Accordingly, asking final year students to indicate the extent to which they believe they meet the learning outcomes of a subject will give an indication of the perceived value-add of the course material and pedagogy employed from the perspective of the student.…”

An Introductory Pharmacy Practice Experience Based on a Medication Therapy Management Service Model

“…12,13 Originally developed to assess training programs, Kirkpatrick's framework is a systematic approach to evaluation involving examination of a course on multiple levels using different data sources. The current study did not include a level 4 evaluation, which focuses on the impact of student behavioral changes to their future place of employment.…”

“…The current study did not include a level 4 evaluation, which focuses on the impact of student behavioral changes to their future place of employment. 7,12,14 The authors chose Kirkpatrick's framework in favor of other available evaluation approaches because they were most familiar with it. 14 …”

Process Evaluation of an Applied Neuroeconomics Extension Course in Consulting and Management Based on Kirkpatrick's Model

“…Particularly, the course was formulated with the goal of developing psychological literacy skills in basic principles of human behavior, to help students from economics programs apply this knowledge to better understand and manage the community members' financial challenges on an individual basis. A process evaluation of the program was undertaken to guide future educational initiatives, following prior positive indications that the program could provide relevant feedback to institutions of higher education (Praslova, 2010).…”