Determining Indicators of High-Quality Application Activities for Team-Based Learning

Abstract

Objective. To determine the indicators of quality for application activities in pharmacy team-based learning (TBL). Methods. A modified Delphi process was conducted with pharmacy TBL experts. Twenty-three experts met the inclusion criteria, including having at least four years of TBL experience, designing at least eight TBL sessions, training others to use TBL, and authoring a peer-reviewed TBL pharmacy paper. In round 1, panelists responded to five open-ended questions about their successful TBL applications activities, including satisfaction with the activity and methods for creating positive student outcomes. In round 2, panelists indicated their level of agreement with the round 1 quality indicators using a four-point Likert rating. Consensus was set at 80% strongly agree/agree. In an open comment period, panelists provided suggestions to help expand the indicator descriptions. Indicators were verified based on TBL and the education literature. Results. Twenty panelists (87% of those eligible) responded in round 1 and 17 (85% participation) in round 2. Sixteen quality indicators were identified in round 1, with 14 achieving consensus in round 2. "Uses authentic pharmacy challenges or situations" (88% strongly agree/agree) and "incorporates or provides effective feedback to groups" (88% strongly agree/agree) met consensus. However, "has multiple right answers" (76% strongly agree/agree) and "incorporates elements from school specific emphases (eg, faith, underserved)" (53% strongly agree/agree) did not reach consensus. Conclusions. These indicators can assist faculty members in designing application activities to provide high-quality TBL exercises that promote deep thinking and engaged classroom discussion. The indicators could also guide faculty development and quality improvement efforts, such as peer review of application activities.

A key performance indicators redefinition initiative at a school of pharmacy using a modified Delphi consensus technique

“… 34 , 35 The three-round Delphi technique is a common approach in the pharmacy literature and elsewhere, but other approaches including two rounds have also been adopted. 18 , 22 , 26 , 28 , 36 - 38 We deliberately used high consensus criteria (as high as ≥90% in the first round and ≥80% in the second) to identify essential KPIs. Previous studies adopted a consensus threshold of ≥80%, or ≥75% throughout all rounds.…”

“…Previous studies adopted a consensus threshold of ≥80%, or ≥75% throughout all rounds. 18 , 26 , 38 Our study used varying consensus thresholds throughout the three rounds similar to previous research, with the goal to differentiate between KPIs for which consensus was obvious, and those that would generate more internal discussion regarding their inclusion for performance measurement purposes. 17 , 37 , 39 This approach allowed us to clearly identify student satisfaction, scholarly output, and funding as the most critical KPIs to evaluate the school’s performance in round 1.…”

“…Some used Likert scales with varying formats (4-, 5-, 6- or 9-point), while others used dichotomous scales. 17 - 19 , 26 , 36 , 38 , 39 When using Likert-type scales, consensus is often assessed by collapsing responses to the two most extreme options of agreement, technically rendering the scale dichotomous (i.e., agree or strongly agree versus everything else). In our study, we opted for a dichotomous option to obtain unequivocal decisions from participants regarding the essentiality of each KPI.…”

Vancomycin Flight Simulator: A Team-Based Learning Exercise

“…Feedback for testing and TBL performance is provided in real time. High-quality TBL activities in pharmacy education should use authentic pharmacy challenges/situations regarding relevant pharmacy tasks, which encourage rich discussion and provide effective feedback to the groups [ 9 ].…”