Developing the Program Evaluation Utility Standards: Scholarly Foundations and Collaborative Processes

Yarbrough, Donald B.

doi:10.3138/cjpe.349

Cited by 19 publications

(25 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Some confusion about the purpose of the reporting standards was revealed through the Delphi process and some participants identified potential uses beyond reporting by those (e.g., to guide development of education interventions or assist policy makers who may commission evaluations). While synchronous review methods, such as face-to-face workshops [9][10][11]13,18,29 may have identified and resolved this confusion quickly, the asynchronous nature of contribution to a Delphi process meant that queries were documented for in-depth discussion and resolution to clarify the purpose and utility of the standards.…”

Section: Discussionmentioning

confidence: 99%

“…The consensus methodology applied was adapted from those used in the development of reporting standards for diagnostic accuracy studies, 9 randomized controlled trials, 10 observational epidemiological studies, 11 systematic reviews 13 , educational interventions for evidence-based practice 17 and educational program evaluation. 18 Figure 1 summarizes the three-stage approach we employed to develop and refine the reporting standards, described below. This process was overseen by a Project Working Group comprising CG, HJ, MMa, AN and MJ.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE2 Genomics)

Nisselle

Janinski

Martyn

et al. 2021

Genetics in Medicine

View full text Add to dashboard Cite

Purpose: Widespread, quality genomics education for health professionals is required to create a competent genomic workforce. A lack of standards for reporting genomics education and evaluation limits the evidence base for replication and comparison. We therefore undertook a consensus process with the aim of developing a recommended minimum set of information to support consistent reporting of the design, development, delivery and evaluation of genomics education interventions.Methods: Draft standards were derived from the literature (25 items from 21 publications).Thirty-six international experts were purposively recruited and completed three rounds of a modified Delphi process to reach consensus on relevance, clarity, comprehensiveness, utility and design. Results:The final standards include 18 items relating to development and delivery of genomics education interventions, 12 relating to evaluation, and one on stakeholder engagement. Conclusion:These reporting item standards for education and evaluation of genomics (RISE2 Genomics) are intended to be widely applicable across settings and health professions. Their use by those involved in reporting genomics education interventions and evaluation, as well as their adoption by journals and policy makers as the expected standard, will support greater transparency, consistency and comprehensiveness of reporting.Consequently, the evidence base of genomics education will be more robust, enabling highquality genomics education and evaluation across a range of settings.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE2 Genomics)

Nisselle

Janinski

Martyn

et al. 2021

Genetics in Medicine

View full text Add to dashboard Cite

show abstract

“…At present, the JCSEE program evaluation standards are the basis for formulating other standards, and have been recognized by the professional evaluation organization [24]. They are now in their third edition and are still under development [25]. JCSEE has always encouraged its users to make comments and suggestions on their own applications [18,19].…”

Section: The Standards Of Meta-evaluationmentioning

confidence: 99%

Meta-Evaluation for the Evaluation of Environmental Management: Standards and Practices

Yin

Guo

2021

Sustainability

View full text Add to dashboard Cite

Environmental management plays a key role in the sustainable development of cities. The effectiveness of environmental management is commonly examined through some evaluation schemes, but the effectiveness of such evaluation schemes also needs to be verified. As such, in this study, meta-evaluation was creatively introduced into environmental management to improve the evaluation of environmental management (EEM). Meta-evaluation is the evaluation of an evaluation scheme, and can verify and enhance the evaluation quality. First, a set of new meta-evaluation standards and criteria was proposed based on the unique characteristics of environmental management, which made meta-evaluation standards more adaptable and effective. After that, the efficacy of the proposed meta-evaluation standards was verified through their application to two evaluation schemes used in different fields of EEM. Based on meta-evaluation, suggestions for improving these two EEM schemes were also provided. The major contributions of this study are to introduce meta-evaluation into environmental management, establish new evaluation standards, and examine the efficacy of EEM. The research showed that it is critical to carry out meta-evaluation before and/or after the implementation of EEM.

show abstract

“…Validasi produk menggunakan indikator The Design Of Instruction and Evaluation dengan lima aspek penilaian yaitu: (1) materi; (2) konten pendidikan; (3) tulisan; (4) layanan interaktif; (5) penggunaan produk (Rabinowitz et al, 2004). Penilaian kualitas produk menggunakan indikator Joint Committee on Standards for Educational Evaluation dengan 4 aspek penilaian, yaitu: utility, feasibility, accuracy, and propriety (Yarbrough, 2017). Analisis data menggunakan content validity ratio (CVR) dengan taraf konsultasi sebesar 0,3 (Ayre & Scally, 2014) dan kualitas produk menggunakan rumus persentase sekaligus sebagai kategori standar (Maksum, 2018).…”

Section: Metodeunclassified

Kelayakan produk pengembangan tutorial aktivitas fisik bagi anak berkebutuhan khusus saat pandemi

et al. 2021

View full text Add to dashboard Cite

Kebutuhan aktivitas fisik menjadi isu terkini ketika aktivitas di luar rumah dibatasi selama pandemi. Tidak terkecuali untuk para anak berkebutuhan khusus (ABK) yang pada umumnya memiliki keterbatasan pada kecakapan fokus. Untuk itu, perlu dibuatkan media agar memicu ABK memiliki tingkat aktivitas fisik sekaligus melatih kecakapan fokus. Produk yang sedang diujicobakan adalah video tutorial yang dimuat dalam https://fokusabk.com/ sebagai media aktivitas fisik untuk ABK. Tujuan penelitian ini adalah mendeskripsikan validitas dan kualitas https://fokusabk.com/ ditinjau dari para ahli dan orang tua sehingga menjadi tanda kelayakannya digunakan sebagai mediator aktivitas fisik untuk ABK. Metode penelitian yang digunakan adalah deskriptif kuantitatif. Subjek penelitian sebanyak 6 ahli dan 10 orang tua subjek. Validasi produk menggunakan indikator dari The Design of Instruction and Evaluation dan untuk kualitas produk menggunakan indikator Joint Committee on Standards for Educational Evaluation. Analisis data menggunakan content validity ratio (CVR) dan persentase. Hasil penelitian menunjukkan bahwa para ahli dan orang tua memberikan penilaian terhadap produk yang dihasilkan dinyatakan valid (CVR= 0,92-1,00 > 0,3). Berdasarkan para orang tua sebagai subjek, produk hasil pengembangan memiliki kualitas baik sekali (81,2%). Sehingga dapat disimpulkan bahwa https://fokusabk.com/ layak digunakan sebagai mediator aktivitas fisik oleh ABK. Kata kunci: tutorial, aktivitas fisik, anak berkebutuhan khusus.The need for physical activity has become a recent issue when outdoor activities are restricted during a pandemic. No exception for children with special needs who generally have limited focus skills. For this reason, it is necessary to create a media so that it triggers for children with special needs to have a level of physical activity as well as train focus skills. The product being tested is a video tutorial published on https://fokusabk.com/ as a medium for physical activity for crew members. The purpose of this study is to describe the validity and quality of https://fokusabk.com/ in terms of experts and parents so that it is a sign of its eligibility to be used as a mediator of physical activity for children with special needs. The research method used is descriptive quantitative. The research subjects were 6 experts and 10 parents. Product validation uses indicators from The Design of Instruction and Evaluation and for product quality uses indicators from the Joint Committee on Standards for Educational Evaluation. Data analysis used content validity ratio (CVR) and percentage. The results showed that the experts and parents gave an assessment of the resulting product was declared valid (CVR= 0.92-1.00> 0.3). Based on the parents as the subject, the product development results have very good quality (81.2%). So it can be concluded that https://fokusabk.com/ is suitable to be used as a mediator for physical activity by children with special needs.

show abstract

Developing the Program Evaluation Utility Standards: Scholarly Foundations and Collaborative Processes

Cited by 19 publications

References 20 publications

Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE2 Genomics)

Ensuring best practice in genomics education and evaluation: reporting item standards for education and its evaluation in genomics (RISE2 Genomics)

Meta-Evaluation for the Evaluation of Environmental Management: Standards and Practices

Kelayakan produk pengembangan tutorial aktivitas fisik bagi anak berkebutuhan khusus saat pandemi

Contact Info

Product

Resources

About