Moving from guideline to measure to findings: The Australian Evaluation Society and the Learn Evaluation Assessment Portal

Evaluation is a practice with increasing global demand. However, an understanding of who wants to learn evaluation and their learning needs related to evaluator competencies is relatively unexplored. This article describes a research effort designed to address those needs using an evaluator competency self-assessment. Despite their validity challenges, self-assessments can yield valuable information for a variety of audiences. Respondents and their organisations can use this method to understand evaluation capacity strengths and needs and create professional development plans. For those conducting evaluation capacity building in or for organisations, and those developing formal evaluation education programs, self-assessment results can provide a road map for planning, development and delivery. This article describes the process of developing and piloting the Learn Evaluation Assessment Portal, an evaluator competency self-assessment tool, with the Australian Evaluation Society. The article reflects on the lessons learnt from the development and testing of the tool and will be valuable for a range of stakeholders from practitioners to commissioners and in particular Voluntary Organisations of Professional Evaluators which are committed to developing evaluation capacity and are working towards professionalisation of the field.

Making the Learn Evaluation Assessment Portal: Developing a reciprocal tool for learning and research

Wildschut,

Gullickson,

Mason

2024

Framing the future: Exploring the values that underpin the Australian Evaluation Society’s Evaluators’ Professional Learning Competencies

Domvo,

Gullickson,

Wildschut

2024

In the last two decades, evaluator competency frameworks have become ubiquitous. Many have been developed by reviewing the literature and engaging in various research processes to ask evaluation practitioners what they do, in order to determine what evaluators should know and be able to do. Due to this practice focused approach, the underlying philosophical and theoretical assumptions of competencies are rarely questioned. In this article, the authors explore this territory by categorising competencies using two taxonomies. Mertens and Wilson’s 2018 Program evaluation theory and practice provides a philosophical framework, and Schwandt & Gates’ 2021 Evaluating and valuing in social research provides a theoretical framework. The authors apply these frameworks to the three evaluation specific domains of the Australian Evaluation Society (AES) Evaluators’ Professional Learning Competency Framework as refined for the Learn Evaluation Assessment Platform in 2020. Findings of this exploratory study suggest that the AES ’ conception of evaluator competencies aligns primarily with Mertens and Wilson’s pragmatist/neopragmatist paradigm and Schwandt and Gates ’ expanded frame. The authors discuss implications of the results and propose ideas for further improvement and testing of this research approach.

Evaluative attitude in action: Using the Learn Evaluation Assessment Portal in graduate education

Gullickson,

Tanabe,

Davis

et al. 2024

Evaluation students need a sense of how the expertise they bring and the knowledge and skills they are learning relate to the tasks of evaluation and to their own ongoing professional development. Within the Master of Evaluation and the Graduate Certificate in Evaluation at The University of Melbourne’s Assessment and Evaluation Research Centre (formerly the Centre for Program Evaluation), students use an evaluator competencies self-assessment to inform their coursework and practice across three subjects. In this paper, the authors – the coordinator of the subjects and several students – share practices and examples of how the self-assessment added value for their learning. Discussion of the logistics, ethical requirements, and appropriate use given the limitations of self-assessments provide ideas for integrating self-assessment into the design and delivery evaluation capacity building or formal courses to increase ownership of learning for adults who want to expand their evaluation expertise.