In 2013, the Australasian Evaluation Society (AES) launched the Evaluators’ Professional Learning Competency Framework. In 2020, the AES (now Australian Evaluation Society) partnered with learnevaluation.org to develop an online self-assessment for the AES community. The AES Competencies were explicitly designed to support professional learning, not as any kind of assessment instrument. Therefore, work was needed to update the competencies into a measurable format. The authors report the history of the competencies, developments made before and after pilot testing them in the online Learn Evaluation Assessment Portal (LEAP), and the findings from the data collected over two and a half years of using the LEAP. The AES Competencies Framework remains one of the few competency sets with explicit knowledge and skill statements related to the logic of evaluation.