No abstract

No abstract

Any government deciding to invoke widespread change in its higher education sector through implementation of new policies impacts on every institution and all staff and students, often in both the time taken up and the heightened emotions caused. The central phenomenon that this study addresses is the process and consequences of policy changes in higher education in Australia. The aim of this article is to record the research design through the perspective (evaluation research), theoretical framework (program evaluation) and methods (content analysis, descriptive statistical analysis and bibliometric analysis) applied to the investigation of the 2003 federal government higher education reform package. This approach allows both the intended and unintended consequences arising from the policy implementation of three national initiatives focused on learning and teaching in higher education in Australia to surface. As a result, this program evaluation, also known in some disciplines as policy implementation analysis, will demonstrate the applicability of illuminative evaluation as a methodology and reinforce how program evaluation will assist and advise future government reform and policy implementation, and will serve as a legacy for future evaluative research.

Purpose Informed by the critical perspective of dialogic accounting theory, the purpose of this paper is to explore the use of evaluation as a means of enhancing accountability to beneficiaries within nonprofit organisations (NPOs). As a stakeholder group frequently marginalised by traditional accounting practices, the participation of beneficiaries within a NPO’s accountability structure is presented as a means of increasing social justice. Design/methodology/approach The research design used case studies involving two NPOs, examining documents and conducting interviews across three stakeholder groups, within each organisation. Findings Findings reveal that when viewed on beneficiaries’ terms, accountability to beneficiaries, through participative evaluation, needs to consider the particular timeframe of beneficiary engagement within each organisation. This temporal element positions downwards accountability to beneficiaries within NPOs as multi-modal. Research limitations/implications The research poses a limit to statistical generalisability outside of the specific research context. However, the research prioritises theoretical generalisation to social forms and meanings, and as such provides insights for literature. Practical implications In acknowledging that beneficiaries have accountability needs dependent upon their timeframe of participation, NPOs can better target their downwards accountability structures. This research also has practical implications in its attempt to action two of the United Nation’s Sustainable Development Goals. Originality/value This paper makes a contribution to the limited research into nonprofit accountability towards beneficiaries. Dialogic accounting theory is enacted to explore how accountability can be practised on beneficiaries’ terms.

Student Evaluations of Teaching (SETs) have been used to evaluate Higher Education teaching performance for decades. Reporting SET results often involves the extraction of an average for some set of course metrics, which facilitates the comparison of teaching teams across different organisational units. Here, we draw attention to ongoing problems with the naive application of this approach. Firstly, a specific average value may arise from data that demonstrates very different patterns of student satisfaction. Furthermore, the use of distance measures (e.g. an average) for ordinal data can be contested, and finally, issues of multiplicity increasingly plague approaches using hypothesis testing. It is time to advance the methodology of the field. We demonstrate how multinomial distributions and hierarchical Bayesian methods can be used to contextualise the SET scores of a course to different organisational units and student cohorts, and then show how this approach can be used to extract sensible information about how a distribution is changing in time. We present a report designed to facilitate sense-making for the more complex statistical methodology that we propose, and demonstrate how it can be used to ensure that this more complex methodology is still appropriately used in decision making.