Understanding and evaluating teaching effectiveness in the UK higher education sector using experimental design

Abstract

Purpose The purpose of this paper is to demonstrate the power of experimental design as a technique to understand and evaluate the most important factors which influence teaching effectiveness for a postgraduate course in a higher education (HE) context. Design/methodology/approach The methodology involves the execution of a case study in the form of an experiment in a business school setting. The experiment was carried out with the assistance of over 100 postgraduate students from 26 countries. The data were collected over a two year period (2015 and 2016) from a postgraduate course offered by the same tutor for repeatability reasons. Findings The key findings of the experiment have clearly indicated that students’ perceptions of teaching effectiveness based on intuition and guesswork are not identical to the outcomes from a simple designed experiment. Moreover, the results of the experiment provided a greater stimulus for the wider applications of the technique to other processes across the case study HE sector. Research limitations/implications One of the limitations of the study is that the experiment was conducted for a popular postgraduate course. It would be beneficial to understand the results of the experiment for less popular postgraduate courses in the university in order to drive improvements. Moreover, this research was conducted only for postgraduate courses and the results may vary for undergraduate courses. This would be an interesting study to understand the differences in the factors between undergraduate and postgraduate teaching effectiveness. Practical implications The outcome of this experiment would help everyone who is involved in teaching to understand the factors and their influences to improve students’ satisfaction scores during the delivery of teaching. Originality/value This paper shows how experimental design as a pure manufacturing technique can be extended to a HE setting.

Design of experiments in the service industry: a critical literature review and future research directions

“…In 66.7% of the case studies, the number of factors were less than or equal to five, as shown in Figure 8. The studies that included more than ten factors used either the (Antony et al, 2019;Bell et al, 2006) or the Taguchi method (Bleier et al, 2019;Shahin et al, 2012) to identify the most significant factors. Some experimenters utilised Plackett-Burman designs to separate out the vital few factors from the trivial many.…”

“…Concerning the applications of DoE in the higher education sector, Antony et al (2019) and Ree et al (2014) reported the difficulty of measuring responses due to the subjectivity involved among the participants. Antony et al (2014) mentioned that the challenge was the small number of students that participated in the study and the fact that the experiment was confined to only one module.…”

“…Nevertheless, when it comes to the non-manufacturing industry or service industry, there is still a lack of ED applications (Antony et al, 2010). Some possible reasons are researchers' lack of awareness of the methods, difficulty in measuring how services are performed, impact of human behaviour on service results, high levels of noise or uncontrolled factors, challenges of including intangible dimensions, and lack of a clear description of the service process (Antony et al, 2019;Bell et al, 2006;Xie and Goh, 1999). With the introduction of Six Sigma in the service industry, DoE has been used by black belts, master black belts, and other professionals as a support tool in the improvement phase of the define, measure, analyse, improve and control approach (Arafeh et al, 2014).…”

“…Despite all of these challenges, it is clear that quality and process improvement professionals can use DoE to solve problems not only in the manufacturing industry. As shown by Krishnamoorthy and Kapadia (1999), Bell et al (2006), and Antony et al (2019), DoE can also be applied in different service industries without the need to carry out a complete Lean Six Sigma project. For this reason, gathering more detailed information on DoE applications in many different service industries can significantly contribute to improving the development and training of this methodology in the service industry from both academic and industry points of view.…”

Online Teaching in the Context of the Epidemic

“…If not handled properly, it will have a serious impact on the country and society [7,8]. Therefore, the development of online teaching can ensure the progress of teaching, and at the same time, this novel teaching method can also make students happily absorb knowledge [9,10].…”