The Delphi method is used to investigate consensus amongst a panel of experts using repeated rounds of a questionnaire, often in healthcare settings. However, many Delphi studies do not report any investigation into what happens to the stability of consensus or the convergence of agreement between the rounds in the study, which may be of importance. In this paper an accessible analytical approach is outlined using graphical presentations of means and standard deviations to identify what happens between rounds. For Delphi studies where the scale upon which experts are expressing their opinions can be considered to be interval, the mean will represent the group opinion whilst the standard deviation will represent the level of agreement. An example Delphi study from a healthcare setting is used to illustrate the methodology.

The Delphi method is used to investigate consensus amongst a panel of experts using repeated rounds of a questionnaire, often in healthcare settings. However, many Delphi studies do not report any investigation into what happens to the stability of consensus or the convergence of agreement between the rounds in the study, which may be of importance. In this paper an accessible analytical approach is outlined using graphical presentations of means and standard deviations to identify what happens between rounds. For Delphi studies where the scale upon which experts are expressing their opinions can be considered to be interval, the mean will represent the group opinion whilst the standard deviation will represent the level of agreement. An example Delphi study from a healthcare setting is used to illustrate the methodology.

The process of examination marking is complex, requiring examiners to engage in a variety of cognitive operations. While consideration has been given to marking practices in a few specific contexts, those of General Certificate of Secondary Education (GCSE) examiners have yet to receive serious attention. This study's aims, therefore, were: first, to investigate the cognitive strategies used when marking GCSEs; and second, to interpret them within the context of psychological theories of human judgement. Two contrasting GCSE examination papers were considered: Mathematics used a 'points-based' marking scheme, while Business Studies relied on a 'levels-based' scheme. Small groups of experienced examiners marked script samples, and using a concurrent 'think aloud' method, verbal protocols were obtained. Using a semi-structured interview schedule, the examiners were then questioned retrospectively. Qualitative data analysis supported a model of five distinct cognitive marking strategies, which can be interpreted within dual-processing theories of judgement. The implications for GCSE marking are discussed.

No abstract