Estimating Interrater Agreement with the Average Deviation Index: A User’s Guide

Burke, Michael J.; Dunlap, William P.

doi:10.1177/1094428102005002002

Cited by 373 publications

(402 citation statements)

References 13 publications

Supporting

Mentioning

380

Contrasting

Unclassified

Order By: Relevance

“…Interrater agreement between independent observers was assessed by computing the average deviation index (Burke & Dunlap, 2002;Burke, Finkelstein, & Dusig, 1999) for which low values indicate high agreement. Raters' average deviation ranged from 0.13 to 0.30 on a 4-point scale and was thus clearly below the cutoff value of 0.67 (Lüdtke, Trautwein, Kunter, & Baumert, 2006).…”

Section: More Fun-less Learned: Oral History In History Lessonsmentioning

confidence: 99%

Learning Historical Thinking With Oral History Interviews: A Cluster Randomized Controlled Intervention Study of Oral History Interviews in History Lessons

Bertram

Wagner

Trautwein

2017

American Educational Research Journal

View full text Add to dashboard Cite

The present study examined the effectiveness of the oral history approach with respect to students' historical competence. A total of 35 ninth-grade classes (N = 900) in Germany were randomly assigned to one of four conditions-live, video, text, or a (nontreated) control group-in a pretest, posttest, and follow-up design. Comparing the three intervention groups with the control group, the intervention groups scored better on four of the five achievement tests. Comparing the live group with the video and text groups, students in the live condition were more convinced of their learning progress at both measurement points. However, they scored lower than the video/text group on two achievement measures and higher on one at the posttest.

show abstract

Section: More Fun-less Learned: Oral History In History Lessonsmentioning

confidence: 99%

Learning Historical Thinking With Oral History Interviews: A Cluster Randomized Controlled Intervention Study of Oral History Interviews in History Lessons

Bertram

Wagner

Trautwein

2017

American Educational Research Journal

View full text Add to dashboard Cite

show abstract

“…The interpretation of the AD Md value is made applying the formula c/6 in which "c" represents the amplitude of the scale of responses (Burke & Dunlap, 2002). Since the statements regarding the performance of the team were answered on 5-point scale, the maximum discrepancy of responses from the members is defined as dividing 5 by 6, resulting in the value of 0.83.…”

Section: As Problematic (Above 2 In Absolute Values)mentioning

confidence: 99%

Team Performance: Evidence for Validity of a Measure

2016

View full text Add to dashboard Cite

This study aimed to obtain validity evidences of a teamwork performance scale. Team performance was understood as a mesolevel characteristic, resulting of an emerging process. Due to that understanding the proposed instrument should take that aspect into consideration. The empirical data were collected from 276 Ecuadorian teachers organized in 70 educational teams. They answered nine sentences of the scale. Results of exploratory factor analysis showed a unifactorial solution explaining 65.84% of the variance. The measure also has adequate values of reliability (Cronbach Alpha = .93). In addition to these analyses, patterns of variance within and between the groups were verified. The results showed that the variance at the individual level was small when answers of team members were analyzed and was significant when teams were compared. We consider it is important that additional studies be performed in order to identify stability of the factor solution. Key words: job performance; assessment scale; teamwork Desempenho de equipes: evidências de validade de uma medidaResumo O objetivo deste estudo foi desenvolver e verificar evidências de validade de uma medida que permita aferir o desempenho de equipes de trabalho. O desempenho foi entendido como atributo do nível meso, portanto, relativo às equipes, que surge mediante um processo de emersão e, desse modo, a medida deve considerar essa especificidade. Para a realização empírica da pesquisa contou-se com respostas de 276 professores equatorianos organizados em 70 equipes pedagógicas, que responderam a nove afirmativas presentes no instrumento. Os resultados de análise fatorial exploratória mostraram a solução unifatorial como a mais satisfatória, a qual captura 65,84% da variância do fenômeno e possui valores adequados de confiabilidade (alpha de Cronbach de 0,93). Análises de padrões de variância dentro e entre grupos revelaram que a medida permite capturar um fenômeno do nível meso. Recomenda-se a realização de outras pesquisas que verifiquem a estabilidade desta solução. Palavras-chave: desempenho no trabalho; escala de avaliação; equipes de trabalho. Rendimiento de equipos: evidencias de validez de una medida ResumenEl objetivo de este estudio fue desarrollar y verificar evidencias de validez de una medida que permita la medición del rendimiento de equipos de trabajo. El rendimiento fue entendido como un atributo del nivel meso, o sea, relativo a los equipos, que surge mediante un proceso de emersión, y de ese modo la medida propuesta debería llevar en consideración ese aspecto. Para la realización empírica de la investigación se contó con las respuestas dadas por 276 profesores ecuatorianos organizados en 70 equipos pedagógicos, que respondieron a nueve afirmativas contenidas en el instrumento. Los resultados del análisis factorial exploratorio mostraron la solución unifactorial como la más satisfactoria que capturó 65,84% de la varianza del fenómeno y posee valores adecuados de confiabilidad (alpha de Cronbach de 0,93). Análisis de patrones de varia...

show abstract

“…To justify this aggregation, the AD Index was calculated (Average Deviation Index; Burke, Finkelstein, & Dusing, 1999;Burke & Dunlap, 2002). For that, the ratio c/6 proposed by Burke and Dunlap (2002), where "c" stands for the number of response options for the items of each measuring instrument, was used. For the GDS and KMT scales the ratios obtained were 1.17 and .83 respectively.…”

Section: Intragroup Agreement and Intergroup Heterogeneitymentioning

confidence: 99%

The Knowledge Management Processes at Different Stages of Group Development

Pinto

Lourenço

Mónico

2018

Psic.: Teor. e Pesq.

View full text Add to dashboard Cite

-Based on a developmental approach, this study intended to analyze the extent to which the diff erent stages of group development diff er regarding the use of knowledge management processes. The sample comprised 211 teams belonging to a Portuguese military organization. In order to test the hypotheses a multivariate analysis of variance was conducted. The results showed that more mature and cooperative groups (Restructuring/Realization phase) apply to a greater degree the processes of knowledge management, while groups in which there is high intragroup competition and in which members try to "gain power" among themselves (Reframing phase), apply these processes to a lesser degree. The literature review which we carried out regarding working groups/teams and knowledge management allows us to affi rm that there are advantages in studying them in an articulated way (e.g., Ancona & Caldwell, 1992;Argote, Ingram, Levine, & Moreland, 2000;Argote, McEvily, & Reagans, 2003;Brown & Utterback, 1985;Henriksen, 2001;Ingram & Simons, 2002;Jehn & Shah, 1996;Lee, Gillespie, Mann, & Wearing, 2010;Liang, Moreland, & Argote, 1995;Love & Roper, 2009;Nelson & Cooprider, 1996;Zárraga & Bonache, 2003;Zhuge & Shi, 1997). Indeed, by one side, teamwork constitutes a way of managing and organizing the work which can contribute to increase productivity, quality of work life and ensure high levels of quality of products/ services (Rousseau, Aubé, & Savoie, 2006). Knowledge management, on the other hand, is nowadays a unique and intangible resource which can be used by groups and organizations in order to acquire advantages regarding competitiveness (Barney, 1991;Miller & Shamsie, 1996).Although there is no clear consensus on the concept of team development (Kozlowski, 2015), conceptual and empirical literature about the working groups/teams which adopts a developmental approach (e.g., Buzalo & Wheelan, 1999;Garfi eld & Dennis, 2013;Tuckman & Jensen, 1977) tend to show that groups change over time and that the group processes/emerging states (e.g., intragroup confl icts, group emotions) operate diff erently at diff erent levels of group existence (or developmental stages).Studies carried out in the fi eld of knowledge management in the working groups/teams, (e.g., Ancona & Caldwell, 1992;Brown & Utterback, 1985;Cummings, 2004;Huang, 2009;Lee et al., 2010;Prange, 1999;Zhuge & Shi, 1997), as well as the models that we can fi nd in the literature (e.g., Model of Cardoso and Peralta, 2011; Model of Marin-Garcia and Zarate-Martinez, 2008; Model of Zárraga-Oberty and Garcia Falcón, 2003), do not take into account the group temporality, namely the process of group development. Therefore, the role group development plays in the way the group manages knowledge is not clear in the literature.With the aim of contributing to a better understanding of the group functioning and also trying to overcome the referred gap in the literature, this study aims to analyze the diff erences among the diff erent stages of group development regarding

show abstract

Estimating Interrater Agreement with the Average Deviation Index: A User’s Guide

Cited by 373 publications

References 13 publications

Learning Historical Thinking With Oral History Interviews: A Cluster Randomized Controlled Intervention Study of Oral History Interviews in History Lessons

Learning Historical Thinking With Oral History Interviews: A Cluster Randomized Controlled Intervention Study of Oral History Interviews in History Lessons

Team Performance: Evidence for Validity of a Measure

The Knowledge Management Processes at Different Stages of Group Development

Contact Info

Product

Resources

About