Data analysis in medical education research: a multilevel perspective

Abstract

A substantial part of medical education research focuses on learning in teams (e.g., departments, problem-based learning groups) or centres (e.g., clinics, institutions) that are followed over time. Individual students or employees sharing the same team or centre tend to be more similar in learning than students or employees from different teams or centres. In other words, when students or employees are nested within teams or centres, there is a within-team or within-centre correlation that should be taken into account in the analysis of data obtained from individuals in these teams or centres. Further, when individuals are measured several times on the same performance (or other) variable, these repeated measurements tend to be correlated, that is: we are dealing with an intra-individual correlation that should be taken into account when analyzing data obtained from these individuals. In such a study context, many researchers resort to methods that cannot account for intra-team and/or intra-individual correlation and this may result in incorrect conclusions with regard to effects and relations of interest. This comparison paper presents the benefits which result from adopting a proper multilevel perspective on the conceptualization and estimation of effects and relations of interest.

Associations between a health-promoting lifestyle and quality of life among adults with beta-thalassemia major

“…Samples within each center shared a single or multiple random effect(s) that was/were different from center to center. The distribution of this random effect was univariate or multivariate normal with mean 0 and a variance (or covariance matrix) that could change from a random effect to another one [21]. Results were considered significant at the conventional level of p<0.05.…”

Influences of OSCE design on students’ diagnostic reasoning

“…random effects) are not of primary interest. However, they must be included in order to avoid overestimating the inter‐rater reliability and obtaining inappropriate SEs and CIs for (some of) the fixed effects …”

7 Deadly Sins in Educational Research

“…Related to this specific violation is the mistake of treating nondependent data as if they were independent (eg, treating data from 20 participants that are measured 3 times as if data are from 60 participants). 15 The violation of such statistical assumptions has the effect of artificially inflating type 1 errors (false positives), which leads to more statistically significant results than warranted. This outcome threatens the validity of inferences that can be made from statistically significant results and can also result in replication failure.…”