Procrustes-based geometric morphometrics on MRI images: An example of inter-operator bias in 3D landmarks and its impact on big datasets

Abstract

Using 3D anatomical landmarks from adult human head MRIs, we assessed the magnitude of inter-operator differences in Procrustes-based geometric morphometric analyses. An in depth analysis of both absolute and relative error was performed in a subsample of individuals with replicated digitization by three different operators. The effect of inter-operator differences was also explored in a large sample of more than 900 individuals. Although absolute error was not unusual for MRI measurements, including bone landmarks, shape was particularly affected by differences among operators, with up to more than 30% of sample variation accounted for by this type of error. The magnitude of the bias was such that it dominated the main pattern of bone and total (all landmarks included) shape variation, largely surpassing the effect of sex differences between hundreds of men and women. In contrast, however, we found higher reproducibility in soft-tissue nasal landmarks, despite relatively larger errors in estimates of nasal size. Our study exemplifies the assessment of measurement error using geometric morphometrics on landmarks from MRIs and stresses the importance of relating it to total sample variance within the specific methodological framework being used. In summary, precise landmarks may not necessarily imply negligible errors, especially in shape data; indeed, size and shape may be differentially impacted by measurement error and different types of landmarks may have relatively larger or smaller errors. Importantly, and consistently with other recent studies using geometric morphometrics on digital images (which, however, were not specific to MRI data), this study showed that inter-operator biases can be a major source of error in the analysis of large samples, as those that are becoming increasingly common in the 'era of big data'.

Measurement error in μCT‐based three‐dimensional geometric morphometrics introduced by surface generation and landmark data acquisition

“…; Daboul et al. ). Nevertheless, the magnitude of observer error often has been considered small or negligible compared with true biological variability (Richtsmeier et al.…”

“…The preservation and preparation of specimens can induce artefactual variance by altering the natural form of the structures of interest (Lee, 1982 [linear measurements]; Bonneau et al 2012). The variability within repeated measurements performed by the same observer and the variability between different observers can also contribute significantly to measurement error (Ross & Williams, 2008;Robinson & Terhune, 2017; reports of relatively large observer error without tests for statistical significance: Curth et al 2017;Fruciano et al 2017;Daboul et al 2018). Nevertheless, the magnitude of observer error often has been considered small or negligible compared with true biological variability (Richtsmeier et al 1995;O'Higgins & Jones, 1998;Lockwood et al 2002;Pujol et al 2014;Barbeito-Andr es et al 2016).…”

Peer Review #1 of "Landmark-based homologous multi-point warping approach to 3D facial recognition using multiple datasets (v0.1)"

“…The centroid size was not log-transformed, as this transformation made no appreciable difference in the results. Despite their significance, the sizes of the effects being tested are small or similar in relative terms, and we must therefore interpret these effects with caution (Daboul et al 2018). The MANOVA test based on Wilks' Lambda showed a significant result for both slope and intercept (p < 0.000).…”

“Nose-metrics” of wild southern elephant seal (Mirounga leonina) males using image analysis and geometric morphometrics

“…There might be difficulties to obtain reliable landmarks on the proboscis and also measurement error is generally more pronounced in shape data (e.g., Viscosi & Cardini, , and Cardini, ) than in traditional measurements of size. Thus, it was crucial to accurately assess the main sources of error, that could artificially inflate variation, reduce statistical power, and introduce biases (Arnqvist & Maartensson, ; Fruciano, ; Daboul et al., ).…”

“…There might be difficulties to obtain reliable landmarks on the proboscis and also measurement error is generally more pronounced in shape data (e.g., Viscosi &Cardini, 2011, andCardini, 2014) than in traditional measurements of size. Thus, it was crucial to accurately assess the main sources of error, that could artificially inflate variation, reduce statistical power, and introduce biases (Arnqvist & Maartensson, 1998;Fruciano, 2016;Daboul et al, 2018). F I G U R E 3 Scatterplot of between group PC1 (BG-PC1) and residual non-between group PC1 (res-PC1) of SL shape (percentages of total variance accounted for by each axis are in parentheses).…”