Purpose:
To determine if crowdsourced ratings of oculoplastic surgical outcomes provide reliable information compared to professional graders and oculoplastic experts.
Methods:
In this prospective psychometric evaluation, a scale for the rating of postoperative eyelid swelling was constructed using randomly selected images and topic experts. This scale was presented adjacent to 205 test images, including 10% duplicates. Graders were instructed to match the test image to the reference image it most closely resembles. Three sets of graders were solicited: crowdsourced lay people from Amazon Mechanical Turk marketplace, professional graders from the Doheny Image Reading Center (DIRC), and American Society of Ophthalmic Plastic and Reconstructive Surgery surgeons. Performance was assessed by classical correlational analysis and generalizability theory.
Results:
The correlation between scores on the first rating and the second rating for the 19 repeated occurrences was 0.60 for lay observers, 0.80 for DIRC graders and 0.84 for oculoplastic experts. In terms of inter-group rating reliability for all photos, the scores provided by lay observers were correlated with DIRC graders at a level of r = 0.88 and to experts at r = 0.79. The pictures themselves accounted for the greatest amount of variation among all groups. The amount of variation in the scores due to the rater was highest in the lay group at 25%, and was 20% and 21% for DIRC graders and experts, respectively.
Conclusions:
Crowdsourced observers are insufficiently precise to replicate the results of experts in grading postoperative eyelid swelling. DIRC graders performed similarly to experts and present a less resource-intensive option.
Background-Few studies explore factors influencing breast cancer screening and early detection behaviors among immigrant Iranian women residing in the United States.
Increased hemodynamic latency in the visual cortex predicted impaired cognitive function (p<0.05), holding constant demographic and cerebrovascular risk. Increased alcohol use was associated with reduced overall cognitive function (Full Scale IQ 2.8 pts, p<0.05), while cardiac disorders (Full Scale IQ 3.3 IQ pts; p<0.05), high cholesterol (Full Scale IQ 3.9 pts; p<0.05), and years of education (2 IQ pts/year; p<0.001) were associated with higher general cognitive ability. Increased hemodynamic latency was associated with reduced executive functioning (p<0.05) as well as deficits in verbal concept formation (p<0.05) and the ability to synthesize and analyze abstract visual information (p<0.01).Hemodynamic latency is associated with reduced cognitive ability across the lifespan, independently of other demographic and cerebrovascular risk factors.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.