Crowdsourced Assessment of Aesthetic Outcomes after Breast Reconstruction

Abstract

Background: Evaluating the aesthetic success of breast reconstruction can be difficult. Patients, surgeons, and the general population may differ in what constitutes a successful outcome. Recently, crowdsourcing has emerged to accumulate and analyze data on a massive scale. The authors propose that crowdsourcing can be a useful tool to reliably rate aesthetic outcomes of breast reconstruction. Methods: One hundred one deidentified photographs of patients at various stages of breast reconstruction were gathered. Assessment tools included a five-point Likert scale and the transverse rectus abdominis myocutaneous (TRAM) visual assessment scale. Anonymous crowd workers and a group of expert reconstructive surgeons rated an identical set of photographs on the Likert scale. Crowd workers also rated the set of photographs on the TRAM scale. Results: The authors obtained 901 anonymous, layperson evaluations on both Likert and TRAM scales. Crowdsourced assessment data collection took 28.6 hours. Expert assessment took 15 months. Expert and crowdsourced scores were equivalent on the Likert scale (overall interrater reliability, κ = 0.99; 95 percent CI, 0.98 to 0.99). Intrarater reliability among each subcomponent was highly reproducible for the crowd (r = 0.98; 95 percent CI, 0.97 to 0.99) and experts (r = 0.82; 95 percent CI, 0.77 to 0.87). Breast contour and positioning were most predictive of overall aesthetic result. Skin patch and scar were least predictive of overall aesthetic appearance. Conclusions: Aesthetic outcomes rated by crowds were reliable and correlated closely with those by expert surgeons. Crowdsourcing can be a rapid, reliable, and valid way to assess aesthetic outcomes in the breast reconstruction patient.

Visual Perception of Breast Free Flap Size Is Influenced by Radiation Changes of Surrounding Tissue

“…The crowdfunding approach theoretically draws a diverse crowd that minimizes individual biases and confounders. 35 Our findings are by no means definitive assertions of public preference, but it is the largest study of layperson opinion on radiation's effect on autologous breast reconstruction aesthetics. However, utilization of crowdfunding methods inevitably invites the possibility of response and sampling bias, especially in situations that involve monetary rewards.…”

“…It is a powerful approach for gather massive amounts of data that generates large sample sizes with high statistical power. The crowdfunding approach theoretically draws a diverse crowd that minimizes individual biases and confounders [28]. Our findings are by no means definitive assertions of public preference, but it is the largest study of layperson opinion on radiation's effect on autologous breast reconstruction aesthetics.…”

Crowdsourcing in Plastic & Reconstructive Surgery: A Systematic Review with Reporting Recommendations

“…Forty-five articles were ultimately included for data extraction (Table 1). 13–57 Proportionate rater agreement was 91.7% for abstract screening and 92.5% for full-text screening. The Cohen kappa coefficient for interrater reliability was 0.82 for abstract screening and 0.81 for full-text screening, demonstrating sufficient interrater reliability.…”

The Use of Validated Visual Scales in Plastic Surgery: Where Are We Now?

“…In multiple fields of surgery, we are beginning to see the initial return on investment of methods such as crowdsourcing and machine learning. 54,55 Through these methods, the overall cost and timeliness of visual analysis has been reduced dramatically. Furthermore, these methods can be applied to any visual medium, including both performance (video) and outcomes data (two-dimensional photography).…”