Using Artificial Intelligence to Measure Facial Expression following Facial Reanimation Surgery

Abstract

Summary: Social interactions are largely dependent on the interpretation of information conveyed through facial expressions. Although facial reanimation seeks restoration of the facial expression of emotion, outcome measures have not addressed this directly. This study evaluates the use of a machine learning technology to directly measure facial expression before and after facial reanimation surgery. Fifteen study subjects with facial palsy were evaluated both before and after undergoing cross-facial nerve grafting and free gracilis muscle transfer. Eight healthy volunteers were assessed for control comparison. Video footage of subjects with their face in repose and with a posed, closed-lip smile was obtained. The video data were then analyzed using the Noldus FaceReader software application to measure the relative proportions of seven cardinal facial expressions detected within each clip. The facial expression recognition application detected a far greater happy signal in postoperative (42 percent) versus preoperative (13 percent) smile videos (p < 0.0001), compared to 53 percent in videos of control faces smiling. This increase in postoperative happy signal was achieved in exchange for a reduction in the sad signal (15 percent to 9 percent; p = 0.092) and the neutral signal (57 percent to 37 percent; p = 0.0012). For video clips of patients in repose, no significant difference in happy emotion was detected between preoperative (3.1 percent) and postoperative (1.4 percent) states (p = 0.5). This study provides the first proof of concept for the use of a machine learning software application to objectively quantify facial expression before and after surgical reanimation. CLINICAL QUESTION/LEVEL OF EVIDENCE: Diagnostic, IV.

Seminal Studies in Facial Reanimation Surgery: Consensus and Controversies in the Top 50 Most Cited Articles

“…78,79 Alternatively, our group has been working on using artificial intelligence to quantify smile before and after facial reanimation surgery by comparing the subject against thousands of control subjects. 80 This, in our view, is particularly attractive as, by obtaining objectively derived, ordinal data before and after surgery we can quantify the restoration of a natural smile; that ethereal phenomenon that defies reduction to a sum of vectors and excursions. This is, after all, the final arbiter of surgical success.…”

Artificial intelligence in plastic surgery: current developments and future perspectives

“…After rhinoplasty, the facial attractiveness of patients is increased [38] , and perceived age relative to the patient's actual age is decreased [39] , meaning that rhinoplasty reverses signs of facial aging. Using CNN to evaluate the facial attractiveness of treated cleft patients, the AI evaluation of cleft patients was comparable to human ratings [40] .…”

Review on Facial-Recognition-Based Applications in Disease Diagnosis

“…Social lives of individuals with AD, low vision, and autism spectrum disorder patients’ responses would be improved [ 89 ]. Another model to evaluate facial reanimation after facial surgery has also been developed [ 90 ].…”