Set Operation Aided Network for Action Units Detection

Preface: Celebrating the 100th anniversary of chemical sciences in Nanjing University

“…These supervised approaches use artificially defined multiple local facial regions to learn the region-specific AU representations, which is not the case in CLP. CLP is also comparable with SO-Net [40] on the two AU datasets. Notably, CLP outperforms the supervised ResNet-34 pre-trained with ImageNet with consistent improvements on the three AU datasets (+3.2% on BP4D, +6.1% on DISFA, +1.3% on GFT), indicating the superiority of the CLPlearned AU representations.…”

“…1) Comparison with supervised AU detection methods: We compare CLP with ROI [34], EAC-Net [7], DSIN [8], DSIN [8], AU-RCNN [35], ATF [38], IdenNet [39], JAA-Net [9], SRERL [10], SEV-Net [11], SO-Net [40], HMP-PS [12], MAL [45] and the transformer-based [64] AU detection method proposed by Jacob et al [37]. Among the state-ofthe-art supervised AU detection methods, ROI [34], JAA-Net [9], DSIN [8], AU-RCNN [35], ARL [36], SRERL [10] extract the regional facial features with manually selected facial landmarks and learn the region-specific AU features with exclusive CNN branches.…”

“…Tu et al [39] used the AU-and identity-annotated datasets that contain numerous subjects to extract identity-dependent image features, and performed the identity-aware AU Detection in the test phase. SO-Net [40] augmented the training data by adding set operations to both the feature and label spaces. The features are thus generalizable to unseen subjects.…”