No abstract

We propose an end-to-end architecture for facial expression recognition. Our model learns an optimal tree topology for facial landmarks, whose traversal generates a sequence from which we obtain an embedding to feed a sequential learner. The proposed architecture incorporates two main streams, one focusing on landmark positions to learn the structure of the face, while the other focuses on patches around the landmarks to learn texture information. Each stream is followed by an attention mechanism and the outputs are fed to a two-stream fusion component to perform the final classification. We conduct extensive experiments on two largescale publicly available facial expression datasets, AffectNet and FER2013, to evaluate the efficacy of our approach. Our method outperforms other solutions in the area and sets new state-ofthe-art expression recognition rates on these datasets.

Prior work has shown that the order in which different components of the face are learned using a sequential learner can play an important role in the performance of facial expression recognition systems. We propose FaceTopoNet, an endto-end deep model for facial expression recognition, which is capable of learning an effective tree topology of the face. Our model then traverses the learned tree to generate a sequence, which is then used to form an embedding to feed a sequential learner. The devised model adopts one stream for learning structure and one stream for learning texture. The structure stream focuses on the positions of the facial landmarks, while the main focus of the texture stream is on the patches around the landmarks to learn textural information. We then fuse the outputs of the two streams by utilizing an effective attention-based fusion strategy. We perform extensive experiments on four largescale in-the-wild facial expression datasets -namely AffectNet, FER2013, ExpW, and RAF-DB -and one lab-controlled dataset (CK+) to evaluate our approach. FaceTopoNet achieves state-ofthe-art performance on three of the five datasets and obtains competitive results on the other two datasets. We also perform rigorous ablation and sensitivity experiments to evaluate the impact of different components and parameters in our model. Lastly, we perform robustness experiments and demonstrate that FaceTopoNet is more robust against occlusions in comparison to other leading methods in the area.Impact Statement-Systems that perform facial expression recognition (FER) have a wide variety of applications in realworld problems such as human-machine interaction, robotics, mental health management systems, to name but a few. Our proposed solution achieves better recognition rates in comparison to many other methods in the field of FER, allowing for development of more accurate systems. Moreover, our proposed method demonstrates more robustness against occlusions, which can be beneficial for in-the-wild scenarios. Lastly, our model can learn trees that are optimized, yet also generalize well across different datasets and perform considerably better than humandesigned face trees.