Discover the Effective Strategy for Face Recognition Model Compression by Improved Knowledge Distillation

Wang, Mengjiao; Liu, Rujie; Abe, Narishige; Uchida, Hidetsugu; Matsunami, Tomoaki; Yamada, Shigefumi

doi:10.1109/icip.2018.8451808

Cited by 4 publications

(9 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In last few years, a variety of knowledge distillation methods have been widely used for model compression in different visual recognition applications. Specifically, most of the knowledge distillation methods were previously developed for image classification (Li and Hoiem, 2017;Peng et al, 2019b;Bagherinezhad et al, 2018;Chen et al, 2018a;Wang et al, 2019b;Mukherjee et al, 2019;Zhu et al, 2019) and then extended to other visual recognition applications, including face recognition (Luo et al, 2016;Kong et al, 2019;Yan et al, 2019;Ge et al, 2018;Wang et al, 2018bWang et al, , 2019cDuong et al, 2019;Wu et al, 2020;Wang et al, 2017), action recognition (Hao and Zhang, 2019;Thoker and Gall, 2019;Luo et al, 2018;Garcia et al, 2018;Wu et al, 2019b;Zhang et al, 2020), object detection Hong and Yu, 2019;Shmelkov et al, 2017;Wei et al, 2018;Wang et al, 2019d), lane detection (Hou et al, 2019), image or video segmentation (He et al, 2019;Liu et al, 2019g;Mullapudi et al, 2019;Siam et al, 2019;Dou et al, 2020), video classification (Bhardwaj et al, 2019;Zhang and Peng, 2018), pedestrian detection (Shen et al, 2016), facial landmark detection (Dong and Yang, 2019), person re-identification (Wu et al, 2019a)…”

Section: Kd In Visual Recognitionmentioning

confidence: 99%

“…The existing KD-based face recognition methods can not only be easily deployed, but also improve the classification accuracy (Luo et al, 2016;Kong et al, 2019;Yan et al, 2019;Ge et al, 2018;Wang et al, 2018bWang et al, , 2019cDuong et al, 2019;Wang et al, 2017). First of all, these methods focus on the lightweight face recognition with very satisfactory accuracy (Luo et al, 2016;Wang et al, 2018bWang et al, , 2019cDuong et al, 2019). In (Luo et al, 2016), the knowledge from the chosen informative neurons of top hint layer of the teacher network is transferred into the student network.…”

Section: Kd In Visual Recognitionmentioning

confidence: 99%

See 1 more Smart Citation

Knowledge Distillation: A Survey

Gou,

Yu,

Maybank

et al. 2020

Preprint

View full text Add to dashboard Cite

In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, distillation algorithms and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded.

show abstract

Section: Kd In Visual Recognitionmentioning

confidence: 99%

Section: Kd In Visual Recognitionmentioning

confidence: 99%

Knowledge Distillation: A Survey

Gou,

Yu,

Maybank

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…For the later, we use L2 loss for features or normalized features to make a constraint. For feature normalizing, the features are normalized as x/∥x∥ to make them have the same scale, and with [47], the optimization of normalized feature L2 loss LfeatNorm becomes consistent with cosine similarity compared with the feature L2 loss Lfeat, which can make an performance improvement. In all evaluations, only the identity features extracted from low-quality images are used to match.…”

Section: Experiments and Resultsmentioning

confidence: 99%

Boosting Depth-Based Face Recognition from a Quality Perspective

Zhenguo

Gui

Feng

et al. 2019

Sensors

View full text Add to dashboard Cite

Face recognition using depth data has attracted increasing attention from both academia and industry in the past five years. Previous works show a huge performance gap between high-quality and low-quality depth data. Due to the lack of databases and reasonable evaluations on data quality, very few researchers have focused on boosting depth-based face recognition by enhancing data quality or feature representation. In the paper, we carefully collect a new database including high-quality 3D shapes, low-quality depth images and the corresponding color images of the faces of 902 subjects, which have long been missing in the area. With the database, we make a standard evaluation protocol and propose three strategies to train low-quality depth-based face recognition models with the help of high-quality depth data. Our training strategies could serve as baselines for future research, and their feasibility of boosting low-quality depth-based face recognition is validated by extensive experiments.

show abstract

“…In this context, distillation and pruning techniques, exploring the deep model parameters redundancy to remove uncritical information, can be adopted to perform model compression and acceleration, without significantly reducing the recognition performance [103, 104]. This may facilitate the deployment of deep face models on resource‐constrained embedded devices such as mobile devices [105]. Face presentation attack detection for face recognition security – Despite the significant progress in face recognition performance, the widespread adoption of face recognition solutions raises new security concerns.…”

Section: Challenges and Future Research Directionsmentioning

confidence: 99%

Section: Challenges and Future Research Directionsmentioning

confidence: 99%

Face recognition: a novel multi‐level taxonomy based survey

2019

View full text Add to dashboard Cite

In a world where security issues have been gaining growing importance, face recognition systems have attracted increasing attention in multiple application areas, ranging from forensics and surveillance to commerce and entertainment. To help understanding the landscape and abstraction levels relevant for face recognition systems, face recognition taxonomies allow a deeper dissection and comparison of the existing solutions. This paper proposes a new, more encompassing and richer multi-level face recognition taxonomy, facilitating the organization and categorization of available and emerging face recognition solutions; this taxonomy may also guide researchers in the development of more efficient face recognition solutions. The proposed multi-level taxonomy considers levels related to the face structure, feature support and feature extraction approach. Following the proposed taxonomy, a comprehensive survey of representative face recognition solutions is presented. The paper concludes with a discussion on current algorithmic and application related challenges which may define future research directions for face recognition.

show abstract

Discover the Effective Strategy for Face Recognition Model Compression by Improved Knowledge Distillation

Cited by 4 publications

References 5 publications

Knowledge Distillation: A Survey

Knowledge Distillation: A Survey

Boosting Depth-Based Face Recognition from a Quality Perspective

Face recognition: a novel multi‐level taxonomy based survey

Contact Info

Product

Resources

About