EyelashNet

Abstract

Eyelashes play a crucial part in the human facial structure and largely affect the facial attractiveness in modern cosmetic design. However, the appearance and structure of eyelashes can easily induce severe artifacts in high-fidelity multi-view 3D face reconstruction. Unfortunately it is highly challenging to remove eyelashes from portrait images using both traditional and learning-based matting methods due to the delicate nature of eyelashes and the lack of eyelash matting dataset. To this end, we present EyelashNet, the first eyelash matting dataset which contains 5,400 high-quality eyelash matting data captured from real world and 5,272 virtual eyelash matting data created by rendering avatars. Our work consists of a capture stage and an inference stage to automatically capture and annotate eyelashes instead of tedious manual efforts. The capture is based on a specifically-designed fluorescent labeling system. By coloring the eyelashes with a safe and invisible fluorescent substance, our system takes paired photos with colored and normal eyelashes by turning the equipped ultraviolet (UVA) flash on and off. We further correct the alignment between each pair of photos and use a novel alpha matte inference network to extract the eyelash alpha matte. As there is no prior eyelash dataset, we propose a progressive training strategy that progressively fuses captured eyelash data with virtual eyelash data to learn the latent semantics of real eyelashes. As a result, our method can accurately extract eyelash alpha mattes from fuzzy and self-shadow regions such as pupils, which is almost impossible by manual annotations. To validate the advantage of EyelashNet, we present a baseline method based on deep learning that achieves state-of-the-art eyelash matting performance with RGB portrait images as input. We also demonstrate that our work can largely benefit important real applications including high-fidelity personalized avatar and cosmetic design.

Effective Eyebrow Matting with Domain Adaptation

“…6 presents the qualitative comparisons. Our results are visually better than those from Li and Lu [LL20], Xiao et al [XZZ*21] and the baseline network on the Internet images (from row 1 to 4, while the results (from rows 5 to 6) of our method on the synthetic images are visually acceptable compared to Li and Lu [LL20], Xiao et al [XZZ*21] and the baseline network. The results in both Table 3 and Fig.…”

“…We train the baseline network and DAM‐Net on the proposed eyebrow matting dataset. Then we compare against the method of Li and Lu [LL20] (trained on the Adobe Image Matting dataset [XPCH17]), the progressive training method of Xiao et al [XZZ*21] (trained on the eyebrow matting dataset), the state‐of‐the‐art on eyelash matting, and the baseline network, respectively. Specially, we provide trimaps for Li and Lu [LL20] for a fair comparison.…”

“…In addition, we compare the inference speed to previous methods. For a single 512*512 resolution input, GCA [LL20], Xiao et al [XZZ*21], baseline, Sun et al [STT21], and our inference times are 93.3 ms, 90.6 ms, 91.3 ms, 10.5 ms, and 91.7 ms, respectively.…”

“…Because the eyebrow is grown on the skin, it is impossible to build eyebrow matting datasets using the aforementioned existing methods [RRW*09,SB96,AAPS16,DJSXN19]. Likewise, Xiao et al [XZZ*21] may easily induce noises to the skin when coloring the eyebrows as addressed before. Moreover, it is tedious and time‐consuming to manually annotate detailed eyebrows compared with other objects of interest such as humans [CGX*18].…”

“…That is, there exists a domain gap between the virtual data and real data, making the model trained on synthetic data perform poorly on real data. The domain difference can be alleviated by a progressive training strategy [XZZ*21]. However, such a method requires an accurate eyelash mask as priors, which is difficult and labor‐intensive to obtain.…”