Scoot: A Perceptual Metric for Facial Sketches

Abstract

Human visual system has the strong ability to quick assess the perceptual similarity between two facial sketches. However, existing two widely-used facial sketch metrics, e.g., FSIM and SSIM fail to address this perceptual similarity in this field. Recent study in facial modeling area has verified that the inclusion of both structure and texture has a significant positive benefit for face sketch synthesis (FSS). But which statistics are more important, and are helpful for their success? In this paper, we design a perceptual metric, called Structure Co-Occurrence Texture (Scoot), which simultaneously considers the block-level spatial structure and co-occurrence texture statistics. To test the quality of metrics, we propose three novel meta-measures based on various reliable properties. Extensive experiments demonstrate that our Scoot metric exceeds the performance of prior work. Besides, we built the first large scale (152k judgments) human-perception-based sketch database that can evaluate how well a metric is consistent with human perception. Our results suggest that "spatial structure" and "co-occurrence texture" are two generally applicable perceptual features in face sketch synthesis.

Fast eyes detection in thermal images

“…As already mentioned, this method can provide initial training exemplars for the network training or in the inference mode it can operate as one of the member classifiers in an ensemble. It would be also interesting to evaluate performance of other feature descriptors, e.g co-occurrence texture and spatial structure from Scoot facial sketch metric presented in [13], that could serve as a replacement of the SIFT descriptor. Other research direction would be to use the hierarchical dictionaries proposed by Fulkerson in [14] to further improve robustness and detection speed.…”

Beyond Domain Gap: Exploiting Subjectivity in Sketch-Based Person Retrieval

“…We keep artists inaccessible to the photo during drawing (denoted by dashed lines) to guarantee their perspective descriptions, while PKU-Sketch does not enforce this. b: intra-style similarities are larger than intra-id similarities in terms of perceptual metrics SSIM [42] and Scoot [9]. c: indicates a sample that we obtain inconsistent descriptions from different witnesses, and each artist has varied sketching styles.…”

Bridging Unpaired Facial Photos and Sketches by Line-Drawings

“…To alleviate the effect of facial identity, we randomly extract about 10,000 patches of size 256 × 256 from real and generated sketches, respectively, for calculating FIDs. In addition, we use the Scoot metric [28] to measure the similarity between real and synthesised sketches. We also use the Fisherface method for face sketch recognition [7].…”