Neural Photofit: Gaze-based Mental Image Reconstruction

Usable and Fast Interactive Mental Face Reconstruction

“…Recently, Strohm et al [40] proposed a method for gaze-based mental face reconstruction. The subjects observed a set of faces, and the so evoked gaze behaviour was used to predict which features of a face are relevant.…”

“…While their method is less invasive and expensive than fMRI and EEG-based methods, it requires prior knowledge about the target. They later proposed an iterative system [41], removing the need for prior knowledge but they still could only operate in a controlled environment and with high-precision gaze data. Instead, our system operates in the less constrained domain of real faces and only requires simple user feedback.…”

“…This allows us to keep the number of auxiliary images 𝑛 shown per iteration low. Consequently, we set the number of auxiliary faces to show in each iteration to 𝑛 = 6 for our MFRS, which has also been used in related work and proven not to overwhelm users [32,40]. Furthermore, we set the number of iterations used in our final MFRS to 20, which we determined through experimental testing.…”

“…More recently, to avoid the need for special-purpose sensing equipment that is either invasive (EEG) or expensive and not practical for everyday use (fMRI), other works have started to explore passive monitoring of human gaze for mental image reconstruction. While gaze is known to be a strong mediator of cognitive processes, including memory [4], prior works have only achieved limited reconstruction quality so far [34,35,40,41].…”