Reading between the pixels: Photographic steganography for camera display messaging

Abstract

Abstract. We exploit human color metamers to send light-modulated messages less visible to the human eye, but recoverable by cameras. These messages are a key component to camera-display messaging, such as handheld smartphones capturing information from electronic signage. Each color pixel in the display image is modified by a particular color gradient vector. The challenge is to find the color gradient that maximizes camera response, while minimizing human response. The mismatch in human spectral and camera sensitivity curves creates an opportunity for hidden messaging. Our approach does not require knowledge of these sensitivity curves, instead we employ a data-driven method. We learn an ellipsoidal partitioning of the six-dimensional space of colors and color gradients. This partitioning creates metamer sets defined by the base color at the display pixel and the color gradient direction for message encoding. We sample from the resulting metamer sets to find color steps for each base color to embed a binary message into an arbitrary image with reduced visible artifacts. Unlike previous methods that rely on visually obtrusive intensity modulation, we embed with color so that the message is more hidden. Ordinary displays and cameras are used without the need for expensive LEDs or high speed devices. The primary contribution of this work is a framework to map the pixels in an arbitrary image to a metamer pair for steganographic photo messaging.

Some practical constraints and solutions for optical camera communication

“…From the perspective of visual quality, humans prefer to enjoy a normal full-screen viewing experience, while acquiring extra information from the screen to camera channel simultaneously. This kind of CSC is usually called covert CSC or hidden CSC, which is subject to a visual constraint for human eyes [32,58,59,[102][103][104][105][106][107][108]. Unlike the traditional VLC system providing proper illumination while transmitting data simultaneously, a covert CSC system focuses more on the transmission while providing the quality display rather than illumination.…”

“…Nguyen et al [32] adopted content-adaptive encoding techniques to achieve a high-throughput and unnoticeable flicker covert CSC system. Wengrowski et al [105] modified each colour pixel by shifting the base colour to a specific colour gradient which was then used to map pixels in an arbitrary image. Sato et al [106] proposed a blue colour difference modulation scheme and verified its feasibility experimentally.…”