Comparametric HDR (High Dynamic Range) imaging for digital eye glass, wearable cameras, and sousveillance

Abstract

Wearable computing can be used to both extend the range of human perception, and to share sensory experiences with others. For this objective to be made practical, engineering considerations such as form factor, computational power, and power consumption are critical concerns. In this work, we consider the design of a low-power visual seeing aid, and how to implement computationally-intensive computational photography algorithms in a small form factor with low power consumption.We present realtime an FPGA-based HDR (High Dynamic Range) video processing and filtering by integrating tonal and spatial information obtained from multiple different exposures of the same subject matter. In this embodiment the system captures, in rapid succession, sets of three exposures, "dark", "medium", and "light", over and over again, e.g. "dark", "medium", "light", "dark", "medium", "light", and so on, at 60 frames per second. These exposures are used to determine an estimate of the photoquantity every 1/60th of a second (each time a frame comes in, an estimate goes out).This allows us to build a seeing aid that helps people see better in high contrast scenes, for example, while welding, or in outdoor scenes, or scenes where a bright light is shining directly into the eyes of the wearer. Our system is suitable for being built into eyeglasses or small camera-based, lifeglogging, or gesturesensing pendants, and other miniature wearable devices, with low-power and compact circuits that can be easily mounted on the body.

Wearables and Sur(over)-Veillance, Sous(under)-Veillance, Co(So)-Veillance, and MetaVeillance (Veillance of Veillance) for Health and Well-Being

“…Sousveillance, loosely defined as the vulnerable watching the powerful, coveillance (côtéveillance or soveillance or sociaveillance or mediaveillance), loosely defined as the side-to-side gaze between peers via social media and the like, and metaveillance, loosely defined as the sensing of sensors and the sensing of their capacity to sense, are relatively recent concepts emerging around the generalized idea of "veillance," which is defined as purposeful sensing (Janzen and Mann 2014;Reynolds 2011;Bakir 2010;Mann 2002;Ali and Mann 2013;Ganascia 2010;Vertegaal and Shell 2008;Michael and Michael 2012;Bakir 2009;Fernback 2013;Reilly 2015;Cardullo 2014;Ali et al 2013a;Nolan, Mann, and Wellman 2008;Ali et al 2013b. Weber 2012a, 2012bMortensen 2014;Quessada 2010;Manders 2013;Cammaerts 2015).…”

Real-time HDR video imaging on FPGA with compressed comparametric lookup tables

“…Response Function (CCRF) [4] , using compressed LUT and bilinear interpolation [1]. These steps have been automated using a program written in C.…”

“…Its size grows with the number of leaves in the quadtree used to represent the CCRF LUT. 4) Arithmetic circuit is implemented in earlier publication [1], mainly using the DSP slices available on the FPGA. It performs the bilinear interpolation to estimate the location of a pixel pair relative to the CCRF node it is in.…”

“…This work, extended upon the prior HDR architecture and quadtree compression [1] , replaces the weighted sum approach to the new quadtree-based compositing for high quality HDR video production [2], [3] . The proposed compositing circuit is generated by software written, with parameters given by the user as input.…”

Simplified floating-point units for high dynamic range image and video systems

“…Although human eyes can cover a luminance range of more than 10 orders of magnitude (5 orders simultaneously), conventional videos and images can only cover a range of about 2 [1]. Thus, most of the HDR coding standards use floating-point numbers for pixel representation.…”