The Urban Observatory: A Multi-Modal Imaging Platform for the Study of Dynamics in Complex Urban Systems

Abstract

We describe an “Urban Observatory” facility designed for the study of complex urban systems via persistent, synoptic, and granular imaging of dynamical processes in cities. An initial deployment of the facility has been demonstrated in New York City and consists of a suite of imaging systems—both broadband and hyperspectral—sensitive to wavelengths from the visible (∼400 nm) to the infrared (∼13 micron) operating at cadences of ∼0.01–30 Hz (characteristically ∼0.1 Hz). Much like an astronomical survey, the facility generates a large imaging catalog from which we have extracted observables (e.g., time-dependent brightnesses, spectra, temperatures, chemical species, etc.), collecting them in a parallel source catalog. We have demonstrated that, in addition to the urban science of cities as systems, these data are applicable to a myriad of domain-specific scientific inquiries related to urban functioning including energy consumption and end use, environmental impacts of cities, and patterns of life and public health. We show that an Urban Observatory facility of this type has the potential to improve both a city’s operations and the quality of life of its inhabitants.

Pixel-Wise Classification of High-Resolution Ground-Based Urban Hyperspectral Images with Convolutional Neural Networks

“…Using a Visible Near-Infrared (VNIR; ∼0.4-1.0 micron) single slit, scanning spectrograph with 848 spectral channels, deployed via the Urban Observatory (UO; [39,41,42]) in New York City, we obtained three side-facing images of complex urban scenes at high spatial and spectral resolution. With these images, we investigated the use, transferability, and limitations of 1-dimensional Convolutional Neural Networks (CNNs) for pixel-level classification and segmentation of ground-based, remote urban hyperspectral imaging.…”

“…Extensions of our basic 1-dimensional model will be the subject of future work, as will the geolocation of individual pixel contents (see [41]) through the inclusion of topographic LiDAR data, which can be included as an additional feature to leverage for pixel classification using CNNs. The addition of pixel geolocation would also allow for the projection of classification results onto thematic maps of urban environments that can connect this work with those of satellite remote imagery, providing complimentary classification at enhanced spatial and temporal resolution.…”

“…The HSI data used to train and test the performance of segmentation models in this work was acquired with the "Urban Observatory" (UO) facility in New York City (NYC). The UO is a multi-scale, multi-modal observational platform for the study of dynamical processes in cities [39][40][41]. In addition to broadband visible and infrared imaging capabilities, the UO has deployed both VNIR [42,43] and Long Wave Infrared (LWIR) [44] HSI cameras to continuously image large swaths of NYC with persistence and temporal granularity on the order of minutes.…”

District-scale surface temperatures generated from high-resolution longitudinal thermal infrared images

“…Urban ecosystems are the biggest, most dynamic, and most complicated man-made systems, with millions of people interacting and hundreds of governing agencies 1 , 2 . Urban modernization has encouraged a significant portion of the global population to move to urban regions.…”

Sensor Technology and Intelligent Systems in Anorexia Nervosa: Providing Smarter Healthcare Delivery Systems

“…Along with these sensors, we have satellite data, location data, and a geographic information system (GIS) to, for example, map the surrounding built environment of citizens. Although all these data from ambient sensors and geolocation apparently seem disparate, we can use computer vision, signal processing, simulation, machine learning and AI, to analyze a built environment, identify patterns like individual and group human behavior within cities, and predict future patterns [ 147 ].…”