Long-range night/day human identification using active-SWIR imaging

Although Short-Wave Infrared (SWIR) sensors have advantages in terms of robustness in bad weather and low-light conditions, the SWIR images have not been well studied for automated object detection and tracking systems. The majority of previous multi-object tracking studies have focused on pedestrian tracking in visible-spectrum images, but tracking different types of vehicles is also important in city-surveillance scenarios. In addition, the previous studies were based on high-computing-power environments such as GPU workstations or servers, but edge computing should be considered to reduce network bandwidth usage and privacy concerns in city-surveillance scenarios. In this paper, we propose a fast and effective multi-object tracking method, called Multi-Class Distance-based Tracking (MCDTrack), on SWIR images of city-surveillance scenarios in a low-power and low-computation edge-computing environment. Eight-bit integer quantized object detection models are used, and simple distance and IoU-based similarity scores are employed to realize effective multi-object tracking in an edge-computing environment. Our MCDTrack is not only superior to previous multi-object tracking methods but also shows high tracking accuracy of 77.5% MOTA and 80.2% IDF1 although the object detection and tracking are performed on the edge-computing device. Our study results indicate that a robust city-surveillance solution can be developed based on the edge-computing environment and low-frame-rate SWIR images.

Section: Short-wave Infrared Sensorsmentioning

confidence: 99%

Multi-Object Tracking on SWIR Images for City Surveillance in an Edge-Computing Environment

Park¹,

Hong²,

Shim³

et al. 2023

“…Three gait recognition algorithms on the created SWIR dataset are evaluated, with special emphasis on the design of algorithms for segmentation and recognition of individuals in the SWIR domain due to significant operational advantages in contrast to color images. The authors of [ 25 ] presented an active SWIR imaging system for the detection, tracking, and identification of human targets at ranges of several hundred meters. A system was developed with the possibility of generating recognizable facial images, day or nighttime, with recognition capabilities achieved at distances up to 350 m. Paper [ 26 ] deals with pedestrian detection in SWIR images under different visibility conditions—clear sky, haze, and fog.…”

Section: Related Work In Swir Object Detectionmentioning

confidence: 99%

Deep Learning Based SWIR Object Detection in Long-Range Surveillance Systems: An Automated Cross-Spectral Approach

Pavlović

Milanović

Stanković

et al. 2022

SWIR imaging bears considerable advantages over visible-light (color) and thermal images in certain challenging propagation conditions. Thus, the SWIR imaging channel is frequently used in multi-spectral imaging systems (MSIS) for long-range surveillance in combination with color and thermal imaging to improve the probability of correct operation in various day, night and climate conditions. Integration of deep-learning (DL)-based real-time object detection in MSIS enables an increase in efficient utilization for complex long-range surveillance solutions such as border or critical assets control. Unfortunately, a lack of datasets for DL-based object detection models training for the SWIR channel limits their performance. To overcome this, by using the MSIS setting we propose a new cross-spectral automatic data annotation methodology for SWIR channel training dataset creation, in which the visible-light channel provides a source for detecting object types and bounding boxes which are then transformed to the SWIR channel. A mathematical image transformation that overcomes differences between the SWIR and color channel and their image distortion effects for various magnifications are explained in detail. With the proposed cross-spectral methodology, the goal of the paper is to improve object detection in SWIR images captured in challenging outdoor scenes. Experimental tests for two object types (cars and persons) using a state-of-the-art YOLOX model demonstrate that retraining with the proposed automatic cross-spectrally created SWIR image dataset significantly improves average detection precision. We achieved excellent improvements in detection performance in various variants of the YOLOX model (nano, tiny and x).

“…They are also used for visualization through scattering media like mist, fog, or smoke [ 6 , 7 ]. At present, these techniques are commonly applied in diverse fields such as in biomedicine [ 8 ], surveillance, defense [ 9 ], food control [ 10 ], pests/crops monitoring [ 11 ], cultural heritage [ 12 ], machine vision systems [ 13 ], and more. Furthermore, active imaging in short-wave IR (SWIR) and mid-wave IR (MWIR) wavelengths would benefit from eye-safe (1.55-μm band) and low attenuation windows (1.55-μm, 2-μm, and 3–5-μm bands), being particularly suited for remote sensing applications [ 14 ].…”

Section: Introductionmentioning

confidence: 99%

Up-Conversion Sensing of 2D Spatially-Modulated Infrared Information-Carrying Beams with Si-Based Cameras

Torregrosa

Karamehmedović

Maestre

et al. 2020

Up-conversion sensing based on optical heterodyning of an IR (infrared) image with a local oscillator laser wave in a nonlinear optical sum-frequency mixing (SFM) process is a practical solution to circumvent some limitations of IR image sensors in terms of signal-to-noise ratio, speed, resolution, or cooling needs in some demanding applications. In this way, the spectral content of an IR image can become spectrally shifted to the visible/near infrared (VIS/NWIR) and then detected with silicon focal plane arrayed sensors (Si-FPA), such as CCD/CMOS (charge-coupled and complementary metal-oxide-semiconductor devices). This work is an extension of a previous study where we recently introduced this technique in the context of optical communications, in particular in FSOC (free-space optical communications). Herein, we present an image up-conversion system based on a 1064 nm Nd3+: YVO4 solid-state laser with a KTP (potassium titanyl phosphate) nonlinear crystal located intra-cavity where a laser beam at 1550 nm 2D spatially-modulated with a binary Quick Response (QR) code is mixed, giving an up-converted code image at 631 nm that is detected with an Si-based camera. The underlying technology allows for the extension of other IR spectral allocations, construction of compact receivers at low cost, and provides a natural way for increased protection against eavesdropping.