A virtual environment for evaluation of computer vision algorithms under general airborne camera imperfections

Abstract

This paper describes a camera simulation framework for validating machine vision algorithms under general airborne camera imperfections. Lens distortion, image delay, rolling shutter, motion blur, interlacing, vignetting, image noise, and light level are modelled. This is the first simulation that considers all temporal distortions jointly, along with static lens distortions in an online manner. Several innovations are proposed including a motion tracking system allowing the camera to follow the flight log with eligible derivatives. A reverse pipeline, relating each pixel in the output image to pixels in the ideal input image, is developed. It is shown that the inverse lens distortion model and the inverse temporal distortion models are decoupled in this way. A short-time pixel displacement model is proposed to solve for temporal distortions (i.e. delay, rolling shutter, motion blur, and interlacing). Evaluation is done by several means including regenerating an airborne dataset, regenerating the camera path on a calibration pattern, and evaluating the ability of the time displacement model to predict other frames. Qualitative evaluations are also made.

Accommodating the multi-state constraint Kalman filter for visual-inertial navigation in a moving and stationary flight

“…For studying the performance of a navigation algorithm in different speeds, a controlled simulation environment is necessary instead of real repeated tests, so that the errors in repeating the maneuvers, sensor data, and the changing lighting conditions do not outperform the effects of the changes in the speed. A high fidelity simulation environment has been developed, evaluated, 35 and used in this work, 1 based on the path followed by the FSR2015 ASL dataset (lawn-mower sequence) (Figure 1). The overall path takes 20 min of flight in which about 13.5 km is traveled.…”

“…To this end, all of the feature's measurements (available in the augmentation window) are used to estimate its position via Gauss-Newton minimization. 5 Subsequently, the residuals (for j = kÀN, …, kÀ1) are computed (35) where πjjkÀ1 and G pf i are the error of the current estimate for the jth pose and the error in the feature position, respectively. The matrices H π ij and H fij are the corresponding Jacobians, evaluated using πjjkÀ1 and G pf i .…”

Maritime navigational assistance by visual augmentation

“…The rest of the camera parameters represents the conversion from sensor plane to pixel units. Note that more sophisticated models may be developed by taking into consideration mechanical properties of the lens and typical hardware components (Mahmoudi et al, 2021).…”

Predictably Unpredictable

“…Scagnetto et al, (2021) looked into the context-awareness and the end-users of the system. Mahmoudi et al, (2021) studied the challenges of the general airborne camera by modelling the uncertainty and imperfection of the different components. Lens distortion, image delay, rolling shutter, motion blur, interlacing, vignetting, image noise, and light level are modelled.…”