Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Network

Yang, Xin; Chen, Jingyu; Dang, Yuanjie; Luo, Hao; Tang, Yiming; Liao, Chunyuan; Chen, Peng; Cheng, Kwang-Ting

doi:10.1109/tits.2019.2955598

Cited by 61 publications

(47 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, many of the existing approaches consider only generating motion decisions/policies in 2D without utilizing the full maneuverability of UAVs. Most of these methods are based on deep reinforcement learning [ 136 , 137 , 138 , 139 , 140 , 141 , 142 , 143 ] and deep neural networks [ 144 , 145 , 146 , 147 , 148 , 149 , 150 , 151 ].…”

Section: Navigation Techniquesmentioning

confidence: 99%

Towards Fully Autonomous UAVs: A Survey

Elmokadem

Savkin

2021

Sensors

View full text Add to dashboard Cite

Unmanned Aerial Vehicles have undergone rapid developments in recent decades. This has made them very popular for various military and civilian applications allowing us to reach places that were previously hard to reach in addition to saving time and lives. A highly desirable direction when developing unmanned aerial vehicles is towards achieving fully autonomous missions and performing their dedicated tasks with minimum human interaction. Thus, this paper provides a survey of some of the recent developments in the field of unmanned aerial vehicles related to safe autonomous navigation, which is a very critical component in the whole system. A great part of this paper focus on advanced methods capable of producing three-dimensional avoidance maneuvers and safe trajectories. Research challenges related to unmanned aerial vehicle development are also highlighted.

show abstract

Section: Navigation Techniquesmentioning

confidence: 99%

Towards Fully Autonomous UAVs: A Survey

Elmokadem

Savkin

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…However, these areas demand drones capable of providing large amounts of data and analysing data in real-time. In addition, drones must detect obstacles using colour segmentation [38], deep learning and RGB cameras [39,40], depth information and deep learning [41], obstacle-free navigation [42], monocular depth prediction [43], and agile and precise flight [44].…”

Section: Related Workmentioning

confidence: 99%

Towards Autonomous Drone Racing without GPU Using an OAK-D Smart Camera

Rojas-Perez

Martínez-Carranza

2021

Sensors

View full text Add to dashboard Cite

Recent advances have shown for the first time that it is possible to beat a human with an autonomous drone in a drone race. However, this solution relies heavily on external sensors, specifically on the use of a motion capture system. Thus, a truly autonomous solution demands performing computationally intensive tasks such as gate detection, drone localisation, and state estimation. To this end, other solutions rely on specialised hardware such as graphics processing units (GPUs) whose onboard hardware versions are not as powerful as those available for desktop and server computers. An alternative is to combine specialised hardware with smart sensors capable of processing specific tasks on the chip, alleviating the need for the onboard processor to perform these computations. Motivated by this, we present the initial results of adapting a novel smart camera, known as the OpenCV AI Kit or OAK-D, as part of a solution for the ADR running entirely on board. This smart camera performs neural inference on the chip that does not use a GPU. It can also perform depth estimation with a stereo rig and run neural network models using images from a 4K colour camera as the input. Additionally, seeking to limit the payload to 200 g, we present a new 3D-printed design of the camera’s back case, reducing the original weight 40%, thus enabling the drone to carry it in tandem with a host onboard computer, the Intel Stick compute, where we run a controller based on gate detection. The latter is performed with a neural model running on an OAK-D at an operation frequency of 40 Hz, enabling the drone to fly at a speed of 2 m/s. We deem these initial results promising toward the development of a truly autonomous solution that will run intensive computational tasks fully on board.

show abstract

“…Park et al worked on devising a deep sensor fusion framework to result in more accurate consequences (32). Yang et al focused on predicting depth map and its prediction confidence for small drones considering effectiveness and efficiency, by means of probabilistic CNNs and the guidance of sparse depth estimation from a visual odometry (33). Guo et al trained their depth estimator using pixel-perfect synthetic images, which is convenient to obtain compared with the real depth data.…”

Section: Related Workmentioning

confidence: 99%

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Jia

Pei

Yang

et al. 2020

Transportation Research Record: Journal of the Transportation R

View full text Add to dashboard Cite

Depth information from still 2D images plays an important role in automated driving, driving safety, and robotics. Monocular depth estimation is considered as an ill-posed and inherently ambiguous problem in general, and a tight issue is how to obtain global information efficiently since pure convolutional neural networks (CNNs) merely extract the local information. To end that, some previous works utilized conditional random fields (CRFs) to obtain the global information, but it is notoriously difficult to optimize. In this paper, a novel hybrid neural network is proposed to solve that, and concurrently a dense depth map is predicted from the monocular still image. Specifically: first, the deep residual network is utilized to obtain multi-scale local information and then feature correlation (FCL) blocks are used to correlate these features. Finally, the feature selection attention-based mechanism is adopted to fuse the multi-layer features, and the multi-layer recurrent neural networks (RNNs) are utilized with bidirectional long short-term memory (Bi-LSTM) unit as the output layer. Furthermore, a novel logarithm exponential average error (LEAE) is proposed to overcome over-weighted problem. The multi-scale feature correlation network (MFCN) is evaluated on large-scale KITTI benchmarks (LKT), which is a subset of KITTI raw dataset, and NYU depth v2. The experiments indicate that the proposed unified network outperforms existing methods. This method also updates the state-of-the-art performance on LKT datasets. Importantly, the depth estimation method can be widely used for collision risk assessment and avoidance in driving assistance systems or automated pilot systems to achieve safety in a more economical and convenient way.

show abstract

Fast Depth Prediction and Obstacle Avoidance on a Monocular Drone Using Probabilistic Convolutional Neural Network

Cited by 61 publications

References 24 publications

Towards Fully Autonomous UAVs: A Survey

Towards Fully Autonomous UAVs: A Survey

Towards Autonomous Drone Racing without GPU Using an OAK-D Smart Camera

Novel Hybrid Neural Network for Dense Depth Estimation using On-Board Monocular Images

Contact Info

Product

Resources

About