Towards Drone-sourced Live Video Analytics for the Construction Industry

George, Shilpa; Wang, Junjue; Bala, Mihir; Eiszler, Thomas; Pillai, Padmanabhan; Satyanarayanan, Mahadev

doi:10.1145/3301293.3302365

Cited by 13 publications

(7 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On one hand, computer vision accuracy has been improved by deep learning at the cost of increased compute demand. On the other hand, low prices of high-definition network-connected cameras make them widely deployed in traffic monitoring [27], video analytics in retail stores [12], and inspection of warehouses or remote industrial sites [38]. Thus, the camera operators must scale out the compute costs of analyzing ever more camera feeds [2,6,21].…”

Section: Video Streaming For Video Analyticsmentioning

confidence: 99%

Server-Driven Video Streaming for Deep Learning Inference

Pervaiz

Yuan

et al. 2020

Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies,

154

View full text Add to dashboard Cite

Video streaming is crucial for AI applications that gather videos from sources to servers for inference by deep neural nets (DNNs). Unlike traditional video streaming that optimizes visual quality, this new type of video streaming permits aggressive compression/pruning of pixels not relevant to achieving high DNN inference accuracy. However, much of this potential is left unrealized, because current video streaming protocols are driven by the video source (camera) where the compute is rather limited. We advocate that the video streaming protocol should be driven by real-time feedback from the server-side DNN. Our insight is twofold: (1) server-side DNN has more context about the pixels that maximize its inference accuracy; and (2) the DNN's output contains rich information useful to guide video streaming. We present DDS (DNN-Driven Streaming), a concrete design of this approach. DDS continuously sends a low-quality video stream to the server; the server runs the DNN to determine where to re-send with higher quality to increase the inference accuracy. We find that compared to several recent baselines on multiple video genres and vision tasks, DDS maintains higher accuracy while reducing bandwidth usage by upto 59% or improves accuracy by upto 9% with no additional bandwidth usage.

show abstract

Section: Video Streaming For Video Analyticsmentioning

confidence: 99%

Server-Driven Video Streaming for Deep Learning Inference

Pervaiz

Yuan

et al. 2020

Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications, Technologies,

154

View full text Add to dashboard Cite

show abstract

“…In traffic monitoring [2] it is useful to have timely insights, appropriately tuning γi and γi . In construction survey [3] there are no strict time deadlines, so γi = 0.…”

Section: A Mission Scheduling Problem (Msp)mentioning

confidence: 99%

“…Unmanned Aerial Vehicles (UAVs), also called drones, are enabling a wide range of applications in smart cities [1], such as traffic monitoring [2], construction surveys [3], package delivery [4], localization [5], and disaster (including COVID-19) management [6], assisted by 5G wireless roll-out [7]. The mobility, agility, and hovering capabilities of drones allow them to rapidly fly to points of interest (i.e., waypoints) in the city to accomplish specific activities.…”

Section: Introductionmentioning

confidence: 99%

Heuristic Algorithms for Co-scheduling of Edge Analytics and Routes for UAV Fleet Missions

Khochare

Simmhan

Sorbelli

et al. 2021

IEEE INFOCOM 2021 - IEEE Conference on Computer Communications

View full text Add to dashboard Cite

Unmanned Aerial Vehicles (UAVs) or drones are increasingly used for urban applications like traffic monitoring and construction surveys. Autonomous navigation allows drones to visit waypoints and accomplish activities as part of their mission. A common activity is to hover and observe a location using on-board cameras. Advances in Deep Neural Networks (DNNs) allow such videos to be analyzed for automated decision making. UAVs also host edge computing capability for on-board inferencing by such DNNs. To this end, for a fleet of drones, we propose a novel Mission Scheduling Problem (MSP) that coschedules the flight routes to visit and record video at waypoints, and their subsequent on-board edge analytics. The proposed schedule maximizes the utility from the activities while meeting activity deadlines as well as energy and computing constraints. We first prove that MSP is NP-hard and then optimally solve it by formulating a mixed integer linear programming (MILP) problem. Next, we design two efficient heuristic algorithms, JSC and VRC, that provide fast sub-optimal solutions. Evaluation of these three schedulers using real drone traces demonstrate utility-runtime trade-offs under diverse workloads.

show abstract

“…Unmanned aerial vehicles (also known as drones) are gaining momentum to serve as moving eyes in the sky. They fly in the low sky above a target area and perform real-time video surveillance for a wide variety of authorized and even critical activities such as large event (e.g., sports, concert, festival) surveillance, infrastructure safety inspection, search and rescue, wildfire surveillance, law enforcement and public safety assistance, to name many [2][3][4][5][6]. Drones possess unique advantages over its counterparts -cameras pre-deployed on the ground: a drone can cruise at a much higher altitude with a broader field of view (FoV) and an unparalleled perspective (birds' eye view) for surveillance; It is always available to operate on demand.…”

Section: Introductionmentioning

confidence: 99%

“…To this end, a promising paradigm is to offload heavy computing tasks to the edge [1,4,6,15]. As illustrated in Figure 1, the drone is responsible for recording and streaming the captured video to the edge while the edge performs heavy analytical tasks over the received video frames.…”

Section: Introductionmentioning

confidence: 99%

Towards Drone-Sourced Live Video Analytics via Adaptive-yet-Compatible Compression

Guo

Peng

2021

Proceedings of the 22nd International Workshop on Mobile Computing Systems and Applications

View full text Add to dashboard Cite

In this work, we present DCC (Deeper-yet-Compatible Compression), one enabling technique for real-time drone-sourced edge-assisted video analytics built on top of the existing codec. DCC tackles an important technical problem to compress streamed video from the drone to the edge without scarifying accuracy and timeliness of video analytical tasks performed at the edge. DCC is inspired by the fact that not every bit in streamed video is equally valuable to video analytics, which opens new compression room over the conventional analytics-oblivious video codec technology. We exploit drone-specific context and intermediate hints from object detection to pursue adaptive fidelity needed to retain analytical quality. We have prototyped DCC in one showcase application of vehicle detection and validated its efficiency in representative scenarios. DCC has reduced transmission volume by 9.5-fold over the baseline approach and 19-683% over [1] with comparable detection accuracy.

show abstract

Towards Drone-sourced Live Video Analytics for the Construction Industry

Cited by 13 publications

References 16 publications

Server-Driven Video Streaming for Deep Learning Inference

Server-Driven Video Streaming for Deep Learning Inference

Heuristic Algorithms for Co-scheduling of Edge Analytics and Routes for UAV Fleet Missions

Towards Drone-Sourced Live Video Analytics via Adaptive-yet-Compatible Compression

Contact Info

Product

Resources

About