Drones have been applied to a wide range of security and surveillance applications recently. With drones, Internet of Things are extending to 3D space. An interesting question is: Can we conduct person identification (PID) in a drone view? Traditional PID technologies such as RFID and fingerprint/iris/face recognition have their limitations or require close contact to specific devices. Hence, these traditional technologies can not be easily deployed to drones due to dynamic change of view angle and height. In this work, we demonstrate how to retrieve IoT data from users’ wearables and correctly tag them on the human objects captured by a drone camera to identify and track ground human objects. First, we retrieve human objects from videos and conduct coordination transformation to handle the change of drone positions. Second, a fusion algorithm is applied to measure the correlation of video data and inertial data based on the extracted human motion features. Finally, we can couple human objects with their wearable IoT devices, achieving our goal of tagging wearable device data (such as personal profiles) on human objects in a drone view. Our experimental evaluation shows a recognition rate of 99.5% for varying walking paths, and 98.6% when the drone’s camera angle is within 37°. To the best of our knowledge, this is the first work integrating videos from drone cameras and IoT data from inertial sensors.

Drones have been applied to a wide range of security and surveillance applications recently. With drones, Internet of Things are extending to3D space. An interesting question is: Can we conduct person identification(PID) in a drone view? Traditional PID technologies such as RFID and fingerprint/iris/face recognition have their limitations or require close contactto specific devices. Hence, these traditional technologies can not be easily deployed to drones due to dynamic change of view angle and height. In this work,we demonstrate how to retrieve IoT data from users’ and correctly tag themon the human objects captured by a drone camera to identify and track groundhuman objects. First, we retrieve human objects from videos and conduct coordination transformation to handle the change of drone positions. Second,a fusion algorithm is applied to measure the correlation of video data andinertial data based on the extracted human motion features. Finally, we cancouple human objects with their wearable IoT devices, achieving our goal oftagging wearable device data such as personal profiles) on human objects ina drone view. Our experimental evaluation shows a recognition rate of 98.9%.To the best of our knowledge, this is the first work integrating videos fromdrone cameras and IoT data from inertial sensors.

IoT technologies enable millions of devices to transmit their sensor data to the external world. The device–object pairing problem arises when a group of Internet of Things is concurrently tracked by cameras and sensors. While cameras view these things as visual “objects”, these things which are equipped with “sensing devices” also continuously report their status. The challenge is that when visualizing these things on videos, their status needs to be placed properly on the screen. This requires correctly pairing visual objects with their sensing devices. There are many real-life examples. Recognizing a vehicle in videos does not imply that we can read its pedometer and fuel meter inside. Recognizing a pet on screen does not mean that we can correctly read its necklace data. In more critical ICU environments, visualizing all patients and showing their physiological signals on screen would greatly relieve nurses’ burdens. The barrier behind this is that the camera may see an object but not be able to see its carried device, not to mention its sensor readings. This paper addresses the device–object pairing problem and presents a multi-camera, multi-IoT device system that enables visualizing a group of people together with their wearable devices’ data and demonstrating the ability to recover the missing bounding box.

No abstract