ClickBAIT: Click-based Accelerated Incremental Training of Convolutional Neural Networks

Abstract

Today's general-purpose deep convolutional neural networks (CNN) for image classification and object detection are trained offline on large static datasets. Some applications, however, will require training in real-time on live video streams with a human-in-the-loop. We refer to this class of problem as Time-ordered Online Training (ToOT)-these problems will require a consideration of not only the quantity of incoming training data, but the human effort required to tag and use it. In this paper, we define training benefit as a metric to measure the effectiveness of a sequence in using each user interaction. We demonstrate and evaluate a system tailored to performing ToOT in the field, capable of training an image classifier on a live video stream through minimal input from a human operator. We show that by exploiting the time-ordered nature of the video stream through optical flow-based object tracking, we can increase the effectiveness of human actions by about 8 times.

FAIM: Vision and Weight Sensing Fusion Framework for Autonomous Inventory Monitoring in Convenience Stores

“…In addition, in order to achieve high accuracy levels, these models require thousands of images of each object, under different conditions, with different backgrounds and from different angles. These images require labeling of the groundtruth for training, this can either be done manually, creating a labor intensive problem, in a semi-supervised (Teng et al, 2018) or fully automated fashion (Ruiz et al, 2019c).…”

Autonomous Curiosity for Real-Time Training Onboard Robotic Agents

“…The metric in question, then, is the average incremental training benefit [32] of each user interaction during an episode. Qualitatively, incremental training benefit is a measure of the increase in model performance (average precision in this case) that can be attributed to a particular user interaction.…”

“…The trainee is an online training system for object detection with the same architecture, data augmentation, and object tracking strategy as the one presented in [32]. The object detector (SSD) is trained from ground truth bounding box data given by the human user, and object tracking is used to interpolate additional ground truth information between user inputs.…”

“…We chose the single shot object detector (SSD) [15] for its high frame rate as compared to region proposal methods. We follow closely the configuration used by [32], a VGG16-SSD model with 300x300 pixel input, re-structured as a two-class (positive and negative) model from the standard PASCAL VOC 20 classes.…”

“…Improving the benefit of given training events (study) has been widely explored [32,23,22]. Active learning partially addresses the question of curiosity by selecting (subsampling) which datapoints would be most advantageous to annotate [30,35,5].…”