Spatio‐temporal feature fusion based correlative binary relevance for visual object detection

Abstract

Object detection in video is an emerging and challenging research topic in Computer Vision. Recently, deep learning based approaches achieves greater results in object detection. The major issues in detecting multiple objects from the videos are exploiting low‐level visual concepts and temporal information and also exploring the correlation between the objects in the videos. Therefore, to address the above mentioned issues, the spatio‐temporal feature fusion based correlative binary relevance (STFF‐CBR) classifier is proposed to generate a rich vector representation and exploit the label correlation for the object detection in videos. In this article, first, the spatio‐temporal feature fusion (STFF) is proposed to exploit the low‐level visual concepts and temporal information in the videos which significantly improves the object detection performance. Second, correlative binary relevance (CBR) classification approach is proposed to exploit the dependencies between the labels in the video using the nearest neighbor based label dependency graph (LDG). Additionally, feed forward neural network (FFNN) classifier is utilized to increase the classification accuracy of the CBR method. Experimental evaluation shows that the STFF‐CBR classifier model achieves better performance for object detection problem in video against the state‐of‐the‐art methods.