Semantically Smoothed Refinement for Everyday Concept Indexing

Abstract

Abstract. Instead of occurring independently, semantic concepts pairs tend to co-occur within a single image and it is intuitive that concept detection accuracy for visual concepts can be enhanced if concept correlation can be leveraged in some way. In everyday concept detection for visual lifelogging using wearable cameras to automatically record everyday activities, the captured images usually have a diversity of concepts which challenges the performance of concept detection. In this paper a semantically smoothed refinement algorithm is proposed using concept correlations which exploit topic-related concept relationships, modeled externally in a user experiment rather than extracted from training data. Results for initial concept detection are factorized based on semantic smoothness and adjusted in compliance with the extracted concept correlations. Refinement performance is demonstrated in experiments to show the effectiveness of our algorithm and the extracted correlations.

Computer Vision for Lifelogging

“…As introduced in Section 3.2, appropriate selection of semantic concepts as attributes impacts the recognition accuracy of activities. Though manual construction of semantic space such as topic-related concept selection user experiment [92,93], has shown its merits, such method is less flexible when dealing with various activity types which are not existing in the pre-defined activity set. A more feasible solution is to exploit external online resources in order to reflect human knowledge on activity-specific concept selection.…”