Web-Based Learning of Naturalized Color Models for Human-Machine Interaction

Abstract

In recent years, natural verbal and non-verbal human-robot interaction has attracted an increasing interest. Therefore, models for robustly detecting and describing visual attributes of objects such as, e.g., colors are of great importance. However, in order to learn robust models of visual attributes, large data sets are required. Based on the idea to overcome the shortage of annotated training data by acquiring images from the Internet, we propose a method for robustly learning natural color models. Its novel aspects with respect to prior art are: firstly, a randomized HSL transformation that reflects the slight variations and noise of colors observed in real-world imaging sensors; secondly, a probabilistic ranking and selection of the training samples, which removes a considerable amount of outliers from the training data. These two techniques allow us to estimate robust color models that better resemble the variances seen in realworld images. The advantages of the proposed method over the current state-of-the-art technique using the training data without proper transformation and selection are confirmed in experimental evaluations. In combination, for models learned with pLSA-bg and HSL, the proposed techniques reduce the amount of mislabeled objects by 19.87 % on the well-known E-Bay data set.

Color naming models for color selection, image editing and palette design

“…An alternative approach used in computer vision is to construct probabilistic naming models in an automated fashion [30,36]. Given color names as input, a system can query a search engine for images associated with that color term; the image pixels can be used to fit a statistical model of colorname association.…”

“…Future large-scale surveys might vary the background color among black, white, and one or more shades of grey to construct color naming models more sensitive to background contrast. Alternatively, automated methods (e.g., using image search engines [30,36]) might be used to refine color name regions.…”

Focusing computational visual attention in multi-modal human-robot interaction

“…Alternatively, image search engines in the Internet can be used in order to collect huge weakly labelled data sets, in order to learn robust color models (cf. [43]). …”

“…Fig. 3) are learned using the Google-512 data set [43], which was gathered from the Internet for the 11 English basic color terms (see Sec. 2.4).…”

An Assistive Vision System for the Blind That Helps Find Lost Things

“…As a complementary approach, we propose to use visual attributes to help find things in a broader range of scenarios; e.g., to help find a specific colored shirt in a pile of shirts or to find objects that have only been verbally described by other persons. To this end, in our prototype implementation, we use probabilistic models of the 11 basic English color terms [9], see Fig. 1, which can also be used to name the color of an object in front of the camera.…”