Mobile cosmetics advisor: an imaging based mobile service

Abstract

Selecting cosmetics requires visual information and often benefits from the assessments of a cosmetics expert. In this paper we present a unique mobile imaging application that enables women to use their cell phones to get immediate expert advice when selecting personal cosmetic products. We derive the visual information from analysis of camera phone images, and provide the judgment of the cosmetics specialist through use of an expert system. The result is a new paradigm for mobile interactions-image-based information services exploiting the ubiquity of camera phones. The application is designed to work with any handset over any cellular carrier using commonly available MMS and SMS features. Targeted at the unsophisticated consumer, it must be quick and easy to use, not requiring download capabilities or preplanning. Thus, all application processing occurs in the back-end system and not on the handset itself. We present the imaging pipeline technology and a comparison of the services' accuracy with respect to human experts.

Convolutional networks for appearance-based recommendation and visualisation of mascara products

“…For certain product types, however, such as cosmetics or clothing, visual information, such as that captured by in-store cameras or a customer's mobile device, may be all that is needed to make accurate product recommendations. Colour Match [7] is a mobile cosmetic advisory system that uses a colour-based statistical analysis of a calibrated mobile photograph to recommend a foundation product. McAuley et al [8] propose an approach to matching clothing and accessories based on human notions of complementarity by computing an embedding that places the output vectors of a pretrained convolutional neural network (CNN) for two products close together if online shopping data demonstrates a correlation between interests in the two products.…”

PowerSense

“…al. [3] developed Mobile Cosmetics Advisor (MCA) to identify a user's skin tone, where the user holds a sheet of reference skin tone colors near their face and takes a picture using the cellphone. Next, server side only image processing is performed to identify the user's proper skin tone, neglecting cellphone processing capability.…”

Cell phones as imaging sensors

“…Previous work [6,11,13] has addressed our assessment of the color gamut and ability to use a linear transform on sampled values to adequately map in this space. The target itself being the only well-defined element in the image, we chose it to evaluate its own correction.…”

“…We created a mobile imaging-based application in response to a commercial request to help women shop for cosmetics [11]. The goal was to increase women's satisfaction with cosmetics purchases by giving personalized product recommendations.…”