Shape and colour recognition of dishes for the purpose of customer service process automation in a self-service canteen

Abstract

Abstract-In the article a vision system for shape and colour recognition of dishes (plates, bowls, mugs), which can be used to automate the process of customer service in a self-service canteen is described. In consists of three basic components: object segmentation using so-called background model subtraction, shape recognition using geometric invariant moments and SVM classifier, as well as colour recognition using a Gaussian model. In addition, recognition in case of close or abut objects using a distance transform like approach is presented. The solution was evaluated on a dedicated test stand with controlled LED lightning. A 98% accuracy was obtained on over 100 test images, which indicates that the solution could be used in business practise.

Segmentation of Dishes for Customer Service Automation in a Self-service Canteen

“…In particular, the test stand used in the experiments, the method of determining the tray position, as well as dishes segmentation procedure based on background subtraction are discussed. It is worth noting, that the solution has undergone extensive evaluation (their detailed description can be found in work [8]).…”

“…This information is then used to compute the total price of the tray content. These parts of the system are broadly described in the paper [8]. At this point it is worth to mention that for shape recognition so-called shape coefficients It should be noted that the described above analysis can be narrowed down to an area smaller than the entire image frame (i.e.…”

“…However, in this case the separation between brightness and colour is not used. [8]). However, in some special cases, where for example three dishes are close to each other ( Fig.…”

“…This affects the segmentation procedure (right part of Fig. 10) and influences the close or abut objects segmentation procedure described in [8].…”