Tooth-Marked Tongue Recognition Using Multiple Instance Learning and CNN Features

Abstract

Tooth-marked tongue or crenated tongue can provide valuable diagnostic information for traditional Chinese Medicine doctors. However, tooth-marked tongue recognition is challenging. The characteristics of different tongues are multiform and have a great amount of variations, such as different colors, different shapes, and different types of teeth marks. The regions of teeth mark only appear along the lateral borders. Most existing methods make use of concave regions information to classify the tooth-marked tongue which leads to inconstant performance when the region of teeth mark is not concave. In this paper, we try to solve these problems by proposing a three-stage approach which first makes use of concavity information to propose the suspected regions, then use a convolutional neural network to extract deep features and at last use a multiple-instance classifier to make the final decision. Experimental results demonstrate the effectiveness of the proposed method.

Tooth-Marked Tongue Recognition Using Gradient-Weighted Class Activation Maps

“…The first is the result on five-fold cross-validation, which is used to evaluate the performance of the proposed method. The second is the comparison with other works, such as Shao et al [19] and Li et al [6].…”

“…Recently, some researchers have used CNN features to extract tooth-marked features. In [6], a method for extracting features using CNN, using a multi-instance classifier for final classification, was proposed.…”

“…These images were labeled by Chinese medicine experts. Among them, 346 nontooth-marked tongue images were marked as negative examples, and 299 tooth-marked tongue images were marked as positive examples [6].…”

“…We conducted experiments on our dataset using another three methods, mentioned in Section 2.1, which were proposed by Wang [18], Shao [19], and Li [6]. The cross-validation settings used in these experiments were the same as in Section 4.3.…”

“…However, the recognition of tooth-marked tongue is a challenging task for TCM practitioners. The appearance of tooth-marked tongues has a great number of variations, such as different colors, different shapes, and different types of teeth marks [6]. Therefore, clinical effectiveness of the diagnosis heavily depends on the TCM practitioner's experience.…”