Benchmarking human epithelial type 2 interphase cells classification methods on a very large dataset

Abstract

human epithelial type 2 interphase cellsclassification methods on a very large dataset, Artificial Intelligence In Medicine (2015), http://dx.doi.org/10. 1016/j.artmed.2015.08.001 This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain. AbstractObjective. This paper presents benchmarking results of human epithelial type 2 (HEp-2) interphase cell image classification methods on a very large dataset. The indirect immunofluorescence method applied on HEp-2 cells has been the gold standard to identify connective tissue diseases such as systemic lupus erythematosus and Sjögren's syndrome. However, the method su↵ers from numerous issues such as being subjective, time consuming and labour intensive. This has been the main motivation for the development of various computer-aided diagnosis systems whose main task is to automatically classify a given cell image into one of the predefined classes.Methods and material. The benchmarking was performed in the form of an international competition held in conjunction with the International Conference of Image Processing in 2013: fourteen teams, composed of practitioners and researchers in this area, took part in the initiative. The system developed by each team was trained and tested on a very large HEp-2 cell dataset comprising over 68,000 images of HEp-2 cell. The dataset contains cells with six di↵erent staining patterns and two levels of fluorescence intensity. For each method we provide a brief description highlighting the design Results. The staining pattern recognition accuracy attained by the methods varies between 47.91% and slightly above 83.65%. However, the difference between the top performing method and the seventh ranked method is only 5%. In the paper, we also study the performance achieved by fusing the best methods, finding that a recognition rate of 85.60% is reached when the top seven methods are employed.Conclusions. We found that highest performance is obtained when using a strong classifier (typically a kernelised support vector machine) in conjunction with features extracted from local statistics. Furthermore, the misclassification profiles of the di↵erent methods highlight that some staining patterns are intrinsically more di cult to recognize. We also noted that performance is strongly a↵ected by the fluorescence intensity level. Thus, low accuracy is to be expected when analyzing low contrasted images.

HEp-2 staining pattern recognition at cell and specimen levels: Datasets, algorithms and results

“…From Table 3, we can note that 57% of the methods reported in this review (8 over 14) achieve a mean class accuracy higher than 80% on the Task 1. This result is very encouraging, especially when it is compared with performance achieved by the methods participating to the previous contest organized in 2013 (Hobson et al (2015)). Indeed, in that competition only 2 methods over 14 submissions achieved an MCA above 80% on the same dataset.…”

“…Most methods for HEp-2 cell classification typically do not perform classification using directly the extracted features; in fact, they often resort to different strategies for feature encoding or aggregation in order to derive a more compact and representative high level representation. As confirmed by Hobson et al (2015), a typical approach is based on the bag of words paradigm. We also note that the introduction of techniques exploiting enhanced versions of bag of words, such as those based on Fisher vector (Qi), on the the soft assignment (Gragnaniello), as well as on the VLAD (Theodorakopoulos): these approaches appear very promising, scoring among the best four, due to their ability to enrich information conveyed in the high level representation with respect to traditional bag of words, thus allowing a better discrimination among the different classes.…”

“…Nevertheless, we can provide a justification of the general increase of the performance. In Hobson et al (2015), the authors observe that the top performing methods participating to the ICIP2013 competition adopted two main ingredients: features extracted from local statistics of an image (e.g. bag-of-words approach) and a strong classifier.…”

“…In this paper we focus on the staining recognition step: indeed, this task has attracted an increasing number of scientists working in the Pattern Recognition field, mainly due to the competitions organized in the last four years (Foggia 3 et al (2013, 2014); Hobson et al (2015)), where the participants were required to design and implement a system able to classify a single cell belonging to HEp-2 images.…”

Executable thematic special issue on pattern recognition techniques for indirect immunofluorescence images analysis

“…Such interest has been also boosted by a series of benchmarking activities on IIF cell and specimen classification held at leading international conferences in the areas of pattern recognition (ICPR 2012 and ICPR 2014) and image processing (ICIP 2013). In all cases these competitions had strong participation from the scientific community, collecting several dozens of distinct software submissions [5][6][7][8][9]. This special issue describes advances achieved through this series of initiatives, and we are very pleased by their success.…”