An evaluation metric for image segmentation of multiple objects

Polak, Mark J.; Zhang, Hong; Pi, Ming Hong

doi:10.1016/j.imavis.2008.09.008

Cited by 148 publications

(97 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Measuring multi-object segmentation accuracy is an active topic of research with several metrics previously proposed [33][34][35]44]. For the challenge and this study, we adopted several evaluation criteria and devised Matlab implementations.…”

Section: Evaluation Measuresmentioning

confidence: 99%

See 1 more Smart Citation

Leaf segmentation in plant phenotyping: a collation study

Scharr

Minervini

French

et al. 2015

Machine Vision and Applications

242

181

View full text Add to dashboard Cite

Image-based plant phenotyping is a growing application domain of computer vision in agriculture. A key task is the segmentation of all individual leaves in images. Here we focus on the most common rosette model plants Arabidopsis and young tobacco. Although leaves do share appearance and shape characteristics, the presence of occlusions and variability in leaf shape and pose, as well as imaging conditions, render this problem challenging. The aim of this paper is to compare several leaf segmentation solutions on a unique and first of its kind dataset containing images from typical phenotyping experiments. In particular, we report and discuss methods and findings of a collection of submissions for the first Leaf Segmentation Challenge (LSC) of the Computer Vision Problems in Plant Phenotyping (CVPPP) workshop in 2014. Four methods are presented: three segment leaves via processing the distance transform in an unsupervised fashion, and the other via optimal template selection and Chamfer matching. Overall, we find that although separating plant from background can be achieved with satisfactory accuracy (>90% Dice score), individual leaf segmentation and counting remain challenging when leaves overlap. Besides, accuracy is lower for younger leaves. We find also that variability in datasets does affect outcomes. Our findings motivate further investigations and development of specialized algorithms for this particular application, and that challenges of this form are ideally suited for advancing the state of the art. Data are publicly available (http://www.plantphenotyping.org/CVPPP2014-dataset) to support future challenges beyond segmentation within this application domain.

show abstract

Section: Evaluation Measuresmentioning

confidence: 99%

“…We should note that we also considered the Global Consistency Error (GCE) [34] and Object-level Consistency Error (OCE) [44] metrics, which are suited for evaluating segmentation of multiple objects. However, we found that they are harder to interpret and that the SBD is capable of capturing relevant leaf segmentation errors.…”

Section: Evaluation Measuresmentioning

confidence: 99%

Leaf segmentation in plant phenotyping: a collation study

Scharr

Minervini

French

et al. 2015

Machine Vision and Applications

242

181

View full text Add to dashboard Cite

show abstract

“…The Partial and Directed Object-Level Consistency Error (PD_OCE) [29,64] and the Reference Weighted Jaccard (RWJ) [29] metrics quantifies quality based on all generated segments intersecting the reference segment, but have a difference based on the importance (area overlap) of generated segments to the problem. The optimal result for all metrics is zero, with the effective range being [0, 1] (with few exceptions).…”

Section: Metrics and Optimizersmentioning

confidence: 99%

Classifier Directed Data Hybridization for Geographic Sample Supervised Segment Generation

Fourie

Schöepfer

2014

Remote Sensing

View full text Add to dashboard Cite

Quality segment generation is a well-known challenge and research objective within Geographic Object-based Image Analysis (GEOBIA). Although methodological avenues within GEOBIA are diverse, segmentation commonly plays a central role in most approaches, influencing and being influenced by surrounding processes. A general approach using supervised quality measures, specifically user provided reference segments, suggest casting the parameters of a given segmentation algorithm as a multidimensional search problem. In such a sample supervised segment generation approach, spatial metrics observing the user provided reference segments may drive the search process. The search is commonly performed by metaheuristics. A novel sample supervised segment generation approach is presented in this work, where the spectral content of provided reference segments is queried. A one-class classification process using spectral information from inside the provided reference segments is used to generate a probability image, which in turn is employed to direct a hybridization of the original input imagery. Segmentation is performed on such a hybrid image. These processes are adjustable, interdependent and form a part of the search problem. Results are presented detailing the performances of four method variants compared to the generic sample supervised segment generation approach, under various conditions in terms of resultant segment quality, required computing time and search process characteristics. Multiple metrics, metaheuristics and segmentation algorithms are tested with this approach. Using the spectral data contained within user OPEN ACCESS Remote Sens. 2014, 6 11853 provided reference segments to tailor the output generally improves the results in the investigated problem contexts, but at the expense of additional required computing time.

show abstract

“…No single algorithm performed with a fixed-parameter setting is considered to be sufficient for analyzing all time-lapse images, and optimizing algorithms for a variety of images is a tedious task for researchers. Solutions to these problems have been proposed based on the idea of algorithm selection (e.g., Cardoso & Corte-Real, 2005;Zhang, 2006;Polak et al, 2009). An appropriate algorithm with an optimized parameter setting for each task is automatically selected according to unique evaluation metrics of algorithm performance.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, the range of features of intracellular substances cannot be pre-defined, and the diversity in features of intracellular substances may destabilize the result of evaluation. The latter type can evaluate different algorithms by using some metrics based on similarity (or error) measurement between two regions: an automatically segmented region and a manually segmented region, called the reference region or the ground-truth (e.g., Zhang & Gerbrands, 1992;Martin et al, 2001;Jiang et al, 2006;Polak et al, 2009). For example, the number of missegmented pixels (Ysnoff et al, 1977), or the number of segmented targets (Zhang, 1996) is commonly used as an error measurement.…”

Section: Introductionmentioning

confidence: 99%