Performance measures for object detection evaluation

Ozdemir, Bahadir; Aksoy, Selim; Eckert, Sandra; Pesaresi, Martino; Ehrlich, Daniele

doi:10.1016/j.patrec.2009.10.016

Cited by 41 publications

(23 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The reason, and the justification for the VTT, is that the current method does not scale with respect to the richness of the representation. Even for the subtasks in the competitions mentioned earlier, the evaluation of performance, i.e., comparing the output of the system (e.g., estimated bounding boxes) to the ground truth, is not always straightforward and the quality of matches must be assessed (28). Moreover, annotating every image submitted for testing at massive levels of detail is not feasible.…”

Section: Current Evaluation Practicementioning

confidence: 99%

Visual Turing test for computer vision systems

Geman

Hallonquist

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

261

183

View full text Add to dashboard Cite

Today, computer vision systems are tested by their accuracy in detecting and localizing instances of objects. As an alternative, and motivated by the ability of humans to provide far richer descriptions and even tell a story about an image, we construct a "visual Turing test": an operator-assisted device that produces a stochastic sequence of binary questions from a given test image. The query engine proposes a question; the operator either provides the correct answer or rejects the question as ambiguous; the engine proposes the next question ("just-in-time truthing"). The test is then administered to the computer-vision system, one question at a time. After the system's answer is recorded, the system is provided the correct answer and the next question. Parsing is trivial and deterministic; the system being tested requires no natural language processing. The query engine employs statistical constraints, learned from a training set, to produce questions with essentially unpredictable answers-the answer to a question, given the history of questions and their correct answers, is nearly equally likely to be positive or negative. In this sense, the test is only about vision. The system is designed to produce streams of questions that follow natural story lines, from the instantiation of a unique object, through an exploration of its properties, and on to its relationships with other uniquely instantiated objects. G oing back at least to the mid-20th century there has been an active debate about the state of progress in artificial intelligence and how to measure it. Alan Turing (1) proposed that the ultimate test of whether a machine could "think," or think at least as well as a person, was for a human judge to be unable to tell which was which based on natural language conversations in an appropriately cloaked scenario. In a much-discussed variation (sometimes called the "standard interpretation"), the objective is to measure how well a computer can imitate a human (2) in some circumscribed task normally associated with intelligent behavior, although the practical utility of "imitation" as a criterion for performance has also been questioned (3). In fact, the overwhelming focus of the modern artificial intelligence (AI) community has been to assess machine performance more directly by dedicated tests for specific tasks rather than debating about general "thinking" or Turing-like competitions between people and machines.In this paper we implement a new, query-based test for computer vision, one of the most vibrant areas of modern AI research. Throughout this paper we use "computer vision" more or less synonymously with semantic image interpretation-"images to words." However, of course computer vision encompasses a great many other activities; it includes the theory and practice of image formation ("sensors to images"), image processing ("images to images"), mathematical representations, video processing, metric scene reconstruction, and so forth. In fact, it may not be possible to interpret scenes at a semantic leve...

show abstract

Section: Current Evaluation Practicementioning

confidence: 99%

Visual Turing test for computer vision systems

Geman

Hallonquist

et al. 2015

Proc. Natl. Acad. Sci. U.S.A.

261

183

View full text Add to dashboard Cite

show abstract

“…However, since the Hacettepe images consisted of multiple cells, a matching step was required to find correspondences between the resulting segments and the ground truth nuclei. We used an object-based evaluation procedure similar to [35] that was adapted from the work of [36] on range image segmentation evaluation. This procedure used the individual reference objects in the ground truth and the output objects in the produced segmentation, and classified every pair of reference and output objects as correct detections, over-detections, underdetections, missed detections, or false alarms with respect to a threshold t on the amount of overlap between these objects.…”

Section: Evaluation Of Segmentationmentioning

confidence: 99%

Unsupervised segmentation and classification of cervical cell images

Genctav

Aksoy

Önder

2012

Pattern Recognition

Self Cite

232

176

View full text Add to dashboard Cite

a b s t r a c tThe Pap smear test is a manual screening procedure that is used to detect precancerous changes in cervical cells based on color and shape properties of their nuclei and cytoplasms. Automating this procedure is still an open problem due to the complexities of cell structures. In this paper, we propose an unsupervised approach for the segmentation and classification of cervical cells. The segmentation process involves automatic thresholding to separate the cell regions from the background, a multi-scale hierarchical segmentation algorithm to partition these regions based on homogeneity and circularity, and a binary classifier to finalize the separation of nuclei from cytoplasm within the cell regions. Classification is posed as a grouping problem by ranking the cells based on their feature characteristics modeling abnormality degrees. The proposed procedure constructs a tree using hierarchical clustering, and then arranges the cells in a linear order by using an optimal leaf ordering algorithm that maximizes the similarity of adjacent leaves without any requirement for training examples or parameter adjustment. Performance evaluation using two data sets show the effectiveness of the proposed approach in images having inconsistent staining, poor contrast, and overlapping cells.

show abstract

“…Area S yang berada pada matrix persegi Precision dan recall sering digunakan pada literatur untuk mengukur seberapa baik objek yang dideteksi terhadap objek referensi. Recall dapat diinterpretasikan sebagai jumlah objek positif benar dideteksi oleh algoritma, sedangkan precision cenderung mengevaluasi algoritma sebagai positif salah [11]. Accuracy digunakan untuk mengukur tingkat kesesuaian semua status lalu lintas yang benar dengan total status lalu lintas yang dideteksi.…”

Section: Algoritma Monte Carlounclassified

Deteksi Kemacetan Lalu Lintas dengan Menggunakan Algoritma Monte Carlo

Adisalam

2017

IndoJC

View full text Add to dashboard Cite

AbstrakSensor dan kamera telah digunakan sekian lama dalam montoring lalu lintas untuk mendeteksi kemacetan lalu lintas. Kemacetan lalu lintas disebabkan karena perkembangan infrastruktur yang lambat tidak sebanding dengan peningkatan jumlah kendaraan. Penelitian ini bertujuan untuk mendeteksi status lalu lintas berdasarkan citra lalu lintas dengan sudut pandang yang berbeda. Citra lalu lintas diolah dengan menggunakan pengolahan citra menjadi 3 kondisi lalu lintas yaitu lancar, ramai, dan padat. Pengolahan citra merupakan metode untuk mengolah citra. Pengolahan citra dilakukan untuk mengubah citra RGB menjadi citra biner sehingga Monte Carlo dapat diterapkan dengan cara menghitung luas dari area piksel putih. Luas area putih pada citra biner digunakan untuk menentukan status lalu lintas. Hasil testing 1 menunjukkan skenario 1 menghasilkan performa terbaik dengan precision ramai bernilai 44%, recall ramai bernilai 77%, precision lancar bernilai 92%, recall lancar bernilai 73%, dan akurasi bernilai 73%. Hasil testing 2 menunjukkan skenario 2 menghasilkan performa terbaik dengan precision padat bernilai 100%, recall padat bernilai 99%, dan akurasi bernilai 99%.Kata Kunci: grayscale, pengolahan citra, segmen gambar, algoritma Monte Carlo, kemacetan lalu lintas, status lalu lintas Abstract Sensors and cameras have been used for so long in montoring traffic to detect traffic congestion. Traffic congestion is caused due to the slow development of infrastructure is not proportional to the increase in the number of vehicles. This study aims to detect traffic status based on traffic images with different viewpoints. Traffic image is processed by using image processing into 3 traffic conditions that is smooth, crowded, and solid. Image processing is a method for image processing. Image processing is done to convert RGB image into binary image so that Monte Carlo can be applied by calculating the area of the white pixel area. The white area of the binary image is used to determine the status of traffic. The result of testing 1 shows scenario 1 gives the best performance with 44% crowded precision, 77% crowded recall, 92% smooth precision, 73% smooth recall, and the accuracy is 73%. Test results 2 show scenario 2 yields the best performance with 100% solid precision, 99% solid recall, and 99% accuracy.

show abstract

Performance measures for object detection evaluation

Cited by 41 publications

References 27 publications

Visual Turing test for computer vision systems

Visual Turing test for computer vision systems

Unsupervised segmentation and classification of cervical cell images

Deteksi Kemacetan Lalu Lintas dengan Menggunakan Algoritma Monte Carlo

Contact Info

Product

Resources

About