A scoring, truthing, and registration toolkit for ATR evaluation

Stevens, Mark R.; Snorrason, Magnus; Jarratt, Mary; Irvine, John

doi:10.1117/12.487049

Cited by 2 publications

(1 citation statement)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To make matters more complicated, there exist cases of over-segmentation and under-segmentation when multiple algorithm detections correspond to a single truth region or a single detection corresponds to several truth regions, respectively. There are many possible scoring techniques, such as point scoring, centroid scoring, touch scoring, and chamfer distance, each of which defines specific distance metrics from an algorithm decision to an image truth [10].…”

Section: Resultsmentioning

confidence: 99%

Autonomous detection of indoor and outdoor signs

et al. 2005

Self Cite

View full text Add to dashboard Cite

Most goal-oriented mobile robot tasks involve navigation to one or more known locations. This is generally done using GPS coordinates and landmarks outdoors, or wall-following and fiducial marks indoors. Such approaches ignore the rich source of navigation information that is already in place for human navigation in all man-made environments: signs. A mobile robot capable of detecting and reading arbitrary signs could be tasked using directions that are intuitive to humans, and it could report its location relative to intuitive landmarks (a street corner, a person's office, etc.). Such ability would not require active marking of the environment and would be functional in the absence of GPS. In this paper we present an updated version of a system we call Sign Understanding in Support of Autonomous Navigation (SUSAN). This system relies on cues common to most signs, the presence of text, vivid color, and compact shape. By not relying on templates, SUSAN can detect a wide variety of signs: traffic signs, street signs, store-name signs, building directories, room signs, etc. In this paper we focus on the text detection capability. We present results summarizing probability of detection and false alarm rate across many scenes containing signs of very different designs and in a variety of lighting conditions.

show abstract

Section: Resultsmentioning

confidence: 99%

Autonomous detection of indoor and outdoor signs

et al. 2005

Self Cite

View full text Add to dashboard Cite

show abstract

START for evaluation of target detection and tracking

et al. 2005

Self Cite

View full text Add to dashboard Cite

A major challenge for ATR evaluation is developing an accurate image truth that can be compared to an ATR algorithm's decisions to assess performance. While many standard truthing methods and scoring metrics exist for stationary targets in still imagery, techniques for dealing with motion imagery and moving targets are not as prevalent. This is partially due to the fact that the moving imagery / moving targets scenario introduces the data association problem of assigning targets to tracks. Video datasets typically contain far more imagery than static collections, increasing the size of the truthing task. Specifying the types and locations of the targets present for a large number of images is tedious, time consuming, and error prone. In this paper, we present an updated version of a complete truthing system we call the Scoring, Truthing, And Registration Toolkit (START). The application consists of two components: a truthing components that assists in the automated construction of image truth, and a scoring component that assesses the performance of a given algorithm relative to the specified truth. In motion imagery, both stationary and moving targets can be detected and tracked over portions of a motion imagery clip. We summarize the capabilities of START with emphasis on the target tracking and truthing diagnostics. The user manually truths certain key frames, truth for intermediate frames is then inferred and sets of diagnostics verify the quality of the truth. If ambiguous situations are encountered in the intermediate frames, diagnostics flag the problem so that the user can intervene manually. This approach can dramatically reduce the effort required for truthing video data, while maintaining high fidelity in the truth data. We present the results of two user evaluations of START, one addressing the accuracy and the other focusing on the human factors aspects of the design.

show abstract

A scoring, truthing, and registration toolkit for ATR evaluation

Cited by 2 publications

References 10 publications

Autonomous detection of indoor and outdoor signs

Autonomous detection of indoor and outdoor signs

START for evaluation of target detection and tracking

Contact Info

Product

Resources

About