Evaluating Performance in Continuous Context Recognition Using Event-Driven Error Characterisation

Ward, Jamie A.; Lukowicz, Paul; Tröster, Gerhard

doi:10.1007/11752967_16

Cited by 25 publications

(27 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…early detection of an action onset) may be wrongly considered as classification errors. Ward et al propose to explicitly quantify the system performance taking all these aspects into account [19]. They characterized different types of errors as follows (listed in increasing order of importance), 1) Overfill: when the start and stop time of predicted labels are less and greater than actual time, respectively.…”

Section: Classification Methodsmentioning

confidence: 99%

Benchmarking classification techniques using the Opportunity human activity dataset

Sagha

Digumarti

Millán

et al. 2011

2011 IEEE International Conference on Systems, Man, and Cybernetics

View full text Add to dashboard Cite

Abstract-Human activity recognition is a thriving research field. There are lots of studies in different sub-areas of activity recognition proposing different methods. However, unlike other applications, there is lack of established benchmarking problems for activity recognition. Typically, each research group tests and reports the performance of their algorithms on their own datasets using experimental setups specially conceived for that specific purpose. In this work, we introduce a versatile human activity dataset conceived to fill that void. We illustrate its use by presenting comparative results of different classification techniques, and discuss about several metrics that can be used to assess their performance. Being an initial benchmarking, we expect that the possibility to replicate and outperform the presented results will contribute to further advances in state-ofthe-art methods.

show abstract

Section: Classification Methodsmentioning

confidence: 99%

Benchmarking classification techniques using the Opportunity human activity dataset

Sagha

Digumarti

Millán

et al. 2011

2011 IEEE International Conference on Systems, Man, and Cybernetics

View full text Add to dashboard Cite

show abstract

“…We distinguished the six methods in the following way: the four participations were identified by their participation numbers (13,49,51,59), and the two additional methods were identified by letters A and B. The submitted method (published in [54]) uses low-level features and mid-level features calculated on detected and tracked people using [55] and detected objects specific to the dataset (doors, mailboxes etc.).…”

Section: Results Of the Icpr 2012 Harl Competitionmentioning

confidence: 99%

“…Evaluation of this variant needs metrics adapted to the problem. In [13], a measure is proposed based on alignment. It introduces six different error types: insertion, deletion, merge, fragmentation, underfill and overfill.…”

Section: Related Metrics and Datasetsmentioning

confidence: 99%

Evaluation of video activity localizations integrating quality and quantity measurements

Wolf

Lombardi

Mille

et al. 2014

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Evaluating the performance of computer vision algorithms is classically done by reporting classification error or accuracy, if the problem at hand is the classification of an object in an image, the recognition of an activity in a video or the categorization and labeling of the image or video. If in addition the detection of an item in an image or a video, and/or its localization are required, frequently used metrics are Recall and Precision, as well as ROC curves. These metrics give quantitative performance values which are easy to understand and to interpret even by non-experts. However, an inherent problem is the dependency of quantitative performance measures on the quality constraints that we need impose on the detection algorithm. In particular, an important quality parameter of these measures is the spatial or spatio-temporal overlap between a ground-truth item and a detected item, and this needs to be taken into account when interpreting the results.We propose a new performance metric addressing and unifying the qualitative and quantitative aspects of the performance measures. The performance of a detection and recognition algorithm is illustrated intuitively by performance graphs which present quantitative performance values, like Recall, Precision and F-Score, depending on quality constraints of the detection. In order to compare the performance of different computer vision algorithms, a representative single performance measure is computed from the graphs, by integrating out all quality parameters. The evaluation method can be applied to different types of activity detection and recognition algorithms. The performance metric has been tested on several activity recognition algorithms participating in the ICPR 2012 HARL competition.

show abstract

“…The comparison with sample-accurate confusion matrices confirms that the soft alignment is a sensible solution for event spotting performance analyses. For a more detailed analysis of detection errors, the error distribution diagrams [41] could be used.…”

Section: Methodsmentioning

confidence: 99%

Recognition of dietary activity events using on-body sensors

Amft

Tröster

2008

Artificial Intelligence in Medicine

Self Cite

195

148

View full text Add to dashboard Cite

SummaryObjective: An imbalanced diet elevates health risks for many chronic diseases including obesity. Dietary monitoring could contribute vital information to lifestyle coaching and diet management, however, current monitoring solutions are not feasible for a long-term implementation. Towards automatic dietary monitoring, this work targets the continuous recognition of dietary activities using on-body sensors. Methods: An on-body sensing approach was chosen, based on three core activities during intake: arm movements, chewing and swallowing. In three independent evaluation studies the continuous recognition of activity events was investigated and the precision-recall performance analysed. An event recognition procedure was deployed, that addresses multiple challenges of continuous activity recognition, including the dynamic adaptability for variable-length activities and flexible deployment by supporting one to many independent classes. The approach uses a sensitive activity event search followed by a selective refinement of the detection using different information fusion schemes. The method is simple and modular in design and implementation. Results: The recognition procedure was successfully adapted to the investigated dietary activities. Four intake gesture categories from arm movements and two food groups from chewing cycle sounds were detected and identified with a recall of 80-90% and a precision of 50-64%. The detection of individual swallows resulted in 68% recall and 20% precision. Sample-accurate recognition rates were 79% for movements, 86% for chewing and 70% for swallowing. Conclusions: Body movements and chewing sounds can be accurately identified using on-body sensors, demonstrating the feasibility of on-body dietary monitoring. Further investigations are needed to improve the swallowing spotting performance. #

show abstract

Evaluating Performance in Continuous Context Recognition Using Event-Driven Error Characterisation

Cited by 25 publications

References 15 publications

Benchmarking classification techniques using the Opportunity human activity dataset

Benchmarking classification techniques using the Opportunity human activity dataset

Evaluation of video activity localizations integrating quality and quantity measurements

Recognition of dietary activity events using on-body sensors

Contact Info

Product

Resources

About