The effect of task on classification accuracy: Using gesture recognition techniques in free-sketch recognition

Field, Martin J.; Gordon, Sam; Peterson, Eric J.; Robinson, Raquel; Stahovich, Thomas F.; Alvarado, Christine

doi:10.1016/j.cag.2010.07.001

Cited by 15 publications

(10 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is likely that increasing the amount of user-specific training data will increase the accuracy of the system. Such training data has proven beneficial for other recognition tasks, such as hand-drawn symbol recognition (Field et al, 2010).…”

Section: Discussionmentioning

confidence: 99%

Using speech to identify gesture pen strokes in collaborative, multimodal device descriptions

Herold

Stahovich

2011

AIEDAM

View full text Add to dashboard Cite

One challenge in building collaborative design tools that use speech and sketch input is distinguishing gesture pen strokes from those representing device structure, that is, object strokes. In previous work, we developed a gesture/object classifier that uses features computed from the pen strokes and the speech aligned with them. Experiments indicated that the speech features were the most important for distinguishing gestures, thus indicating the critical importance of the speech–sketch alignment. Consequently, we have developed a new alignment technique that employs a two-step process: the speech is first explicitly segmented (primarily into clauses), then the segments are aligned with the pen strokes. Our speech segmentation step is unique in that it uses sketch features for locating segment boundaries in multimodal dialog. In addition, it uses a single classifier to directly combine word-based, prosodic (pause), and sketch-based features. In the second step, segments are initially aligned with strokes based on temporal correlation, and then classifiers are used to detect and correct two common alignment errors. Our two-step technique has proven to be substantially more accurate at alignment than the existing technique that lacked explicit segmentation. It is more important that, for nearly all cases, our new technique results in greater gesture classification accuracy than the existing technique, and performed nearly as well as the benchmark manual speech–sketch alignment.

show abstract

Section: Discussionmentioning

confidence: 99%

Using speech to identify gesture pen strokes in collaborative, multimodal device descriptions

Herold

Stahovich

2011

AIEDAM

View full text Add to dashboard Cite

show abstract

“…The first step is to collect realistic sketches using the data collection component [Blagojevic et al 2008]. Realistic sketches are important as they provides natural inter-gesture spatial and timing information that is beneficial for sketch recognition [Field et al 2009]. For optimal recognition results, examples drawn by 5 or 6 people are needed, with each person drawing at least four examples of each syntax type.…”

Section: Diagram Creation and Editingmentioning

confidence: 99%

Combining Sketching and Traditional Diagram Editing Tools

Plimmer

Delaney

Rodgers

2015

ACM Trans. Intell. Syst. Technol.

View full text Add to dashboard Cite

The least cognitively demanding way to create a diagram is to draw it with a pen. Yet there is also a need for more formal visualizations, that is diagrams created using both traditional keyboard and mouse interaction. Our objective is to allow the creation of diagrams using traditional and stylus-based input. Having two diagram creation interfaces requires that changes to a diagram should be automatically rendered in the other visualization. Because sketches are imprecise, there is always the possibility that conversion between visualizations results in a lack of syntactic consistency between the two visualizations. We propose methods for converting diagrams between forms, checking them for equivalence and rectifying inconsistencies. As a result of our theoretical contributions, we present an intelligent software system allowing users to create and edit diagrams in sketch or formal mode. Our proof of concept tool supports diagrams with connected and spatial syntactic elements. Two user studies show that this approach is viable and participants found the software easy to use. We conclude that supporting such diagram creation is now possible in practice.

show abstract

“…Studies have shown that in situ collection of data can have a significant positive effect on recognizer accuracy (Schmieder et al, 2009), especially when user data is included in training (Field et al, 2009). Therefore, although most of our data are collected using this method, one set is similar to most other publically available data set of individual shapes.…”

Section: Data Collectionmentioning

confidence: 99%

RATA.Gesture: A gesture recognizer developed using data mining

Chang

Blagojevic

Plimmer

2012

AIEDAM

View full text Add to dashboard Cite

Although many approaches to digital ink recognition have been proposed, most lack the flexibility and adaptability to provide acceptable recognition rates across a variety of problem spaces. This project uses a systematic approach of data mining analysis to build a gesture recognizer for sketched diagrams. A wide range of algorithms was tested, and those with the best performance were chosen for further tuning and analysis. Our resulting recognizer, RATA.Gesture, is an ensemble of four algorithms. We evaluated it against four popular gesture recognizers with three data sets; one of our own and two from other projects. Except for recognizer-data set pairs (e.g., PaleoSketch recognizer and PaleoSketch data set) the results show that it outperforms the other recognizers. This demonstrates the potential of this approach to produce flexible and accurate recognizers.

show abstract

The effect of task on classification accuracy: Using gesture recognition techniques in free-sketch recognition

Cited by 15 publications

References 16 publications

Using speech to identify gesture pen strokes in collaborative, multimodal device descriptions

Using speech to identify gesture pen strokes in collaborative, multimodal device descriptions

Combining Sketching and Traditional Diagram Editing Tools

RATA.Gesture: A gesture recognizer developed using data mining

Contact Info

Product

Resources

About