Context aware 3D gesture recognition for games and virtual reality

LaViola, Joseph J.

doi:10.1145/2776880.2792711

Cited by 26 publications

(7 citation statements)

References 150 publications

(123 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, we go over the design of such interfaces, their deployment in field experiments, and discuss how their operations work in the field. Our interfaces drew from existing research on human and robot interactions (Sakamoto et al, 2009;Cacace et al, 2016;LaViola, 2015;Bashyal and Venayagamoorthy, 2008), iterating upon them to meet the large scale needs of the field experiment.…”

Section: The Developed User Interfacesmentioning

confidence: 99%

“…Regarding our sketch classifier, the work done in Taranta et al (2017) was utilized as it is shown as adaptable to many forms of signal input and relies on little training data, ideal for our combination of 2D and 3D gesture data. Where hand tracking was available, such as in the augmented reality interface, we utilized principles outlined in LaViola (2015) as the ability to navigate and control virtual assets in VR could be translated well to the swarm network.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Command and Control of a Large Scale Swarm Using Natural Human Interfaces

Williamson¹,

Taranta²,

Moolenaar³

et al. 2023

View full text Add to dashboard Cite

While swarm robotics contain varying levels of autonomy, it is still important for human operators to command and control the swarm before and during mission operations. As such, humanin- the-loop interfaces must be developed that are intuitive to the operator and able to relay accurate commands to flight and ground robots in a large-scale swarm network. Furthermore, the tremendous amount of information gathered by the swarm must be sorted and presented back to the user in a way that reduces cognitive load so that they may determine the relevancy of the information and its effect on the mission. In this paper, we present sketch-based and augmented reality interfaces that both allow for command and control (C2) of unmanned ground and flight vehicles and for processing of data that the swarm has gathered in the field. These interfaces were tested in two field experiments with multiple aerial and ground robots deployed on a mission. Finally, we discuss the results of the field experiments, lessons learned, and areas for future work.

show abstract

Section: The Developed User Interfacesmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Command and Control of a Large Scale Swarm Using Natural Human Interfaces

Williamson¹,

Taranta²,

Moolenaar³

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Step (2), the first stage-classifier is trained, while the rest of stage-classifiers are trained one by one in Step (3). During training of the first stage, the initial negatives are randomly cropped, and all the rest are acquired using hard example mining techniques (Step (2.4)).…”

Section: Multiresolution Hog Feature For Different Stage-mentioning

confidence: 99%

“…Hand detection refers to determining the hands location and their shapes. It works as a prerequisite step for various hand gesture recognition systems [1,2] that have been widely studied, due to their potential application in entertainment and virtual reality [3], medical systems, and assistive technologies, as well as in crisis management and disaster relief [4]. However, hand detection is never an easy task due to the hand deformation [5], the sensitivity of skin colors to lighting conditions [6], and the complicated environments for practical applications.…”

Section: Introductionmentioning

confidence: 99%

Hand Detection Using Cascade of Softmax Classifiers

Zhao

Zheng

Song

2018

Advances in Multimedia

View full text Add to dashboard Cite

Sliding-window based multiclass hand posture detections are often performed by detecting postures of each predefined category using an independent detector, which makes it lack efficiency and results in high postures confusion rates in real-time applications. To tackle such problems, in this work, an efficient cascade detector that integrates multiple softmax-based binary (SftB) models and a softmax-based multiclass (SftM) model is investigated to perform multiclass posture detection in parallel. The SftB models are used to distinguish the predefined postures from the background regions, and the SftM model is applied to discriminate among all the predefined hand posture categories. Another usage of the cascade structure is that it could effectively decompose the complexity of background pattern space and therefore improve the detection accuracy. In addition, to balance the detection accuracy and efficiency, the HOG features of increasing resolutions will be adopted by classifiers of increasing stage-levels in the cascade structure. The experiments are implemented under various scenarios with complicated background and challenging lightings. Results show the superiority of the proposed SftB classifiers over the traditional binary classifiers such as logistic regression, as well as the accuracy and efficiency improvements brought by the softmax-based cascade architecture compared with the noncascade multiclass softmax detectors.

show abstract

“…Thus, the preferred method of interaction during analysis and human insight phases prior to or after distributed processing of the evidence and event reconstruction algorithms. Virtual and Augmented Reality Virtual Reality (VR) [45][46][47] is defined in [48] as "a three-dimensional simulation of the real world or an imaginary world allowing the user to have a sense of physical presence and to manipulate 3D objects, in real-time, inside three-dimensional computer-generated environments." In [49], authors point out the possibility of exhibiting concepts that a user might not be able to view otherwise and the immersive nature of VR can aid in education thus, can be inferred for investigators as well.…”

Section: Illimitable Space System (Iss)mentioning

confidence: 99%

Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

Mokhov,

Song,

Singh

et al. 2018

Preprint

View full text Add to dashboard Cite

Visualization requirements in Forensic Lucid have to do with different levels of case knowledge abstraction, representation, aggregation, as well as the operational aspects as the final long-term goal of this proposal. It encompasses anything from the finer detailed representation of hierarchical contexts to Forensic Lucid programs, to the documented evidence and its management, its linkage to programs, to evaluation, and to the management of GIPSY software networks. This includes an ability to arbitrarily switch between those views combined with usable multimodal interaction. The purpose is to determine how the findings can be applied to Forensic Lucid and investigation case management. It is also natural to want a convenient and usable evidence visualization, its semantic linkage and the reasoning machinery for it. Thus, we propose a scalable management, visualization, and evaluation of digital evidence using the modified interactive 3D documentary system -Illimitable Space System -(ISS) to represent, semantically link, and provide a usable interface to digital investigators that is navigable via different multimodal interaction techniques using Computer Vision techniques including gestures, as well as eye-gaze and audio.

show abstract

Context aware 3D gesture recognition for games and virtual reality

Cited by 26 publications

References 150 publications

Command and Control of a Large Scale Swarm Using Natural Human Interfaces

Command and Control of a Large Scale Swarm Using Natural Human Interfaces

Hand Detection Using Cascade of Softmax Classifiers

Toward Multimodal Interaction in Scalable Visual Digital Evidence Visualization Using Computer Vision Techniques and ISS

Contact Info

Product

Resources

About