Camera mouse implementation using 3D head pose estimation by monocular video camera and 2D to 3D point and line correspondences

Nabati, Masoomeh; Behrad, Alireza

doi:10.1109/istel.2010.5734136

Cited by 10 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The machine learning model is trained manually using Adaboost algorithm [15] based on the skin colour feature. Point feature is tracked over the video frame by Lucas Kanade algorithm [16] & rotation matrix and translation vector is used to calculate the location of pointer for creating click events [17]. A system is proposed in [18], where head movement is used to control the cursor and speech recognition is used to perform mouse action.…”

Section: A Interaction With Desktop Computermentioning

confidence: 99%

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Islam¹,

Aadeeb²,

Khan³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Human-mobile interaction is aimed at facilitating interaction with the smartphone devices. The conventional way to interact with mobile devices is through manual input where most of the applications are made assuming that the end user has full control over their hand movements. However, this assumption excludes people who are unable to use their hands or have suffered limb damage. In this paper, we proposed a nose and teeth based interaction system, which allows the users to control their mobile devices completely hands free. The proposed system uses the front facing camera of the smartphone to track the position of the nose for cursor control on the smartphone screen. The system detects teeth for performing the touch screen events such as tap, scroll, long press, and drag. Viola-Jones algorithm is used to detect the face and teeth based on the Haar features. After detecting the face, the nose position is calculated and tracked continuously using Lucas Kanade's method for optical flow estimation. All the touch screen events have been implemented in the system so that the user can execute all the operations of the smartphone. To evaluate the performance and the effect of (smartphone) device type on the execution time, the proposed system was installed in 3 smartphone devices and 7 trials for each device were performed by 3 different able-bodied elderly persons. The result shows a significant success rate for the detection of nose and teeth, and for the execution of the operations. The execution time of each operation slightly varies by 0.72s on average because of the configuration of the smartphones.INDEX TERMS HCI, human-mobile interaction, gesture operations, disabled user, accessibility, mobile device, smartphone.

show abstract

Section: A Interaction With Desktop Computermentioning

confidence: 99%

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Islam¹,

Aadeeb²,

Khan³

et al. 2021

IEEE Access

View full text Add to dashboard Cite

show abstract

“…To calculate the translation vector we present the problem as optimization problem as follow [26]: 19) where (x j ,y j ) is the estimated coordinates of virtual points, N is the number of virtual points, and ) , ( j j y x are calculated using the following equation: 20) where F presents the lens distortion function and [X hj , Y hj , Z hj ] are the 3D coordinates of virtual points in head coordinate system. By solving the optimization problem of Eq.…”

Section: B) Estimating Translation Vectormentioning

confidence: 99%

“…In our previous work [20] we discussed about the algorithm for the calculation of 3D head pose using four artificial rectangular points on the face. To remove the need for marking artificial points on the face and increasing the accuracy of the camera mouse, in this work we utilized a new algorithm to obtain the coordinates of the four arbitrary virtual points on the face area in each frame.…”

mentioning

confidence: 99%

“…However the precise detection and tracking of these points is mandatory for the robust camera mouse implementation. In our previous work [20] we used four artificial points for this purpose, however to increase the efficiency of algorithm we used four arbitrary virtual and rectangular points on the face in this work. The points are virtual and their coordinates are arbitrary on the face.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Robust facial 2D motion model estimation for 3D head pose extraction and automatic camera mouse implementation

Nabati

Behrad

2010

2010 5th International Symposium on Telecommunications

Self Cite

View full text Add to dashboard Cite

In this paper, we present a novel approach to 3D head pose estimation from monocular camera images for the control of mouse pointer movements on the screen and clicking events. This work is motivated by the goal of providing a noncontact instrument to control the mouse pointer on a PC system for handicapped people with severe disabilities using low-cost and widely available hardware. The required information is derived from video data captured using a monocular web camera mounted on the computer monitor. Our approach proceeds in six stages. First, the face area is extracted using Haar-like features and AdaBoost algorithm. Second, the locations of the point features are detected and tracked over video frames by LK algorithm. Third, the 2D transformation model between consecutive frames is estimated by matching features and robust RANSAC algorithm. Fourth, the estimated 2D transformation model is applied to four supposed points on the face area. Then, the 3D rotation matrix and translation vector between the web camera and 3D head pose are estimated using four points correspondences. Finally, the 3D rotation and translation matrix is applied for estimating the mouse pointer movements on the PC screen and clicking events. Experimental results showed the promise of the algorithm.

show abstract

“…3 Most successful eye-tracking techniques in commercial areas pose either the requirement of a high-cost image capturing device (e.g., camera and lens) or very limited operation within a strongly controlled situation. 4,5 Recently, much interest in eye-tracking technology is being generated in the areas of HCI for web usability, advertising, smart TV, and mobile applications. However, the high cost or the limitation of conventional eye-tracking techniques can hardly be employed in low-cost commercial applications.…”

Section: Introductionmentioning

confidence: 99%

Evolutionary adaptive eye tracking for low-cost human computer interaction applications

Shen

Shin

Sung

et al. 2013

J. Electron. Imaging

View full text Add to dashboard Cite

We present an evolutionary adaptive eye-tracking framework aiming for low-cost human computer interaction. The main focus is to guarantee eye-tracking performance without using high-cost devices and strongly controlled situations. The performance optimization of eye tracking is formulated into the dynamic control problem of deciding on an eye tracking algorithm structure and associated thresholds/parameters, where the dynamic control space is denoted by genotype and phenotype spaces. The evolutionary algorithm is responsible for exploring the genotype control space, and the reinforcement learning algorithm organizes the evolved genotype into a reactive phenotype. The evolutionary algorithm encodes an eye-tracking scheme as a genetic code based on image variation analysis. Then, the reinforcement learning algorithm defines internal states in a phenotype control space limited by the perceived genetic code and carries out interactive adaptations. The proposed method can achieve optimal performance by compromising the difficulty in the real-time performance of the evolutionary algorithm and the drawback of the huge search space of the reinforcement learning algorithm. Extensive experiments were carried out using webcam image sequences and yielded very encouraging results. The framework can be readily applied to other low-cost vision-based human computer interactions in solving their intrinsic brittleness in unstable operational environments.

show abstract

Camera mouse implementation using 3D head pose estimation by monocular video camera and 2D to 3D point and line correspondences

Cited by 10 publications

References 18 publications

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Developing a Novel Hands-Free Interaction Technique Based on Nose and Teeth Movements for Using Mobile Devices

Robust facial 2D motion model estimation for 3D head pose extraction and automatic camera mouse implementation

Evolutionary adaptive eye tracking for low-cost human computer interaction applications

Contact Info

Product

Resources

About