Mouse Operations using Finger Tracking

Abstract

Computers are the electronic brains of the era, technology's greatest gift to mankind. The functioning of these electronic brains is controlled by humans. Interaction between the computer and humans is possible with the help of various devices such as keyboard, mouse, joystick, light pen, track ball, barcode reader, etc. Of these devices, the mouse performs various crucial functions in the most user-friendly way. This paper conveys work to implement the same functionalities without the use of external, bulky devices. Hand gestures can be used for natural and intuitive interaction of the user with a computer [1]. This paper intends to replicate different hand gestures as mouse functionalities using image processing tools and techniques. General Termscursor movements, hand gestures, morphological operations.

Real-time virtual mouse system using RGB-D images and fingertip detection

“…In recent years, traditional camera-based approaches that detect the area of the hand and recognize hand gestures have been developed [1,2,6,13,17,19,21,24,25,31,34]. These approaches had obvious detection difficulty when the light levels were changed or a complex background was used and required a fixed distance from the camera to the users.…”

“…Real-time fingertip detection and tracking can be applied in computer vision using virtual mice [1,3,15,17,19,25,31]. Despite significant improvements in recent years, virtual mouse systems are limited in certain aspects.…”

“…Despite significant improvements in recent years, virtual mouse systems are limited in certain aspects. The approaches in [1,17,19,25,31] use complex models and achieve real-time performance; however, they are limited by complex backgrounds, low light levels, and the distance from the camera to the hand. In [3], users must wear color pointers for finger tracking, and the mouse control systems are based on the color detection.…”

“…In this experiment, ten subjects made various rapid gestures to evaluate the detection accuracy. The dataset was recorded with various size of monitor resolution to prove that our model is more compatible with real application, instead of using a fit resolution as presented in [1,17,19,25,31]. There are four computer resolutions as follows: 1280 × 1024 with 200 cases, followed by 1600 × 1200 100 cases, 1680 × 1050 200 cases and finally 1900 × 1200 100 cases.…”

“…The details of the gesture comparisons are listed in Table 2. Table 2, it can be seen that the main drawback of [1,17,19,25] is that they use a traditional camera with RGB image. Therefore, their systems only work with an unchanging background and a fixed distance, while our proposed system and [15] can overcomes these disadvantages by using the RGB-D Kinect sensor.…”