HMOG: New Behavioral Biometric Features for Continuous Authentication of Smartphone Users

Abstract

We introduce Hand Movement, Orientation, and Grasp (HMOG), a set of behavioral features to continuously authenticate smartphone users. HMOG features unobtrusively capture subtle micro-movement and orientation dynamics resulting from how a user grasps, holds, and taps on the smartphone. We evaluated authentication and biometric key generation (BKG) performance of HMOG features on data collected from 100 subjects typing on a virtual keyboard. Data was collected under two conditions: sitting and walking. We achieved authentication EERs as low as 7.16% (walking) and 10.05% (sitting) when we combined HMOG, tap, and keystroke features. We performed experiments to investigate why HMOG features perform well during walking. Our results suggest that this is due to the ability of HMOG features to capture distinctive body movements caused by walking, in addition to the hand-movement dynamics from taps. With BKG, we achieved EERs of 15.1% using HMOG combined with taps. In comparison, BKG using tap, key hold, and swipe features had EERs between 25.7% and 34.2%. We also analyzed the energy consumption of HMOG feature extraction and computation. Our analysis shows that HMOG features extracted at 16Hz sensor sampling rate incurred a minor overhead of 7.9% without sacrificing authentication accuracy. Two points distinguish our work from current literature: 1) we present the results of a comprehensive evaluation of three types of features (HMOG, keystroke, and tap) and their combinations under the same experimental conditions; and 2) we analyze the features from three perspectives (authentication, BKG, and energy consumption on smartphones).

Convolutional Neural Networks for User Identification Based on Motion Sensors Represented as Images

“…Some works modify the standard pipeline by adding a feature selection step before classification. The approach described by Sitova et al [20] uses Principal Component Analysis for feature selection. The authors extracted statistical features specifically designed for tap gestures on the touchscreen of the mobile device.…”

“…The data recorded by these sensors during the interaction of the user with the mobile device can be used as biometric data to identify the user. Indeed, one-time or continuous user identification based on the data collected by the motion sensors of a mobile device is an actively studied task [8][9][10][11][12][13][14][15][16][17][18][19][20][21][22], that emerged after the integration of motion sensors into commonly used mobile devices.…”

A comprehensive overview of biometric fusion

“…User interaction with the touch screen of their smartphone is analyzed to develop a set of different strokes that are classified using the KNN and SVM classifiers. A combination of hand movement, orientation, and grasp (HMOG) features have also been used to continuously authenticate a user on a smartphone [146]. Data collected from the accelerometer, gyroscope, and magnetometer is combined to perform unobtrusive authentication.…”

On Capturing Older Adults’ Smartphone Keyboard Interaction as a Means for Behavioral Change Under Emotional Stimuli Within i-PROGNOSIS Framework

“…In fact, since the smartphone is embedded with the accelerometer and gyroscope sensor, more information can be used on the pattern analysis [11], [20], [21]. Some of the main keyboard dynamics are based on latencies of the keystrokes (e.g., time between keystrokes or the length of time that each keystroke is pressed), revealing that the typing patterns of the same individuals vary over time and are affected by other factors, such as stress or gradual changes in cognitive or physical function [22]; thus, keyboard dynamics can provide relevant behavioral information about the affective/cognitive state of the user.…”