Smartphones have ubiquitously integrated into our home and work environments, however, the user normally relies on explicit but inefficient identification processes in a controlled environment. Therefore, when the device is stolen, the attacker can have access to the user's personal information and services against the stored password/s. As a result of this potential scenario, this work demonstrates the possibilities of legitimate user identification in a semicontrolled environment through the built-in smartphones motion dynamics captured by two different sensors. This is a two-fold process, sub-activity recognition based user identification using the wavelet kernel extreme learning machine (KELM) to first recognize the performed activity and then identifies whether the recognized activity is performed by the legitimate user or impostor. Prior to the identification; Extended Sammon Projection (ESP) method has been deployed to reduce the redundancy between the features. To validate the proposed system, we first collected data from 20 users walking with their device freely placed in one of their pants pockets (front right, front left, back right and back left pocket). Through extensive experimentation's using overall and averageone-subject-cross-validation, we demonstrate that together time and frequency domain features optimized by ESP to train the KELM is an effective system to identify the legitimate user or an impostor with 97 − 98% overall accuracy.