Smartphones are ubiquitously integrated into our home and work environment and users frequently use them as the portal to cloud-based secure services. Since smartphones can easily be stolen or coopted, the advent of smartwatches provides an intriguing platform legitimate user identification for applications like online banking and many other cloud-based services. However, to access security-critical online services, it is highly desirable to accurately identifying the legitimate user accessing such services and data whether coming from the cloud or any other source. Such identification must be done in an automatic and non-bypassable way. For such applications, this work proposes a two-fold feasibility study; (1) activity recognition and (2) gait-based legitimate user identification based on individual activity. To achieve the above-said goals, the first aim of this work was to propose a semicontrolled environment system which overcomes the limitations of users’ age, gender, and smartwatch wearing style. The second aim of this work was to investigate the ambulatory activity performed by any user. Thus, this paper proposes a novel system for implicit and continuous legitimate user identification based on their behavioral characteristics by leveraging the sensors already ubiquitously built into smartwatches. The design system gives legitimate user identification using machine learning techniques and multiple sensory data with 98.68% accuracy.