With a plethora of wearable IoT devices available today, we can easily monitor human activities, many of which are unconscious or subconscious. Interestingly, some of these activities exhibit distinct patterns for each individual, which can provide an opportunity to extract useful features for user authentication. Among those activities, walking is one of the most rudimentary and mundane activity. Considering each individual's unique walking pattern, gait, which is the pattern of limb movements during locomotion, can be utilized as a biometric feature for user authentication. In this paper, we propose a lightweight seamless authentication framework based on gait (LiSA-G) that can authenticate and identify users on the widely available commercial smartwatches. Unlike the existing works, our proposed framework extracts not only the statistical features but also the human-action-related features from the collected sensor data in order to more accurately and efficiently reveal distinct patterns. Our experimental results show that our framework achieves a higher authentication accuracy (i.e., an average equal error rate (EER) of 8.2%) in comparison with the existing works while requiring fewer features and less amount of sensor data. This makes our framework more practical and rapidly deployable in the wearable IoT systems with limited computing power and energy capacity.