Commonly used sensors like accelerometers, gyroscopes, surface electromyography sensors, etc., which provide a convenient and practical solution for human activity recognition (HAR), have gained extensive attention. However, which kind of sensor can provide adequate information in achieving a satisfactory performance, or whether the position of a single sensor would play a significant effect on the performance in HAR are sparsely studied. In this paper, a comparative study to fully investigate the performance of the aforementioned sensors for classifying four activities (walking, tooth brushing, face washing, drinking) is explored. Sensors are spatially distributed over the human body, and subjects are categorized into three groups (able-bodied people, stroke survivors, and the union of both). Performances of using accelerometer, gyroscope, sEMG, and their combination in each group are evaluated by adopting the Support Vector Machine classifier with the Leave-One-Subject-Out Cross-Validation technique, and the optimal sensor position for each kind of sensor is presented based on the accuracy. Experimental results show that using the accelerometer could obtain the best performance in each group. The highest accuracy of HAR involving stroke survivors was 95.84 ± 1.75% (mean ± standard error), achieved by the accelerometer attached to the extensor carpi ulnaris. Furthermore, taking the practical application of HAR into consideration, a novel approach to distinguish various activities of stroke survivors based on a pre-trained HAR model built on healthy subjects is proposed, the highest accuracy of which is 77.89 ± 4.81% (mean ± standard error) with the accelerometer attached to the extensor carpi ulnaris.