Optical fiber Bragg sensor (FBG) has great potential in the field of flexible wearable device for tracking human gestures. Due to the different human size, different wearable tensions cause errors inevitably during tracking joint movements. We have designed a flexible wearable smart sleeve with four heads of FBG and spandex polyurethane fibers. The spandex polyurethane fibers sewed with the Optical fiber Bragg sensor (FBG) has great potential in the field of flexible wearable device for tracking human gestures. Due to the different human size, different wearable tensions cause errors inevitably during tracking arm joint movements. We have designed a flexible wearable smart sleeve with four heads of FBG and spandex polyurethane fibers. The spandex polyurethane fibers sewed with the flexible fabric sleeve convert the elbow yawing, wrist pitching and rolling into the axial strain of FBG. The measuring system has been developed to deduce the personalized sensitivity by a dynamic calibration method. For a male and a female, the dynamic calibration, verification and tracking tests were carried out. From the male’s experimental data, the relative errors between the verification sensitivity and the personalized sensitivity are 1.93%, 5.85% and 7.16%, and the average relative errors between the tracking sensitivity and the personalized sensitivity are 7.09%, 5.58% and 2.52%, respectively. And from the data of the female’s experiment, the relative errors between the verification sensitivity and the personalized sensitivity are 0.25%, 5.0% and 6.75%, and the average relative errors between the tracking sensitivity and the personalized sensitivity are 0.99%, 5.56% and 6.95%, respectively. The experimental data have showed this wearable smart sleeve and the measuring system work well. The research results can be used to develop FBG sensing system of monitoring joint movements for different human size on-line.