Basketball is a sport that requires high athletes’ skills and physical fitness and is deeply loved by the people in our country. This paper studies the application of neural network-based motion sensors in basketball technology and physical fitness evaluation system. The ideal effect of the system is to scientifically analyze relevant data through intelligent algorithms and provide more accurate diagnosis suggestions. Recognizing human movements requires collecting various data of the human body through motion sensors. The data acquisition components of this system are based on considerations of portability and power consumption and are equipped with equipment with strong computing power to realize the functions of data preprocessing, training, and recognition of the recognition model. The system only needs to send the data in the data collector to the computing device; it can effectively realize the action recognition and judge whether the athlete’s technical action and physical fitness level meet the standard. From the experimental data, the pass rate of the subjects in the 1000-meter run was 83.3%, and the excellent rate was 10%; the pass rate in the 1-mile run was 90%, and the excellent rate was 6.7%; and the pass rate in the 20-meter round trip was only at 56.67%; it can be seen that there is still room for improvement in the reaction speed and agility of most subjects. According to intelligent data analysis, athletes can better understand where they have shortcomings and improve their physical fitness and basketball skills through targeted training.