Deep learning has aided in the improvement of diagnosis identification, evaluation, and the interpretation of muscle ultrasound images, which may benefit clinical personnel. Muscle ultrasound images presents challenges such as low image quality due to noise, insufficient data, and different characteristics between skeletal and smooth muscles that can affect the effectiveness of deep learning results. From 2018 to 2020, deep learning has the improved solutions used to overcome these challenges; however, deep learning solutions for ultrasound images have not been compared to the conditions and strategies used to comprehend the current state of knowledge for handling skeletal and smooth muscle ultrasound images. This study aims to look at the challenges and trends of deep learning performance, especially in regard to overcoming muscle ultrasound image problems such as low image quality, muscle movement in skeletal muscles, and muscle thickness in smooth muscles. Skeletal muscle segmentation presents difficulties due to the regular movement of muscles and resulting noise, recording data through skipped connections, and modified layers required for upsampling. In skeletal muscle classification, the problems faced are area-specific, thus making a cropping strategy useful. Furthermore, there is no need to add additional layer modifications for smooth muscle segmentation as muscle thickness is the main problem in such cases.