To explore the correlation between Fried Frailty Phenotype (FFP) and the muscle thickness and quality of local muscle, and to provide a reasonable basis for the application of ultrasound measurement in the frailty assessment. A total of 150 people (age ≥ 65 years, 58 women, 92 men) were included from the First Hospital Affiliated to Nanjing Medical University. They were divided into Normal group (40 cases), Prefrailty group (69 cases) and Frailty group (41 cases). The thickness and the quality of local muscle were detected by ultrasound. Participants in the prefrailty group had a higher grayscale value of the vastus lateralis muscle, indicating the deterioration of muscle quality. At the frailty stage, the muscle thickness and quality of the vastus lateralis muscle and the anterior tibialis muscle decreased significantly compared with the normal and the prefrailty group. Pearson's correlation analysis also showed FFP was negatively correlated with muscle thickness and quality of the lower limbs. In multiple regression model, FFP was positively associated with gray value (Vastus lateralis muscle:β =0.457, p < 0.001; Anterior tibialis muscle: β = 0.220, p = 0.037) and inversely associated with muscle thickness (Vastus lateralis muscle:β = −0.973, p = 0.031; Anterior tibialis muscle: β = −4.551, p = 0.004) in the frailty stage. Together, FFP was closely related to muscle thickness and quality, especially vastus lateralis muscle. Moreover, Muscle quality has deteriorated in the prefrailty stage, which is earlier than muscle thickness.