Non-invasive and easy alternative methods to indicate skeletal muscle mass index (SMI) have not been established when dual energy X-ray absorptiometry (DXA) or bioelectrical impedance analysis (BIA) cannot be performed. This study aims to construct a prediction model including gastrocnemius thickness using ultrasonography for skeletal muscle mass index (SMI). Total of 193 Japanese aged ≥65 years participated. SMI was measured by BIA, and subcutaneous fat thickness and gastrocnemius thickness in the medial gastrocnemius were measured by using ultrasonography, and age, gender and body mass index (BMI), grip strength, and gait speed were collected. The stepwise multiple regression analysis was conducted, which incorporated SMI as a dependent variable and age, gender, BMI, gastrocnemius thickness, and other factors as independent variables. Gender, BMI, and gastrocnemius thickness were included as significant factors, and the formula: SMI = 1.27 × gender (men: 1, women: 0) + 0.18 × BMI + 0.09 × gastrocnemius thickness (mm) + 1.3 was shown as the prediction model for SMI (R = 0.89, R2 = 0.8, adjusted R2 = 0.8, p < 0.001). The prediction model for SMI had high accuracy and could be a non-invasive and easy alternative method to predict SMI in Japanese older adults.