This study analyzed the distorted perception of weight in adults aged 20 years or older and investigated the difference in the amount of physical activity and health-related quality of life (HRQOL). This study examined 21,326 adults regarding their body mass index (BMI), subjective body recognition (SBR), physical activity (according to the Global Physical Activity Questionnaire; GPAQ), and HRQOL (EuroQol-5 Dimension; EQ-5D) from the 7th Korea National Health and Nutrition Examination Survey (2016–2018). Independent t-test, analysis of variance (ANOVA), chi-square test, and multiple regression analysis were conducted. The group with the same BMI and SBR significantly showed a long time of activity vigorous-intensity (F = 21.25, p = 0.003) and moderate-intensity time (F = 17.24, p < 0.001). In the ‘BMI = SBR’ group, the sub-group with normal BMI and normal SBR showed the highest vigorous-intensity (mean ± SD = 7.20 ± 26.05, F = 37.86, p < 0.001) and moderate-intensity (mean ± SD = 13.89 ± 30.18, F = 43.27, p < 0.001) activity times. The sub-group with normal BMI and normal SBR had the highest percentage of responding as normal in the five EQ-5D sub-items. For the score of the EQ-5D Index, the highest score was shown in the group that felt subjectively more obese than the actual BMI (F = 56.83, p < 0.001). In the ‘BMI = SBR’ group, these factors related to physical activity (vigorous-intensity, moderated-intensity, and walking) are factors influencing health-related quality of life in this regression model (F = 396.57, p < 0.001, R2 = 0.165). Various health promotion programs and policy recommendations to reduce the distorted perception of weight are required.