Improvement of primary care for patients with type 2 diabetes mellitus (T2DM) through the promotion of good knowledge, attitude, and practice is of paramount importance for preventing its related complications. This study aimed to assess the levels of knowledge, attitude, and practice and associated factors among patients with T2DM. This was a cross-sectional multicenter hospital-based study that included 979 patients from 8 health facilities in Tanzania. A standardized semi-structured interviewer-administered questionnaire was used to extract the required data. Factor analysis was used to determine the level of knowledge, attitude, and practice. Multivariable analysis under binary logistic regression analysis was used to determine the predictors of knowledge, attitude, and practice. P<0.05 was considered significant. The levels of adequate knowledge, positive attitude, and appropriate practice were 62.1%, 54%, and 30.9%, respectively. Being self-employed (AOR = 1.74, 95% CI = 0.28–0.91, p = 0.040) predicted adequate knowledge. Being male (AOR = 1.46, 95% CI = 1.06–2.01, p = 0.021 and visiting regional hospitals (AOR = 2.17, 95% CI = 1.33–2.51, p = 0.013) were predictors of positive attitude. Residing in rural areas and not having adequate knowledge of diabetes were less likely associated with appropriate practice. This study has shown a significantly low level of appropriate practice among patients with T2DM towards general issues on diabetes, risk factors, and related complications. Therefore, emphasis should be placed on improving good practices that can help prevent related complications.