Tuberculosis (TB) represents a health problem in Colombia, and its control is focused on the search for contacts and treatment of TB cases underscoring the role of latent tuberculosis infection (LTBI) as a reservoir of Mycobacterium tuberculosis. The burden of LTBI in Colombia is unknown. We aimed to estimate the prevalence of LTBI and identify the associated risk factors. In this cross-sectional study, we recruited participants from four health care centers in Cali, Colombia. The participants were eligible if they were aged between 14 and 70 years, and all participants answered a survey evaluating their medical history and sociodemographic and lifestyle factors. LTBI status was based on tuberculin skin test (TST) positivity using two thresholds: ≥10 mm (TST-10) and ≥15 mm (TST-15). The magnitude of the associations between independent factors and dependent outcomes (LTBI status and TST induration) were evaluated by logistic regression and generalized linear models, respectively. A total of 589 individuals were included with TST positivity rates of 25.3% (TST-10) and 13.2% (TST-15). Logistic regression showed that being between age 40 and 69 years (OR = 7.28, 95% CI [1.62–32.7]), being male (OR = 1.71, 95% CI [1.04–2.84]), being employed (OR = 1.56, 95% CI [1.02–2.38]), and having a low intake of alcohol (OR = 2.40, 95% CI [1.13–5.11]) were risk factors for TST positivity, while living in the north zone (OR = 0.32, 95% CI [0.18–0.55]), living in the suburb zone (OR = 0.28, 95% CI [0.15–0.52]) and having a secondary education (OR = 0.49 95% CI [0.29–0.83]) lowered the risk of TST positivity. The generalized linear model showed that the previous predictors, as well as a low body mass index, had an effect on TST reaction size. The LTBI prevalence found in the population was moderate, reflecting the continuous transmission of M. tuberculosis. Social factors seem to play a decisive role in the risk of LTBI. Employed males, who are over 40 years of age, are overweight, have a lower level of education and have a low intake of alcohol (50–100 mL, once/week) should be a priority group for prophylactic treatment as a strategy for TB control in this city.