Background: People living with HIV (PLHIV) with latent tuberculosis infection (LTBI) are at increased risk of tuberculosis (TB) reactivation compared to the HIV-negative population. Lithuania belongs to the 18 high-priority TB countries in the European region. The aim of this study was to compare the prevalence of LTBI and LTBI-related risk factors between PLHIV and HIV-uninfected populations. Methods: A cross-sectional study was conducted in three Lithuanian Infectious Diseases centres from August 2018 to May 2022 using the interferon gamma release assay (IGRA) and tuberculin skin test (TST) in Vilnius, and IGRA only in Siauliai and Klaipeda. Cohen’s kappa was used to assess IGRA and TST agreement. A structured questionnaire was completed by the study participants. LTBI-related risk factors were identified using a multivariable logistic regression model. Results: In total, 391 PLHIV and 443 HIV-uninfected individuals enrolled, with a median age of 41 (IQR 36–48) and 43 (IQR 36–50), consisting of 69.8% and 65.5% male, respectively. The prevalence of LTBI defined by positive IGRA and/or TST among PLHIV was higher compared to that in the HIV-uninfected population (20.5% vs. 15.3%; OR 1.42; 95% CI 1.02–2.03; p = 0.04). The concordance between IGRA and TST was fair: kappa = 0.23 (95% CI 0.09–0.34). In multivariable analyses, association with injecting drug use (IDU) (ORa 2.25, 95% CI 1.27–3.99, p = 0.01) and imprisonment (ORa 1.99, 95% CI 1.13–3.52, p = 0.02) in all participants, IDU (ORa 2.37, 95% CI 1.09–5.15; p = 0.029) in PLHIV and a history of contact with an active TB patient (ORa 3.33, 95% CI 1.53–7.24; p = 0.002) in HIV-uninfected individuals were significant associations evidenced by LTBI. Conclusions: The prevalence of LTBI among PLHIV in Lithuania is higher compared to that in the HIV-uninfected population and the European average. The association with IDU in PLHIV emphasizes the need for integrated HIV, TB and substance abuse treatment to provide patient-centred care.