A series of novel Li+ doped Bi3.46Ho0.04Yb0.5Ti3O12 (BHYTO: xLi, 0 ≤ x ≤ 0.15) upconversion phosphors were prepared through a sol-gel-sintering method. There exist three emission bands centered at 545 nm, 658 nm, and 756 nm in the upconversion emission spectra at 980 nm excitation, corresponding to energy transitions of 5F4/5S2 → 5I8, 5F5 → 5I8 and 5F4/5S2 → 5I7 of Ho3+, and the upconversion emission intensity of BHYTO: 0.05Li is about 2.2 times stronger than that of BHYTO samples. The luminescent lifetime of the strongest emission (545 nm) is in the range of 45.25 to 65.99 μs for the different BHYTO: xLi phosphors. The energy transfers during the upconversion pumping process from Yb3+ to Ho3+ are mainly responsible for all the emissions, each belonging to a double-photon process. Li+ mainly entered into the interspace sites or occupied Bi3+ sites in Bi4Ti3O12 host during the fabrication process according to its dosage, and the possibility is very low for Li+ to take part in the energy transfer process directly due to its lack of matching levels with 4f of Ho3+ and Yb3+. However, Li+ doping can not only increase the size of crystal grains to improve crystallinity through XRD analysis, but also reduced oxygen vacancies to decrease the number of quenching centers through XPS analysis. The improved crystallinity and reduced quenching centers are proposed to be the main causes for the enhanced upconversion luminescence of the Li+ doped BHYTO phosphor.