Colloidal quantum dots (CQDs) are extremely promising infrared optoelectronic materials for efficient solar cells owing to their strong infrared absorption with tunable spectra. However, the liquid-state ligand exchange of CQDs using ammonium acetate (AA) as an additive generally resulted in intensive charge-transport barriers within the CQD solids. This is induced by the highbandgap PbI2 matrix, which considerably affects the charge-carrier extraction of CQD solar cells (CQDSCs), and thus their photovoltaic performance. Herein, dimethylammonium iodide (DMAI) was used as an additive instead for the liquidstate ligand exchange, substantially eliminating the PbI2 matrix capping the CQDs and simultaneously restraining CQD fusion during the ligand exchange, thereby reducing the barriers for the charge-carrier transport within the CQD solids. Extensive experimental studies and theoretical calculations were performed to link the surface chemistry of the CQDs with the charge-carrier dynamics within the CQD solids and full solar cell devices. The theoretical calculation results reveal that DMAI which possess small dissociation energy could finely regulate the ligand exchange of CQDs, resulting in the suppressed energetic disorder and diminished charge-transport barriers in the CQD solids compared to those of the CQD solids prepared using AA. The DMAI-treated quantum dots were characterized and analyzed by transmission electron microscopy, X-ray photoelectron spectroscopy, and 2D grazing-incidence wide-and small-angle X-ray scattering spectrometry. The results show PbI2-related Bragg peaks in the AA-treated CQD solid films, indicating a thick layer of PbI2 crystal matrix being formed in the CQD solids, whereas there was no obvious PbI2 signal observed in DMAI-treated CQD solids. These results also demonstrate that DMAI provides additional I − , improving the surface passivation of the CQDs and reducing trap-assisted recombination. For the infrared photovoltaic applications, the CQDSC devices were fabricated, which shows that the photovoltaic performance of CQDSCs was significantly improved. The power conversion efficiency of DMAI-based CQDSCs was improved by 17.8% compared with that of the AA-based CQDSC. The charge-carrier dynamics in both CQD solids and full solar cell devices were analyzed in detail, revealing that the improved photovoltaic performance in DMAI-based CQDSCs was attributed to the facilitated charge-carrier transport within the CQD solids and suppressed trap-assisted recombination, resulting from eliminated charge-transport barriers and improved surface passivation of CQDs, respectively. This work provides a new avenue to controlling the surface chemistry of infrared CQDs and a feasible approach to substantially diminishing the charge transfer barriers of CQD solids for infrared solar cells.