Colloidal quantum dots (CQDs) are cutting-edge optoelectronic semiconductor nanocrystals that enable short-wave infrared (SWIR) vision by a widely tunable SWIR light absorption. Thanks to the advances in CQD surface ligand engineering, SWIR detectors and emitters will soon find their way into products. The CQD-based optoelectronic devices are being optimized by adapting the size of CQDs and selection of the ligands, and yet, the measurement schemes of energy band structure based on different ligands and processes of ligand exchange are not systematically studied. In this work, we systematically characterize the energy band structure of PbS (absorbing at different SWIR wavelengths) and InAs with various ligands for both solid-state and liquid-phase ligand exchange (LPLE) processes [solid-state ligand exchange (SSLE) and LPLE] by using ultraviolet photoelectron spectroscopy. The deduced energy band structures reveal that the apparent energy difference between the Fermi and valence band maximum, |E F − E VBM |, largely depends on the physical density and distribution of the CQDs within the probing area. Transmission electron microscopy images, X-ray photoelectron spectroscopy, atomic force microscopy, and variable angle spectroscopic ellipsometry reveal details of the CQD distribution, surface elemental profile, and topologies and how they affect the observed energy band structure. We demonstrate that the multistep coating improves the CQD distribution and packing density, resulting in more reliable and reproducible results that represent the bulk CQD film energy band structure. The comparison of solid and liquid phase ligand-exchanged PbS and InAs SWIR CDQs energetics indicates that the LPLE ensures more uniform dispersion and a high packing density of CQDs regardless of the solution concentration. The photoemissiondeduced energy band structures are validated by fabricating thin-film photodiodes using SWIR SSLE PbS and LPLE In(As,P) CQDs. The Fermi-referenced band structures of the fabricated full photodiode stacks including band offsets and bending are discussed to improve our understanding of the device working principles and to further optimize the devices.