This paper explores poly-silicon-germanium (poly-SiGe) avalanche photo-sensors (APSs) involving a device of heterojunction structures. A low pressure chemical vapor deposition (LPCVD) technique was used to deposit epitaxial poly-SiGe thin films. The thin films were subjected to annealing after the deposition. Our research shows that the most optimal thin films can be obtained at 800 °C for 30 min annealing in the hydrogen atmosphere. Under a 3-μW/cm2 incident light (with a wavelength of 550 nm) and up to 27-V biased voltage, the APS with a n+-n-p-p+ alloy/SiO2/Si-substrate structure using the better annealed poly-SiGe film process showed improved performance by nearly 70%, 96% in responsivity, and 85% in quantum efficiency, when compared to the non-annealed APS. The optimal avalanche multiplication factor curve of the APS developed under the exponent of n = 3 condition can be improved with an increase in uniformity corresponding to the APS-junction voltage. This finding is promising and can be adopted in future photo-sensing and optical communication applications.