We investigate the microstructure and optical properties of Ge films on Si substrates prepared at low temperature by DC magnetron sputtering and the effect of in situ annealing on them. With increasing growth temperature, Ge films undergo a transition from amorphous to microcrystalline, then to polycrystalline. After annealing, these thin films transform into polycrystalline films with the (111) preferred orientation and identical crystal sizes. The surfaces of the amorphous and microcrystalline Ge films are severely coarsened, whereas the polycrystalline Ge film still displays a smooth surface. The growth mechanisms of Ge films with different crystalline phases in the annealing process are discussed, which can explain their morphology evolutions. Additionally, their infrared absorptions are enhanced after annealing, and this is useful for fabricating high-efficiency Si-based solar cells.