The development of new materials for shortwavelength infrared (SWIR) optical sensors is of high importance for the fast development of different applications, as, for example, Internet of Things, road safety, and pollution monitoring. Group IV SiGe provides more sustainable As-, Cd-, and Pb-free nanomaterials that are cheaper and ecologic and offer easy integration with CMOS technology. This Review is on Ge and SiGe quantum dots/nanocrystals (QDs/NCs) embedded in dielectrics for VIS-SWIR photodetection, in which we highlight and discuss photocurrent mechanisms, correlation of photodetection parameters and characteristics with crystalline structure, morphology and energy bandgap, and applications as photodetectors, optical sensors, phototransistors, and solar cells. The embedding matrix induces NC surface passivation, stress field, and nanocrystallization effects and brings specific advantages depending on the matrix material. SiGe NCs in oxides for VIS-SWIR sensing represents a niche domain, showing high photosensitivity (photocurrent) in SWIR up to 1.8 μm at room temperature and 2 μm at 100 K, deeper in SWIR than Ge. By alloying Ge with a small content of Si, NC thermal stability is much improved as the detrimental Ge fast diffusion in oxides is hindered and SWIR photosensing is enhanced due to light absorption in Ge-rich SiGe NCs.