Continuous patient monitoring of newborns in the neonatal intensive care unit (NICU) is often performed with wired sensors which can be cumbersome, can interfere with parental bonding, and can irritate the patient's fragile skin. Non-contact video-based patient monitoring systems are therefore a preferrable solution. While a multitude of high-performing machine vision technologies have been successfully implemented on an adult population, such methods often fail in neonatal population. In this thesis, we assess state-of-the-art adult-based methods to bridge the gap to an understudied neonatal population in the NICU environment. To this end, several important machine vision concepts are investigated, including scene understanding, image classification, face detection, semantic segmentation, motion detection, face tracking, and heart rate estimation. In each of these areas, we assess the state-of-the-art and identify its applicability to a neonatal population. In cases where serious limitations are observed, this thesis pushes the state-of-the-art