Although preterm birth constitutes a risk factor for postpartum depressive symptomatology, perinatal depression (PND) has not been investigated extensively in fathers of very low (VLBW) and extremely low birth weight (ELBW) infants. This study explored paternal depression levels at 3, 9, and 12 months of infant corrected age, investigating also the predictive role played by the severity of prematurity, maternal and paternal PND levels, and parenting stress. We recruited 153 fathers of 33 ELBW, 42 VLBW, and 78 full-term (FT) infants, respectively. Depression was investigated by the Edinburgh Postnatal Depression Scale (EPDS) and distress by the Parenting Stress Index-Short Form-PSI-SF (Total and subscales: Parental Distress, Parent–Child Dysfunctional Interaction, and Difficult Child). ELBW fathers showed a significant decrease (improvement) in EPDS, total PSI-SF, and Parental Distress mean scores after 3 months. Paternal EPDS scores at 12 months were significantly predicted by VLBW and FT infants’ birth weight categories, fathers’ EPDS scores at 3 and 9 months, Parent–Child Dysfunctional Interaction subscale at 3 months, and Difficult Child subscale at 9 months. This study strengthens the relevance of including early routine screening and parenting support for fathers in perinatal health services, with particular attention to fathers who might be more vulnerable to mental health difficulties due to severely preterm birth.