Residential and nonresidential fathers are taking an increasingly greater role in their children’s lives, and father involvement predicts positive child outcomes across development. This study utilized data from a large sample of racially and ethnically diverse fathers of low to middle income (n = 1,112) to test if paternal stress is associated with lower father involvement in early infancy and whether perceived social support is protective. Exploratory analyses examined whether the effects of stress on father involvement differ depending on whether a father lives with his child. Multiple regression analyses tested associations between three different forms of stress (perceived general stress, financial stress, major life events) and father involvement measured as time spent with infant, father confidence, and father provision of tangible support, and to examine whether social support moderated the relation. Controlling for residential status, results showed that perceived and financial stress in fathers were significantly associated with lower father confidence in parenting and lower provision of tangible support. In addition, higher paternal perceived stress related to less time spent with infants. Some patterns differed for residential versus nonresidential fathers. Further, only perceived available affectional social support emerged as protective. These results highlight the importance of identifying the unique types of stress that affect father involvement in the first few months of life, what may be protective, and a possible need to distinguish between residential and nonresidential fathers as they enter parenthood.