Four experiments examined how faces compete with physically salient stimuli for the control of attention in 4-month-old, 6-month-old, and 8-month-old infants (N = 117 total). Three computational models were used to quantify physical salience. We presented infants with visual search arrays containing a face and familiar object(s), such as shoes and flowers. Six- and 8-month-old infants looked first and longest at faces; their looking was not strongly influenced by physical salience. In contrast, 4-month-old infants showed a visual preference for the face only when the arrays contained 2 items and the competitor was relatively low in salience. When the arrays contained many items or the only competitor was relatively high in salience, 4-month-old infants’ looks were more often directed at the most salient item. Thus, over ages of 4 to 8 months, physical salience has a decreasing influence and faces have an increasing influence on where and how long infants look.