Top-down attention is hypothesized to dynamically allocate limited neural resources to task-relevant computations. According to this view, sensory neurons are driven not only by stimuli but also by feedback signals from higher brain areas that adapt the sensory code to the goals of the organism and its belief about the state of the environment. Here we formalize this view by optimizing a model of population coding in the visual cortex for maximally accurate perceptual inference at minimal activity cost. The resulting optimality predictions reproduce measured properties of attentional modulation in the visual system and generate novel hypotheses about the functional role of top-down feedback, response variability, and noise correlations. Our results suggest that a range of seemingly disparate attentional phenomena can be derived from a general theory combining probabilistic inference with efficient coding in a dynamic environment.