Cortical visual processing transforms features of the external world into increasingly complex and specialized neuronal representations. These transformations arise in part through target-specific routing of information; however, within-area computations may also contribute to area-specific function. Here, we sought to determine whether higher-order visual cortical areas LM, AL, PM, and AM have specialized anatomical and physiological properties by using a combination of whole-cell recordings and optogenetic stimulation of V1 axons in vitro. We discovered area-specific differences in the strength of recruitment of interneurons through feed-forward and recurrent pathways, as well as differences in cell-intrinsic properties and interneuron densities. These differences were most striking when comparing across medial and lateral areas, suggesting that these areas have distinct profiles for net excitability and integration of V1 inputs. Thus, cortical areas are not defined simply by the information they receive, but also by area-specific circuit properties that enable specialized filtering of these inputs.