The A allele of the FRAS1-related extracellular matrix protein 3 (FREM3) rs7676614 single nucleotide polymorphism (SNP) was linked to major depressive disorder (MDD) in an early genome-wide association study (GWAS), and to symptoms of psychomotor retardation in a follow-up investigation. In line with significant overlap between age- and depression-related molecular pathways, parallel work has shown that FREM3 expression in postmortem human brain decreases with age. Here, we probe the effect of rs7676614 on amygdala reactivity and perceptual processing speed, both of which are altered in depression and aging. Amygdala reactivity was assessed using a face-matching BOLD fMRI paradigm in 365 Caucasian participants in the Duke Neurogenetics Study (DNS) (192 women, mean age 19.7 ± 1.2). Perceptual processing speed was indexed by reaction times in the same task and the Trail Making Test (TMT). The effect of rs7676614 on FREM3 mRNA brain expression levels was probed in a postmortem cohort of 169 Caucasian individuals (44 women, mean age 50.8 ± 14.9). The A allele of rs7676614 was associated with blunted amygdala reactivity to faces, slower reaction times in the face-matching condition (p < 0.04), as well as marginally slower performance on TMT Part B (p = 0.056). In the postmortem cohort, the T allele of rs6537170 (proxy for the rs7676614 A allele), was associated with trend-level reductions in gene expression in Brodmann areas 11 and 47 (p = 0.066), reminiscent of patterns characteristic of older age. The low-expressing allele of another FREM3 SNP (rs1391187) was similarly associated with reduced amygdala reactivity and slower TMT Part B speed, in addition to reduced BA47 activity and extraversion (p < 0.05). Together, these results suggest common genetic variation associated with reduced FREM3 expression may confer risk for a subtype of depression characterized by reduced reactivity to environmental stimuli and slower perceptual processing speed, possibly suggestive of accelerated aging.