Cattle temperament has been considered by farmers as a key breeding goal due to its relevance for cattlemen’s safety, animal welfare, resilience, and longevity and its association with many economically important traits (e.g., production and meat quality). The definition of proper statistical models, accurate variance component estimates, and knowledge on the genetic background of the indicator trait evaluated are of great importance for accurately predicting the genetic merit of breeding animals. Therefore, 266,029 American Angus cattle with yearling temperament records (1–6 score) were used to evaluate statistical models and estimate variance components; investigate the association of sex and farm management with temperament; assess the weighted correlation of estimated breeding values for temperament and productive, reproductive efficiency and resilience traits; and perform a weighted single-step genome-wide association analysis using 69,559 animals genotyped for 54,609 single-nucleotide polymorphisms. Sex and extrinsic factors were significantly associated with temperament, including conception type, age of dam, birth season, and additional animal–human interactions. Similar results were observed among models including only the direct additive genetic effect and when adding other maternal effects. Estimated heritability of temperament was equal to 0.39 on the liability scale. Favorable genetic correlations were observed between temperament and other relevant traits, including growth, feed efficiency, meat quality, and reproductive traits. The highest approximated genetic correlations were observed between temperament and growth traits (weaning weight, 0.28; yearling weight, 0.28). Altogether, we identified 11 genomic regions, located across nine chromosomes including BTAX, explaining 3.33% of the total additive genetic variance. The candidate genes identified were enriched in pathways related to vision, which could be associated with reception of stimulus and/or cognitive abilities. This study encompasses large and diverse phenotypic, genomic, and pedigree datasets of US Angus cattle. Yearling temperament is a highly heritable and polygenic trait that can be improved through genetic selection. Direct selection for temperament is not expected to result in unfavorable responses on other relevant traits due to the favorable or low genetic correlations observed. In summary, this study contributes to a better understanding of the impact of maternal effects, extrinsic factors, and various genomic regions associated with yearling temperament in North American Angus cattle.