Phenolamides (PAs) are specialized (secondary) metabolites mainly synthesized by BAHD N-acyltransferases. Here, we report metabolic profiling coupled with association and linkage mapping of 11 PAs in rice (Oryza sativa). We identified 22 loci affecting PAs in leaves and 16 loci affecting PAs in seeds. We identified eight BAHD N-acyltransferases located on five chromosomes with diverse specificities, including four aromatic amine N-acyltransferases. We show that genetic variation in PAs is determined, at least in part, by allelic variation in the tissue specificity of expression of the BAHD genes responsible for their biosynthesis. Tryptamine hydroxycinnamoyl transferase 1/2 (Os-THT1/2) and tryptamine benzoyl transferase 1/2 (Os-TBT1/2) were found to be bifunctional tryptamine/tyramine N-acyltransferases. The specificity of Os-THT1 and Os-TBT1 for agmatine involved four tandem arginine residues, which have not been identified as specificity determinants for other plant BAHD transferases, illustrating the versatility of plant BAHD transferases in acquiring new acyl acceptor specificities. With phylogenetic analysis, we identified both divergent and convergent evolution of N-acyltransferases in plants, and we suggest that the BAHD family of tryptamine/tyramine N-acyltransferases evolved conservatively in monocots, especially in Gramineae. Our work demonstrates that omics-assisted gene-to-metabolite analysis provides a useful tool for bulk gene identification and crop genetic improvement.