Background The diversified and high value-added utilization of cotton by products can promote the sustainable development of modern agriculture. Differences in potential nutrients among varieties can be explained by variations in the composition and abundance of fatty acids, polyphenols, carbohydrates, amino acids, and organic acids. Therefore, the analysis of metabolite species and relationships in cottonseed is meaningful for the development of cotton byproducts. Results In this study, the metabolomes of three representative cotton cultivars of different species were compared using untargeted GC-TOF/MS analysis. A total of 263 metabolites were identified from 705 peaks, and their levels were compared across cultivars. Principal component analysis and OPLS-DA clearly distinguish these samples based on metabolites. There were significant differences in the contents of amino acids, carbohydrates, organic acids, flavonoids, and lipids in G. hirsutum TM-1 compared with G. arboreum Shixiya1 and G. barbadense Hai7124. Notably, the bioactive nutrient compound catechin obtained from the differential metabolites significantly accumulated in TM-1. Furthermore, a comprehensive analysis using catechin and oil-related traits was conducted in core collections of Gossypium hirsutum. The results revealed the reliability of the GC-TOF/MS analysis, as well as that catechin content has a negative association with myristic acid, palmitic acid, stearic acid, oleic acid, linoleic acid, arachidic acid, and total fatty acids. Conclusion These findings suggest that untargeted GC-TOF/MS analysis could provide a new method for investigating the underlying plant biochemistry of nutrient variation in cottonseed, and that catechin content has a negative association with oil-related traits in cottonseed. This study may pave the way to exploit the value of cotton byproducts.