The genus Hippophae (sea buckthorn) is widely cultivated and consumed in Asia and Europe. The fruit color is an important appearance and commercial trait for sea buckthorn, which is closely related to the biosynthesis and accumulation of various nutrients and pigments. The fruit colors of sea buckthorn are varied, which appear as yellow, orange, red, and brown. However, the nutrients and pigments forming different the fruit colors of sea buckthorn remain unclear. To investigate the mechanism of pigmentation of sea buckthorn fruit, integrative analyses of the transcriptome and targeted metabolome, including the carotenoids, flavonoids, and chlorophylls, were performed in five sea buckthorn varieties with different fruit colors. A total of 209 flavonoids and 41 carotenoids were identified in five sea buckthorn fruits of different colors. The types and contents of flavonoids and carotenoids in the five sea buckthorn fruits were significantly different. Interestingly, we only found a high content of chlorophyll (772.7 mg/kg) in the sea buckthorn fruit with a brown color. The quantities and relative proportions of the flavonoids, carotenoids, and chlorophyll led to the different colors of the sea buckthorn fruits. Using a weighted gene co-expression network analysis (WGCNA), the key genes related to the carotenoids and chlorophyll metabolism were identified. The high content of chlorophylls in the brown fruit was closely related to the downregulated expression of key genes in the chlorophyll degradation pathway, including SGR, SGRL, PPH, NYC1, and HCAR. Our results provide new insights into the roles of flavonoids, carotenoids, and chlorophylls in the formation of fruit color in sea buckthorn.