As a crucial stage in the processing of black tea, rolling plays a significant role in both the color transformation and the quality development of the tea. In this process, the production of theaflavins, thearubigins, and theabrownins is a primary factor contributing to the alteration in color of rolled leaves. Herein, tea pigments are selected as the key quality indicators during rolling of black tea, aiming to establish rapid detection methods for them. A machine vision system is employed to extract nine color feature variables from the images of samples subjected to varying rolling times. Then, the tea pigment content in the corresponding samples is determined using a UV-visible spectrophotometer. In the meantime, the correlation between color variables and tea pigments is discussed. Additionally, Z-score and PCA are used to eliminate the magnitude difference and redundant information in original data. Finally, the quantitative prediction models of tea pigments based on the images’ color features are established by using PLSR, SVR, and ELM. The data show that the Z-score–PCA–ELM model has the best prediction effect for tea pigments. The Rp values for the model prediction sets are all over 0.96, and the RPD values are all greater than 3.50. In this study, rapid determination methods for tea pigments during rolling of black tea are established. These methods offer significant technical support for the digital production of black tea.