Ternary polymer solar cells (PSCs) are currently the simplest and most efficient way to further improve the device performance in PSCs. To find high-performance organic photovoltaic materials, the established connection between the material structure and device performance before fabrication is of great significance. Herein, firstly, a database of the photovoltaic performance in 874 experimental PSCs reported in the literature is established, and three different fingerprint expressions of a molecular structure are explored as input features; the results show that long fingerprints of 2D atom pairs can contain more effective information and improve the accuracy of the models. Through supervised learning, five machine learning (ML) models were trained to build a mapping of the photovoltaic performance improvement relationship from binary to ternary PSCs. The GBDT model had the best predictive ability and generalization. Eighteen key structural features from a non-fullerene acceptor and the third components that affect the device’s PCE were screened based on this model, including a nitrile group with lone-pair electron, a halogen atom, an oxygen atom, etc. Interestingly, the structural features for the enhanced device’s PCE were essentially increased by the Jsc or FF. More importantly, the reliability of the ML model was further verified by preparing the highly efficient PSCs. Taking the PM6:BTP-eC9:PY-IT ternary PSC as an example, the PCE prediction (18.03%) by the model was in good agreement with the experimental results (17.78%), the relative prediction error was 1.41%, and the relative error between all experimental results and predicted results was less than 5%. These results indicate that ML is a useful tool for exploring the photovoltaic performance improvement of PSCs and accelerating the design and application with highly efficient non-fullerene materials.