Benchmarking Machine Learning Models for Polymer Informatics: An Example of Glass Transition Temperature

Abstract

In the field of polymer informatics, utilizing machine learning (ML) techniques to evaluate the glass transition temperature T g and other properties of polymers has attracted extensive attention. This data-centric approach is much more efficient and practical than the laborious experimental measurements when encountered a daunting number of polymer structures. Various ML models are demonstrated to perform well for T g prediction. Nevertheless, they are trained on different data sets, using different structure representations, and based on different feature engineering methods. Thus, the critical question arises on selecting a proper ML model to better handle the T g prediction with generalization ability. To provide a fair comparison of different ML techniques and examine the key factors that affect the model performance, we carry out a systematic benchmark study by compiling 79 different ML models and training them on a large and diverse data set. The three major components in setting up an ML model are structure representations, feature representations, and ML algorithms. In terms of polymer structure representation, we consider the polymer monomer, repeat unit, and oligomer with longer chain structure. Based on that feature, representation is calculated, including Morgan fingerprinting with or without substructure frequency, RDKit descriptors, molecular embedding, molecular graph, etc. Afterward, the obtained feature input is trained using different ML algorithms, such as deep neural networks, convolutional neural networks, random forest, support vector machine, LASSO regression, and Gaussian process regression. We evaluate the performance of these ML models using a holdout test set and an extra unlabeled data set from high-throughput molecular dynamics simulation. The ML model’s generalization ability on an unlabeled data set is especially focused, and the model’s sensitivity to topology and the molecular weight of polymers is also taken into consideration. This benchmark study provides not only a guideline for the T g prediction task but also a useful reference for other polymer informatics tasks.

Discovery of Innovative Polymers for Next-Generation Gas-Separation Membranes using Interpretable Machine Learning

“…We have reached a similar conclusion from our polymer informatics benchmark study on polymer glass transition 44 . In short, we believe that deep learning techniques, even standard multilayer perceptrons, have much broader applicability to small datasets of chemical features than previously assumed.…”

“…Firstly, our study reveals that fixed chemical descriptors or fingerprints are both excellent representations for predicting gas permeabilities of polymer membranes. Corroborating our recent benchmark study on polymer glass-transition temperature 44 , we conclude that the choice of chemical representation generally plays a limited role in each ML model's performance, as long as sufficient chemical substructures are captured. Additional features, such as microstructure, could be considered in future ML models, given the importance of microstructural characteristics such as FVEs in solution-diffusion transport theory of membranes 67 .…”

“…ML models for prediction of gas permeability from chemistry must utilize a descriptive and appropriate input to represent the polymer 44 . Thus, in Step 2, RDKit 54 is used to calculate two different chemical representations of each polymer's repeating unit: its molecular descriptors and its MFF 55 .…”

“…Machine learning (ML) is a promising data-centric approach for prediction of gas permeabilities by learning a functional model based on polymer chemistry 40,41 . ML methods using chemical inputs have been successfully applied to accurately predicting many polymer properties including glass transition temperature [42][43][44] , thermal conductivity 45 , dielectric constants 46 , organic photovoltaic properties 47,48 , and transport properties [49][50][51] . The primary challenge for learning a generalizable ML model is training on robust and diverse data, which requires compiling multiple databases with the most recent literature values and imputing missing values 40 .…”

“…We impute missing permeabilities using multivariable imputation by chained equations (MICE) 56 , and we then train multitask supervised ML models to establish synthesis-property relations for these polymer membranes. While various supervised ML models have been used in polymer informatics, including recurrent neural networks, support vector machines, gaussian processes, and others, we choose to focus our study to random forest (RF) regression and deep neural networks (DNN), which have demonstrated outstanding performance in our recent benchmark study 44 . Due to the high variance of DNNs, we perform bootstrap ensembling to further improve our predictions (achieving testing R 2 ~0.90).…”