The construction industry is one of the main producers of greenhouse gasses (GHG). With the looming consequences of climate change, sustainability measures including quantifying the amount of air pollution during a construction project have become an important project objective in the construction industry. A major contributor to air pollution during construction projects is the use of heavy equipment. Therefore, efficient operation and management can substantially reduce a project’s carbon footprint and other environmental harms. Using unintrusive and indirect methods to predict on-road vehicle emissions has been a widely researched topic. Nevertheless, the same is not true in the case of construction equipment. This paper describes the development and deployment of a framework that uses machine learning (ML) methods to predict the level of emissions from heavy construction equipment. Data is collected via an Internet of Things (IoT) approach with accelerometer and gyroscope sensors as data collection nodes. The developed framework was validated using an excavator performing real-world construction work. A portable emission measurement system (PEMS) was used along with the inertial sensors to record the amount of CO, NOX, CO2, SO2, and CH4 pollution emitted by the equipment. Different ML algorithms were developed and compared to identify the best model to predict emission levels from inertial sensors data. The results show that Random Forest with the coefficient of determination (R2) of 0.94, 0.91, and 0.94, and normalized root-mean-square error (NRMSE) of 4.25, 6.42, and 5.17 for CO, NOX, and CO2, respectively, was the best algorithm among different models evaluated in this study.