Linear model decision trees as surrogates in optimization of engineering applications

Ammari, Bashar L.; Johnson, Emma S.; Stinchfield, Georgia; Kim, Taehun; Bynum, Michael; Hart, William E.; Pulsipher, Joshua; Laird, Carl D.

doi:10.1016/j.compchemeng.2023.108347

Cited by 15 publications

(2 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…OMLT is a software package that allows for the optimization of trained surrogate models (high-level representations) of Linear Trees (LT), ANN and GBT. 49 In OMLT, LT are reformulated using generalized disjunctive programming (GDP) 50 and different transformations such as Big-M, Convex Hull, among others. While the optimization formulation used for GBT is based on ENTMOOT (see section 3.3).…”

Section: Omlt (Optimization and Machine Learning Toolkit)mentioning

confidence: 99%

Process Systems Engineering Tools for Optimization of Trained Machine Learning Models: Comparative and Perspective

López-Flores,

Ramírez-Márquez,

Ponce-Ortega

2024

Ind. Eng. Chem. Res.

View full text Add to dashboard Cite

This article studies the relevance of innovative Process Systems Engineering (PSE) tools that can reformulate trained machine learning models that are driven by advances in computational technologies, showcasing a pivotal transformation in chemical engineering methodologies. The article also delves into how trained machine learning models are reformulated and optimized to refine engineering decisions as it provides a novel analysis of tools to develop machine learning models by reformulating them, and optimizing them in PSE, thus highlighting their significance and applications. It offers a comprehensive comparison of several cutting-edge tools, including JANOS, MeLOn, ENTMOOT, reluMIP, OptiCL, Gurobi Machine Learning, OMLT, and PySCIPOpt-ML, highlighting their distinct abilities for performance and decision-making. Furthermore, challenges related to the explicit formulation of the main machine learning models are discussed. Guidance is provided to select the appropriate tool according to users' requirements. Additionally, a comparative study of the tools is presented using a case study to analyze and compare the size and type of formulations, the optimal solution, and the computation times.

show abstract

Section: Omlt (Optimization and Machine Learning Toolkit)mentioning

confidence: 99%

Process Systems Engineering Tools for Optimization of Trained Machine Learning Models: Comparative and Perspective

López-Flores,

Ramírez-Márquez,

Ponce-Ortega

2024

Ind. Eng. Chem. Res.

View full text Add to dashboard Cite

show abstract

“…Decision trees are used in the financial sector for risk assessment and credit evaluation. Decision tree regression is also widely used in industrial production to anticipate product sales, optimize production processes, and detect production management concerns such as defects [47][48][49][50][51].…”

Section: Decision Tree Regressionmentioning

confidence: 99%

A Pork Price Prediction Model Based on a Combined Sparrow Search Algorithm and Classification and Regression Trees Model

Qin,

Yang,

Zhang

2023

Applied Sciences

View full text Add to dashboard Cite

The frequent fluctuation of pork prices has seriously affected the sustainable development of the pork industry. The accurate prediction of pork prices can not only help pork practitioners make scientific decisions but also help them to avoid market risks, which is the only way to promote the healthy development of the pork industry. Therefore, to improve the prediction accuracy of pork prices, this paper first combines the Sparrow Search Algorithm (SSA) and traditional machine learning model, Classification and Regression Trees (CART), to establish an SSA-CART optimization model for predicting pork prices. Secondly, based on the Sichuan pork price data during the 12th Five-Year Plan period, the linear correlation between piglet, corn, fattening pig feed, and pork price was measured using the Pearson correlation coefficient. Thirdly, the MAE fitness value was calculated by combining the validation set and training set, and the hyperparameter “MinLeafSize” was optimized via the SSA. Finally, a comparative analysis of the prediction performance of the White Shark Optimizer (WSO)-CART model, CART model, and Simulated Annealing (SA)-CART model demonstrated that the SSA-CART model has the best prediction of pork price (compared with a single decision tree, R2 increased by 9.236%), which is conducive to providing support for pork price prediction. The accurate prediction of pork prices with an optimized machine learning model is of great practical significance for stabilizing pig production, ensuring the sustainable growth of farmers’ income, and promoting sound economic development.

show abstract

Global optimization: a machine learning approach

Bertsimas,

Margaritis

2024

J Glob Optim

View full text Add to dashboard Cite

Many approaches for addressing global optimization problems typically rely on relaxations of nonlinear constraints over specific mathematical primitives. This is restricting in applications with constraints that are implicit or consist of more general primitives. Trying to address such limitations, Bertsimas and Ozturk (2023) proposed OCTHaGOn as a way of solving very general global optimization problems by approximating the nonlinear constraints using hyperplane-based decision-trees and then using those trees to construct a unified MIO approximation of the original problem. We provide extensions to this approach, by (i) approximating the original problem using other MIO-representable ML models besides decision trees, such as gradient boosted trees, multi layer perceptrons and suport vector machines (ii) proposing adaptive sampling procedures for more accurate ML-based constraint approximations, (iii) utilizing robust optimization to account for the uncertainty of the sample-dependent training of the ML models, (iv) leveraging a family of relaxations to address the infeasibilities of the final MIO approximation. We then test the enhanced framework in 81 global optimization instances. We show improvements in solution feasibility and optimality in the majority of instances. We also compare against BARON, showing improved optimality gaps and solution times in more than 9 instances.

show abstract

Linear model decision trees as surrogates in optimization of engineering applications

Cited by 15 publications

References 37 publications

Process Systems Engineering Tools for Optimization of Trained Machine Learning Models: Comparative and Perspective

Process Systems Engineering Tools for Optimization of Trained Machine Learning Models: Comparative and Perspective

A Pork Price Prediction Model Based on a Combined Sparrow Search Algorithm and Classification and Regression Trees Model

Global optimization: a machine learning approach

Contact Info

Product

Resources

About