Flow boiling in microchannel heat sinks provides superior cooling capacity and high heat transfer rates thanks to their favorable surface area-to-volume ratio and the utilization of the coolant’s latent heat. Despite its advantages, the transition of flow boiling in microchannel heat sinks from the laboratory to commercial use faces significant challenges. One key challenge is the lack of established methods for accurately estimating the heat transfer coefficient. Additionally, there has been notable neglect in addressing the optimization problem for finding the best flow conditions to maximize the heat transfer coefficient. Therefore in this study, five machine learning models (SVM, GPR, kNN, RF, and MLP) were initially developed to estimate the heat transfer coefficient. Performance evaluation showed that the MLP model outperformed the others based on R2, RMSE, and MAPE metrics. Subsequently, an optimization problem was formulated to determine the optimum microchannel dimensions and flow condition via dimensionless numbers that would maximize the heat transfer coefficient. The optimization results indicated microchannel dimensions of 300 µm in width (w) and 60 µm in height (h) as well as dimensionless numbers, namely Boiling number (Bo) of 0.000103, Weber number (We) of 0.1012, Reynolds number (Re) of 65.278, vapor quality (χe) of 0.0104, and Prandtl number (Pr) of 2.3757. In these optimized conditions, the heat transfer coefficient reached 34.497 kW/m2K.