The present study aims to offer new numerical solutions and optimisation strategies for the fluid flow and heat transfer behaviour at a stagnation point through a nonlinear sheet that is expanding or contracting in water-based hybrid nanofluids. Most hybrid nanofluids typically use metallic nanoparticles. However, we deliver a new approach by combining single- and multi-walled carbon nanotubes (SWCNTs-MWCNTs). The flow is presumptively steady, laminar, and surrounded by a constant temperature of the ambient and body walls. By using similarity variables, a model of partial differential equations (PDEs) with the magnetohydrodynamics (MHD) effect on the momentum equation is converted into a model of non-dimensional ordinary differential equations (ODEs). Then, the dimensionless first-order ODEs are solved numerically using the MATLAB R2022b bvp4C program. In order to explore the range of computational solutions and physical quantities, several dimensionless variables are manipulated, including the magnetic parameter, the stretching/shrinking parameter, and the volume fraction parameters of hybrid and mono carbon nanotubes. To enhance the originality and effectiveness of this study for practical applications, we optimise the heat transfer coefficient via the response surface methodology (RSM). We apply a face-centred central composite design (CCF) and perform the CCF using Minitab. All of our findings are presented and illustrated in tabular and graphic form. We have made notable contributions in the disciplines of mathematical analysis and fluid dynamics. From our observations, we find that multiple solutions appear when the magnetic parameter is less than 1. We also detect double solutions in the shrinking region. Furthermore, the increase in the magnetic parameter and SWCNTs-MWCNTs volume fraction parameter increases both the skin friction coefficient and the local Nusselt number. To compare the performance of hybrid nanofluids and mono nanofluids, we note that hybrid nanofluids work better than single nanofluids both in skin friction and heat transfer coefficients.