The growing need to machine challenging materials, especially nickel-based superalloys found in vital aerospace and automotive parts, is evident. This study examines the machining of 15.5 mm Nimonic-263 superalloy using Abrasive Waterjet Machining (AWJM). Nine trial sets, repeated thrice, are comprehensively evaluated using an L9 orthogonal array design to analyze crucial machining variables: the impact of Stand-off Distance, Waterjet Pressure, and Traverse Speed on Kerf Angle, Machining Time, and Material Removal Rate. Statistical significance is evaluated through multi-parametric analysis of variance, and quadratic multiple linear regression models are formulated to correlate output responses with machining variables. The JAYA optimization algorithm is introduced to optimize the machining process, aiming to minimize Kerf Angle and Machining Time while maximizing Material Removal Rate. A comparison with Ant-Lion, Genetic, and Multi-Verse Optimization algorithms highlights JAYA’s effectiveness. A confirmation test validates the JAYA algorithm’s output, confirming its superior performance. This research aids in optimizing machining parameters for challenging materials, benefiting critical components in various industries, especially aerospace.