Burning fossil fuels causes toxic gas emissions to increase, therefore, scientists are trying to find alternative green fuels. One of the important alternative fuels is biodiesel. However, using eco-friendly primary materials is a main factor. Sustainable catalysts should have high performance, good activity, easy separation from reaction cells, and regenerability. In this study, to solve the mentioned problem NaOH@Graphene oxide-Fe3O4 as a magnetic catalyst was used for the first time to generate biodiesel from waste cooking oil. The crystal structure, functional groups, surface area and morphology of catalyst were studied by XRD, FTIR, BET, and FESEM techniques. The response surface methodology based central composite design (RSM-CCD) was used for biodiesel production via ultrasonic technique. The maximum biodiesel yield was 95.88% in the following operation: 10.52:1 molar ratio of methanol to oil, a catalyst weight of 3.76 wt%, a voltage of 49.58 kHz, and a time of 33.29 min. The physiochemical characterization of biodiesel was based to ASTM standard. The magnetic catalyst was high standstill to free fatty acid due to the five cycle’s regeneration. The kinetic study results possess good agreement with first-order kinetics as well as the activation energy and Arrhenius constant are 49.2 kJ/min and 16.47 * 1010 min−1, respectively.