This paper proposes a sensorless speed control strategy for a permanent magnet synchronous motor system. Sliding mode control with a whale optimization algorithm was developed for robustness and chattering reduction. To estimate the position and speed of the rotor, an extended Kalman filter using Gaussian process regression was designed. In this controller, the whale optimization method adjusts the switching gain to minimize the tracking error. However, it provides chattering reduction and robustness, owing to the adaptive gain. The extended Kalman estimator calculates the rotor speed by using the current and voltage of the motor as an observer. The observer ensures the high reliability and low cost of the controller. The noise covariance and weight matrices that validated the performance of the estimation were optimized using a regression algorithm. The Gaussian process regression was trained to approximate the best covariance and matrices from the results of the motor controller execution. The performance of the proposed method was demonstrated through simulations under several conditions of tracking speed and load torque changes.