This paper presents a general theoretical framework of spline adaptive filtering based on a normalized version of orthogonal gradient adaptive algorithm. A nonlinear spline adaptive filter normally consists of a linear combination with a memory-less function and a spline function for adaptive approach. We explain how the adaptive linear filter and spline control points are derived in a straightforward iterative gradient-based method. In order to improve the convergence characteristics, the normalized version of orthogonal gradient adaptive algorithm is introduced by the orthogonal projection along with the gradient adaptive algorithm. In addition, a simple form of adaptation algorithm is introduced how to obtain a lower bound on the excess mean square error (MSE) in a theoretical basis. Convergence and stability analysis based on the MSE criterion are proven in terms of the excess MSE. Simulation results reveal that the proposed algorithm achieves more robustness compared with the conventional spline adaptive filtering algorithm.