To solve the system of linear equations is one of the hottest topics in iterative methods. The system of linear equations occurs in business, engineering, social and in sensitive research areas like medicine, therefore applying efficient matrix solvers to such systems is crucial. In this paper, an improved iterative scheme using successive overrelaxation has been constructed. The proposed iterative method converges well when a linear system’s matrix is M-matrix, Symmetric positive definite with some conditions, irreducibly diagonally dominant, strictly diagonally dominant, and H-matrix. Such type of linear system of equations does arise usually from ordinary differential equations and partial differential equations. The improved iterative scheme has decreased spectral radius, improved stability and reduced the number of iterations. To show the effectiveness of the improved scheme, it is compared with the refinement of generalized successive over-relaxation and generalized successive over-relaxation method with the help of numerical experiments using MATLAB software.