An implicit large eddy simulation (iLES) method based on a spectral vanishing viscosity (SVV) method has been well established in the framework of the spectral/ hp element method (which is called the SVV-iLES method), and the SVV regularization introduces proper dissipation to the numerical scheme, which in turn reduces the truncation error and the uncertainty in computation. However, SVV-iLES is still limited to moderate Reynolds numbers due to the huge computational cost in resolving the inner part of the boundary layer. By modeling the near-wall layer, a wall-model coupling strategy, tailored to the features of the SVV-iLES, is proposed to substantially reduce the computational cost. This work is the first to construct the wall model of SVV-iLES based on the spectral/ hp element method, and the practice rules can be regarded as guidelines for the application of the wall model to iLES with high order schemes. Numerical investigations for a well-established benchmark problem of the turbulent channel flow are implemented by the proposed wall-modeled SVV-iLES. The results obtained are consistent with the reference direct numerical simulation (DNS) data at different Reynolds numbers considered. The effect of the time-averaged velocity at the matching interface is examined, and the results suggest that the instantaneous velocity can be directly used in the algebraic wall-model, which in turn could speed up the computation. Additionally, a comparison between the classical log-law and Reichardt's law models reveals that there are no significant differences in statistical quantities in the logarithmic region. Finally, it was found that, at similar grid resolution, the p-type refinement shows better agreement with the reference DNS results.