The coarse-grained (CG) model serves as a powerful tool for the simulation of polymer systems; its reliability depends on the accurate representation of both structural and dynamical properties. However, strong correlations between structural and dynamical properties on different scales and also a strong memory effect, enforced by chain connectivity between monomers in polymer systems, render developing a chemically specific systematic CG model a formidable task. In this study, we report a systematic CG approach that combines the iterative Boltzmann inversion (IBI) method and the generalized Langevin equation (GLE) dynamics. Structural properties are ensured by using conservative CG potentials derived from the IBI method. To retrieve the correct dynamical properties in the system, we demonstrate that using a combination of a Rouse-type delta function and a time-dependent short-time kernel in the GLE simulation is practically efficient. The former can be used to adjust the long-time diffusion dynamics, and the latter can be reconstructed from an iterative procedure according to the velocity autocorrelation function (ACF) from all-atomistic (AA) simulations. Taking the polystyrene as an example, we show that not only structural properties of radial distribution function, intramolecular bond, and angle distributions can be reproduced but also dynamical properties of mean-square displacement, velocity ACF, and force ACF resulted from our CG model have quantitative agreement with the reference AA model. In addition, reasonable agreements are observed in other collective properties between our GLE-CG model and the AA simulations as well.