An oscillating second-grade fluid through a rectangular cross duct is studied. A traditional integer time derivative in the kinematic tensors is substituted by a fractional operator that considers the memory characteristics. To treat the fractional governing equation, an analytical method was obtained. To analyze the impact of the parameters more intuitively, the difference method was applied to determine the numerical expression and draw with the help of computer simulation. To reduce the cost of the amount of computation and storage, a fast scheme was proposed, one which can greatly improve the calculation speed. To verify the correctness of the difference scheme, the contrast between the numerical expression and the exact expression—constructed by introducing a source term—was given and the superiority of the fast scheme is discussed. Furthermore, the influences of the involved parameters, including the parameter of retardation time, fractional parameter, magnetic parameter, and oscillatory frequency parameter, on the distributions of velocity and shear force at the wall surface with oscillatory flow are analyzed in detail.