We consider the wave equation in a medium with a rapidly varying speed of propagation. We construct a multiscale scheme based on the heterogeneous multiscale method, which can compute the correct coarse behavior of wave pulses traveling in the medium, at a computational cost essentially independent of the size of the small scale variations. This is verified by theoretical results and numerical examples.