A P-adic Exact Scientific Computational Library (ESCL) for rational matrix operations has been developed over the past few years. The effort has been focusing on converting all rational number operations to integer calculation, and fully taking advantage of the fast integer multiplication of modern computer architectures. In this paper, we report our progress on parallel implementation of P-adic arithmetic by means of a multiple modulus rational system related to the Chinese remainder theorem. Experimental results are given to illustrate computational efficiency.