Identification of cancer driver modules or pathways is a key step in understanding cancer pathogenesis and exploring patient-specific treatments. Numerous studies have shown that some genes with low mutation frequency are also important for the cancer progression, while previous research have focused on identifying high-frequency mutation genes. In this study, we propose a new framework with a new metric to identify driver modules with low-frequency mutation genes, called iCDModule. Inspired by the gravity model, we integrate the coverage and mutual exclusivity in mutation information, define a new metric between gene pairs, called mutation impact distance, to help identifying potential driver genes sets, including those have extremely low mutation rates but play an important role in functional networks. A genetic network is constructed by combining the defined mutation impact distance and then the driver module identification problem is formalized as the maximum clique solution problem, and an improved ant colony optimization algorithm is used to solve it. iCDModule is applied to TCGA breast cancer, glioblastoma, ovarian cancer to test performance. Experiments show that it can accurately identify known cancer driver modules and pathways, and also detect driver modules containing low-frequency mutation genes. iCDModule is significantly better than other existing methods in identifying driver modules.