Type 2 diabetes mellitus (T2DM) is a common endocrine disorder which remains a large challenge for clinicians. Previous studies have suggested that mitochondrial dysfunction plays an active role in T2DM progression, but a detailed mechanism is still elusive. In the current study, two Han Chinese families with maternally inherited T2DM were evaluated using clinical, genetic, molecular, and biochemical analyses. The mitochondrial genomes were PCR amplified and sequenced. Phylogenetic and bioinformatic analyses were used to assess the potential pathogenicity of mitochondrial DNA (mtDNA) mutations. Interestingly, the matrilineal relatives of these pedigrees exhibited variable severity of T2DM, in particular, the age at onset of T2DM varied from 26 to 65 years, with an average of 49 years. Sequence analysis revealed the presence of ND4 G11696A mutation, which resulted in the substitution of an isoleucine for valine at amino acid (AA) position 312. Indeed, this mutation was present in homoplasmy only in the maternal lineage, not in other members of these families, as well as 200 controls. Furthermore, the m.C5601T in the tRNAAla and novel m.T5813C in the tRNACys, showing high evolutional conservation, may contribute to the phenotypic expression of ND4 G11696A mutation. In addition, biochemical analysis revealed that cells with ND4 G11696A mutation exhibited higher levels of reactive oxygen species (ROS) productions than the controls. In contrast, the levels of mitochondrial membrane potential (MMP), ATP, mtDNA copy number (mtDNA-CN), Complex I activity, and NAD+/NADH ratio significantly decreased in cell lines carrying the m.G11696A and tRNA mutations, suggesting that these mutations affected the respiratory chain function and led to mitochondrial dysfunction that was involved in T2DM. Thus, our study broadened the clinical phenotypes of m.G11696A mutation.