Ecological restoration of mining areas has mainly focused on the succession dynamics of vegetation and the fate of microbial communities remains poorly understood. We examined changes in soil characteristics and plant and microbial communities with increasing reclamation period in an open coal mine. Bacterial, archaeal and fungal communities were assessed by tag-encoded 454 pyrosequencing. At the phylum level, Proteobacteria, Crenarchaeota, and Ascomycota had the highest detected relative abundance within bacteria, archaea, and fungi, respectively. Partial regressions and canonical correspondence analysis demonstrated that vegetation played a major role in bacterial and archaeal diversity and assemblies, and soil characteristics, especially nitrogen, were important for fungal diversity and assemblies. Spearman rank correlation indicated that bacterial and archaeal communities showed synergistic succession with plants; whereas, fungal communities showed no such pattern. Overall, our data suggest that there are different drivers of bacterial, archaeal and fungal succession during secondary succession in a reclaimed open mine.