As an evolutionarily conserved multi-protein complex, the Mediator complex modulates the association between transcription factors and RNA polymerase II to precisely regulate gene transcription. Although numerous studies have shown the diverse functions of Mediator complex in plant development, flowering, hormone signaling, and biotic stress response, its roles in the Abscisic acid (ABA) signaling pathway and abiotic stress response remain largely unclear. It has been recognized that the phytohormone, ABA, plays a predominant role in regulating plant adaption to various abiotic stresses as ABA can trigger extensive changes in the transcriptome to help the plants respond to environmental stimuli. Over the past decade, the Mediator complex has been revealed to play key roles in not only regulating the ABA signaling transduction but also in the abiotic stress responses. In this review, we will summarize current knowledge of the Mediator complex in regulating the plants’ response to ABA as well as to the abiotic stresses of cold, drought and high salinity. We will particularly emphasize the involvement of multi-functional subunits of MED25, MED18, MED16, and CDK8 in response to ABA and environmental perturbation. Additionally, we will discuss potential research directions available for further deciphering the role of Mediator complex in regulating ABA and other abiotic stress responses.