Objective To compare the efficacy and safety of sedation with dexmedetomidine vs. midazolam for patients with acute cardiogenic pulmonary edema and hypoxemia during the treatment of non-invasive ventilation (NIV). Methods The intensive care unit (ICU) patients treated in our hospital between March 2008 and August 2011 who had acute pulmonary edema and hyoxemia in NIV failure due to patient refusal to continue the NIV sessions (due to discomfort) were enrolled in this study. The patients were divided into two groups by the random numerical table method. They were treated with either midazolam (29 cases) or dexmedetomidine (33 cases). The patients were sedated (Ramsay scale 2-3) by a continuous perfusion of midazolam or dexmedetomidine during the NIV session. Cardiorespiratory and ventilatory parameters, the results of the blood gas analysis, and adverse events were prospectively recorded. The main outcome measure was the percentage of endotracheal intubation during NIV. Secondary endpoints included the duration of non-invasive mechanical ventilation, length of ICU stay, and adverse events. Results In both groups of patients, the expected sedative scores were obtained. The cardiorespiratory symptoms and signs (oxygenation index, pH value, and respiratory rate) were significantly improved in both groups. In the dexmedetomidine-treated group, the patients had a further decreased percentage of failure of NIV requiring endotracheal intubation (ETI) and a more prolonged mean time to ETI (p=0.042, p=0.024). Furthermore, when compared with the group treated with midazolam, the overall duration of mechanical ventilation and the duration of ICU hospitalization in the group treated with dexmedetomidine were markedly decreased, and weaning from mechanical ventilation was easier (p=0.010, p=0.042). Despite the fact that more dexmedetomidine-treated patients developed bradycardia (18.2% vs. 0, p=0.016), no patients required an intervention or interruption of study drug infusion. Conversely, the incidence of respiratory infections and vomiting was lower in the dexmedetomidine-treated patients (p=0.026, p=0.010). Conclusion Dexmedetomidine led to a more desired level of awaking sedation, shortened the duration of mechanical ventilation and the length of the ICU stay, and further reduced the prevalence of nosocomial infection for NIV sedation in patients with acute cardiogenic pulmonary edema. It appears to provide several advantages and safe control compared with the γ-amino butyric acid (GABA) agonist midazolam.