We sought to prospectively investigate the accuracy of an artificial intelligence (AI)-based tool for left ventricular ejection fraction (LVEF) assessment using a hand-held ultrasound device (HUD) in COVID-19 patients and to examine whether reduced LVEF predicts the composite endpoint of in-hospital death, advanced ventilatory support, shock, myocardial injury, and acute decompensated heart failure. COVID-19 patients were evaluated with a real-time LVEF assessment using an HUD equipped with an AI-based tool vs. assessment by a blinded fellowship-trained echocardiographer. Among 42 patients, those with LVEF < 50% were older with more comorbidities and unfavorable exam characteristics. An excellent correlation was demonstrated between the AI and the echocardiographer LVEF assessment (0.774, p < 0.001). Substantial agreement was demonstrated between the two assessments (kappa = 0.797, p < 0.001). The sensitivity, specificity, PPV, and NPV of the HUD for this threshold were 72.7% 100%, 100%, and 91.2%, respectively. AI-based LVEF < 50% was associated with worse composite endpoints; unadjusted OR = 11.11 (95% CI 2.25–54.94), p = 0.003; adjusted OR = 6.40 (95% CI 1.07–38.09, p = 0.041). An AI-based algorithm incorporated into an HUD can be utilized reliably as a decision support tool for automatic real-time LVEF assessment among COVID-19 patients and may identify patients at risk for unfavorable outcomes. Future larger cohorts should verify the association with outcomes.