Background We determine the predictive value of transthoracic echocardiographic (TTE) metrics for clinical deterioration within 5 days in adults with intermediate-risk pulmonary embolism (PE). Methods This was a prospective observational study of intermediate-risk PE patients. To determine associations of TTE and clinical predictors with clinical deterioration, we used univariable analysis, Youden’s index for optimal thresholds, and multivariable analyses to report odds ratios (ORs) or area under the curve (AUC). Results Of 306 intermediate-risk PE patients, 115 (37.6%) experienced clinical deterioration. PE patients who had clinical deterioration within 5 days had greater baseline right ventricle (RV) dilatation and worse systolic function than the group without clinical deterioration as indicated by the following: RV basal diameter 4.46 ± 0.77 versus 4.20 ± 0.77 cm; RV/LV basal width ratio 1.14 ± 0.29 versus 1.02 ± 0.24; tricuspid annular plane systolic excursion (TAPSE) 1.56 ± 0.55 versus 1.80 ± 0.52 cm; and RV systolic excursion velocity 10.40 ± 3.58 versus 12.1 ± 12.5 cm/s, respectively. Optimal thresholds for predicting clinical deterioration were: RV basal width 3.9 cm (OR 2.85 [1.64, 4.97]), RV-to-left ventricle (RV/LV) ratio 1.08 (OR 3.32 [2.07, 5.33]), TAPSE 1.98 cm (OR 3.3 [2.06, 5.3]), systolic excursion velocity 10.10 cm/s (OR 2.85 [1.75, 4.63]), and natriuretic peptide 190 pg/mL (OR 2.89 [1.81, 4.62]). Significant independent predictors were: transient hypotension 6.1 (2.2, 18.9), highest heart rate 1.02 (1.00, 1.03), highest respiratory rate 1.02 (1.00, 1.04), and RV/LV ratio 1.29 (1.14, 1.47). By logistic regression and random forest analyses, AUCs were 0.80 (0.73, 0.87) and 0.78 (0.70, 0.85), respectively. Conclusions Basal RV, RV/LV ratio, and RV systolic function measurements were significantly different between intermediate-risk PE patients grouped by subsequent clinical deterioration.