Background: It is unclear whether the use of clinical prediction rules is sufficient to rule out infective endocarditis (IE) in patients with Staphylococcus aureus bacteremia (SAB) without an echocardiogram evaluation, either transthoracic (TTE) and/or transesophageal (TEE). Our primary purpose was to test the usefulness of PREDICT, POSITIVE, and VIRSTA scores to rule out IE without echocardiography. Our secondary purpose was to evaluate whether not performing an echocardiogram evaluation is associated with higher mortality. Methods: We conducted a unicentric retrospective cohort including all patients with a first SAB episode from January 2015 to December 2020. IE was defined according to modified Duke criteria. We predefined threshold cutoff points to consider that IE was ruled out by means of the mentioned scores. To assess 30-day mortality, we used a multivariable regression model considering performing an echocardiogram as covariate. Results: Out of 404 patients, IE was diagnosed in 50 (12.4%). Prevalence of IE within patients with negative PREDICT, POSITIVE, and VIRSTA scores was: 3.6% (95% CI 0.1–6.9%), 4.9% (95% CI 2.2–7.7%), and 2.2% (95% CI 0.2–4.3%), respectively. Patients with negative VIRSTA and negative TTE had an IE prevalence of 0.9% (95% CI 0–2.8%). Performing an echocardiogram was independently associated with lower 30-day mortality (OR 0.24 95% CI 0.10–0.54, p = 0.001). Conclusion: PREDICT and POSITIVE scores were not sufficient to rule out IE without TEE. In patients with negative VIRSTA score, it was doubtful if IE could be discarded with a negative TTE. Not performing an echocardiogram was associated with worse outcomes, which might be related to presence of occult IE. Further studies are needed to assess the usefulness of clinical prediction rules in avoiding echocardiographic evaluation in SAB patients.