Magneto-elastic (straintronic) switching of bistable magnetostrictive nanomagnets is an extremely energyefficient switching methodology for (magnetic) binary switches that has recently attracted widespread attention because of its potential application in ultra-low-power digital computing hardware. Unfortunately, this modality of switching is also error very prone at room temperature. Theoretical studies of switching error probability of magneto-elastic switches have predicted probabilities ranging from 10 -8 -10 -3 at room temperature for ideal, defect-free nanomagnets, but experiments with real nanomagnets show a much higher probability that exceeds 0.1 in some cases. To understand what causes this large difference, we have theoretically studied the effect of common defects (that occur during fabrication) on magneto-elastic switching probability in the presence of room-temperature thermal noise. Surprisingly, we found that even small defects increase the switching error probabilities by orders of magnitude. This could limit or preclude the application of magneto-elastic (straintronic) binary switches in either Boolean logic or memory, despite their excellent energy-efficiency, and restrict them to non-Boolean (e.g. neuromorphic, stochastic) computing applications. We also studied the difference between magneto-elastic switching with a stress pulse of constant amplitude and sinusoidal time-varying amplitude (e.g. due to a surface acoustic wave) and found that the latter method is more reliable and generates lower switching error probabilities in most cases, provided the time variation is reasonably slow.