Robustness and stability of the Continuum Strong Discontinuity Approach (CSDA) to material failure are addressed. After identification of lack of symmetry of the finite element formulation and material softening in the constitutive model as possible causes of loss of robustness, two remedies are proposed: 1) a symmetric version of the elementary enriched finite element with embedded discontinuities, and 2) an implicit-explicit integration of the internal variable, in the constitutive model, that renders the tangent constitutive algorithmic operator positive definite and constant. The combination of both developments leads to finite element formulations with constant and non-singular tangent structural stiffness, these allowing dramatic improvements in terms of robustness and computational costs. After assessing the convergence properties of the new strategies, three-dimensional numerical simulations of failure problems illustrate the performance of the proposed procedures.