We consider estimation in moment condition models and show that under squared error loss and bounds on identification strength, asymptotically admissible (i.e. undominated) estimators must be Lipschitz functions of the sample moments. GMM estimators are in general discontinuous in the sample moment function, and are thus inadmissible under weak identification.We show, by contrast, that quasi-Bayes posterior means and bagged, or bootstrap aggregated, GMM estimators have superior continuity properties, while results in the literaure imply that they are equivalent to GMM when identification is strong. In simulations calibrated to published instrumental variables specifications, we find that these alternatives often outperform GMM. Hence, quasi-Bayes and bagged GMM present attractive alternatives to GMM.