Purpose The present study sought to determine the diagnostic accuracy of FSH level, testicular volume, and testicular histology in predicting the successful sperm retrieval (SSR) in a large cohort of patients with non-obstructive azoospermia undergoing conventional testicular sperm extraction (TESE). Methods We retrospectively evaluated 356 patients with nonobstructive azoospermia between June 2004 and July 2009. Binary logistic regression was used to evaluate the diagnostic accuracy of our predicting model, identifying sperm retrieval rate as binary dependent variable. The predictive accuracy of all variables individually evaluated was quantified with area under curve (AUC) estimates derived from receiver operating characteristic (ROC) curve.Results The mean patients' age was 36.8 years. Testicular sperm were retrieved in 158 out of 356 patients (44.3 %). Histological diagnosis of Sertoli cell only syndrome (SCO) was obtained in 216 patients (60.6 %), while 55 patients (15.4 %) had maturation arrest (MA) and 85 (23.8 %) had hypospermatogenesis (HYPO). The binary logistic regression model was statistically significant (χ 2 = 96.792, p < 0.0001) and correctly classified 72.8 % of cases with 46.8 % sensitivity and 93.4 % specificity, positive predictive value (PPV) 85.06 %, negative predictive value (NPV) 68.7 %, +likelihood ratio (LR) 7.13, and −LR 0.57. Only testicular histology was significant to the model, while FSH and testicular volume were not. Sperm retrieval rate (SRR) was significantly higher in patients with HYPO compared to patients with SCO or MA (88.2 vs 30.5 and 30.9 %, respectively, p < 0.0001) Conclusions This study demonstrates that including testicular histology in a model for predicting sperm retrieval increases its diagnostic accuracy. As histology is not available prior to TESE, this model applies only to patients with previous testicular surgery.