Using data from 3,534 men and 1,350 women who participated in the 2001 and 2002 National Household Surveys on Drug Use and Health (NSDUH), and who indicated past-year criminal justice involvement, this study evaluated the accuracy of the K6, a screening tool for severe psychological distress associated with serious mental illness, with and without additional questions on past-year psychiatric treatment. We used the Composite International Diagnostic Interview Schedule-Short Form (CIDI-SF) as the gold standard. We found the unmodified K6 to be about equally accurate for men and women with ROC-AUC values for both at .88 and sensitivity scores in the range of 62.4 to 74.0 for men and 70.3 to 80.9 for women. Inclusion of psychiatric treatment indicators increased sensitivity but decreased specificity of the K6 for both men and women and consequently did not improve overall performance. Based on these findings, we tentatively recommend use of the K6 with criminal justice populations, although we recommend more rigorous studies to establish an optimum threshold and to evaluate whether the K10 (the longer version of the K6) can improve on the sensitivity and positive predictive values obtained with the K6.