The expectation that punishment be effective at controlling crime is a longstanding convention in the U.S., and no doubt elsewhere. While the history of American punishment has not been shaped entirely by the question of efficacy, it has played a predominant role in justifying penal policy for over 200 years. The question has become even more salient in policy decision-making of late, as research has begun to certify and consolidate findings on what is effective at reducing recidivism. What is lacking in this ongoing conversation, however, is a critique of this penal policy question and the answers it generates in the form of recidivism rates. The current paper fills this void by interrogating the claims of the evaluation literature, namely that better proof of what is effective is available and that more research is still needed. The questions and findings of 19th, 20th, and 21st Century seekers of what is effective in the American adult penal system are recounted and analyzed using several data sources. They include government reports, professional association meeting minutes, legislative documents, scholarly reports, individual studies, research reviews, and statistical analysis of systematic reviews. Ultimately, an overarching narrative is provided that deepens and challenges our understanding of what is known about what is effective.