According to the evolutionary sceptic, the fact that our cognitive faculties evolved radically undermines their reliability. A number of evolutionary epistemologists have sought to refute this kind of scepticism. This paper accepts the success of these attempts, yet argues that refuting the evolutionary sceptic is not enough to put any particular domain of beliefs -notably scientific beliefs, which include belief in Darwinian evolution -on a firm footing. The paper thus sets out to contribute to this positive justificatory project, underdeveloped in the literature. In contrast to a 'wholesale' approach, attempting to secure justification for all of our beliefs on the grounds that our belief-forming mechanisms evolved to track truth, we propose a 'piecemeal' approach of assessing the reliability of particular belief-forming mechanisms in particular domains. This stands in contrast to the more familiar attempt to transfer warrant obtained for one domain (e.g., common-sense beliefs) to another (e.g., scientific beliefs) by showing how one is somehow an extension of the other. We offer a naturalist reply to the charge of circularity by appealing to reliabilist work on the problem of induction, notably Peter Lipton's distinction between self-certifying and non-self-certifying inductive arguments. We show how, for scientific beliefs, a non-self-certifying argument might be made for the reliability of our cognitive faculties in that domain. We call this strategy Humean Bootstrapping.