Exploration is both an important part of a firm's innovation strategy and an activity that involves a high degree of uncertainty. This paper investigates a duality in the exploratory component of R&D activity with regard to innovation failure: while exploration is likely to increase firms' exposure to failure, it might also provide learning opportunities to reduce failure. Our study contributes to the innovation management and organisational learning literatures by demonstrating the value of exploratory R&D for enabling two types of learning mechanisms. The first, experience-based learning, is based on the learning opportunities derived from accumulated experience in exploratory R&D: it involves improvements to procedures associated with experimentation and provides guidance for current exploration and to navigate the search space.The second, inferential-based learning, is based on the learning opportunities derived from current exploratory R&D efforts, which are associated with improved interpretation of ill-defined problems and timely responses to unstructured information. We draw on a longitudinal dataset of 2,226 Spanish manufacturing companies and show that, when past experience is associated with current exploration, innovation failure in the conception phase is reduced. We also find an inverted U-shaped relation between current exploratory R&D and innovation failure, in both the conception and implementation phases of innovation projects, showing that increasing levels of investment in current exploration activities attenuate the initial positive association between exploratory R&D and failure.