Astronomical observations have established that extremely compact, massive objects are common in the universe. It is generally accepted that these objects are, in all likelihood, black holes. As observational technology has improved, it has become possible to test this hypothesis in ever greater detail. In particular, it is or will be possible to measure the properties of orbits deep in the strong field of a black hole candidate (using x-ray timing or future gravitational-wave measurements) and to test whether they have the characteristics of black hole orbits in general relativity. Past work has shown that, in principle, such measurements can be used to map the spacetime of a massive compact object, testing in particular whether the object's multipolar structure satisfies the rather strict constraints imposed by the black hole hypothesis. Performing such a test in practice requires that we be able to compare against objects with the "wrong" multipole structure. In this paper, we present tools for constructing the spacetimes of bumpy black holes: objects that are almost black holes, but that have some multipoles with the wrong value. In this first analysis, we focus on objects with no angular momentum. Generalization to bumpy Kerr black holes should be straightforward, albeit labor intensive. Our construction has two particularly desirable properties. First, the spacetimes which we present are good deep into the strong field of the object -we do not use a "large r" expansion (except to make contact with weak field intuition). Second, our spacetimes reduce to the exact black hole spacetimes of general relativity in a natural way, by dialing the "bumpiness" of the black hole to zero. We propose that bumpy black holes can be used as the foundation for a null experiment: if black hole candidates are indeed the black holes of general relativity, their bumpiness should be zero. By comparing the properties of orbits in a bumpy spacetime with those measured by an astrophysical source, observations should be able to test this hypothesis, stringently testing whether they are in fact the black holes of general relativity.