We explore the properties of test-particle orbits in bumpy spacetimes -stationary, reflectionsymmetric, asymptotically flat solutions of Einstein equations that have a non-Kerr (anomalous) higher-order multipole-moment structure but can be tuned arbitrarily close to the Kerr metric. Future detectors should observe gravitational waves generated during inspirals of compact objects into supermassive central bodies. If the central body deviates from the Kerr metric, this will manifest itself in the emitted waves. Here, we explore some of the features of orbits in non-Kerr spacetimes that might lead to observable signatures. As a basis for this analysis, we use a family of exact solutions proposed by Manko and Novikov which deviate from the Kerr metric in the quadrupole and higher moments, but we also compare our results to other work in the literature. We examine isolating integrals of the orbits and find that the majority of geodesic orbits have an approximate fourth constant of the motion (in addition to the energy, angular momentum, and rest mass) and the resulting orbits are triperiodic to high precision. We also find that this fourth integral can be lost for certain orbits in some oblately deformed Manko-Novikov spacetimes, leading to ergodic motion. However, compact objects will probably not end up on these chaotic orbits in nature. We compute the location of the innermost stable circular orbit (ISCO) and find that the behavior of an orbit in the approach to the ISCO can be qualitatively different depending on whether the location of the ISCO is determined by the onset of an instability in the radial or vertical direction. Finally, we compute periapsis and orbital-plane precessions for nearly circular and nearly equatorial orbits in both the strong and weak field, and discuss weak-field precessions for eccentric equatorial orbits.