We give a generalisation of Shimizu's lemma to complex or quaternionic hyperbolic space in any dimension for groups of isometries containing an arbitrary parabolic map. This completes a project begun by Kamiya (Hiroshima Math J 13: [501][502][503][504][505][506] 1983). It generalises earlier work of Kamiya, Inkang Kim and Parker. The analogous result for real hyperbolic space is due to Waterman (Adv Math 101:87-113, 1993).