We generalize our recent method for constructing Killing tensors of the second rank to conformal Killing tensors. The method is intended for foliated spacetimes of arbitrary dimension m, which have a set of conformal Killing vectors. It applies to foliations of a more general structure than in previous literature. The basic idea is to start with reducible Killing tensors in slices constructed from a set of conformal Killing vectors and the induced metric, and then lift them to the whole manifold. Integrability conditions are derived that ensure this, and a constructive lifting procedure is presented. The resulting conformal Killing tensor may be irreducible. It is shown that subdomains of foliation slices suitable for the method are fundamental photon surfaces if some additional photon region inequality is satisfied. Thus our procedure also opens the way to obtain a simple general analytical expression for the boundary of the gravitational shadow. We apply this technique to electrovacuum, and N = 2, 4, 8 supergravity black holes, providing a new easy way to establish the existence of exact and conformal Killing tensors.