In this article we generalize the concepts that were used in the PhD thesis of Drudge to classify Cameron-Liebler line classes in PG(n, q), n ≥ 3, to Cameron-Liebler sets of k-spaces in PG(n, q) and AG(n, q). In his PhD thesis, Drudge proved that every Cameron-Liebler line class in PG(n, q) intersects every 3-dimensional subspace in a Cameron-Liebler line class in that subspace. We are using the generalization of this result for sets of k-spaces in PG(n, q) and AG(n, q). Together with a basic counting argument this gives a very strong non-existence condition, n ≥ 3k + 3. is condition can also be improved for k-sets in AG(n, q), with n ≥ 2k + 2.