We investigate how the motive of hyper-Kähler varieties is controlled by weight-2 (or surface-like) motives via tensor operations. In the first part, we study the Voevodsky motive of singular moduli spaces of semistable sheaves on K3 and abelian surfaces as well as the Chow motive of their crepant resolutions, when they exist. We show that these motives are in the tensor subcategory generated by the motive of the surface, provided that a crepant resolution exists. This extends a recent result of Bülles to the O'Grady-10 situation. In the non-commutative setting, similar results are proved for the Chow motive of moduli spaces of (semi-)stable objects of the K3 category of a cubic fourfold. As a consequence, we provide abundant examples of hyper-Kähler varieties of O'Grady-10 deformation type satisfying the standard conjectures. In the second part, we study the André motive of projective hyper-Kähler varieties. We attach to any such variety its defect group, an algebraic group which acts on the cohomology and measures the difference between the full motive and its weight-2 part. When the second Betti number is not 3, we show that the defect group is a natural complement of the Mumford-Tate group inside the motivic Galois group, and that it is deformation invariant. We prove the triviality of this group for all known examples of projective hyper-Kähler varieties, so that in each case the full motive is controlled by its weight-2 part. As applications, we show that for any variety motivated by a product of known hyper-Kähler varieties, all Hodge and Tate classes are motivated, the motivated Mumford-Tate conjecture holds, and the André motive is abelian. This last point completes a recent work of Soldatenkov and provides a different proof for some of his results.