We propose a method for manipulating wave propagation in phononic lattices by employing local vibro-impact (VI) nonlinearities to scatter energy across the underling linear band structure of the lattice, and transfer energy from lower to higher optical bands. Inspired by recent developments in the field of nonlinear targeted energy transfer (TET) using non-resonant energy exchanges, we achieve this using spatially localized VI forces that redistribute energy across the linear spectrum of the lattice in a non-resonant fashion. First, a 1-dimensional (1D), 2-band phononic lattice with embedded VI unit cells is computationally studied to demonstrate that energy is scattered in the wavenumber domain, and this nonlinear scattering mechanism depends on the energy of the propagating wave. Next, a 4-band lattice is studied with a similar technique to demonstrate the concept of inter-band targeted energy transfer (IBTET) and to establish analogous scaling relations with respect to energy. To interpret the results of IBTET, we study the nonlinear normal modes (NNMs) of a reduced order model (ROM) of the VI unit cell in the 4-band lattice, using the method of numerical continuation. Interestingly, the slope of the frequency-energy branches of the ROM corresponding to the 1:1 resonance NNM matches remarkably well with the dependence of IBTET to input energy in the 4-band lattice. In both phononic lattices, it is shown that there exists a maximum energy transfer at moderate input energies, followed by a power law decay of relative energy transfer either to the wavenumber domain or between bands on input energy; this power law dependence is additionally validated by the ROM. Moreover, relations between the dynamics of the VI lattice and the NNMs of the underlying Hamiltonian system provide physical interpretations for the relative energy transfers. Hence, we present a predictive framework to computationally explore non-resonant energy transfers across the linear band structure of phononic lattices with local strong non-smooth nonlinearities and provide a comprehensive physics-based interpretation of these energy transfers based on the nonlinear dynamics of the lower-dimensional ROM.