The measurement of the Transmission Matrix (TM) of a scattering medium is of great interest for imaging. It can be acquired directly by interferometry using an internal reference wavefront. Unfortunately, internal reference fields are scattered by the medium which results in a speckle that makes the TM measurement heterogeneous across the output field of view. We demonstrate how to correct for this effect using the intrinsic properties of the TM. For thin scattering media, we exploit the memory effect of the medium and the reference speckle to create a corrected TM. For highly scattering media where the memory effect is negligible, we use complementary reference speckles to compose a new TM, not compromised by the speckled reference anymore. Using this correction, we demonstrate large field of view second harmonic generation imaging through thick biological media.