X-ray and neutron diffraction studies succeeded in the 1960s to determine the principal structural properties of congruent lithium niobate. However, the nature of the intrinsic defects related to the non-stoichiometry of this material remained an object of controversial discussion. In addition, the incorporation mechanism for dopants in the crystal lattice, showing a solubility range from about 0.1 mol% for rare earths to 9 mol% for some elements (e.g., Ti and Mg), stayed unresolved. Various different models for the formation of these defect structures were developed and required experimental verification. In this paper, we review the outstanding role of nuclear physics based methods in the process of unveiling the kind of intrinsic defects formed in congruent lithium niobate and the rules governing the incorporation of dopants. Complementary results in the isostructural compound lithium tantalate are reviewed for the case of the ferroelectric-paraelectric phase transition. We focus especially on the use of ion beam analysis under channeling conditions for the direct determination of dopant lattice sites and intrinsic defects and on Perturbed Angular Correlation measurements probing the local environment of dopants in the host lattice yielding independent and complementary information.