We investigate the Landau-Zener tunneling (LZT) of a self-interacting two-level system in which the coupling between the levels is nonreciprocal. In such a non-Hermitian system, when we adjust the energy bias between two levels very slowly, i.e., in adiabatic limit, we find that a quantum state can still closely follow the eigenstate solution until it encounters the exceptional points (EPs), at which two eigenvalues and their corresponding eigenvectors coalesce. In the absence of the nonlinear self-interaction, we can obtain the explicit expressions of the eigenvectors and eigenvalues and have analytically deduced the adiabatic LZT probability according to a conserved integral value at EPs. In the presence of the nonlinear interaction, the dynamics of the adiabatic evolutions are explicitly demonstrated with the help of classical trajectories in the plane of two reduced variables associated with the populations difference and relative phase. We find surprisingly that the nonzero adiabatic tunneling probabilities can not be correctly predicted by the classical action at EPs. We finally plot a phase diagram for large ranges of nonreciprocity and nonlinear interaction parameters to explicitly demonstrate where the adiabaticity breaks. Our theory has been certificated by numerical results and important implications are discussed.