When describing the low-energy physics of bosons in a double-well potential with a high barrier between the wells and sufficiently weak atom-atom interactions, one can to a good approximation ignore the high energy states and thereby obtain an effective two-mode model. Here, we show that the regime in which the two-mode model is valid can be extended by adding an on-site three-body interaction term and a three-body interaction-induced tunneling term to the two-mode Hamiltonian. These terms effectively account for virtual transitions to the higher energy states. We determine appropriate strengths of the three-body terms by an optimization of the minimal value of the wave function overlap within a certain time window. Considering different initial states with three or four atoms, we find that the resulting model accurately captures the dynamics of the system for parameters where the two-mode model without the three-body terms is poor. We also investigate the dependence of the strengths of the three-body terms on the barrier height and the atom-atom interaction strength. The optimal three-body interaction strengths depend on the initial state of the system.