A retention behavior model for self-rectifying TaO/HfOx- and TaO/AlOx-based resistive random-access memory (RRAM) is proposed. Trapping-type RRAM can have a high resistance state (HRS) and a low resistance state (LRS); the degradation in a LRS is usually more severe than that in a HRS, because the LRS during the SET process is limited by the internal resistor layer. However, if TaO/AlOx elements are stacked in layers, the LRS retention can be improved. The LRS retention time estimated by extrapolation method is more than 5 years at room temperature. Both TaO/HfOx- and TaO/AlOx-based RRAM structures have the same capping layer of TaO, and the activation energy levels of both types of structures are 0.38 eV. Moreover, the additional AlOx switching layer of a TaO/AlOx structure creates a higher O diffusion barrier that can substantially enhance retention, and the TaO/AlOx structure also shows a quite stable LRS under biased conditions.