Several simulations strategies have emerged to predict the permeability of solutes across membranes, which is important for many biological or industrial processes such as drug design. The widespread inhomogeneous solubility-diffusion (ISD) model is based on the Smoluchowski equation and describes permeation as purely diffusive. The counting method, which counts membrane transitions in a long molecular dynamics (MD) trajectory, is free of this diffusive assumption, but it lacks sufficient statistics when the permeation involves high free energy barriers. Metadynamics and variations thereof can overcome such barriers, but they generally lack the kinetics information. The milestoning framework has been used to describe permeation as a rare event, but it still relies on the Markovian assumption between the milestones. Replica Exchange Transition Interface Sampling (RETIS) has been shown to be an effective method for sampling rare events while simultaneously describing the kinetics without assumptions. This paper is the first permeation application of RETIS on an all-atom lipid bilayer consisting of 1-palmitoyl-2-oleoyl-sn-glycero-3-phosphocholine (POPC) to compute the entrance, escape and complete transition of molecular oxygen. Conventional MD was performed as a benchmark, and the MD rates from counting were converted to rate constants, giving good agreement with the RETIS values. Moreover, a correction factor was derived to convert the collective order parameter in RETIS, which was aimed to improve efficiency, to a single-particle order parameter. With this work, we showed how the exact kinetics of drug molecules permeation can be assessed with RETIS even if the permeation is truly a rare event or if the permeation is non-Markovian. RETIS will therefore be a valuable tool for future permeation studies.