Deterministic ion-photon qubit exchange -a highly desirable building block for quantum information networks -is often assumed to require strong coupling, namely having the single-photon Rabi frequency be the fastest rate in the system. Yet the two native photon-atom gates demonstrated to date (C-phase and SWAP) require only Purcell enhancement that corresponds to high single-atom cooperativities, not strong coupling. This implies that small mode volume cavities -which are challenging to incorporate with ions due to the difficulty of trapping them close to dielectric surfacesare not mandatory. Instead, larger cavities that are more compatible with the trap apparatus are enough, as long as their numerical aperture is high enough to maintain a small mode area at the ion's position. Here we outline the details of a scheme for a deterministic ion-photon SWAP gate based on realistic cavity-QED systems with 171 Yb + , 40 Ca + and 138 Ba + ions. We define the optimal coupling and detuning parameters and simulate the resulting fidelities and efficiencies of the gate, demonstrating that highly efficient photon-ion two-qubit gates indeed do not require strong coupling and are practically attainable with current experimental capabilities.I.