Algebraic diagrammatic construction (ADC) theory is a computationally efficient and accurate approach for simulating electronic excitations in chemical systems. However, for the simulations of excited states in molecules with unpaired electrons the performance of ADC methods can be affected by the spin contamination in unrestricted Hartree-Fock (UHF) reference wavefunctions. In this work, we benchmark the accuracy of ADC methods for electron attachment and ionization of open-shell molecules with the UHF reference orbitals (EA/IP-ADC/UHF) and develop an approach to quantify the spin contamination in the charged excited states. Following this assessment, we demonstrate that the spin contamination can be reduced by combining EA/IP-ADC with the reference orbitals from restricted open-shell Hartree-Fock (ROHF) or orbital-optimized Møller-Plesset perturbation (OMP) theories. Our numerical results demonstrate that for open-shell systems with strong spin contamination in the UHF reference the third-order EA/IP-ADC methods with the ROHF or OMP reference orbitals are similar in accuracy to equation-of-motion coupled cluster theory with single and double excitations.