Strong spin-orbit coupling (SOC) can result in ground states with non-trivial topological properties. The situation is even richer in magnetic systems where the magnetic ordering can potentially have strong influence over the electronic band structure. The class of AMnBi2 (A = Sr, Ca) compounds are important in this context as they are known to host massive Dirac fermions with strongly anisotropic dispersion, which is believed to be due to the interplay between strong SOC and magnetic degrees of freedom. We report the optical conductivity of YbMnBi2, a newly discovered member of this family and a proposed Weyl semi-metal (WSM) candidate with broken time reversal symmetry. Together with density functional theory (DFT) band structure calculations, we show that the complex conductivity can be interpreted as the sum of an intra-band Drude response and inter-band transitions. We argue that the canting of the magnetic moments that has been proposed to be essential for the realization of the WSM in an otherwise antiferromagnetically ordered system is not necessary to explain the optical conductivity. We believe our data is explained qualitatively by the uncanted magnetic structure with a small offset of the chemical potential from strict stochiometry. We find no definitive evidence of a bulk Weyl nodes. Instead we see signatures of a gapped Dirac dispersion, common in other members of AMnBi2 family or compounds with similar 2D network of Bi atoms. We speculate that the evidence for a WSM seen in ARPES arises through a surface magnetic phase. Such an assumption reconciles all known experimental data.