Cs2AgBiBr6 has attracted much interest as a potential lead‐free alternative for perovskite solar cells. Although this material offers encouraging optoelectronic features, severe bottlenecks limit the performance of the resulting solar cells to a power conversion efficiency of below 3%. Here, the performance‐limiting factors of this material are investigated in full solar cells featuring various architectures. It is found that the photovoltaic parameters of Cs2AgBiBr6‐based solar cells strongly depend on the scan speed of the J/V measurements, suggesting a strong impact of ionic conductivity in the material. Moreover, a sign change of the photocurrent for bias voltages above 0.9 V during the measurement of the external quantum efficiency (EQE) is revealed, which can be explained by non‐selective contacts. The radiative loss of the VOC from sensitive subgap‐EQE measurements is calculated and it is revealed that the loss is caused by a low external luminescence yield and therefore a high non‐radiative recombination, supported by the first report of a strongly red shifted electroluminescence signal between 800 and 1000 nm. Altogether, these results point to a poor selectivity of the contacts and charge transport layers, caused by poor energy level alignment that can be overcome by optimizing the architecture of the solar cell.