Integrated reference electrodes allow to deconvolute voltage contributions of anode and cathode and contribute to a better understanding of CO2 electrolyzers. However, in zero‐gap cell configurations, this integration can be challenging and obtaining error‐free data with such a setup is a non‐trivial task. This study compares five different methods to integrate a reference electrode into an alkaline zero‐gap CO2 electrolysis cell. Sources of error and measures to circumvent them are investigated and finite‐element simulation is used to gain a better understanding of observed effects. Placing a reference electrode into the inactive area of the cell is found to be a reliable method, as long as the placement of electrodes is sufficiently controlled. Sandwiching a wire quasi‐reference electrode between two membranes is especially useful for electrochemical impedance spectroscopy; however, it can affect the overall cell performance. Contacting the catalyst layer from the backside with a salt‐bridge is promising for localized measurements if sufficient reproducibility can be ensured.