We show how the temporal drift of CCD amplifier bias can cause significant relative magnitude estimation error in speckle interferometric observations of multiple-star systems. When amplifier bias varies over time, the estimation error arises if the time between acquisition of dark-frame calibration data and science data is long relative to the timescale over which the bias changes. Using analysis, we show that while detectortemperature drift over time causes a variation in accumulated dark current and a residual bias in calibrated imagery, only amplifier bias variations cause a residual bias in the estimated energy spectrum. We then use telescope data taken specifically to investigate this phenomenon to show that for the detector used, temporal bias drift can cause residual energy spectrum bias as large or larger than the mean value of the noise energy spectrum. Finally, we use a computer simulation to demonstrate the effect of residual bias on differential magnitude estimation. A supplemental calibration technique is described in the appendices.