The inevitable existence of static internal imperfections and residual interactions in some quantum computer architectures results in internal decoherence, dissipation and destructive unitary shifts of active algorithms. By exact numerical simulations, we determine the relative importance and origin of these errors for a Josephson charge-qubit quantum computer. In particular we determine that the dynamics of a CNOT gate interacting with its idle neighboring qubits via native residual coupling behaves much like a perturbed kicked top in the exponential decay regime, where fidelity decay is only weakly dependent on perturbation strength. This means that retroactive removal of gate errors (whether unitary or non-unitary) may not be possible, and that effective error correction schemes must operate concurrently with the implementation of subcomponents of the gate.