Compensation has been a common while unacknowledged strategy in the design of computing‐in‐memory (CIM) schemes. It enables efficient CIM designs by intentionally letting the sum of capacitances or conductances of two or more rows or columns in the memory array equal, thus resulting in a concise mathematical formula regarding the memory cell data and the input data, which constitute computing primitives. Here, the capacitance and conductance compensation methods are reviewed that have been used for CIM designs based on static random‐access memory (SRAM) in combination with capacitors and nonvolatile resistive memory, respectively, and uncover the underlying principles and their application to CIM. It is hoped this effort will help recognize the compensation methods as a building block for CIM designs, and will be an inspiration to developing more CIM schemes that are more compact in area, more efficient in energy consumption, and capable of solving more complicated problems.