Water treatment has experienced a surge in the adoption of membrane separation technology. Covalent organic frameworks (COFs), a class of metal‐free and open‐framework materials, have emerged as potential membrane materials owing to their interconnected periodic porosity, tunability, and chemical stability. However, the challenges associated with processing COF powders into self‐standing membranes have spurred the emergence of COF composite membranes. This review article highlights the rationale behind developing COF composite membranes and their categories, including mixed matrix membranes (MMMs) and thin film composite (TFC) membranes. The common fabrication techniques of each category are presented. In addition, the influence of COF additives on the performance of the resultant composite membranes is systematically discussed, with a focus on the recent progress in applying COF composite membranes in the separation of different categories of water pollutants, including organic ions/molecules, toxic solvents, proteins, toxic heavy metals, and radionuclides.