A critical challenge in environmental remediation is the design of adsorbents with proper pore size for the removal of organic pollutants. Three covalent organic frameworks (COFs) with different pore sizes were successfully prepared by a room-temperature solution-suspension method and used to remove a typical aryl-organophosphorus flame retardant [triphenyl phosphate (TPhP)] from aqueous solution. The prepared COFs showed strong acid resistance and thermal stability. The 1,3,5-triformylphloroglucinol (TFP) reacted with benzidine (BD) (COF2) and exhibited the highest sorption capacity for TPhP, followed by the reaction of TFP and 4,4″-diamino- p-terphenyl (DT) (COF3), and the reaction of TFP and p-phenylenediamine (COF1). Their adsorption equilibriums were achieved within 12 h, and COFs with a larger pore size have higher initial sorption rate but longer time to reach sorption equilibrium. According to the Langmuir fitting, the maximum sorption capacities of three COFs for TPhP were 86.1, 387.2, and 371.2 mg/g, respectively. The density functional theory calculation verified that COF1 with a small pore size prevents TPhP molecules from entering the pores, resulting in extremely low sorption capacity, whereas a relatively too large pore size (COF3) will decrease the sorption energy, which is also not conducive to the adsorption of TPhP. Moreover, the prepared COFs can selectively adsorb TPhP in the presence of coexisting compounds and had high removal of TPhP from actual municipal wastewater, showing a promising application potential for selective removal of micropollutants from water by precisely controlling the COF structure.