In this study, phosphorus-modified alumina with large pore size was synthesized through a coprecipitation method. The carbon-covered, phosphorus-modified alumina with large pores was prepared by impregnating with glucose and carbonizing to further improve the adsorption of organic dyes. The morphology and structure of these composites were characterized by various analysis methods, and Rhodamine B (RhB) adsorption was also examined in aqueous media. The results showed that the specific surface area and pore size of the phosphorus-modified alumina sample AP7 (prepared with a P/Al molar ratio of 0.07) reached 496.2 m2·g−1 and 21.9 nm, while the specific surface area and pore size of the carbon-covered phosphorus-modified alumina sample CAP7–27 (prepared by using AP7 as a carrier for glucose at a glucose/Al molar ratio of 0.27) reached 435.3 m2·g−1 and 21.2 nm. The adsorption experiment of RhB revealed that CAP7–27 had not only an equilibrium adsorption capacity of 198 mg·g−1, but also an adsorption rate of 162.5 mg·g−1 in 5 min. These superior adsorption effects can be attributed to the similar pore structures of CAP7–27 with those of alumina and the specific properties with those of carbon materials. Finally, the kinetic properties of these composites were also studied, which were found to be consistent with a pseudo-second-order kinetic model and Langmuir model for isothermal adsorption analysis. This study indicates that the prepared nanomaterials are expected to be promising candidates for efficient adsorption of toxic dyes.