The minerals in raw anthracite significantly limit the preparation of coal‐based graphene. Raw and demineralized anthracites were graphitized and coal‐based graphene was prepared by improved Hummers’ oxidation‐reduction method. The morphologies, crystal structures and spectroscopic characteristics of the products in various stages were characterized by scanning electron microscopy, transmission electron microscopy, X‐ray diffraction, Fourier transform infrared spectroscopy, X‐ray photoelectron spectroscopy and Raman spectroscopy. The results show that demineralized or not, anthracite can be used to prepare graphene, but the minerals in anthracite physically inhibit the directional development of graphene sheets. And the residual high temperature conversion products of the minerals embedded between graphene sheets will lead to multiple irregular pore defects on the surface of graphene, which will have strongly unfavorable effects on the properties of coal‐based graphene materials. Demineralization of raw anthracite can reduce the defective pores and oxygen‐containing functional groups on the surface of coal‐based graphene and improves its morphology and properties.