Various nanoparticles have been applied as chemical demulsifiers to separate the crude-oil-in-water emulsion in the petroleum industry, including graphene oxide (GO). In this study, the Janus amphiphilic graphene oxide (JGO) was prepared by asymmetrical chemical modification on one side of the GO surface with n-octylamine. The JGO structure was verified by Fourier-transform infrared spectra (FTIR), transmission electron microscopy (TEM), and contact angle measurements. Compared with GO, JGO showed a superior ability to break the heavy oil-in-water emulsion with a demulsification efficiency reaching up to 98.25% at the optimal concentration (40 mg/L). The effects of pH and temperature on the JGO’s demulsification efficiency were also investigated. Based on the results of interfacial dilatational rheology measurement and molecular dynamic simulation, it was speculated that the intensive interaction between JGO and asphaltenes should be responsible for the excellent demulsification performance of JGO. This work not only provided a potential high-performance demulsifier for the separation of crude-oil-in-water emulsion, but also proposed novel insights to the mechanism of GO-based demulsifiers.