The Industrial Internet of Things (IIoT) plays an important role in digital transformation of traditional industries towards Industry 4.0. By connecting sensors, instruments and other industry devices to the Internet, IIoT facilitates the data collection, data analysis, and automated control, thereby improving the productivity and efficiency of the business as well as the resulting economic benefits. Due to the complex IIoT infrastructure, anomaly detection becomes an important tool to ensure the success of IIoT. Due to the nature of IIoT, graph-level anomaly detection has been a promising means to detect and predict anomalies in many different domains such as transportation, energy and factory, as well as for dynamically evolving networks. This paper provides a useful investigation on graph neural networks (GNN) for anomaly detection in IIoTenabled smart transportation, smart energy and smart factory. In addition to the GNN-empowered anomaly detection solutions on point, contextual, and collective types of anomalies, useful datasets, challenges and open issues for each type of anomalies in the three identified industry sectors (i.e., smart transportation, smart energy and smart factory) are also provided and discussed, which will be useful for future research in this area. To demonstrate the use of GNN in concrete scenarios, we show three case studies in smart transportation, smart energy, and smart factory, respectively.