A strategy for increasing the accuracy rate of internal combustion engine (ICE) fault diagnosis based on the probabilistic graphical model is proposed. In this method, a three-layer network with inference of probability is constructed, and both the material conditions and the signals collected from different engine parts are considered as the inputs of the system. Machine signals measured by sensors were processed in order to diagnose potential faults, which were presented as probabilities based on the components in layer 1, fault categories in layer 2, and fault symptoms in layer 3. The diagnosis model was built by using nodes and arcs, and the results depended on the connections between the fault categories and symptoms. The parameters of the network represented quantitative probabilistic relationships among all layers, and the conditional probabilities of each type of fault and relevant symptoms were summarized. Fault cases were simulated on a 12-cylinder diesel engine, and three fault types that often occur on ICEs were tested based on five different fault symptoms with different loads, respectively. The diagnostic capability of the method was investigated, reporting high accuracy rates.