Machine learning methods based on classifiers are more robust to system complexity, but they ignore the relations that exist in the data due to the physical laws governing the behavior of the system. In this paper we discuss how (partial) knowledge about the physical system can be integrated in the machine learning process. We focus on classification based diagnosis. We show how the partially known model is integrated in the classification algorithm, and how the new algorithm differs from a machine learning classifier based on neural networks. We demonstrate that by integrating the partial system knowledge, the cross-entropy optimization problem used for learning a classifier can be expressed as a set of regression problems in terms of the parameters of the model representing the unknown behavior, followed by a simpler classifier learning. We showcase our approach when diagnosis faults for a rail switch system.