Nowadays, intelligent connectionist systems such as artificial neural networks have been proved very powerful in a wide area of applications. Consequently, the ability to interpret their structure was always a desirable feature for experts. In this field, the neural logic networks (NLN) by their definition are able to represent complex human logic and provide knowledge discovery. However, under contemporary methodologies, the training of these networks may often result in non-comprehensible or poorly designed structures. In this work, we propose an evolutionary system that uses current advances in genetic programming that overcome these drawbacks and produces neural logic networks that can be arbitrarily connected and are easily interpretable into expert rules. To accomplish this task, we guide the genetic programming process using a context-free grammar and we encode indirectly the neural logic networks into the genetic programming individuals. We test the proposed system in two problems of medical diagnosis. Our results are examined both in terms of the solution interpretability that can lead in knowledge discovery, and in terms of the achieved accuracy. We draw conclusions about the effectiveness of the system and we propose further research directions.