In this paper, we propose an unsupervised lightweight network with a single layer for ear print recognition. We refer to this method by MDFNet because it relies on gradient Magnitude and Direction alongside with responses of data-driven Filters. At first, we align ear using Convolution Neural Network (CNN) and Principal Component Analysis (PCA). MDFNet starts by generating a filter bank from training images using PCA. This is followed by a twofold layer, which comprises two operations namely convolution using learned filters and computation of gradient image. To prevent over-fitting, a binary hashing process is done by combining different filter responses into a single feature map. Then, we separately construct histograms for each of gradient magnitude and direction according to the feature map. These histograms are then normalized, using power-L 2 rule, to cope with illumination disparity. Several fusion rules are evaluated to combine the two histograms. The main novelty of MDFNet lies in its simple architecture, effectiveness, the good compromise between processing time and performance it provides along with its high robustness to occlusion. We conduct extensive experiments on three public datasets namely AWE, AMI and IIT Delhi II. Experimental results demonstrate the effectiveness of MDFNet, which achieves high recognition rates (82.5%, 97.67% and 98.96%, respectively), and outperformed several state of the art methods with a high robustness to occlusion. Experiments revealed also the actual need for considering ear alignment.