Gene Biomarkers is valuable to the medical mechanism and drug targets. But it is very hard to find the accurate biomarkers. The traditional bioinfomatical methods and machine learning both focus on binary classified methods. However, it is also important to find precised biomarkers for one disease consisting of many subtypes. Deep learning can deal well with multi-classification. But, finding gene biomarker is short of good related methods. LRP is one of interpretable methods in deep learning. It is more valid for MLP-likelihood networks. Since LRP method just finds the features for one sample not for a group composed of many samples with the common features. In this paper,we propose an improved LRP to represent features for a group, not for a single sample. In addition, to make up the inaccuracy of the method, we propose to use the feature intersection of genes from two MLP-likehood models as the biomarkers. To find the two MLP-likelihood models, we construct a two-way MLP-likelihood network ConvAttMLPA, inspired by NFM network. Then, select any two models from MLP, NFM and ConvAttnMLPA, to compute feature genes based on improved LRP. To validate this method, we propose an evaluation method and based on it, find the best genes as biomarkers.