Phishing attack, as a significant security concern in cyberspace, has continuously threatened organizations and Internet users. For organizations, the rise in the number of phishing target brands has instilled distrust and dissatisfaction in legitimate Internet users and even damaged brand equity. Therefore, more fine-grained phishing detection mechanisms are urgently needed. In this study, we propose PTI-NN, an effective model based on neural networks that uses category features and images to identify the target brands of phishing websites. We furthermore contribute a new dataset including 3,500 phishing websites and present thirty phishing category features, which facilitate pertinent phishing detection in the field of cyber security. In the proposed PTI-NN, an embedding-based DNN is constructed to process the category features, a 2D-CNN is constructed to process the images, and finally, a fully connected layer is used to predict the target brand of phishing websites. The experimental results show that our proposed model is able to classify seventy phishing-targeted brands with a high accuracy of 91.10%, which showcases the effectiveness of our method on the identification of phishing target brands.