This paper presents a deep neural network (DNN)‐based model to recognize fishing vessels. In Taiwan, the vast majority of small fishing vessels are not equipped with an automatic identification system (AIS). As a consequence, the staff in a fishing port administration become heavily loaded when monitoring and managing the fishing vessels accessing a port. The workload is expected to be eased using this work. For the first time in the literature, a captured fishing vessel image was converted to a 128‐dimensional embedding for recognition purposes. The presented model gave a false positive rate (FPR) as low as 1.13% and an accuracy up to 99.47% at threshold = 0.772379. Finally, all the performance metrics, namely, the true positive rate (TPR), the FPR, precision and accuracy, are actually functions of the threshold which can be specified by users to meet specific requirements. © 2022 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.