This research is aimed at the development of automatic drug image verification functions. Our verification system is composed of two stages. The first stage is an arbitrarily axis-aligned object detector, which is mainly based on a deep residual network and feature pyramid network (FPN). The detector predicts the rotation bounding boxes for drugs using multiscale feature maps generated by the FPN. Then, the rotation bounding boxes are axially aligned, and the drug image is cropped according to the axis-aligned bounding box. The second stage is a feature matcher, which is based on a feature embedding network. The embedding feature extracted by the feature embedding network is combined with the geometric feature obtained by the arbitrarily axis-aligned object detector to determine whether the drug in the image belongs to the category specified by the user. The database used in this research is an image database created by imaging drugs provided by domestic local medical centers. Our verification system achieved a false positive rate (FPR) of 0.047% in verification tasks of drugs of 21 categories.