Tracking insect movement in a social group (such as ants) is challenging, because they are not only visually identical but also likely to perform intensive body contact and sudden movement adjustment (start/stop, direction turning). To address this challenge, we introduced an online multi-object tracking framework by combining both the motion and appearance information of ants. We obtained the appearance descriptors by using the ResNet model for offline training on a small (N=50) sample dataset. For online association, cosine similarity metric computes the matching degree between historical appearance sequences of the trajectory and the current detection. We validated our method in both indoor (lab-setup) and outdoor video sequences. The results show that the accuracy and precision of the model are 99.22%±0.37% and 91.93%±1.46% across 46041 testing samples, with real-time tracking performance. Additionally, we offered a public dataset of ant tracking with 46091 samples for future research in relevant domains.