Tracking the movement of all individual group members in their natural environment remains a challenging task. Using advances in computer vision and Deep Learning, we developed and tested a semi-automated in situ tracking system to reconstruct simultaneous three-dimensional trajectories of marked individuals in social groups of a coral-reef fish. Our system has a temporal resolution of 10s of milliseconds, allowing for multiple 30-min tracking sessions that have been repeated over weeks to months. We present the technique and illustrate its application for Dascyllus marginatus, a planktivorous damselfish that lives in social groups associated with branching corals. Our technique identified all individuals 85-100% of the time, with a mean spatial error of $ 1.3 cm. It provides a cost-effective semi-automated tool for in situ research on movements and foraging of individuals within small site-attached groups of animals in their natural environment.