The ease of use and advancements in drone technology is resulting in the widespread application of Unmanned Aerial Vehicles (UAVs) to diverse fields, making it a booming technology. Among UAVs' several applications, livestock agriculture is one of the most promising, where UAVs facilitate various operations for efficient animal management. But the field is characterized by multiple environmental, technical, economic, and strategic challenges. However, the use of advanced technological techniques like Artificial Intelligence (AI), Internet of Things (IoT), Machine Learning (ML), Deep Learning (DL), advanced sensors, etc. along with the assurance of animal welfare while operating the UAVs, can lead to widespread adoption of drone technology amongst livestock farmers. This paper discusses livestock management research where UAVs monitor farm animals via detection, counting, tracking animals, etc. In this article, an attempt has been made to elucidate different aspects and broader issues around livestock management while highlighting the associated challenges, opportunities, and prospects. This work is the first review paper on the subject matter with all the necessary information and analysis, to the best of our knowledge. Therefore, the article promises to provide interested researchers with detailed information about the field, guiding future research.