As a part of the smart city revolution, crowd management is an emerging trend and it can enhance the quality of life. Unmanned ariel vehicles (UAVs) can help in making the crowd management process more efficient and more accurate. UAVs can monitor and collect environmental-related surveillance data and share real-time information with each other and with the decision makers. However, the battery-operated UAVs communicate over the open public channel making the privacy and security of the UAVs a crucial element in mission-critical applications. The weaknesses of the existing scheme pave the way to design a new lightweight authentication scheme for UAV environments. In this article, we present a symmetric key primitive-based scheme and provide authentication among a user and a UAV through an intermediate control center. Due to usage of symmetric key and elliptic curve cryptography, the proposed scheme fulfils the performance requirements of the UAVs. The security of the proposed scheme is substantiated through BAN logic, along with a discussion on security features extended by the proposed scheme. The performance and security comparisons show that the proposed scheme provides adequate security and efficiency and can be practically deployed in real UAV environments.