Drone swarms have gained a lot of popularity in recent times because, as a group, drones can perform highly intelligent tasks. Drone swarms are strongly inspired by the flocking behavior of birds, insects, and schools of fish, where all the members work in a coordinated manner to achieve a common goal. Since each drone is an independent entity, automating the control of a swarm is difficult. Previous works propose various swarming models with either centralized or distributed control. With distributed control, each drone makes its own decisions based on a small set of rules to accomplish swarm behavior, whereas in centralized control, one drone acts as the leader, who knows the final destination and the path to follow; it specifies the trajectories and velocities for the rest of the drones. Almost all the work in the area of swarming models follows Reynolds’ model, which has three basic rules. For GPS-aided settings, state-of-the-art proposals are not mature enough to handle complex environments with obstacles where primarily local decisions are taken. We propose a new set of rules and a game-theoretic method to set the values of the hyperparameters to design robust swarming algorithms for such scenarios. Similarly, the area of realistic swarming in GPS-denied environments is very sparse, and no work simultaneously handles obstacles and ensures that the drones stay in a confined zone and move along with the swarm. Our proposed solution SmrtSwarm solves all of these problems. It is the first comprehensive model that enables swarming in all kinds of decentralized environments regardless of GPS signal availability and obstacles. We achieve this by using a stereo camera and a novel algorithm that quickly identifies drones in depth maps and infers their velocities and identities with reference to itself. We implement our algorithms on the Unity gaming engine and study them using exhaustive simulations. We simulate 15-node swarms and observe cohesive swarming behavior without seeing any collisions or drones drifting apart. We also implement our algorithms on a Beaglebone Black board and show that even in a GPS-denied setting, we can sustain a frame rate of 75 FPS, much more than what is required in practical settings.