Autonomous blimps have potential applications in surveillance, monitoring, and advertising. Due to their lower payload capacities and possible unstable flight profile, small blimps have been mostly used in indoor applications. However, recent advancements in their design and control have increased the prospects of deploying them for outdoor applications. This study presents a literature review of various aspects that encompass the unique elements of blimps' design and operations followed by a discussion on the modern applications. The review focuses on advancements made in the fundamental attributes of blimps, including design, propulsion, navigation, and control. The review reveals that recent successes in using blimps for various missions, where heavier-than-air platforms have been usually used, highlight the potential for blimps to offer a lightweight and low-risk alternative. The relatively stable operation of blimps in low winds and longer-duration hovering capability can make them a preferred platform for rescue, source localization, and space applications. However, their stability in extreme environments is a challenge, in addition to their lower mobility and problems in accurately predicting control parameters. Thus, this review concludes with an overview of blimps' futuristic applications of miniature blimps and recommendations to address the challenges in using them in real-world missions.