Several bio-inspired underwater robots have been demonstrated in the last few years that can horizontally swim using different smart actuators. However, very few works have been presented on robots which can swim vertically, have a payload and resemble a jellyfish-like creature. In this work, we present the design, fabrication, and performance characterization of a new tethered robotic jellyfish, which is based on inflatable soft pneumatic composite (SPC) actuators. These soft actuators use compressed air to expand and contract, which help the robot to swim vertically in water. The soft actuators consist of elastomeric air chambers and very thin steel springs, which contribute to gaining faster motion of the biomimetic robot. A prototype of 220 mm in diameter and consisting of eight actuating units was fabricated and tested underwater in a fish tank. It reached a height of 400 mm within 2.5 s while carrying a dead weight of 100 g when tested at 70 psi (483 kPa) pressure. This high performance (160 mm/s on average speed) suggests that faster motion with a payload can be achieved by using SPC actuators. The inflatable structures help to flap the bell segments as well as in buoyancy effect for rapid vertical motion. The major achievement of this work is the ability to demonstrate a novel use of inflatable structures and biomimetic flapping wings for fast motion in water. The experimental and deduced data from this work can be used for the design of future small unmanned underwater vehicles (UUVs). This work adds a new robot to the design space of biomimetic jellyfish-like soft robots. Such kind of vehicle design might also be useful for transporting objects underwater effectively.