To measure environmental noises, many noise measurement and mapping systems have been studied. However, these systems are costly because many measurement points are required to construct a detailed and three-dimensional noise map. In recent years, with advances in unmanned aerial vehicle (UAV) technology, a multirotor aircraft has been frequently used as general use. Although it could be applied to acoustic measurements, it causes loud noise as it must always rotate its propellers during flying. Herein, a noise and spatial mapping system with a blimp robot is proposed. The proposed system achieved a silent, slow, and omnidirectional movement with a balloon filled with helium gas. Furthermore, the simultaneous localization and mapping (SLAM) technique is applied for the system's positional tracking and surrounding spatial mapping with a stereo camera. To evaluate our system, three experiments were conducted. First, the propeller rotational noises of the proposed system were compared to a general recreational-use multirotor. Next, the acoustical effects of a blimp, such as reflection and diffraction, were measured to decide the microphone position. Finally, a preliminary experiment was conducted to construct a simple three-dimensional noise map in a large experimental room. The results show that the proposed system could construct a three-dimensional indoor noise map by combining the sound information and the positional information.