We demonstrate a highly sensitive acoustic vibration sensor based on a tapered-tip optical fiber acting as a microcantilever. The tapered-tip fiber has a unique output profile that exhibits a circular fringe pattern, whose distribution is highly sensitive to the vibration of the fiber tip. A piezo transducer is used for the acoustic excitation of the fiber microcantilever, which results in a periodic bending of the tip and thereby a significant output power modulation. Using a multimode readout fiber connected to an electric spectrum analyzer, we measured the amplitude of these power modulations over the 10–50 kHz range and observed resonances over certain frequency ranges. Two types of tapered-tip fibers were fabricated with diameter values of 1.5 µm and 1.8 µm and their frequency responses were compared with a non-tapered fiber tip. Thanks to the resonance effect as well as the sensitive fringe pattern of the tapered-tip fibers, the limit of detection and the sensitivity of the fiber sensor were obtained as 0.1 nm and 15.7 V/nm, respectively, which were significantly better than the values obtained with the non-tapered fiber tip (i.e., 1.1 nm and 0.12 V/nm, respectively). The sensor is highly sensitive, easy to fabricate, low-cost, and can detect sub-nanometer displacements, which makes it a promising tool for vibration sensing, particularly in the photoacoustic sensing of greenhouse gases.