Covalent organic frameworks (COFs) are a novel family of porous crystalline materials utilized in various advanced applications. However, applying COFs as a hazardous organic acid gas sensor is substantial but still challenging. Herein, a phenylenediamine-based covalent organic framework (TPDA-TPB COF) featuring excellent crystallinity, ultrastable thermal stability, and high surface area was successfully constructed. Then, the TPDA-TPB COF-modified quartz crystal microbalance (QCM) sensor is fabricated by immobilizing the TPDA-TPB COF thin film on the gold-QCM chip. The fabricated TPDA-TPB COF-modified QCM sensor demonstrates a rapid response, excellent reproducibility, high selectivity, and sensitivity to formic gas, arising from hydrogen-bonding interactions between formic acid and the outermost layer of the TPDA-TPB COF, as determined by extensive analysis and density functional theory calculations. The basic sites of the TPDA-TPB COF, which are numerous due to its high nitrogen content, and the carboxylic acid groups present in formic acid exhibit efficient interactions. The sensitivity of the TPDA-TPB COF-modified QCM sensor was found to be 7.75 Hz ppm −1 at standard room temperature and pressure conditions, with a limit of detection (LOD) of formic acid down to 1.18 ppm, which is significantly below the workplace olfactory threshold limit of 5.0 ppm established by the Occupational Safety and Health Administration. The TPDA-TPB COF-modified QCM sensor exhibits remarkable detecting capabilities, making it highly attractive for detecting organic acid vapors in diverse applications that require superior performance.