Heartbeat inter-beat interval (IBI) extraction is a crucial technology for unobtrusive vital sign monitoring, yet its precision and robustness remain challenging. A promising approach is fusing heartbeat signals from different types of unobtrusive sensors. This paper introduces HSF-IBI, a novel and universal framework for unobtrusive IBI extraction using heterogeneous sensor fusion. Specifically, harmonic summation (HarSum) is employed for calculating the average heart rate, which in turn guides the selection of the optimal band selection (OBS), the basic sequential algorithmic scheme (BSAS)-based template group extraction, and the template matching (TM) procedure. The optimal IBIs are determined by evaluating the signal quality index (SQI) for each heartbeat. The algorithm is morphology-independent and can be adapted to different sensors. The proposed algorithm framework is evaluated on a self-collected dataset including 19 healthy participants and an open-source dataset including 34 healthy participants, both containing heterogeneous sensors. The experimental results demonstrate that (1) the proposed framework successfully integrates data from heterogeneous sensors, leading to detection rate enhancements of 6.25 % and 5.21 % on two datasets, and (2) the proposed framework achieves superior accuracy over existing IBI extraction methods, with mean absolute errors (MAEs) of 5.25 ms and 4.56 ms on two datasets.