Damping is an important factor contributing to errors in the measurement of rotational inertia using the torsion pendulum method. Identifying the system damping allows for minimizing the measurement errors of rotational inertia, and accurate continuous sampling of torsional vibration angular displacement is the key to realizing system damping identification. To address this issue, this paper proposes a novel method for measuring the rotational inertia of rigid bodies based on monocular vision and the torsion pendulum method. In this study, a mathematical model of torsional oscillation under a linear damping condition is established, and an analytical relationship between the damping coefficient, torsional period, and measured rotational inertia is obtained. A high-speed industrial camera is used to continuously photograph the markers on a torsion vibration motion test bench. After several data processing steps, including image preprocessing, edge detection, and feature extraction, with the aid of a geometric model of the imaging system, the angular displacement of each frame of the image corresponding to the torsion vibration motion is calculated. From the characteristic points on the angular displacement curve, the period and amplitude modulation parameters of the torsion vibration motion can be obtained, and finally the rotational inertia of the load can be derived. The experimental results demonstrate that the proposed method and system described in this paper can achieve accurate measurements of the rotational inertia of objects. Within the range of 0–100 × 10−3 kg·m2, the standard deviation of the measurements is better than 0.90 × 10−4 kg·m2, and the absolute value of the measurement error is less than 2.00 × 10−4 kg·m2. Compared to conventional torsion pendulum methods, the proposed method effectively identifies damping using machine vision, thereby significantly reducing measurement errors caused by damping. The system has a simple structure, low cost, and promising prospects for practical applications.