To improve the mechanization levels of red cluster pepper harvesting in hilly and mountainous regions of southwest China, a crawler-type harvester is developed to suit the local planting and growth characteristics of red cluster pepper and to facilitate mechanized picking, conveying, and collecting processes. The design, supported by theoretical calculations and structural analysis, includes detailed studies of the picking, conveying, and collecting mechanisms, as well as the hydraulic system. Factors affecting picking efficiency were identified. ADAMS simulation was used to determine the optimum rotational speed range for the spring-tooth roller by analyzing its trajectory. A prototype was then built and field tested with forward speed and the spring-tooth roller’s rotational speed as variables to assess impurity, damage, loss, and hanging rates. Data from these tests were analyzed using Design Expert software, which created a mathematical model relating the test indices to the two variables. Optimum parameters were identified, resulting in a harvester configuration that achieved an average productivity of 0.21 ha·h−1 at a forward speed of 1.75 m·s−1 and a roller rotational speed of 181 r·min−1. The impurity rate was 26.7%, the loss rate was 6.1%, the damage rate was 2.3%, and the hanging rate was 4.2%, conforming to the industry standard DG/T 114-2019. This research provides a viable solution for mechanized harvesting of red cluster pepper in hilly and mountainous regions with small planting plots.