Refined tobacco plant information extraction is the basis of efficient yield estimation. Tobacco planting in mountainous plateau areas in China is characterized by scattered distribution, uneven growth, and mixed/intercropping crops. Thus, it is difficult to accurately extract information on the tobacco plants. The study area is Beipanjiang topographic fracture area in China, using the smart phantom 4 Pro v2.0 quadrotor unmanned aerial vehicle to collect the images of tobacco planting area in the study area. By screening the visible light band, Excess Green Index, Normalized Green Red Difference Vegetation Index, and Excess Green Minus Excess Red Index were used to obtain the best color index calculation method for tobacco plants. Low-pass filtering was used to enhance tobacco plant information and suppress noise from weeds, corn plants, and rocks. Combined with field measurements of tobacco plant data, the computer interactive interpretation method performed gray-level segmentation on the enhanced image and extracted tobacco plant information. This method is suitable for identifying tobacco plants in mountainous plateau areas. The detection rates of the test and verification areas were 96.61% and 97.69%, and the completeness was 95.66% and 96.53%, respectively. This study can provide fine data support for refined tobacco plantation management in the terrain broken area with large exposed rock area and irregular planting land.