Drought stress is one of the main threats to poplar plant growth and has a negative impact on plant yield. Currently, high-throughput plant phenotyping has been widely studied as a rapid and nondestructive tool for analyzing the growth status of plants, such as water and nutrient content. In this study, a combination of computer vision and deep learning was used for drought-stressed poplar sapling phenotyping. Four varieties of poplar saplings were cultivated, and 5 different irrigation treatments were applied. Color images of the plant samples were captured for analysis. Two tasks, including leaf posture calculation and drought stress identification, were conducted. First, instance segmentation was used to extract the regions of the leaf, petiole, and midvein. A dataset augmentation method was created for reducing manual annotation costs. The horizontal angles of the fitted lines of the petiole and midvein were calculated for leaf posture digitization. Second, multitask learning models were proposed for simultaneously determining the stress level and poplar variety. The mean absolute errors of the angle calculations were 10.7° and 8.2° for the petiole and midvein, respectively. Drought stress increased the horizontal angle of leaves. Moreover, using raw images as the input, the multitask MobileNet achieved the highest accuracy (99% for variety identification and 76% for stress level classification), outperforming widely used single-task deep learning models (stress level classification accuracies of <70% on the prediction dataset). The plant phenotyping methods presented in this study could be further used for drought-stress-resistant poplar plant screening and precise irrigation decision-making.