Accurately estimating the breast muscle weight of broilers is important for poultry production. However, existing related methods are plagued by cumbersome processes and limited automation. To address these issues, this study proposed an efficient method for predicting the breast muscle weight of broilers. First, because existing deep learning models struggle to strike a balance between accuracy and memory consumption, this study designed a multistage attention enhancement fusion segmentation network (MAEFNet) to automatically acquire pectoral muscle mask images from X-ray images. MAEFNet employs the pruned MobileNetV3 as the encoder to efficiently capture features and adopts a novel decoder to enhance and fuse the effective features at various stages. Next, the selected shape features were automatically extracted from the mask images. Finally, these features, including live weight, were input to the SVR (Support Vector Regression) model to predict breast muscle weight. MAEFNet achieved the highest intersection over union (96.35%) with the lowest parameter count (1.51 M) compared to the other segmentation models. The SVR model performed best (R2 = 0.8810) compared to the other prediction models in the five-fold cross-validation. The research findings can be applied to broiler production and breeding, reducing measurement costs, and enhancing breeding efficiency.