Accurately estimating large-area crop yields, especially for soybeans, is essential for addressing global food security challenges. This study introduces a deep learning framework that focuses on precise county-level soybean yield estimation in the United States. It utilizes a wide range of multi-variable remote sensing data. The model used in this study is a state-of-the-art CNN-BiGRU model, which is enhanced by the GOA and a novel attention mechanism (GCBA). This model excels in handling intricate time series and diverse remote sensing datasets. Compared to five leading machine learning and deep learning models, our GCBA model demonstrates superior performance, particularly in the 2019 and 2020 evaluations, achieving remarkable R2, RMSE, MAE and MAPE values. This sets a new benchmark in yield estimation accuracy. Importantly, the study highlights the significance of integrating multi-source remote sensing data. It reveals that synthesizing information from various sensors and incorporating photosynthesis-related parameters significantly enhances yield estimation precision. These advancements not only provide transformative insights for precision agricultural management but also establish a solid scientific foundation for informed decision-making in global agricultural production and food security.