In recent years, unmanned aerial vehicle (UAV) remote sensing systems have advanced rapidly, enabling the effective assessment of crop growth through the processing and integration of multimodal data from diverse sensors mounted on UAVs. UAV-derived multimodal data encompass both multi-source remote sensing data and multi-source non-remote sensing data. This study employs Image Guided Filtering Fusion (GFF) to obtain high-resolution multispectral images (HR-MSs) and selects three vegetation indices (VIs) based on correlation analysis and feature reduction in HR-MS for multi-source sensing data. As a supplement to remote sensing data, multi-source non-remote sensing data incorporate two meteorological conditions: temperature and precipitation. This research aims to establish remote sensing quantitative monitoring models for four crucial growth-physiological indicators during rapeseed (Brassica napus L.) seedling stages, namely, leaf area index (LAI), above ground biomass (AGB), leaf nitrogen content (LNC), and chlorophyll content (SPAD). To validate the monitoring effectiveness of multimodal data, the study constructs four model frameworks based on multimodal data input and employs Support Vector Regression (SVR), Partial Least Squares (PLS), Backpropagation Neural Network (BPNN), and Nonlinear Model Regression (NMR) machine learning models to create winter rapeseed quantitative monitoring models. The findings reveal that the model framework, which integrates multi-source remote sensing data and non-remote sensing data, exhibits the highest average precision (R2 = 0.7454), which is 28%, 14.6%, and 3.7% higher than that of the other three model frameworks, enhancing the model’s robustness by incorporating meteorological data. Furthermore, SVR consistently performs well across various multimodal model frameworks, effectively evaluating the vigor of rapeseed seedlings and providing a valuable reference for rapid, non-destructive monitoring of winter rapeseed.