Two multivariate models were compared to assess their yield predictability based on long-term (1980–2021) rice yield and weather datasets over eleven districts of Karnataka. Simple multiple linear regression (SMLR) and artificial neural network models (ANN) were calibrated (1980–2019 data) and validated (2019–2020 data), and yields were forecasted (2021). An intercomparison of the models revealed better yield predictability with ANN, as the observed deviations were smaller (−37.1 to 21.3%, 4% mean deviation) compared to SMLR (−2.5 to 35.0%, 16% mean deviation). Further, district-wise yield forecasting using ANN indicated an underestimation of yield, with higher errors in Mysuru (−0.2%), Uttara Kannada (−1.5%), Hassan (−0.1%), Ballari (−1.5%), and Belagavi (−15.3%) and overestimations in the remaining districts (0.0 to 4.2%) in 2018. Likewise, in 2019 the yields were underestimated in Kodagu (−0.6%), Shivamogga (−0.1%), Davanagere (−0.7%), Hassan (−0.2%), Ballari (−5.1%), and Belagavi (−10.8%) and overestimated for the other five districts (0.0 to 4.8%). Such model yield underestimations are related to the farmers’ yield improvement practices carried out under adverse weather conditions, which were not considered by the model while forecasting. As the deviations are in an acceptable range, they prove the better applicability of ANN for yield forecasting and crop management planning in addition to its use for regional agricultural policy making.