The research into rainfall-runoff plays a very important role in water resource management. However, runoff simulation is a challenging task due to its complex formation mechanism, time-varying characteristics and nonlinear hydrological dynamic process. In this study, a nonlinear autoregressive model with exogenous input (NARX) is used to simulate the runoff in the Linyi watershed located in the northeastern part of the Huaihe river basin. In order to better evaluate the performance of NARX, a distributed hydrological model, TOPX, is used to simulate the discharge as a reference, and runoff classification by cluster analysis is used to further improve the accuracy of runoff simulation. Based on the four statistics indexes of the Nash–Sutcliffe efficiency (NSE), correlation coefficient (CC), root mean square error (RMSE) and mean relative bias (Bias), the NARX model is capable of simulating the rainfall-runoff dynamic process satisfactorily, although there is a little underestimation of the peak flow. After runoff classification, underestimation has been improved, and discharge simulation driven by NARX based on runoff classification (C-NARX) is well consistent with the observation. It is feasible to take it as a promising method, which also can be seen as a good reference and replacement for the current rainfall-runoff simulation.