Complexity and dynamical analysis in neural systems play an important role in the application of optimization problem and associative memory. In this paper, we establish a delayed neural system with external stimulations. The complex dynamical behaviors induced by external simulations are investigated employing theoretical analysis and numerical simulation. Firstly, we illustrate number of equilibria by the saddle-node bifurcation of nontrivial equilibria. It implies that the neural system has one/three equilibria for the external stimulation. Then, analyzing characteristic equation to find Hopf bifurcation, we obtain the equilibrium’s stability and illustrate periodic activity induced by the external stimulations and time delay. The neural system exhibits a periodic activity with the increased delay. Further, the external stimulations can induce and suppress the periodic activity. The system dynamics can be transformed from quiescent state (i.e., the stable equilibrium) to periodic activity and then quiescent state with stimulation increasing. Finally, inspired by ubiquitous rhythm in living organisms, we introduce periodic stimulations with low frequency as rhythm activity from sensory organs and other regions. The neural system subjected by the periodic stimulations exhibits some interesting activities, such as periodic spiking, subthreshold oscillation, and bursting-like activity. Moreover, the subthreshold oscillation can switch its position with delay increasing. The neural system may employ time delay to realize Winner-Take-All functionality.