In this paper, we considered a delayed differential equation modeling two-neuron system with both inertial terms and time delay. By analyzing the distribution of the eigenvalues of the corresponding transcendental characteristic equation of its linearized equation, local stability criteria are derived for various model parameters and time delay. By choosing time delay as a bifurcation parameter, the model is found to undergo a sequence of Hopf bifurcation. Furthermore, the direction and the stability of the bifurcating periodic solutions are determined by using the normal form theory and the center manifold theorem. Also, resonant codimension-two bifurcation is found to occur in this model. Some numerical examples are finally given for justifying the theoretical results. Chaotic behavior of this inertial two-neuron system with time delay is found also through numerical simulation, in which some phase plots, waveform plots, power spectra and Lyapunov exponent are computed and presented.