This paper presents a systematic investigation of a ZnMgO/InN core-shell nanorods heterojunction device on a p-Si substrate. Here we demonstrated the heteroepitaxial growth of the well-aligned ZnMgO/InN core-shell nanorods structure, which enabled an increased heterojunction area to improve the carrier injection efficiency of nanodevices by plasma-assisted molecular beam epitaxy combined with metal-organic chemical vapor deposition. In situ X-ray photoelectron spectroscopy measurements were performed on the ZnMgO nanorods, the interface of ZnMgO/InN and the InN core-shell nanorods to fully understand the structure and working mechanism of the heterojunction device. The current transport mechanism has been discussed in terms of the characteristics of current-voltage and the energy band diagram of the n-InN/ZnMgO/p-Si heterojunction. At a low forward voltage, the current transport followed the dependence of I ∼ V(1.47), which was attributed to the deep-level assisted tunneling. When the forward voltage was larger than 10 V, the current followed the relation of I ∼ V(2) because of the radiative recombination process. In accordance with the above conclusion, the near-infrared electroluminescence of the diode could be observed after the forward bias voltage up to 11.6 V at room temperature. In addition, the size quantization effect and the intrinsic electron accumulation of the InN core-shell nanorods were investigated to explain the blueshift and broadened bandwidth. Furthermore, the light output power of about 0.6 microwatt at a fixed wavelength of 1500 nm indicated that our study will further provide a useful route for realizing the near-infrared electroluminescence of InN on Si substrate.