On-chip photonic neural network (PNN) is emerging as an attractive solution for artificial neural networks due to its high computing density, low energy consumption, and compact size. Matrix-vector multiplication (MVM) plays a key role in on-chip PNN, which can achieve high-speed multiply-accumulate operation. Most of the current schemes implement MVM by adopting wavelength division multiplexing technology to accumulate the power of different wavelengths together, resulting in using a mass of laser sources. Additionally, real-number-field MVM is inevitable for realizing precise PNNs, while limited by the nature of light, effective solutions to perform negative value computing are still inadequate. Here, we propose and demonstrate a PNN accelerator based on wavelength and mode hybrid multiplexing technology to reduce the use of multi-wavelength lasers, which can satisfactorily play real-number-field computing (including positive and negative domain) based on a newly presented transformation mapping approach, avoiding the demanding experimental setup and the sacrificing weight modulation depth. As a proof-of-concept, a fabricated accelerator for image convolution and letter pattern detection has been demonstrated successfully, achieving a computing density of 1.37 TOPS/mm2 under the 22.38 Gbaud modulation rate.