Owing to the disparity between the computing power and hardware development in electronic neural networks, optical diffraction networks have emerged as crucial technologies for various applications, including target recognition, because of their high speed, low power consumption, and large bandwidth. However, traditional optical diffraction networks and electronic neural networks are limited by long training durations and hardware requirements for complex applications. To overcome these constraints, this paper proposes an innovative opto-electronic hybrid system that combines optical diffraction networks with electronic neural networks. Using scattering layers to replace the diffraction layers in traditional optical diffraction networks, this hybrid system circumvents the challenging training process associated with diffraction layers. Spectral outputs of the optical diffraction network were processed using a simple backpropagation neural network, forming an opto-electronic hybrid network exhibiting exceptional performance with minimal data. For three-class target recognition, this network attains a classification accuracy of 93.3% within a substantially short training time of 9.2 s using only 100 data samples (training: 70 and testing: 30). Furthermore, it demonstrates exceptional insensitivity to position errors in scattering elements, enhancing its robustness. Therefore, the proposed opto-electronic hybrid network presents substantial application prospects in the fields of machine vision, face recognition, and remote sensing.