A novel technique is proposed to implement optical signal-to-noise ratio (OSNR) estimation by using an improved binary particle swarm optimization (IBPSO) and deep neural network (DNN) based on amplitude histograms (AHs) of signals obtained after constant modulus algorithm (CMA) equalization in an optical coherent system. For existing OSNR estimation models of DNN and AHs, sparse AHs with valid features of original data are selected by IBPSO algorithm to replace the original, and the sparse sets are used as input vector to train and test the particle swarm optimization (PSO) optimized DNN (PSO-DNN) network structure. Numerical simulations have been carried out in the OSNR ranges from 10 dB to 30 dB for 112 Gbps PM-RZ-QPSK and 112 Gbps PM-NRZ-16QAM signals, and results show that the proposed algorithm achieves a high OSNR estimation accuracy with the maximum estimation error is less than 0.5 dB. In addition, the simulation results with different data input into the deep neural network structure show that the mean OSNR estimation error is 0.29 dB and 0.39 dB under original data and 0.29 dB and 0.37 dB under sparse data for the two signals, respectively. In the future dynamic optical network, it is of more practical significance to reconstruct the original signal and analyze the data using sparse observation information in the face of multiple impairment and serious interference. The proposed technique has the potential to be applied for optical performance monitoring (OPM) and is helpful for better management of optical networks.Photonics 2019, 6, 111 2 of 17 network resources. This phenomenon improves the transparency of an optical system but increases the complexity of network management [1]. The capacity of optical networks has been growing steadily, and the network architectures are becoming more dynamic, complex, and transparent. Optical signals in high-speed optical fiber networks are vulnerable to various transmission impairments, which can change dynamically with time. Therefore, an appropriate monitoring mechanism must be established throughout the optical network to provide accurate and real-time information on the quality of transmission links and the health of optical signals. optical performance monitoring (OPM) technology detects the performance parameters of transmission links or optical network nodes in optical networks, and understands the network status and signal transmission status in real time, and therefore processes and ensures the normal network operation and signal transmission. It is of great significance to guarantee the performance of signal transmission and optimize the allocation and management of network resources. Therefore, in order to realize the next generation dynamic reconfigurable optical network, it is necessary to monitor the important parameters, isolate faults in time, and optimize the processing to ensure the transmission quality of the network [2,3].The optical signal-to-noise ratio (OSNR) is one of the key parameters of OPM. It can realize fault management of an optical ...