Featured Application: This work can good candidate for parameter estimation and channel equalization in elastic optical networks (EON).Abstract: A joint compensation scheme based on pilot symbols aiding adaptive Kalman filter (AKF) for phase noise and polarization cross-talk is proposed and investigated via numerical simulation and experimental demonstration. In the proposed scheme, the optimizing parameter Q is adaptively adjusted according to signal parameters or channel conditions. This improvement avoids the drawback of conventional extended Kalman filter (EKF), and its performance is strongly dependent on Q. Another improvement is that the convergence speed of AKF is improved. Pilot quadrature phase shift keyin (QPSK) symbols are inserted into 16 quadrature amplitude modulation (QAM) signals periodically. Besides accelerating convergence speed, the employment of pilot symbols also improves the tracking capability of AKF. The format ratio between pilot symbols and payload symbols is suggested under different system environments, for instance, optical signal-to-noise rate, polarization rotation frequency drift rate, and laser linewidth. With the proposed scheme, it has excellent tolerance to initial parameter Q and dramatically improves the performances in convergence speed, polarization rotation frequency drift rate tracking, and carrier phase recovery. Both the numerical simulation and experimental demonstration achieve convergence improvement for around 40 times than the original AKF. Additionally, the improvements in tracking ability are also demonstrated.Regarding polarization tracking, the traditional algorithms are the constant modulus algorithm (CMA) and its variants [11][12][13]. The CMA algorithm is capable of tracking polarization rotation of up to 1 Mrad/s [14], which is not enough for meeting the requirements imposed by dynamic or burst optical detection scenarios.For coherent optical communication system, both polarization tracking and carrier phase recovery are essential. The algorithms that were mentioned above such as CMA+BPS can be used for polarization tracking and carrier phase recovery. While we also have another choice, the extended Kalman filter (EKF) has been demonstrated in coherent optical transmission systems [15][16][17][18][19]. This is a joint scheme and can be used for polarization tracking and carrier phase recovery simultaneously, the overall computational complexity is lower than CMA+BPS. However, it is commonly known that the tuning parameters' Q value is significant for determining EKF performance. The higher laser linewidth, the larger Q value should be configured theoretically. The same rule exists in the case of tracking polarization rotation. Therefore, an adaptive Kalman filter (AKF) scheme is expected, in which Q value can change along with the conditions in an intelligent manner [20]. The convergence speed should be improved further in the case of high-order modulation format to meet the requirement of the next-generation dynamic optical transmission.Based on our prev...