Abstract-In this paper, we demonstrate wavelength conversion of advanced modulation formats such as 10-GBd 64-QAM and 256-QAM with high conversion efficiency over a 29-nm spectral window by using four-wave mixing in an AlGaAs-On-Insulator (AlGaAsOI) nano-waveguide. A thorough characterization of the wavelength converter is reported, including the optimization of the AlGaAsOI nano-waveguide in terms of conversion efficiency and associated bandwidth and the analysis of the impact of the converter pump quality and power as well as the signal input power. The optimized converter enables generating idlers with optical signal-to-noise ratio (OSNR) above 30 dB over a 29-nm bandwidth leading to error-free conversion of 64-QAM and 256-QAM with OSNR penalty below 1.0 dB and 2.0 dB respectively. The generated idlers exhibit an OSNR margin to the chosen forward error correction thresholds of >3 dB and >7 dB for 64-QAM and 256-QAM, respectively, that can be used for transmission after conversion.