We derived the formula for the detection probability, signal-to-noise ratio (SNR), and average bit error rate (BER) for the signal orbital angular momentum (OAM) state carried via non-uniformly correlated high-order Bessel–Gaussian beam propagation in a turbulent atmosphere. The wavelength, receiver aperture, beam width, strength of the turbulent atmosphere, and topological charge effect on detection probability, SNR, and average BER of the signal OAM state were demonstrated numerically. The results show that the signal OAM state with low topological charge, a small receiver aperture, a narrow beam width, and a long wavelength can improve the performance of optical communications systems under conditions of weak atmospheric turbulence. Our results will be useful in long-distance free space optical (FSO) communications.