Sub-terahertz (THz)-band communication system has drawn much attention as a promising technology to provide future-proof high data rate services. In a photonics-based sub-THz communication system, the generation of THz wave using free running lasers enables us to make a simple, cost effective and frequency tunable implementation. On the other hand, commercially available lasers have relatively broader linewidth and a large carrier frequency offset (CFO).To mitigate the performance degradations due to phase noise and CFO, a carrier recovery digital signal processing (DSP) algorithm is studied for a sub-THz transmission system. We propose a novel phase estimation algorithm to avoid cyclic slips while minimizing phase estimation error to improve BER performance. Our proposed phase recovery DSP algorithm is demonstrated in a 16-quadrature amplitude modulation (QAM) in a 0.3 THz band photonics-based transmission system. Experimental results show that the measured BER are improved from 8.810 -3 to 3.610 -3 in a 120 Gb/s 16-QAM transmission using the proposed algorithm. A wide range of CFO estimation is also supported for a sub-THz wireless transmission system using off-the-shelf lasers. Recovery of a CFO between -5 GHz and 5 GHz was also successfully demonstrated.