This paper reviews recent progress on different high-speed optical short- and medium-reach transmission systems. Furthermore, a comprehensive tutorial on high-performance, low-cost, and advanced optical transceiver (TRx) paradigms is presented. In this context, recent advances in high-performance digital signal processing algorithms and innovative optoelectronic components are extensively discussed. Moreover, based on the growing increase in the dynamic environment and the heterogeneous nature of different applications and services to be supported by the systems, we discuss the reconfigurable and sliceable TRxs that can be employed. The associated technical challenges of various system algorithms are reviewed, and we proffer viable solutions to address them.