The wavelength-division-multiplexed passive optical network (WDM-PON) has been generally regarded as a promising solution to the next-generation access network that will be required to deliver services over 40 Gb∕s. However, fiber dispersion often limits the capacity and reach of WDM-PONs. Compared with dispersion compensation fiber, which is bulky and expensive with significant power loss, digital signal processing is a more suitable way to mitigate chromatic dispersion in PONs. Furthermore, expense is a critical concern in the WDM-PON, due to its need for a large number of lasers and a complex wavelength control mechanism. One practical solution is to reuse the downstream (DS) signal as the carrier for the upstream (US) modulation. In this case, the residual DS signal after remodulation can seriously degrade US transmission. In addition, system performance can be deteriorated by the unwanted reflection as uplinks and downlinks share one wavelength. In this paper, we propose using modified duobinary (MD) coding in the DS to improve its dispersion tolerance and reduce the crosstalk between DS and US induced by remodulation and reflection. MD is a correlative level code that can reduce signal bandwidth and achieve DC balance. We demonstrate a 15 km WDM-PON delivering a 40 Gb∕s MD-coded signal in the downlink and a 10 Gb∕s on-off keying signal in the uplink. Compared with no coding, the maximal allowable extinction ratio of the DS signal (ER d ) is improved by 4 dB. Moreover, the reflection tolerance of the uplink and downlink is enhanced by 5 and 4 dB, respectively. In addition, investigations on the use of different equalizers in the DS to further suppress fiber dispersion confirm that the superior performance of nonlinear equalization in MD-coded transmission and that the network reach can be extended to 25 km by a nonlinear decision feedback equalizer. Index Terms-Dispersion; Passive optical network (PON); Reflection; Remodulation; Wavelength division multiplexing (WDM).Recently, we proposed the application of a dicode level code, which alters the signal spectrum to have zero content at the direct current (DC), in a wavelength-reused WDM-PON with 10 Gb∕s downlink and 2.5 Gb∕s uplink [15]. The results verify the great ability of dicode coding in suppressing remodulation noise and reflection noise, but the network capacity needs to be further upgraded to meet the rapidly growing bandwidth demand. For a higher transmission rate, such as 40 Gb∕s, the impact of fiber dispersion becomes much more significant and limits the http://dx.