We propose an autoencoding sequence-based transceiver for communication over dispersive channels with intensity modulation and direct detection (IM/DD), designed as a bidirectional deep recurrent neural network (BRNN). The receiver uses a sliding window technique to allow for efficient data stream estimation. We find that this sliding window BRNN (SBRNN), based on end-to-end deep learning of the communication system, achieves a significant bit-error-rate reduction at all examined distances in comparison to previous block-based autoencoders implemented as feed-forward neural networks (FFNNs), leading to an increase of the transmission distance. We also compare the end-to-end SBRNN with a state-of-the-art IM/DD solution based on two level pulse amplitude modulation with an FFNN receiver, simultaneously processing multiple received symbols and approximating nonlinear Volterra equalization. Our results show that the SBRNN outperforms such systems at both 42 and 84 Gb/s, while training fewer parameters. Our novel SBRNN design aims at tailoring the end-to-end deep learning-based systems for communication over nonlinear channels with memory, such as the optical IM/DD fiber channel.blocks are not included in the design structure. Thus, the inter-block interference is treated as extra noise and, as a consequence, the achievable performance, in terms of CD that can be compensated and hence transmission distance, of such systems is limited by the block size.In this work, we address the limitations of the FFNN design for communication over nonlinear channels with memory by implementing sequence-based end-to-end deep learning transceivers using recurrent neural networks (RNN) [1,9]. RNNs have been recently demonstrated as a viable receiver-only signal processing solution in passive optical networks (PON) [10]. We use RNNs to design an end-to-end optimized fiber-optic system resilient to the nonlinearities and inter-symbol interference (ISI) present in IM/DD communications over dispersive channels. More specifically, since CD causes ISI from both preceding and succeeding symbols, we employ in our design bidirectional RNNs (BRNN) [11]. We operate the trained BRNN transceivers in a sliding window sequence estimation scheme (SBRNN), which allows us to estimate the data stream efficiently [12,13]. In contrast to [12,13], where the neural network processing is only at the receiver side, in our work we employ BRNN structures at both transmitter and receiver to allow end-to-end optimization of the transmission over the communication channel. Two variations of the recurrent cell in the RNN structure are examined, a straightforward vanilla concatenation as well as the long short-term memory gated recurrent unit structure (LSTM-GRU), specifically designed to handle long term dependencies in the sequence [1,[14][15][16]. We find that the LSTM-GRU design has a slightly superior bit error rate (BER) performance compared to the vanilla SBRNN, however at a higher computational cost.Our study shows that both SBRNN systems, specifically design...