Bandlimited optical intensity channels, arising in applications such as indoor infrared communications and visible light communications (VLC), require that all signals satisfy a bandwidth constraint as well as average, peak and non-negative amplitude constraints. However, the signaling designed for conventional radio frequency (RF) electrical channels cannot be applied directly, since they take energy constraints into consideration instead of amplitude constraints. In addition, conventional transmission techniques optimized for broad-band optical channels such as fiber optics, terrestrial/satellite-to-satellite free-space optical (FSO) communications are typically not bandwidth efficient.In this thesis, a two-dimensional signal space for bandlimited optical intensity channels is presented. A novel feature of this model is that the non-negativity and peak amplitude constraints are relaxed. The signal space parameterizes the likelihood of a negative or peak amplitude excursions in the output. Although the intensity channel only supports non-negative amplitudes, the impact of clipping on system performance is shown to be negligible if the likelihood of negative amplitude excursion is small enough. For a given signal space, a tractable approximation approach using a finite series is applied to accurately compute the likelihood of clipping under average and peak optical power constraints. The uncoded asymptotic optical power and iv spectral efficiencies using two-dimensional lattice constellations are computed. The Monte-Carlo (MC) simulation results show that for a given average or peak optical power, schemes designed in the presented signal space haver higher spectral efficiency than M -ary pulse amplitude modulation (PAM) using previously established tech- g 1 (t) first branch receive filterg 2 (t) second branch receive filterh(t) channel response I(t) instantaneous optical intensity n(t) channel noise n 1,i output sample noise after filter g 1 (t) at i-th symbol interval n 2,i output sample noise after filter g 2 (t) at i-th symbol interval n c,1,i clipping distortion after filter g 1 (t) at i-th symbol interval n c,2,i clipping distortion after filter g 2 (t) at i-th symbol interval q variable defined as the ratio of t to symbol period T Q(·) tail probability of the standard normal distribution R residual sum r 1 (t) output waveform of filter g 1 (t)