Space division multiple access (SDMA) is widely used in modern wireless communication systems to substantially increase spectral efficiency and attainable channel capacity. The SDMA technique implements an antenna array to separate different users according to their directions through beamforming. Adaptive beamforming techniques and direction of arrival (DOA) estimation methods have been extensively investigated and adopted in SDMA to adapt the array beam to each user while nulling out multiuser interference (MUI), thereby increasing the channel capacity. Recently, another spatial dimension, named array orientation, has been explored, which provides an additional degree of freedom for further increasing channel capacity and spectral efficiency of SDMA. The orientational beamforming (OBF) system is employed to exploit the new spatial dimension, which spatially filters signals according to the orientations of the transmitting and receiving arrays regardless of the signals' directions. Therefore, it can effectively separate co-channel users lying in close directions, which, together with low co-channel interference in the array orientation dimension of the spatial domain, further increase the attainable channel capacity. However, essential techniques required for the OBF system to achieve the expected increase in channel capacity have remained to be investigated in-depth so far. This thesis mainly focuses on new techniques for the OBF system to fill the gap.Considering the coexistence of the OBF system and other wireless communication systems, there are two kinds of interference that primarily deteriorate the performance of the OBF system, which are directional interference and MUI. Therefore, this thesis first studies algorithms for suppressing these two kinds of interference in the OBF system. For directional interference suppression, three adaptive OBF algorithms, namely the orientational maximum signal-to-interferenceplus-noise ratio (O-MSINR) and orientational linearly constrained minimum variance (O-LCMV) algorithms, and the orientational generalized sidelobe canceller (O-GSC),