Being an efficient image reconstruction and recognition algorithm, two-dimensional PCA (2DPCA) has an obvious disadvantage in that it treats the rows and columns of images unequally. To exploit the other lateral information of images, alternative 2DPCA (A2DPCA) and a series of bilateral 2DPCA algorithms have been proposed. This paper proposes a new algorithm named direct bilateral 2DPCA (DB2DPCA) by fusing bilateral information from images directly—that is, we concatenate the projection results of 2DPCA and A2DPCA together as the projection result of DB2DPCA and we average between the reconstruction results of 2DPCA and A2DPCA as the reconstruction result of DB2DPCA. The relationships between DB2DPCA and related algorithms are discussed under some extreme conditions when images are reshaped. To test the proposed algorithm, we conduct experiments of image reconstruction and recognition on two face databases, a handwritten character database and a palmprint database. The performances of different algorithms are evaluated by reconstruction errors and classification accuracies. Experimental results show that DB2DPCA generally outperforms competing algorithms both in image reconstruction and recognition. Additional experiments on reordered and reshaped databases further demonstrate the superiority of the proposed algorithm. In conclusion, DB2DPCA is a rather simple but highly effective algorithm for image reconstruction and recognition.