Integrating devices with nanostructures is considered a promising strategy to improve the performance of solar energy harvesting devices such as photovoltaic (PV) devices and photo-electrochemical (PEC) solar water splitting devices. Extensive efforts have been exerted to improve the power conversion efficiencies (PCE) of such devices by utilizing novel nanostructures to revolutionize device structural designs. The thicknesses of light absorber and material consumption can be substantially reduced because of light trapping with nanostructures. Meanwhile, the utilization of nanostructures can also result in more effective carrier collection by shortening the photogenerated carrier collection path length. Nevertheless, performance optimization of nanostructured solar energy harvesting devices requires a rational design of various aspects of the nanostructures, such as their shape, aspect ratio, periodicity, etc. Without this, the utilization of nanostructures can lead to compromised device performance as the incorporation of these structures can result in defects and additional carrier recombination. The design guidelines of solar energy harvesting devices are summarized, including thin film non-uniformity on nanostructures, surface recombination, parasitic absorption, and the importance of uniform distribution of photo-generated carriers. A systematic view of the design concerns will assist better understanding of device physics and benefit the fabrication of high performance devices in the future.