In this paper, we characterized and reviewed the emergence of fundamental and extended losses that limit the efficiency of a photovoltaic (PV) system. Although there is an upper theoretical bound to the power conversion efficiency of solar cells, i.e., the Shockley Queisser limit, in a practical environment, the consideration of inevitable losses in a whole PV system is imperative to optimally harvest solar energy. In this regard, this study quantifies the losses from a PV cell level to the whole PV system. It was perceived that reported losses on the PV cell level included the low energy bandgap, thermalization, recombination (surface and bulk recombination), optical absorption, space charge region, finite thickness, and metal contact loss, and it was determined that cutting techniques mainly constrained the power conversion efficiency of the solar cell. Furthermore, the detailed PV array losses were classified as mismatch power losses, dust accumulation losses, temperature effects, material quality losses, and ohmic wiring losses. The unavoidable system losses were quantified as inverter losses, maximum power point tracking losses, battery losses, and polarization losses. The study also provides insights into potential approaches to combat these losses and can become a useful guide to better visualize the overall phenomenology of a PV System.