In an attempt to mitigate and balance economic, environmental, and societal externalities that ports exert, port sustainability is becoming increasingly important as a key priority. This work embarks on an effort to explore the boundaries, provide a holistic view, and establish a knowledge map of contemporary research in the field of port externalities and impacts, by conducting a systematic literature review (SLR) by carrying out a three-stage procedure (planning, execution, and reporting), enhancing objectivity, while limiting errors and bias. The literature under study pertains to research domains focusing on: (a) identifying port externalities and impacts, (b) utilizing metrics and indicators to measure and quantify the positive or negative impact of port externalities, and (c) promoting an inclusive framework for sustainable port development. Our results include, among others, the classification of methodologies and particularly, of indicators established per type of externality to measure and monitor sustainable port performance, as well as identification of trends and gaps within the contemporary literature. We find considerable room for exploring new concepts and research paths within the domain of port impacts and externalities, both within individual dimensions of sustainability (economic, social, environmental), and on a multidisciplinary level.