Water quality and pollution is the main environmental concern for ports and adjacent coastal waters. Therefore, the development of Port Environmental Management systems often relies on water pollution monitoring. Computer vision is a powerful and versatile tool for an exhaustive and systematic monitoring task. An investigation has been conducted at the Port of Palma de Mallorca (Spain) to assess the feasibility and evaluate the main opportunities and difficulties of the implementation of water pollution monitoring based on computer vision. Experiments on surface slicks and marine litter identification based on random image sets have been conducted. The reliability and development requirements of the method have been evaluated, concluding that computer vision is suitable for these monitoring tasks. Several computer vision techniques based on convolutional neural networks were assessed, finding that Image Classification is the most adequate for marine pollution monitoring tasks due to its high accuracy rates and low training requirements. Image set size for initial training and the possibility to improve accuracy through retraining with increased image sets were considered due to the difficulty in obtaining port spill images. Thus, we have found that progressive implementation can not only offer functional monitoring systems in a shorter time frame but also reduce the total development cost for a system with the same accuracy level.