With the continuous development of smart ports, the construction of port information systems has become the construction focus of the future development of such ports. However, the existing performance evaluation system for port information systems represents a research gap. It cannot effectively achieve specific improvement in the construction of information systems for port services and capabilities. In this paper, an evaluation index system was forged from the three perspectives of the operating level, management capability, and economic efficiency. As a first step, starting from the current situation of Beibu Gulf Port (Port B) construction, the three dimensions of the evaluation index system were determined using the scheme of literature review, questionnaire research, and fieldwork. Additionally, through adversarial interpretive structural modeling (AISM), it is concluded that the port loading and unloading operation capability and service level are the fundamental factors for measuring the construction performance of the container terminal operation system (CTOS). The results are used as input to construct an analytic network process (ANP) model to obtain the index weights. Finally, the gray clustering method (GCM) is introduced to construct a quantitative evaluation model to quantitatively evaluate the construction performance of the Port B CTOS. The quantitative benefits brought by the construction of the CTOS are finally verified. The scores of the corresponding dimensions of the port before and after the construction of the CTOS system in Beibu Gulf Port are obtained experimentally. The conclusion shows that the construction of CTOS makes the port operating level module improve the most, from 40.023 to 70.733 points; at the same time, it is found that two aspects, i.e., green security and economic benefits, in current construction are directions requiring further work in future port construction. Finally, a visualized quantitative analysis and evaluation method for the performance of smart port construction is proposed.