The construction of coastal infrastructure alters the natural hydrodynamics, leading to irreversible changes in coastal morphology. Furthermore, there has been an increasing concern about global climate change in recent years, which requires examining how climatic shifts impact the mechanisms that govern oceanic processes, the trends in morphological changes, the extent of the impacts, and the corresponding weightings assigned to coastal infrastructure. This study aims to assess the impacts of climate change on the wave distribution and coastal morphology around the two breakwaters perpendicular to the shoreline of Hsinchu Fishery Harbor in Taiwan at the end of the 21st century. The findings reveal that, by the end of the century, during extreme climatic events, such as typhoons (increasing the maximum wind speed and extreme sea levels), the wave heights around the Hsinchu Fishery Harbor, compared to the present day, may increase by 5.94% to 81.25%. Regarding the potential coastal morphological changes, erosion trends are evident around the harbor, but there is a pronounced tendency toward accretion in the sheltered area. The impact range and weightings of the Hsinchu Fishery Harbor breakwaters under climate change on coastal morphology were then determined based on historical bathymetric data and simulated bathymetric changes using the empirical orthogonal function (EOF) method. Under different-intensity climate change scenarios, the EOF analysis indicates no significant differences in the impact range and weightings on coastal morphological changes. In light of the analysis results, it is evident that structures have a much more substantial impact on coastal morphological change than climate change does.