In comparison with coastal zones, islands are even more vulnerable to anthropogenic disturbance, especially to tourism and tourism-induced activities. Despite a great number of studies on either island tourism or island vulnerability reviewed in this paper, knowledge and practice of the impact from tourism upon island ecological vulnerability (IEV) still needs to be expanded. In this contribution, the IEV of four administrative regions in Zhoushan, China is assessed between 2012 and 2017 based on an "exposure (E)-sensitivity (S)-adaptive capacity (A)" framework and by means of coupling coordination degree modeling (CCDM) for determination of the overall development level of E-S-A subsystems in each region. The assessment results show that: (1) An index system consisting of 1 objective, 3 sub-objectives, 7 elements, and 20 indicators can be established and tested to reflect the IEV to tourism; (2) As the most attractive tourist destinations, Shengsi and Putuo inevitably have the highest IEV values; (3) Dinghai's moderate low level of IEV comes as a surprise, due to its direct connectivity to its neighboring coastal city of Ningbo; (4) The more balanced the coupling coordination degree (CCD) values among E-S-A subsystems are, the higher the IEV values in the four tourist destinations of Zhoushan. In conclusion, tourism can be a double-edged sword for islands, the overall benefits of which outweigh the negative impacts upon island ecological conditions.