Due to the repeated bearing of mechanical operations and natural factors, the container will suffer various types of damage during use. Adopting effective container damage detection methods plays a vital role in prolonging the service life and using function. This paper proposes a multitype damage detection model for containers based on transfer learning and MobileNetV2. In addition, a data set containing nine typical types of container damage is established. To ensure the validity and practicability of the model, we conducted tests and verifications in the actual port environment. The results show that the model can identify multiple types of container damage. Compared with the existing models, the damage detection model proposed in this paper can ensure the identification effect of various types of container damage, which is more suitable for the actual container detection situation. This method can provide a new idea of damage detection for container management in ports.