The Deep Residual Network (ResNet) learning model is known to achieve better accuracy and requiring shorter training time compared to other pre-trained learning models for image classifications and recognition. In this paper, the use of ResNet networks for semantic segmentation of coral reefs images was explored. Three ResNet networks (ResNet-18, ResNet-50, and ResNet-101) were evaluated and compared using 900 images as training dataset and 38 images as test dataset. The last three layers of the pre-trained ResNets were replaced with a set of layers that classified each pixel of the images into four classes: 'dead', 'alive', 'sand' and 'unknown'. A Softmax layer was introduced to reduce the imbalanced defects. Then, DeepLabv3+ employed the Atrous convolution to extract the features computed by applied CNN and segment the pixels of the object. ResNet-101 was shown to achieve better results compared to ResNet-18 and ResNet-50. Further analysis of the results implied that the class weightage assignment needs to be improved and more a larger training dataset should be acquired.