Abstract. The historical castles (castellated walls), which are cultural heritages in Japan, require regular maintenance, and it is necessary to record the arrangement of individual wall stones in the maintenance work. Recently, image processing techniques are practiced to optimize maintenance and management of the infrastructure assets. In the previous study, we proposed an automatic method for efficiently extracting individual wall stone polygons by improved multiscale image segmentation technique. However, the problem has remained that wall stone polygons could not be extracted properly when there were no clear gaps or boundaries between stones. To address this problem, we improved the multiscale image segmentation technique used in our previous studies. The first improvement is that in the region growing process, selecting the best combination of a plurality of objects instead of two. The second improvement is the modification of the shape criterion to be used. Besides, we proposed three-stage Stacked cGAN for wall stone edge detection that enables us to complement areas with weak or broken boundaries of stone edges. This approach is composed of a coarse-to-fine based image-to-edges translation network. The edge images derived from this method are used as the additional channel in multiscale image segmentation with a higher weight compared to the other RGB channels. It was confirmed that the separation performance of individual wall stone polygons was improved by the proposed method. Furthermore, the proposed method is highly effective to reduce the difficulty in setting of the scale parameter, which is usually sensitive to segmentation results and requires trial and error.