The method of real-time estimation of weather, especially the amount of rainfall, by analyzing CCTV images is much cheaper than one using the existing expensive weather observation equipment. In this paper, we propose a method to find an estimation model function which has its input as CCTV images and output as the amount of rainfall. From the CCTV images, we propose an algorithm for selecting the number and size of the region of interest optimized for rainfall estimation, generating a data pattern graph showing a clear distinction from the number of region of interest, clustering the pattern data graphs, and estimating the amount of rainfall. Experiments using real CCTV images show that the estimation accuracy is over 80%.