We describe a collaborative analysis study involving numerical models and observation data for the Tokyo metropolitan area, called the ULTIMATE (ULTra-sIte for Measuring Atmosphere of Tokyo Metropolitan Environment) project. It evaluates cloud microphysics schemes of numerical models using extensive observation data for the Tokyo area. We have access to a variety of remote sensing and in-situ data for the Tokyo area for operational and research purposes, particularly by enhancing observations for ground validation of the EarthCARE satellite, which is set to launch in 2023. This study focuses on using the dual-polarization Doppler weather radar, operated by the Japan Meteorological Agency. In terms of numerical models, we use and compare multi-models with various cloud microphysics schemes, including a global non-hydrostatic model, NICAM (Non-hydrostatic Icosahedral Atmospheric Model), and the regional model of the Japan Meteorological Agency, ASUCA (A System based on a Unified Concept for Atmosphere), together with a regional model, SCALE (Scalable Computing for Advanced Library and Environment) developed by RIKEN. In particular, because NICAM can be used as both a global and a regional model, we can immediately test the improved scheme on a global scale for its effect on climatology and the evaluation of climate sensitivity.This paper introduces the methodology for evaluating numerical models by the dual-polarization radar using the observation simulator and compares numerical model results with observations. We found discrepancies in the signals of rain and graupel between the observation and the simulations. In future studies, we will use the present results to improve the cloud microphysics scheme, which will be tested on a global model.