The finite-difference frequency-domain (FDFD) method is an effective method for numerical simulation of electromagnetic fields. It has great advantages in dealing with electromagnetic scattering problems of complex structures and complex media. This method can transform the frequency-domain Maxwell equations into a linear system for solution by difference operation on the spatial grid. However, high-precision differential calculations can result in more memory consumption and a decrease in computational speed. In previous reports, subgridding technique is often used to solve such problems, where mesh refinement is only performed in local areas, while coarse mesh partitioning is still used in other areas. However, the refinement area can only be manually set, lacking flexibility and accuracy. Therefore, we propose a novel FDFD method based on adaptive grids, which uses the cartesian tree-based hierarchical grids to discrete the spatial domain. It can automatically refine the local grids according to the geometrical characteristic of the model to improve the accuracy of specific areas, without significantly increasing the number of unknowns, and has strong flexibility while improving the calculation efficiency. In this study, we use two levels of grids for adaptive grids construction, with a mesh size ratio of 3:1. Using second-order interpolation to handle the transmission problem of electromagnetic field components at different grid boundaries. The simulation results show that the computation speed of the adaptive grids FDFD system is faster than that of structured grids.