Accurate differentiation of energy consumption information of residential users is of great significance for load planning, scheduling, operation and management of power system, and is the basic premise for realizing intelligent perception of energy system and energy saving and carbon reduction. Considering that the conventional single-layer clustering method has limited clustering stability and clustering effect, this paper takes the key family feature factors as the modified feature quantity of quadratic clustering, and proposes a study of user energy characteristics based on double-layer clustering and modification. Firstly, the user’s energy consumption data is collected and pre-processed, and the user’s energy consumption curve is clustered and analyzed by using the integrated clustering algorithm based on voting and the advantages of each member algorithm. Then, the key family characteristic factors are obtained, and the results of one-layer clustering and key family characteristic factors are combined to carry out two-layer clustering of the same category of users in the form of questionnaire survey. Finally, the nonlinear mapping capability of Support Vector Machine (SVM) is used to reverse correct the results of the one-layer clustering. The actual algorithm data of the residents’ demand response experiment in a southeastern province are compared. The results show that compared with the single-layer clustering algorithm, the proposed method can accurately distinguish the energy consumption characteristics and adjustable potential of different users, and correct the wrong clustering results in the single-layer clustering. The clustering stability and clustering effect have been effectively improved.The example results show that the clustering results modified by SVM can better mine and distinguish user energy characteristics, and can be used to solve the problem of the current demand response clustering algorithm not being able to comprehensively and objectively describe the participation willingness and response-ability of residential users in the implementation process. It can also provide a basis for peak shaving and power grid frequency regulation.