The current research of Charpy impact mainly focuses on obtaining the ductile brittle transition temperature of materials by experiments. Compared with experiments, numerical simulation can study many problems with harsh conditions. However, there are still few studies on the influence of geometric factors such as side grooves. In this paper, the geometry of standard Charpy impact test is designed. Specimens with different widths and side grooves are tested. The finite element model of Charpy impact was established by ABAQUS software. Use test results and simulation results to verify each other. The effects of sample width, side groove depth and side groove bottom fillet on the impact fracture resistance of the sample were studied. The results show that the specimen width is positively correlated with the impact toughness of the specimen. The side groove greatly reduces the impact toughness of the material; the toughness of side groove decreases with the increase of depth; the fracture toughness of side groove decreases with the increase of fillet at the bottom of side groove. The proportion of toughness energy to impact energy of samples was analyzed. The results show that the toughness energy accounts for about 70% of the impact energy of the sample, which has little to do with the geometric characteristics of the sample. This study presents a reliable method for studying Charpy impact tests. The influence of geometric parameters is obtained, which provides a reference method for the study of impact toughness of high toughness materials.