The assessment of rock brittleness holds significant importance for understanding and predicting the mechanical properties and engineering behavior of rocks. Due to the lack of a unified definition of rock brittleness, numerous evaluation methods for brittleness indexes have been proposed by scholars both domestically and internationally in recent decades, resulting in diverse evaluation outcomes. In this study, we first summarize the existing rock brittleness evaluation methods and highlight their respective advantages and disadvantages. Subsequently, considering the pre-peak plastic deformation of the rock mass, the pre-peak brittleness index factor is introduced. Furthermore, taking into account the total energy consumed by the rock mass for failure after the peak, the post-peak brittleness index factor is proposed. These two components of the brittleness index describe the characteristics of different stages of the stress-strain curve, leading to the development of a novel brittleness index. The proposed method is then applied to evaluate the brittleness of both red-bed sandstone and cyan sandstone, revealing the variation of rock brittleness under different working conditions. Finally, three existing evaluation methods are selected to validate the rationality of the proposed method. The results demonstrate that for red-bed sandstone, the proposed brittleness index exhibits maximum values under natural conditions at all confining pressures. The four brittleness indexes consistently characterize the brittleness of red-bed sandstone under natural conditions. Under saturated conditions, the brittleness indexes exhibit different patterns of variation. For cyan sandstone, the three brittleness indexes—B7, B9, and Bnew—exhibit a similar trend in characterizing the brittleness of cyan sandstone under natural conditions and freezing-thawing conditions, while the trend of B17 is essentially opposite to that of the previous three indexes. The research findings provide guidance for the assessment of sandstone brittleness.