Infrared radiation (IR) features that are influenced by infrared emissivity ε and physical temperature Td have been successfully applied to the early-warning of landslides. Although the infrared emissivity of a rock is a key parameter to determine its thermal radiation properties, the effect of particle size on the infrared emissivity of rock fragments is unknown. So in this paper, granite, marble, and sandstone were used as examples to conduct infrared imaging experiments on rock fragments. Their equivalent emissivity was used to interpret the detected infrared emission, including that from indoor backgrounds. In addition, the characteristics of changes in equivalent emissivity were discussed with reference to changes in observation direction and zenith angle. Then, a computation model of equivalent emissivity based on multiple observation directions and zenith angles was built to reveal the change in equivalent emissivity with particle sizes. The result indicates that the indoor background radiation has a predominant direction just above the rock fragments. The maximum deviation of infrared brightness temperature (IBT) was 0.260 K, and the maximum deviation of equivalent emissivity among different observation directions and zenith angles was 0.0065. After eliminating the influence of directional and angle effects with the operation of normalization, the general law of equivalent emissivity for all rock fragments that change with particle size is consistent. The maximum equivalent emissivity occurs at particle size 5 mm in the condition of particle size larger than 1 mm, while the equivalent emissivity changes inversely with particle size in the condition of particle size smaller than 1 mm. Above all, this study contributes new cognitions to Remote Sensing Rock Mechanics, and provides valuable evidence for better thermal infrared remote sensing monitoring on loose slope landslides.