The objective of this study was to evaluate the effects of brown rice particle size on the apparent total tract digestibility (ATTD) of energy and nutrients in diets fed to pigs at four different stages and determine the optimal particle size (OPS) of brown rice for young pigs and adult sows. Eighteen weanling piglets (initial body weight (BW):10.2 ± 0.4 kg), 18 growing barrows (initial BW: 35.6 ± 1.5 kg), 24 gestating sows (initial BW: 220 ± 2.8 kg), and 24 lactating sows (initial BW: 208 ± 3.8 kg) were allotted to 1 of 3 or 4 diets based on completely randomized design with six replicates per diet.Within each stage, brown rice-soybean meal diets were formulated, and the only difference among diets was the brown rice used was ground to the specified particle size.Each stage lasted 19 days, including 7 days for cage adaptation, 7 days for diet adaptation, and 5 days for total feces and urine collection. For weanling and growing pigs, the results showed that pigs fed brown rice milled to 600 μm had a greater ATTD of dry matter (DM), gross energy (GE), and crude protein (CP) than pigs fed brown rice ground to 800 μm. However, there was no improvement in the ATTD of energy and nutrients for pigs fed brown rice milled to 600 μm versus 400 μm. The concentration of nitrogen (N) in feces significantly reduced (p < 0.01) as brown rice particle size decreased from 800 to 400 μm. However, there were no differences in phosphorus (P) output and absorbed P among diets. For gestating and lactating sows, a reduction in particle size from 1,000 to 800 μm significantly improved (p < 0.01) the ATTD of DM, GE, and CP in diets. However, there was also no improvement in the ATTD of energy and nutrients for pigs fed brown rice milled from 800 to 400 μm. In conclusion, considering the energy required for milling and nutrient digestibility, milling brown rice to 600 and 800 μm are recommended in diets for young pigs and adult sows, respectively. The OPS of brown rice for pigs at different physiological stages should be considered to economically and accurately formulate diets.