Rice is an important food crop extensively cultivated worldwide, and rice’s grain yield should be improved to meet future food demand. Grain number per panicle is the main trait that determines the grain yield in rice, and other panicle-related traits influence the grain number. To study the genetic diversity, 50 diverse Indian-origin germplasm were evaluated for grain number per panicle and other panicle traits for two consecutive seasons (Rabi 2019 and Kharif 2020). The maximum genotypic and phenotypic coefficient of variation was obtained for the number of spikelets and filled grains per panicle. The genotypes were grouped into eight clusters with Mahalanobis’ D2 analysis and six groups using Principal component analysis. Based on, per se, performance for grain number per panicle and genetic distances, six parents were selected and subjected to full diallel mating. The genotypes CB12132, IET 28749, and BPT 5204 were the best general combiners for the number of filled grains per panicle and other panicle branching traits, viz., the number of primary and secondary branches per panicle. The hybrid BPT 5204 × CB 12132 identified as the best specific combination for most of the studied panicle traits. The additive gene effects were high for the number of filled grains per panicle, the number of primary branches, and secondary branches, whereas non-additive gene action was high for the number of productive tillers and grain yield per plant. The information obtained from this study will be useful in rice breeding programs to improve panicle traits, especially the grain number, which would result in higher grain yield.